Research Knowledge about Online Survey
- Posted by IRL.Vietnam
- On May 4, 2023
To celebrate and promote the contribution of data, research, and insights to the development of Vietnam’s society, Indochina Research Vietnam (IRL), as a member of European Society for Opinion and Market Research (ESOMAR), celebrates the International Market Research Day by launching series of articles to promote ESOMAR guidelines and research standards for more ethical, professional, and high-quality research practices. Online Survey is the first topic sharing by IRL in these series.
Market research is the fact of gathering, analysing and interpreting information about individual needs and preferences. It is both a science and an art to help businesses and institutions better understand their targeted populations and to make informed decisions about the demand and viability of a new service, programme, or product. As among the longest standing market research agency in Vietnam, Indochina Research (IRL) Vietnam has provided research services to support clients from various sectors and industries to capture the perceptions and interests of Vietnamese people.
Online Survey Prospect in Vietnam
With the current growing preference for online surveys due to it perceived fast turn-around and cost-effectiveness, there raises both challenges and opportunities to ensure data quality and validity. One of IRL Vietnam action is to explore the behaviour of online respondents. In a large-scaled face-to-face nationwide Population survey of 4,747 samples in 2022, we asked whether they had ever completed any online survey before, and results shows that only a small proportion (2.7%) had done so, significantly skewed to younger generations (93%). This indicates that although the overall internet penetration rate in Vietnam was approximately 74% (World Bank, 2021)[1] but the reach and coverage level of online survey methodology remain limited.
[1] World Bank (2021) ” Individuals using the Internet (% of population) – Vietnam”, The World Bank Data Portal, accessed at https://data.worldbank.org/indicator/IT.NET.USER.ZS?locations=VN, 01 May 2023.
Besides the low reach, the skewedness[1] towards younger generations of Gen Y (47%) and Gen Z (46%) could lead to a potential bias in the data collected through online surveys, as the perspectives and experiences of older age groups may be underrepresented. For example, the population of over 59 years old (Boomers) are invisible in our experiment.
Another bias also observed with urbanity ratio when the online survey ratio is 48% vs urban: 52% rural while the actual ratio is 35 : 65% (GSO, 2019)[2].
[1] Small sample size
[2] GSO (2019) “Công bố kết quả Tổng điều tra dân số 2019”, GSO Online Data Portal, 13 July, accessed at https://datacollection.gso.gov.vn/dieutralaodongvieclam/cong-bo-ket-qua-tong-dieu-tra-dan-so-2019, 01 May 2023.
Thus, researchers need to take into account this reality when commissioning online research, and consider other modes, such as Face to Face (CAPI), Telephone (CATI), or mix-method, when collecting data in Vietnam, to ensure reaching a representative sample of the population.
Online Survey Behaviour – 4 out of 10 do not fully read all questions and answers
In this experiment, we also asked respondents with previous experience in online survey (n=131) about their survey behaviour. Survey behaviour refers to the actions, attitudes, and responses of individuals when an individual participates in a survey or answer a questionnaire.
We wanted to measure levels of satisficing behaviour, to understand if online respondents were likely to provide quick, satisfactory answers rather than taking the time to carefully consider each question, like it would be done when assisted by an interviewer. Our questionnaire described different levels of decision-making effort and the extent to which individuals were willing to settle for a less-than-optimal answer, with 4 codes accordingly to the three categories of satisficing behaviour, as specified in the Table below, including Optimal answers, weak Satisficing, Strong satisficing.
It is reported that 6 in 10 (61%) respondents self-declared that they carefully evaluated all possible alternatives and selected the most applicable one while 32% stated that they asserted less level of effort when completing the questionnaire when scanning through the answers quickly and settling for an option that is good enough. It is worth noting that 7% confessed that they randomly choose the answers, which would greatly impact the quality of the data.
Understanding the different levels of satisficing behaviour can help design surveys (collection method and questionnaire) that minimize the impact on data quality. Different techniques such as randomizing response options, reducing survey length, and simplifying question wording, should be employed to encourage respondents to put more effort into their answers and reduce the likelihood of satisficing. Other strategies to ensure the quality of your survey data and improve the reliability and accuracy of findings are conducting face-to-face survey pilots to observe respondents behaviour and get feedbacks on unclear questions, improve language, or adapt question types.
Concluding Remarks
There is potential for online surveys to become more popular in Vietnam, however, it is important to recognise and critically evaluate the limitations and potential biases of this method, including:
- Coverage bias: Online surveys can only reach people who have access and are willing to answer survey via internet, which results in data bias and the exclusion of some rural, or marginalised, vulnerable, populations.
- Self-selection bias: Online surveys rely on people voluntarily choosing to participate which can lead to self-selection bias and skew the results to population looking for rewards (incentive) in compensation.
- Sampling bias: Even with efforts to make the survey representative of the population, there may still be sampling bias in the data. This means that the sample of people who participate in the survey may not accurately reflect the larger population (in geographic area for instance), which can lead to inaccurate conclusions being drawn.
- Response bias: People may respond differently to questions depending on the mode of survey administration, which can lead to response bias. For example, people may be more likely to give kind of opposite views in an online survey than they would be in a face-to-face interviews.
Furthermore, it is also challenging for online surveys to address geographic legal, ethical, and practical considerations and control sample quality. To know more about online survey and quality aspects, we are happy to share the ESOMAR 5European Society for Opinion and Market Research), guidelines as below.
Overall, while online surveys can be a valuable tool, it is important to consider their limitations and potential biases when commissioning such research and interpreting the results. Combining online surveys with other modes of data collection, such as F2F CAPI or CATI, can help to mitigate some of these weaknesses and provide a more complete picture of the population.
Please contact us for all your research projects in South East Asia
[email protected].
Indochina Research, Regional know-how you can trust!
0 Comments