Response vs non-response bias in surveys + [Examples] (2023)

When conducting surveys, one of the things researchers need to avoid to get accurate results is bias from response and non-response. These biases can come from the researcher or from the respondents.

For researchers, this can come from indirectly putting words in the respondent's mouth, or narrowing your choices to what you want. In this way, they have no choice but to choose what the seeker wants.

Respondents can also be the source of bias by intentionally providing subjective answers to questions posed by researchers. However, there are several reasons why a respondent may provide biased answers to questions.

What is response bias?

Response bias is a situation where a respondent or participant provides inaccurate or incorrect answers to a question. In research, it is very common for participant self-reports, questionnaires, surveys, interviews, etc. Response bias can be caused by several factors, mainly the human factor, since robots do not necessarily give a direct answer to questions.

Response bias cannot be completely avoided as it is fundamental to any survey. Giving biased answers to questions can be intentional or accidental. Whatever the case, it completely interferes with the search and renders the collected data unusable.

Types of Response Bias

Social Desirability Bias

Social desirability bias is a type of response bias in which respondents are more likely to provide socially acceptable answers than sincere answers to questions. The questions in this case are often phrased in a certain way that can lead people to give in to social conditioning.

Even when people have different personal opinions, in an attempt not to look bad, they give the socially acceptable answer. This could also refer to cases where people want to form a class or belong to an elite group, i.e. lie in order to present themselves better.

A respondent is likely to choose "no" in this case, even if they have already cheated on an exam and don't think there is anything wrong with it. This is because society has conditioned cheating to be viewed as a bad thing.

demand characteristics

This occurs in cases where respondents give a specific answer to a question because they think that is the answer the survey requires. For example, a company that wants to improve the quality of its product could send out a survey to find out what changes customers should make about the product's design.

In this case, the customer may not have any suggestions for improving the questionnaire. But since he thinks the company needs answers, there will be an answer anyway.

Few respondents care about brand identity. However, many will choose "yes" because they think that's what the brand wants to hear.

Extreme reaction

Extreme response bias occurs when respondents give exaggerated answers to a question. This can be done to make yourself look good or to make someone else look bad on the quiz.

For example, a respondent who has been to a restaurant might be asked how the food was and s/he gives an exaggerated response, detailing how awful it was. Some may even do this to get an extra plate of food as compensation at a restaurant.

Let's say a philanthropist gives away $5,000 to those in need and this is a survey question. Most people will choose the "More than enough" option to represent needs, desires, and the pursuit of compassion, even if it's very small.

This is because the “Too Small” option can make the respondent appear proud and dissatisfied.

Neutral answer

This happens when the respondent gives a neutral answer to all the questions asked. Many respondents do this when they are undecided or have no idea of ​​an answer to the question being asked.So they give neutral answers to all questions. This type of reaction is just as bad as the extreme reaction.

The image below is an example of neutral response bias.

toleration biased

toleration biasedIt is a form of response bias in which respondents give affirmative answers to all questions asked. In some cases, they don't bother to read the questions before choosing affirmative answers.This type of response bias is usually the respondent's fault and is usually intentional. This response bias is usually easy to spot, as it leads to a number of contradictory statements from respondents.

By choosing a positive answer to all questions, the respondent made a contradictory statement. How can you be an atheist and still believe in God?

deviant bias

Dissent bias is the opposite of agreement bias and is a form of response bias in which respondents give negative answers to all questions asked. This answer bias is mostly intentional as they generally don't try to read the questions.

This can happen because the questions are too much for the respondent to answer. So at a certain point, respondents just start picking random answers to the questions.

Reasons for response bias

• emotional questions

Some questions can appeal to a person's emotions and cause them to give biased answers. This is due to the emotional nature of humans.

For example, here's an emotional approach to asking someone if they plan on having children soon.

Her parents are getting older and want to see their grandchildren. Could you imagine having a child soon? 🇧🇷

The questionnaire might as well have asked if the respondent was going to have a child soon. But in order to address the person's emotions, the current condition of the parents was taken into account.

• reward questions

Many online survey platforms reward respondents for taking surveys. Before taking a paid survey, respondents typically go through a pre-selection phase to confirm that they are eligible to take the survey.

At this point, respondents want to qualify for these surveys so they can complete them and be rewarded. Hence, they resort to lies for rewards.

For example, a survey company may need people between the ages of 40 and 60 to conduct a survey. They then ask respondents if they fall into that age group during the pre-selection phase.

Some respondents will say yes, even if they don't fall into that age bracket, just to qualify for the award.

• Complex questions:

Respondents often find it difficult to answer some questions because they are so complex. So you just pick a random answer to the question.

The response bias in this case is usually intentional and may be related to multiple choice questions in the education system. When students are graded and asked to choose an answer from the options, they can choose a random answer if they aren't sure which is correct.

How to avoid answer bias

• Stop asking emotional questions. Questionnaires must stop asking questions emotionally in order to only give respondents objective answers.
• keep it simple The questions should be simple enough for a layperson to understand. Don't use large grammar or confusing sentences.
• Embrace anonymity. Respondents are known to be more direct and honest when anonymous. In this way, there is no need for a social desirability bias since the answer cannot be attributed to them.
• Add response validation. Researchers must validate responses so respondents cannot provide biased answers to a question.

For example, if a respondent answers "yes/no" to a question, he/she should not answer "yes/no" to another question if it contradicts the first answer. You can do this withLogic function in Formplus form builder.

• Allow respondents to save and continue later. Some respondents choose random answers to a question when they are tired and the questions are still too many.

By being able to save and continue later, they can take a break when they're tired and react properly when they're ready.

Disadvantages of Response Bias

• Response bias produces irrelevant data. Researchers often cannot work with response biases because they are often wrong. If they use them anyway, just formulate your own answers.
• This leads to incorrect statements or conclusions. If the researcher fails to notice the gaps in the data collected and the green light to use it, an incorrect conclusion will be drawn.
• This leads to an expensive data collection process. Researchers may need to go through another data collection process if the first is irrelevant. So it costs more money and time.
• This can cause a company to suffer a loss. Many companies use the opinions of respondents to make relevant decisions, for example to launch a new product. Response bias answers can influence a business decision that may ultimately be wrong.

What is non-response bias?

Non-response bias is a type of bias that occurs when people are unwilling or unable to respond to a survey because of some factor that makes them different from the people who responded. The difference between non-responders and responders is often a factor influencing non-response.

Non-response bias, sometimes referred to as participation bias, can result from poor survey construction and poor questionnaire alignment. It could also be due to an irrelevant decision by the defendant.

For example, a poll asking for the best brand of alcohol aimed at older religious people is unlikely to get an answer. In other cases, the survey may not even reach the target participant, e.g. B. an email that ended up in the spam folder.

Examples of non-response bias

example one

Let's imagine a database of 1000 email addresses of older people who only use their email accounts to contact their children. In most cases we can see that these people only know how to email and read what they were taught by their children.

Now imagine a person who is about to open a new club and needs to conduct a competitive analysis by asking a few questions about existing nightclubs. It is clear that if this survey is sent to these 1000 email addresses, about 90% will not respond.

A nightclub will not go down as well with older people as with young people. Well, unless it's an older people's club.

example two

Many young people watch adult videos on their internet-enabled devices, but most of them are too shy to talk about it. So take a poll with questions like: How often do you watch adult videos? What is your favorite adult video site? may not get as many replies as it should.

That's because they're too embarrassed to talk about it. A better way to get more answers to such questions is to allow them to remain anonymous when answering these questions. Even so, some will choose not to respond for fear of being exposed in the future.

Reasons for non-response bias

• Request for confidential information

Consider a survey that measures the rate at which separated parents comply with child support policies. A parent who does not pay child support regularly will feel very uncomfortable completing this survey.

Hence dealing with a bias that skewed the data towards a more law-abiding net sample than the original sample. This reaction is evident, and we can also say that polls that specifically state their involvement in a government agency are likely to receive a non-response bias.

• Invitation by email/SPAM

Sometimes, researchers are often the cause of non-response bias because they don't properly pre-test their invites. Pre-testing is very important when sending email invitations, which is why email sending platforms allow you to send a test email to yourself first.

This way you can confirm that the email plays well on both mobile and PC. It's well known that most young adults respond to email on their mobile devices, and when mobile search isn't working well, responses from smartphone users will drop dramatically.

Sometimes the email ends up in spam, so potential respondents can't see it.

• wrong audience

Choosing the right audience is very important when sending out surveys. For example, if you send a survey about weekly hours worked and earnings to college students or unemployed recent graduate students, you may not get as many responses as you would with employees.

The same applies if a survey on the number of exam hours is not answered by non-students.

Disadvantages of non-response bias

• This invalidates the results of an investigation or survey.
• This can lead to larger discrepancies in the estimates because the sample size the researcher ends up with is smaller than expected.
• This can lead to unsuccessful searches.

How to avoid non-response bias in polls and polls

• Use closed questions

closed questionsare usually more direct and do not require descriptive responses from respondents. Therefore, it is easy for them to give their answers.

In this case, respondents will not give up questions because they find it difficult to give answers. You can also control the type of replies you receive.

Easily create closed questions with Formplus' Choices feature. For example:

1. How many glasses of water do you drink daily?
2. less than 1
3. 1
4. 2
5. 3
6. 4 or more

• Your questions must be very neutral

Don't try to put words in your conversation partner's mouth by limiting the options to only answers that suit you. Researchers must put their personal opinions aside when creating a questionnaire.

That said, don't ask;

How good was President Trump's State of the Union address?

• Big
• very good
• Better than any other President.

Instead say:

What was President Trump's State of the Union address like?

• Arm
• Messe
• Geb
• very good
• Big.

In the first example highlighted above, the options limited respondents' responses to only saying something positive about President Trump. This may not sit well with other non-Trump supporters and they may decline to respond.

The second example, on the other hand, leaves room for both positive and negative opinions.

• Avoid double questions

Double questions are questions that address more than one topic, but allow only one answer. This is usually very confusing and can discourage respondents from answering.

An example of a duplicate question would be the following;

"Do you think our chef and waiter were great?".

“Do you think our chef is great” and “Do you think our waiter is great?” were combined.

Aside from confusing the respondent, not even the researcher can draw adequate conclusions from this type of data.

• Understand your target audience

It's important to understand your audience in order to know how the question should be asked. For example, when dealing with older people, the words should be formal as this is more appealing to them.

In younger people, this may not be the case.

• Make sure your options cover the possible answers you need.

Always make sure your options are comprehensive and cover all possible answers.

For example, when asking a person their gender, having male and female options is no longer enough.

This is because some people identify as non-binary, transgender, etc. Therefore, other non-binary genders may react if you add these options to the options available.

• provide incentives

Paid surveys are one way to incentivize respondents. This way you can get the answers you need for your search.

You do not need to control the survey as many companies are available to assist you in handling paid surveys.

Conclusion

Bias can be one of the things that cannot be avoided when conducting surveys. However, they can be reduced by making a conscious effort to prevent them.

The first step in doing this is to understand the response bias and non-response bias that can be introduced by either the researcher or the respondent. Sometimes these biases can be unintentional, as in the case where a respondent forgets to fill out a survey.