Response vs non-response bias in surveys + [Examples] (2023)

When conducting surveys, one of the things researchers need to avoid to get accurate results is bias from response and non-response. These biases can come from the researcher or from the respondents.

For researchers, this can come from indirectly putting words in the respondent's mouth, or narrowing your choices to what you want. In this way, they have no choice but to choose what the seeker wants.

Respondents can also be the source of bias by intentionally providing subjective answers to questions posed by researchers. However, there are several reasons why a respondent may provide biased answers to questions.

What is response bias?

Response bias is a situation where a respondent or participant provides inaccurate or incorrect answers to a question. In research, it is very common for participant self-reports, questionnaires, surveys, interviews, etc. Response bias can be caused by several factors, mainly the human factor, since robots do not necessarily give a direct answer to questions.

Response bias cannot be completely avoided as it is fundamental to any survey. Giving biased answers to questions can be intentional or accidental. Whatever the case, it completely interferes with the search and renders the collected data unusable.

Types of Response Bias

Social Desirability Bias

Social desirability bias is a type of response bias in which respondents are more likely to provide socially acceptable answers than sincere answers to questions. The questions in this case are often phrased in a certain way that can lead people to give in to social conditioning.

Even when people have different personal opinions, in an attempt not to look bad, they give the socially acceptable answer. This could also refer to cases where people want to form a class or belong to an elite group, i.e. lie in order to present themselves better.

Response vs non-response bias in surveys + [Examples] (1)

A respondent is likely to choose "no" in this case, even if they have already cheated on an exam and don't think there is anything wrong with it. This is because society has conditioned cheating to be viewed as a bad thing.

demand characteristics

This occurs in cases where respondents give a specific answer to a question because they think that is the answer the survey requires. For example, a company that wants to improve the quality of its product could send out a survey to find out what changes customers should make about the product's design.

In this case, the customer may not have any suggestions for improving the questionnaire. But since he thinks the company needs answers, there will be an answer anyway.

Response vs non-response bias in surveys + [Examples] (2)

Few respondents care about brand identity. However, many will choose "yes" because they think that's what the brand wants to hear.

Extreme reaction

Extreme response bias occurs when respondents give exaggerated answers to a question. This can be done to make yourself look good or to make someone else look bad on the quiz.

For example, a respondent who has been to a restaurant might be asked how the food was and s/he gives an exaggerated response, detailing how awful it was. Some may even do this to get an extra plate of food as compensation at a restaurant.

Response vs non-response bias in surveys + [Examples] (3)

(Video) What is non-response bias?

Let's say a philanthropist gives away $5,000 to those in need and this is a survey question. Most people will choose the "More than enough" option to represent needs, desires, and the pursuit of compassion, even if it's very small.

This is because the “Too Small” option can make the respondent appear proud and dissatisfied.

Neutral answer

This happens when the respondent gives a neutral answer to all the questions asked. Many respondents do this when they are undecided or have no idea of ​​an answer to the question being asked.So they give neutral answers to all questions. This type of reaction is just as bad as the extreme reaction.

The image below is an example of neutral response bias.

Response vs non-response bias in surveys + [Examples] (4)

toleration biased

toleration biasedIt is a form of response bias in which respondents give affirmative answers to all questions asked. In some cases, they don't bother to read the questions before choosing affirmative answers.This type of response bias is usually the respondent's fault and is usually intentional. This response bias is usually easy to spot, as it leads to a number of contradictory statements from respondents.

Response vs non-response bias in surveys + [Examples] (5)

By choosing a positive answer to all questions, the respondent made a contradictory statement. How can you be an atheist and still believe in God?

deviant bias

Dissent bias is the opposite of agreement bias and is a form of response bias in which respondents give negative answers to all questions asked. This answer bias is mostly intentional as they generally don't try to read the questions.

This can happen because the questions are too much for the respondent to answer. So at a certain point, respondents just start picking random answers to the questions.

Reasons for response bias

  • emotional questions

Some questions can appeal to a person's emotions and cause them to give biased answers. This is due to the emotional nature of humans.

For example, here's an emotional approach to asking someone if they plan on having children soon.

Her parents are getting older and want to see their grandchildren. Could you imagine having a child soon? 🇧🇷

The questionnaire might as well have asked if the respondent was going to have a child soon. But in order to address the person's emotions, the current condition of the parents was taken into account.

  • reward questions

Many online survey platforms reward respondents for taking surveys. Before taking a paid survey, respondents typically go through a pre-selection phase to confirm that they are eligible to take the survey.

At this point, respondents want to qualify for these surveys so they can complete them and be rewarded. Hence, they resort to lies for rewards.

(Video) Examples of bias in surveys | Study design | AP Statistics | Khan Academy

For example, a survey company may need people between the ages of 40 and 60 to conduct a survey. They then ask respondents if they fall into that age group during the pre-selection phase.

Some respondents will say yes, even if they don't fall into that age bracket, just to qualify for the award.

  • Complex questions:

Respondents often find it difficult to answer some questions because they are so complex. So you just pick a random answer to the question.

The response bias in this case is usually intentional and may be related to multiple choice questions in the education system. When students are graded and asked to choose an answer from the options, they can choose a random answer if they aren't sure which is correct.

How to avoid answer bias

  • Stop asking emotional questions. Questionnaires must stop asking questions emotionally in order to only give respondents objective answers.
  • keep it simple The questions should be simple enough for a layperson to understand. Don't use large grammar or confusing sentences.
  • Embrace anonymity. Respondents are known to be more direct and honest when anonymous. In this way, there is no need for a social desirability bias since the answer cannot be attributed to them.
  • Add response validation. Researchers must validate responses so respondents cannot provide biased answers to a question.

For example, if a respondent answers "yes/no" to a question, he/she should not answer "yes/no" to another question if it contradicts the first answer. You can do this withLogic function in Formplus form builder.

  • Allow respondents to save and continue later. Some respondents choose random answers to a question when they are tired and the questions are still too many.

By being able to save and continue later, they can take a break when they're tired and react properly when they're ready.

Disadvantages of Response Bias

  • Response bias produces irrelevant data. Researchers often cannot work with response biases because they are often wrong. If they use them anyway, just formulate your own answers.
  • This leads to incorrect statements or conclusions. If the researcher fails to notice the gaps in the data collected and the green light to use it, an incorrect conclusion will be drawn.
  • This leads to an expensive data collection process. Researchers may need to go through another data collection process if the first is irrelevant. So it costs more money and time.
  • This can cause a company to suffer a loss. Many companies use the opinions of respondents to make relevant decisions, for example to launch a new product. Response bias answers can influence a business decision that may ultimately be wrong.

What is non-response bias?

Non-response bias is a type of bias that occurs when people are unwilling or unable to respond to a survey because of some factor that makes them different from the people who responded. The difference between non-responders and responders is often a factor influencing non-response.

Non-response bias, sometimes referred to as participation bias, can result from poor survey construction and poor questionnaire alignment. It could also be due to an irrelevant decision by the defendant.

For example, a poll asking for the best brand of alcohol aimed at older religious people is unlikely to get an answer. In other cases, the survey may not even reach the target participant, e.g. B. an email that ended up in the spam folder.

Examples of non-response bias

example one

Let's imagine a database of 1000 email addresses of older people who only use their email accounts to contact their children. In most cases we can see that these people only know how to email and read what they were taught by their children.

Now imagine a person who is about to open a new club and needs to conduct a competitive analysis by asking a few questions about existing nightclubs. It is clear that if this survey is sent to these 1000 email addresses, about 90% will not respond.

A nightclub will not go down as well with older people as with young people. Well, unless it's an older people's club.

example two

Many young people watch adult videos on their internet-enabled devices, but most of them are too shy to talk about it. So take a poll with questions like: How often do you watch adult videos? What is your favorite adult video site? may not get as many replies as it should.

That's because they're too embarrassed to talk about it. A better way to get more answers to such questions is to allow them to remain anonymous when answering these questions. Even so, some will choose not to respond for fear of being exposed in the future.

Reasons for non-response bias

  • Request for confidential information

Consider a survey that measures the rate at which separated parents comply with child support policies. A parent who does not pay child support regularly will feel very uncomfortable completing this survey.

Hence dealing with a bias that skewed the data towards a more law-abiding net sample than the original sample. This reaction is evident, and we can also say that polls that specifically state their involvement in a government agency are likely to receive a non-response bias.

(Video) 2.1 Objective 6: Identify Sampling Bias, the Nonresponse Bias, and the Response Bias of a Study

  • Invitation by email/SPAM

Sometimes, researchers are often the cause of non-response bias because they don't properly pre-test their invites. Pre-testing is very important when sending email invitations, which is why email sending platforms allow you to send a test email to yourself first.

This way you can confirm that the email plays well on both mobile and PC. It's well known that most young adults respond to email on their mobile devices, and when mobile search isn't working well, responses from smartphone users will drop dramatically.

Sometimes the email ends up in spam, so potential respondents can't see it.

  • wrong audience

Choosing the right audience is very important when sending out surveys. For example, if you send a survey about weekly hours worked and earnings to college students or unemployed recent graduate students, you may not get as many responses as you would with employees.

The same applies if a survey on the number of exam hours is not answered by non-students.

Disadvantages of non-response bias

  • This invalidates the results of an investigation or survey.
  • This can lead to larger discrepancies in the estimates because the sample size the researcher ends up with is smaller than expected.
  • This can lead to unsuccessful searches.

How to avoid non-response bias in polls and polls

  • Use closed questions

closed questionsare usually more direct and do not require descriptive responses from respondents. Therefore, it is easy for them to give their answers.

In this case, respondents will not give up questions because they find it difficult to give answers. You can also control the type of replies you receive.

Easily create closed questions with Formplus' Choices feature. For example:

Response vs non-response bias in surveys + [Examples] (6)

  1. How many glasses of water do you drink daily?
  2. less than 1
  3. 1
  4. 2
  5. 3
  6. 4 or more
  • Your questions must be very neutral

Don't try to put words in your conversation partner's mouth by limiting the options to only answers that suit you. Researchers must put their personal opinions aside when creating a questionnaire.

That said, don't ask;

How good was President Trump's State of the Union address?

  • Big
  • very good
  • Better than any other President.

Instead say:

What was President Trump's State of the Union address like?

  • Arm
  • Messe
  • Geb
  • very good
  • Big.

Response vs non-response bias in surveys + [Examples] (7)

In the first example highlighted above, the options limited respondents' responses to only saying something positive about President Trump. This may not sit well with other non-Trump supporters and they may decline to respond.

(Video) 2 2 1 Nonresponse Bias

The second example, on the other hand, leaves room for both positive and negative opinions.

  • Avoid double questions

Double questions are questions that address more than one topic, but allow only one answer. This is usually very confusing and can discourage respondents from answering.

An example of a duplicate question would be the following;

"Do you think our chef and waiter were great?".

“Do you think our chef is great” and “Do you think our waiter is great?” were combined.

Response vs non-response bias in surveys + [Examples] (8)

Aside from confusing the respondent, not even the researcher can draw adequate conclusions from this type of data.

  • Understand your target audience

It's important to understand your audience in order to know how the question should be asked. For example, when dealing with older people, the words should be formal as this is more appealing to them.

In younger people, this may not be the case.

  • Make sure your options cover the possible answers you need.

Always make sure your options are comprehensive and cover all possible answers.

For example, when asking a person their gender, having male and female options is no longer enough.

This is because some people identify as non-binary, transgender, etc. Therefore, other non-binary genders may react if you add these options to the options available.

Response vs non-response bias in surveys + [Examples] (9)

  • provide incentives

Paid surveys are one way to incentivize respondents. This way you can get the answers you need for your search.

You do not need to control the survey as many companies are available to assist you in handling paid surveys.

Conclusion

Bias can be one of the things that cannot be avoided when conducting surveys. However, they can be reduced by making a conscious effort to prevent them.

(Video) Undercoverage, Non Response, Response Bias

The first step in doing this is to understand the response bias and non-response bias that can be introduced by either the researcher or the respondent. Sometimes these biases can be unintentional, as in the case where a respondent forgets to fill out a survey.

FAQs

What are some examples of response bias? ›

Examples include the phrasing of questions in surveys, the demeanor of the researcher, the way the experiment is conducted, or the desires of the participant to be a good experimental subject and to provide socially desirable responses may affect the response in some way.

What are some examples of non response bias? ›

Forgotten surveys

You receive less than half of the survey responses you expected. This would be considered nonresponse bias because participants simply forgot to take your survey and you're left with a sample that no longer represents the population for your study.

Which two major issues can be caused by non response in surveys? ›

Nonresponse bias can lead to several issues:
  • Because the obtained sample size doesn't correspond to the intended sample size, nonresponse bias increases sampling error.
  • Results are not representative of the target population, as respondents are systematically different from nonrespondents.
Nov 9, 2022

What are examples of non response error? ›

Nonresponse Error

Nonresponse errors result when the individuals who complete the interview are somehow systematically different than those who were unable to be contacted and those who chose not to participate. For example, consider an interview project that is being conducted on political participation.

What is an example of response bias in surveys? ›

An example would be if your question asks about customer satisfaction, and the options given are Very Satisfied, Satisfied and Dissatisfied. In this instance there is bias that can affect results. To avoid bias here, you could balance the survey questions by including two of each of the positive and negative options.

What is response vs non-response bias? ›

To be clear, “response bias” is not the opposite of “non-response bias.” To review, non-response bias focuses on what happens when those who receive your survey choose not to respond. Response bias is about societal or survey constructs that can impact the actual quality of the survey answers.

What are the 5 examples of bias? ›

Reduce your unconscious bias by learning more about the five largest types of bias:
  • Similarity Bias. Similarity bias means that we often prefer things that are like us over things that are different than us. ...
  • Expedience Bias. ...
  • Experience Bias. ...
  • Distance Bias. ...
  • Safety Bias.
Feb 25, 2021

What is non-response error in surveys? ›

DEFINITION: Non-response error is the error that occurs when the survey fails to get a response to one, or possibly all, of the questions. SOURCES: Primary source: CODED-Statistical concept.

What are three bias examples? ›

Confirmation bias, sampling bias, and brilliance bias are three examples that can affect our ability to critically engage with information. Jono Hey of Sketchplanations walks us through these cognitive bias examples, to help us better understand how they influence our day-to-day lives.

What factors can cause response bias in a sample? ›

Response bias (also known as survey bias) is defined as the tendency in respondents to answer untruthfully or inaccurately. It often occurs when participants are asked to self-report on behaviors, but can also be caused by poor survey design.

How do you deal with non response bias in surveys? ›

How to reduce nonresponse bias
  1. Keep your surveys short and simple. Simplicity is key. ...
  2. Set expectations with your participants. ...
  3. Re-examine survey timing and distribution method. ...
  4. Provide an incentive to complete the survey. ...
  5. Send a friendly reminder. ...
  6. Close the loop.

What are three types of error that affect responses in surveys? ›

Coverage errors: Not targeting the right people. Sampling error: Inevitable random fluctuations you get when surveying only a part of the sample frame. Non-response error: Systematic difference from those who don't respond to all or some questions.

What is item non response and how can it affect survey results? ›

“Item nonresponse” designates the units' refusal to respond to specific items. With item nonresponse, the unit is recorded in the data set but with at least one variable missing. These types of nonresponse can affect the quality of the survey separately or jointly and increase the total number of survey errors.

What is item non response and how can it effect survey results? ›

Therefore, an item is missing if the researcher interprets it as such, and decides that some kind of treatment (e.g., imputation) is required. Thus, item nonresponse is defined as the failure to obtain information for a question in an interview or questionnaire, so data are missing (see also Groves 1989).

Which occur due to non response from respondents is an example of? ›

Answer: Non-response bias is a type of bias that occurs when people are unwilling or unable to respond to a survey due to a factor that makes them differ greatly from people who respond. ...

How do you explain response bias? ›

Response bias is a general term used to describe a number of different conditions or factors that cue respondents to provide inaccurate or false answers during surveys or interviews. These factors range from the interviewer's perceived social position or appearance to the the phrasing of questions in surveys.

What is response bias in research? ›

The response bias refers to our tendency to provide inaccurate, or even false, answers to self-report questions, such as those asked on surveys or in structured interviews.

What is an example of response rate? ›

For example, if there are 1000 eligible sample units, and we are able to contact 850 of them, and to interview 600 of them, then the response rate is 60%, while the contact rate is 850/1000 or 85%, and the cooperation rate is 600/850 or 70.6%.

What are two kinds of response bias? ›

Acquiescence bias is when all answers are chosen in agreement, and dissent bias is when all answers are chosen in disagreement. These types of response bias can be evidenced in research through the analysis of collected results.

Does non response bias affect validity? ›

Response rates lack both validity and reliability as a proxy measure of nonresponse bias. Response rates lack validity in that there is not even a moderate correlation with nonresponse bias (Groves 2006).

What are the four types of response error? ›

Three classes of response error are respondent, nonresponse, and interviewer. Respondent error occurs when respondents provide incorrect or incomplete data.

What are the most common examples of bias? ›

We explore these common biases in detail below.
  1. Gender bias. Gender bias, the favoring of one gender over another, is also often referred to as sexism. ...
  2. Ageism. ...
  3. Name bias. ...
  4. Beauty bias. ...
  5. Halo effect. ...
  6. Horns effect. ...
  7. Confirmation bias. ...
  8. Conformity bias.
Oct 9, 2022

What are the most common biases? ›

10 Common Biases That Affect How We Make Everyday Decisions
  • The Dunning-Kruger Effect. ...
  • The Sunk Cost Fallacy Bias. ...
  • Optimism and Pessimism Bias. ...
  • The Framing Effect Bias. ...
  • Confirmation Bias. ...
  • Reactance. ...
  • Self-Serving Bias. ...
  • Hindsight Bias.
Jun 25, 2021

What is the effect of non response bias? ›

The effect of nonresponse is to confound the behavioral parameters of interest with parameters that determine response. Nonresponse bias is not unique to Internet surveys but the potential problem is quite severe for Web-based surveys that have low response rates and nonrandom recruitment procedures.

What are some examples of bias for students? ›

For example, you might be biased to think that another student who has dirty, torn clothes might be from a poor family, when maybe they just had an accident that day or spilled their lunch on their shirt.

What type of error is response bias? ›

Response bias refers to the ways respondents may be unduly influenced when providing answers on a survey. Bias is an issue that affects the accuracy of the survey data obtained and is the result of participants' inability or unwillingness to answer questions precisely or truthfully.

What is the best way to minimize non response bias? ›

Any estimate from a study can be subject to bias due to nonresponse across one or more stages. The best way to avoid bias is to improve response rates by using methods such as intensive refusal conversion techniques, incentives, multiple modes of data collection, flexible scheduling, and interviewer training.

How do you overcome respondent bias? ›

Prevent bias in surveys

Avoid framing survey questions that influence respondents to answer a certain way. Be clear and concise. Keep your language simple to avoid misinterpretation. Provide straightforward answers for better assessment results.

What is the best strategy to avoid bias? ›

Avoiding Bias
  • Use Third Person Point of View. ...
  • Choose Words Carefully When Making Comparisons. ...
  • Be Specific When Writing About People. ...
  • Use People First Language. ...
  • Use Gender Neutral Phrases. ...
  • Use Inclusive or Preferred Personal Pronouns. ...
  • Check for Gender Assumptions.

What are the three 3 sources of survey error? ›

While headline response rate is commonly used as an indication of survey quality, there are four potential areas of survey error. These are coverage, sampling, measurement and response.

What are the two most common errors in conducting a survey? ›

As a world-leading company providing the best survey software, we prepared a list of the 7 most common errors you have to avoid when creating a survey.
...
  • Too Many Questions in One Survey. ...
  • Using Too Many Open-Ended Questions. ...
  • Skipping the Introduction. ...
  • Too Many Choices. ...
  • Leading Questions. ...
  • Question Addressed to Everyone.
Nov 2, 2022

What are the two major sources of error in survey research? ›

The four main sources of errors in survey research are as follows: Sampling error; Measurement error; Coverage error; Non-response error.

Does non response reduce sample size? ›

In probability sampling, non-response reduces sample size, affecting the calculation of sampling error and confidence intervals. More importantly, in both probability and non-probability sampling, non-response is a source of potential bias.

Does increasing sample size reduce non response bias? ›

Reducing coverage error or nonresponse error increases the representativeness of the sample. While increasing sample size can reduce sampling error, it will not necessarily increase representativeness or reduce systematic error called “bias.”

What are the causes of non response in sample survey? ›

It can be determined by multiple causes: the statistical unit does not receive the questionnaire or is not contacted by the interviewer (noncontact), • the contacted unit does not respond because unable , e.g. due to language problems (inability), • he/she explicitly does not cooperate (refusal).

Which type of survey research is most susceptible to non response? ›

However, this trend is less true today as more people choose to only have cell phones and do not install land lines that would be included in telephone directories. Mail surveys are less costly still but generally have even lower response rates—making them most susceptible to non-response bias.

What is common response bias? ›

Response bias refers to several factors that can lead someone to respond falsely or inaccurately to a question. Self-report questions, such as those asked on surveys or in structured interviews, are particularly prone to this type of bias.

Which is an example of a response? ›

Example of a stimulus and a response: If you accidentally touch a hot object, you automatically withdraw your hand. The heat of the hot object is the stimulus and you, withdrawing your hand is the response to the stimulus. Q.

What are the 4 types of response bias? ›

The eight types of response bias you'll need to be aware of in order to build surveys that minimize the risk of inaccurate responses are:
  • Demand bias.
  • Social desirability bias.
  • Dissent bias.
  • Agreement bias.
  • Extreme responses.
  • Neutral responding.
  • Personal bias.
  • Non-response bias.

How do you avoid response bias in surveys? ›

5 Tips For Avoiding Response Bias
  1. Make sure that your language is appropriate for your audience. ...
  2. Don't make the mistake of asking two questions at once. ...
  3. Avoid inherent bias in your questions. ...
  4. Do your research and provide enough options. ...
  5. Make sure you target the right audience.

Why is response bias a problem? ›

Because respondents are not actually answering the questions truthfully, response bias distorts study results, threatening the validity of your research. Response bias is a common type of research bias.

What are 3 common biases? ›

Three types of bias can be distinguished: information bias, selection bias, and confounding. These three types of bias and their potential solutions are discussed using various examples.

What is a non response bias? ›

Nonresponse bias happens when a surveyor does not respond to your survey or survey question because they are unable or do not want to complete it. While the reasoning for not completing or responding to the survey can vary, nonresponse bias can make your survey data less accurate.

What are response bias factors? ›

The response bias refers to our tendency to provide inaccurate, or even false, answers to self-report questions, such as those asked on surveys or in structured interviews.

Videos

1. Who Didn't Respond Analyzing Nonresponse Bias in Survey Data
(IDEAs That Work Resources)
2. Bias associated with nonresponse
(Embraced Wisdom Resource Group)
3. Survey Response Biases in User Research
(NNgroup)
4. Survey Response Bias And How To Avoid Them
(involve_me)
5. What is Response Bias? Reasons, Types and how do you get rid of it
(Marketing91)
6. Q. How does survey non-response cause bias?
(Monika Wahi)
Top Articles
Latest Posts
Article information

Author: Tish Haag

Last Updated: 04/18/2023

Views: 6234

Rating: 4.7 / 5 (47 voted)

Reviews: 94% of readers found this page helpful

Author information

Name: Tish Haag

Birthday: 1999-11-18

Address: 30256 Tara Expressway, Kutchburgh, VT 92892-0078

Phone: +4215847628708

Job: Internal Consulting Engineer

Hobby: Roller skating, Roller skating, Kayaking, Flying, Graffiti, Ghost hunting, scrapbook

Introduction: My name is Tish Haag, I am a excited, delightful, curious, beautiful, agreeable, enchanting, fancy person who loves writing and wants to share my knowledge and understanding with you.