Section 4. Survey Data#
Survey data refers to informational data gathered through a sample of the population. This data can cover a wide variety of topics including demographics, activities, behaviors, and opinions. Surveys can be completed by the individual unaided, or through a more formal interview.
Health Equity and Survey Data
|
Common Types of Survey Data#
The table below describes common types of survey media, questions and sampling methods.
If you are already familiar with survey data media, questions, and sampling methods, please continue to the next section. Otherwise click here.
Common Types of Survey Media
| Types | Common Usage | Example |
|---|---|---|
| Written Surveys | A set of written survey questions was distributed to respondents either physically or electronically. | A survey link was sent to all the participants in a CPR training course. |
| Verbal Surveys | A verbal survey is conducted by a survey administrator face-to-face or in-person. | A survey was conducted using random phone numbers in the 319 area code, after a possible chemical exposure. |
| Mixed-Mode Surveys | A survey mode combining verbal and written surveys | A group of residents in an assisted living facility are asked to fill out yes/no questions about their daily living activities on a written form. The daytime nursing staff then follows up with verbal interviews to acquire additional details from the residents. |
Common Types of Survey Questions
| Types | Common Usage | Example |
|---|---|---|
| Open-ended Questions | Questions that survey respondents can answer with their own words, may allow for more free expression of a respondent's opinion and motivation. The answers to these questions may require manual coding or parsing. |
|
| Closed-ended Questions: Ordered Choices | These are questions that are often numeric, and ask the respondent to rank the answers on a numerical scale of the order of importance. These include Likert scales, and often some of the easiest questions for respondents to answer. |
|
| Closed-ended Questions: Unordered Choices | These include questions like multiple choice questions. | What are your preferred methods for losing weight?
|
| Closed-ended Questions: Other | Other types of closed-ended questions include hybrid questions, either of closed or open types. | What has been your overall experience with the clinical training?
|
Common Types of Survey Sampling Methods
| Types | Common Usage | Example |
|---|---|---|
| Probability Sampling: Simple random sampling (SRS) | Any n persons out of a population size of N have an equal chance of being selected. | Out of a population size of 100, where the desired survey size is 10. The population is shuffled and 10 people are randomly selected. |
| Probability Sampling: Stratified random sampling with uniform allocation | The population is stratified into any number of homogenous groups, and then a sample is selected from each group. | The population is made up of 100 people. The population is stratified into a) 20 adults over age 50, and b) 80 adults under age 50. Five people are then selected randomly from each group. This is stratified sampling with uniform allocation. An alternative allocation strategy could be proportional to the stratum size. In this case, if the desired sample size is 10, proportional allocation would sample two adults >50 and eight adults <50 instead of five from each group. |
| Probability Sampling: Cluster sampling | In internet surveys, the sample is selected from people from a particular user group or domain. In non-internet surveys, the clustering technique may sample people from a similar geographic area, such as a city block or zip code. | A researcher wishes to interview several participants who are PrEP users. They create a cluster sample from several self-identified users on a PrEP sub-reddit. |
| Probability Sampling: Systematic sampling | A random sample of a population based on selecting every kth participant. | A researcher is studying behaviors around vaccines. They create an Internet survey that prompts every 10th user on a free vaccine sign-up page. |
| Non-probability Sampling: Quota sampling | Setting quotas for certain demographic or other criteria, and sampling from those groups until quotas are met. | A researcher is studying attitudes about diabetes among young people. They survey the first 10 participants from each OMB race category (White, Black or African American, American Indian or Alaska Native, Asian, and Native Hawaiian or Other Pacific Islander). |
| Non-probability Sampling: Snowball sampling | Usually only used when finding a sufficient sample size is too difficult, timely or expensive, and relies on referrals from the respondents that meet the initial criteria. | A researcher is studying LGBT+ American Indian teen attitudes about mental health. She regularly works with small after-school LGBT+ groups, and uses their contacts to find additional participants. |
| Non-probability Sampling: Judgment sampling | A sample based on the judgment of the researcher. | A researcher studying food-borne illness, selects a random sample from a group of weekend mall goers. |
Health Equity Considerations#
Users of survey data should be aware of the design features of the data they are using. Below we review some of the common challenges in survey design and execution which may introduce potential biases.
Challenge |
Challenge Description |
Heath Equity Example |
Recommended Best Practice |
|---|---|---|---|
Privacy and Anonymity |
The primary concern here is balancing privacy and accuracy. On anonymous surveys, respondents may be more likely to be candid and truthful. However, when surveys are not anonymous, respondents may be less likely to respond as they want to protect personal information and opinions. With less identifiable features, like demographics, it can be difficult to ensure there is a representative sample. The challenge lies in finding the right balance, and it will depend on a case-to-case basis. |
CC is a researcher studying the opinions and experiences of LGBT+ youth on bullying. They do a non-probability sample of after-school programs affiliated with a non-profit. After reviewing the results, CC finds that almost all of the students completed the survey by putting in the name of their after-school program which may lead to being able to identify participants’ locations. |
Review survey collection methodology. Ensure you understand how data were collected in regards to privacy and the potential impact of anonymity on survey responses. If survey data does not contain demographic information, you may be unable to confirm a representative sample and should communicate this limitation to consumers of your analysis results. |
Language and Literacy |
Several issues may occur with surveys and language and literacy. Firstly, respondents may not be familiar with the language or terminologies used or, they may have reading difficulties. It is also important to not sway respondents in one way or another with biased language. |
DJ is a researcher studying disease outbreaks in migrant worker groups. DJ makes a survey available in English and Spanish. However, after handing out the survey at various locations, DJ finds that many workers are not able to understand the scientific names of the diseases. Thus, the survey had to be retranslated to a more casual tone to gain more participants. |
Review survey questions to assess potential impact of language difficulties and the impact they may have on your particular analysis. Review survey collection methodology to determine the approach to participants with language or literacy limitations. |
Technological Access |
If surveys exist only online, there could be access challenges. Additionally, device type should be considered, and ensure that surveys work on a variety of mobile devices and operating systems. |
MO pays for an iOS developer to build a simple survey application, trying to gather information for heart disease patients. However, because the app only works for Apple users, MO is unable to gather data from patients who have Android or other devices. |
Ensure a clear understanding of how the data collected and what technical constraints could have affected responses. Review survey collection methodology to determine the approach to minimizing barriers to access. If barriers were present (eg must use a mobile device or particular platform), consider how these barriers may correlate with your topic of interest. |
Cultural Norms and Values |
The concerns with cultural norms and values in surveys is complex. Primarily, it is important to consider whether a question is appropriate to ask. Different cultures may have different expectations as to what and how certain topics are discussed. These could be issues around sexuality, gender and gender roles, food, religion, and more. Additionally, if a survey must be translated, special care should be given to not introduce additional biases, and ensure the translation is accurate and culturally sensitive. |
MU surveys illicit substance use among Asian Americans. Because the survey has potentially identifiable information in it, MU is finding it difficult to gather any information from some of the participants, as they are uncomfortable sharing information with an unknown party. |
Review the data collection methodology and consider how cultural norms may have affected the responses of study participants, particularly those in underrepresented groups. If these norms relate to your intended area of study, this limitation should be communicated to consumers of your analysis. |
Sampling Bias (especially in non-Probability Sampling) |
This is a common concern in any survey. It occurs when the survey population is not representative, and certain groups or individuals are more likely to be represented over others. |
AT is a researcher investigating hand-washing habits in restaurant workers. However, AT only sends the surveys to restaurants with lower health department ratings, thus potentially lowering the response rate. |
Assess the representativeness of the survey population relative to the target population of interest. Where appropriate, perform weighting as per Unit 4 to mitigate this imbalance. |
Case Study Example#
Case study is for illustrative purposes and does not represent a specific study from the literature.
Scenario: YG is a researcher wishing to understand the prevalence of diabetes and related lifestyle and family history factors in rural communities.
Data Source: YG performs a phone survey with lifestyle, family history, and dietary questions, along with laboratory data from a participating research hospital.
Analytic Method: The analytic method uses primarily descriptive statistics.
Results: YG was able to identify lifestyle and family history prevalence trends related to diabetes in rural settings.
Health Equity Considerations:
Common factors that impact Health Equity in surveys are data availability and willingness of participants to participate in surveys. The reasons for this could vary from the local culture of the region being surveyed (i.e. lack of desire to share personal information for research) or even socioeconomic status of the participants (i.e. participants make work long hours or in the evenings when the phone surveys are performed).
Other issues to factor in include response bias, where participants respond to what they think the survey takers want to hear versus the actual answers.
Using a statistical method such as raking, YG may be able to account for gaps in the data by adjusting the weights for each data point so it the demographics align with the population of that region.
Considerations for Project Planning
|