Avoiding common mistakes in survey design

So, you've chosen survey research to answer your research question, you've considered ethical issues and planned to obtain informed consent, and you're ready to write your survey questions! Here's how to make them make sense and avoid some common pitfalls. 

1. Ask only the questions necessary to answer your research question. It is a lot of work for respondents to volunteer their time to answer your survey, and it is a lot of work for you to analyze it, and for your readers to read about it. Spend the time to thoughtfully design your survey so that every question you ask your respondents is necessary to answer one of your research questions. Start by mapping your core research questions onto what survey questions will be needed to answer them. Delete survey questions that do not directly help answer one or more core research questions. Similarly, if you find you are not answering some of your research questions with current survey content, add survey questions that do address your research questions. 
   ​
2. Choose standard scale labels. Do not invent your own. Use standard, easily interpretable scale labels, balanced across the (usually 5 point) scale. Select the appropriate wording for extent, frequency, etc. See this helpful list from Vagias at Clemson University.  
   
   
3. Specify the time period you're asking about. Do you mean in the last week, in an average week, over the last 5 years? People are usually better at recalling the immediate past more accurately, so this is probably a better time frame to specify. 
   
   ​Example: If asking respondents how much they agree or disagree with the statement:
   
   "Climate change is caused mostly by natural causes," you must specify
   
   "Climate change nowadays (or another word indicating the present time) is caused mostly by natural causes."
   
   Otherwise the respondent could answer True if they know that all climate change over all of the Earth's history until the last few centuries was caused by natural causes, so most climate change is indeed naturally-caused, but this is probably not answering the question you intended to ask. 
   
   
4. Ask exactly one question at a time (avoid double-barreled questions). Make sure there is only one way to interpret the question you ask. For example, 
   
   "Policy makers and companies can make a difference to climate change, not my individual actions."
   
   is ambiguously worded, because there is no way to distinguish between the contrast being made (what if I think policymakers but not companies are important, or vice versa?), and there is no way to disagree with the framing (what if I think both policymakers and my individual actions are important?). Better options (depending on what you're trying to elicit) would be: 
   
   "Policymakers can make a big difference for climate change" or
   "Policymakers can make a bigger difference for climate change than my individual actions" -and the same for companies, separately. 
   
   
5. Use consistent wording throughout. Unless you are testing the effect of wording in your study (e.g., are people more worried about "climate change," "global warming," or "the climate crisis"?), do not use different words to mean the same thing in different parts of your survey. This introduces unnecessary error and is likely to confuse your respondents. Pick the label that most accurately or commonly represents the idea you're interested in to your survey audience, and use it consistently. It is a good idea to define it at first use if there's any chance it's unfamiliar to your audience.