KIM NICHOLAS
  • Home
  • About
  • Book
    • UNDER THE SKY WE MAKE
    • Book Seminar
    • Teach UNDER THE SKY WE MAKE
    • Discussion Questions
    • Book Clubs
    • Support the Book
    • Press Kit & Images
    • Request from Local Bookstore/Library
    • How to order outside US/Canada
    • Behind the Scenes
    • If My Book Were Music
  • Research
    • Lab Members
    • Peer-Reviewed Publications
    • Flying Less >
      • The Takeoff of Staying on the Ground
      • Policy Briefs
      • Ingen ny tid för avgång
      • Academics Flying Less
    • Radically Reducing Lund's Emissions
    • Climate Solutions >
      • What Can I Do? 2 >
        • What Can I Do?
        • High School Teaching Materials
        • Fyra klimatsmarta livsstilsval
        • Press Release: 4 Lifestyle Choices That Most Reduce Your Carbon Footprint
        • The Climate Mitigation Gap: Study & Video Abstract
        • Study FAQs
      • Climate Science 101
      • Climate Policy >
        • IPCC Report on 1.5°
        • Kims Klimatval
        • COP21 (Paris Agreement)
      • Farmer adaptation
      • Harnessing biodiversity
    • Climate Education
    • Sustainable Land >
      • Global land use
      • European farming systems
      • Swedish land use
      • Ecosystem Services & OPERAs
      • REDD+
      • Land Acquisitions
    • Sustainable Food >
      • Urban Food Forestry
      • Local food in Iceland
      • One Great Meal
      • Dietary choices & climate change
      • Crop yields & climate
    • Wine, Climate, & Sustainability >
      • Wine & Climate: Impacts & Solutions
      • Wine Diversity for Climate Adaptation
      • Wine yields & quality under climate change
      • Farmer climate adaptation
      • Vineyard ecosystems & landscapes
      • European Wine Case Studies (OPERAs)
    • For Kids (K-12)
  • Writing
    • Newsletter
    • Peer-Reviewed Publications
    • Magazines & Popular Science
    • Blog
  • Speaking
  • Teaching
    • Teaching Overview
    • Climate Change Curriculum
    • We Can Fix It World Cafe >
      • We Can Fix It World Cafe 2017
      • We Can Fix It World Cafe 2016
      • We Can Fix It World Cafe 2015
      • We Can Fix It World Cafe 2014
    • Courses >
      • Writing for Change >
        • Course Readings
        • Apply
        • Course Information
    • Advice for Students
    • Peer Writing Tutors >
      • Instructions for Peer Tutors
      • Apply to be a writing tutor!
    • Scholarship of Teaching and Learning
    • Early Career
    • R Tutorials >
      • R tutorial 1: Basic calculations and graphs
      • R tutorial 2: Data Visualization
    • Student-Led Exams >
      • Simplified Self Grading
      • DIY Exam Teaching Notes
      • Peer Grading
      • Self Grading
  • Activism
  • Contact

Avoiding common mistakes in survey design

2/13/2019

 
So, you've chosen survey research to answer your research question, you've considered ethical issues and planned to obtain informed consent, and you're ready to write your survey questions! Here's how to make them make sense and avoid some common pitfalls. 

  1. Ask only the questions necessary to answer your research question. It is a lot of work for respondents to volunteer their time to answer your survey, and it is a lot of work for you to analyze it, and for your readers to read about it. Spend the time to thoughtfully design your survey so that every question you ask your respondents is necessary to answer one of your research questions. Start by mapping your core research questions onto what survey questions will be needed to answer them. Delete survey questions that do not directly help answer one or more core research questions. Similarly, if you find you are not answering some of your research questions with current survey content, add survey questions that do address your research questions. 
    ​
  2. Choose standard scale labels. Do not invent your own. Use standard, easily interpretable scale labels, balanced across the (usually 5 point) scale. Select the appropriate wording for extent, frequency, etc. See this helpful list from Vagias at Clemson University.  

  3. Specify the time period you're asking about. Do you mean in the last week, in an average week, over the last 5 years? People are usually better at recalling the immediate past more accurately, so this is probably a better time frame to specify. 

    ​Example: If asking respondents how much they agree or disagree with the statement:

    "Climate change is caused mostly by natural causes," you must specify

    "Climate change nowadays (or another word indicating the present time) is caused mostly by natural causes."

    Otherwise the respondent could answer True if they know that all climate change over all of the Earth's history until the last few centuries was caused by natural causes, so most climate change is indeed naturally-caused, but this is probably not answering the question you intended to ask. 

  4. Ask exactly one question at a time (avoid double-barreled questions). Make sure there is only one way to interpret the question you ask. For example, 

    "Policy makers and companies can make a difference to climate change, not my individual actions."

    is ambiguously worded, because there is no way to distinguish between the contrast being made (what if I think policymakers but not companies are important, or vice versa?), and there is no way to disagree with the framing (what if I think both policymakers and my individual actions are important?). Better options (depending on what you're trying to elicit) would be: 

    "Policymakers can make a big difference for climate change" or
    "Policymakers can make a bigger difference for climate change than my individual actions"
    -and the same for companies, separately. 

  5. Use consistent wording throughout. Unless you are testing the effect of wording in your study (e.g., are people more worried about "climate change," "global warming," or "the climate crisis"?), do not use different words to mean the same thing in different parts of your survey. This introduces unnecessary error and is likely to confuse your respondents. Pick the label that most accurately or commonly represents the idea you're interested in to your survey audience, and use it consistently. It is a good idea to define it at first use if there's any chance it's unfamiliar to your audience. 

Comments are closed.

    RSS Feed

    Categories

    All
    Academia
    Advice
    Agriculture
    Book
    Climate Change
    Conferences
    COP21
    Divestment
    Early Career
    Ecosystem Services
    Grantwriting
    Policy
    Research
    Scicomm
    Teaching
    Wine
    Writing

    Archives

    February 2023
    June 2022
    August 2021
    May 2021
    March 2021
    January 2021
    November 2020
    October 2019
    September 2019
    May 2019
    February 2019
    July 2018
    January 2018
    November 2016
    April 2016
    March 2016
    December 2015
    November 2015
    August 2015
    July 2015
    May 2015
    March 2015
    February 2015
    January 2015
    November 2014
    October 2014
    September 2014
    August 2014
    July 2014

    RSS Feed


Connect

Picture
Picture

  • Home
  • About
  • Book
    • UNDER THE SKY WE MAKE
    • Book Seminar
    • Teach UNDER THE SKY WE MAKE
    • Discussion Questions
    • Book Clubs
    • Support the Book
    • Press Kit & Images
    • Request from Local Bookstore/Library
    • How to order outside US/Canada
    • Behind the Scenes
    • If My Book Were Music
  • Research
    • Lab Members
    • Peer-Reviewed Publications
    • Flying Less >
      • The Takeoff of Staying on the Ground
      • Policy Briefs
      • Ingen ny tid för avgång
      • Academics Flying Less
    • Radically Reducing Lund's Emissions
    • Climate Solutions >
      • What Can I Do? 2 >
        • What Can I Do?
        • High School Teaching Materials
        • Fyra klimatsmarta livsstilsval
        • Press Release: 4 Lifestyle Choices That Most Reduce Your Carbon Footprint
        • The Climate Mitigation Gap: Study & Video Abstract
        • Study FAQs
      • Climate Science 101
      • Climate Policy >
        • IPCC Report on 1.5°
        • Kims Klimatval
        • COP21 (Paris Agreement)
      • Farmer adaptation
      • Harnessing biodiversity
    • Climate Education
    • Sustainable Land >
      • Global land use
      • European farming systems
      • Swedish land use
      • Ecosystem Services & OPERAs
      • REDD+
      • Land Acquisitions
    • Sustainable Food >
      • Urban Food Forestry
      • Local food in Iceland
      • One Great Meal
      • Dietary choices & climate change
      • Crop yields & climate
    • Wine, Climate, & Sustainability >
      • Wine & Climate: Impacts & Solutions
      • Wine Diversity for Climate Adaptation
      • Wine yields & quality under climate change
      • Farmer climate adaptation
      • Vineyard ecosystems & landscapes
      • European Wine Case Studies (OPERAs)
    • For Kids (K-12)
  • Writing
    • Newsletter
    • Peer-Reviewed Publications
    • Magazines & Popular Science
    • Blog
  • Speaking
  • Teaching
    • Teaching Overview
    • Climate Change Curriculum
    • We Can Fix It World Cafe >
      • We Can Fix It World Cafe 2017
      • We Can Fix It World Cafe 2016
      • We Can Fix It World Cafe 2015
      • We Can Fix It World Cafe 2014
    • Courses >
      • Writing for Change >
        • Course Readings
        • Apply
        • Course Information
    • Advice for Students
    • Peer Writing Tutors >
      • Instructions for Peer Tutors
      • Apply to be a writing tutor!
    • Scholarship of Teaching and Learning
    • Early Career
    • R Tutorials >
      • R tutorial 1: Basic calculations and graphs
      • R tutorial 2: Data Visualization
    • Student-Led Exams >
      • Simplified Self Grading
      • DIY Exam Teaching Notes
      • Peer Grading
      • Self Grading
  • Activism
  • Contact