This is a tough and thankless job, but science depends on it! Here are a few principles I keep in mind when suggesting (to journals that ask for them) or soliciting (when I'm an editor) peer reviewers.
When identifying reviewers for a particular paper, I try to find a balance of:
Where to find reviewers?
(Along those lines- if you’re publishing make sure you’re giving back to the community by serving as a peer reviewer and/or editor yourself! Read my guide to writing a solid peer review or how to get started, and register as a potential reviewer with journals in your field).
See guidelines for picking reviewers: https://methodsblog.wordpress.com/2015/10/15/preferred-reviewers/
For the PNAS guidelines see here: http://m.pnas.org/site/authors/coi.xhtml
Springer, Conflict of Interest: http://www.springer.com/authors/manuscript+guidelines?SGWID=0-40162-6-795522-0
Article on COI in medicine: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2246405/
I remember being confused about what was expected of scientific authorship in grad school. My mentor Pam Matson had a helpful rule of thumb: there are three things you can do to contribute to a scientific paper: (1) have the idea, (2) get the money, and (3) do the work. At least two of these three are required for authorship. (Thus, under this model, a PI who has an idea and gets funding to support a PhD student on that theme would be expected to be a coauthor on all resulting papers.)
I appreciate having clear guidelines and expectations for authorship, so I was glad to come across the authorship guidelines from the Vancouver Convention. Basically, they recommend 4 criteria for authorship (all four criteria must be met for authorship):
1. Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND
2. Drafting the work or revising it critically for important intellectual content; AND
3. Final approval of the version to be published; AND
4. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
This is the model I aim to follow in my collaborations. Thus, I expect myself and all authors to make a substantial intellectual contribution (#1) and contribute to writing and editing the manuscript (#2).
I interpret #3 above as the lead (first) author has responsibility to solicit and integrate input from all authors in making revisions, and obtain their approval before sending to the journal. I interpret this responsibility as applying at three stages:
1. During drafting of a manuscript, until all authors approve the MS being submitted to the journal;
2. During peer review, when the lead author takes primary responsibility for addressing comments from peer review, with input from all authors, and gets approval from all authors for the version to re-submit to the journal (this stage repeated as necessary if there is more than one round of peer review); and
3. During copyediting, when the lead author shares the typeset and corrected final proof with all authors for their approval before submitting for processing and publication.
I think all three of these stages are important in order to ensure that the last round (approval before publication) is sufficiently met, so that all authors are in a position to take ethical responsibility for the work (#4).
(See my tips on how to work with revisions suggested by reviewers here.)
When working on revisions, and especially with large and diffuse author groups, the lead author has to herd the cats and balance between giving everyone opportunity for input, and making decisions about the most appropriate direction for the paper (especially when coauthors or reviewers may have contradictory suggestions). After giving all authors a chance for input, during revisions the lead author might send around a version that incorporates changes suggested and say something like,
“Thanks for all your comments, which have been incorporated in the attached version. I had to balance between suggestions X and Y, which I did by Z; I hope everyone is satisfied with this approach. I would like to submit on X date (eg 1 week in the future). Please reply with either (a) any critical changes needed for accuracy or (b) your approval to submit. Thanks!”
It's especially essential to receive positive affirmation (i.e., a verbal or written OK to submit) from each author for the final version to be published.
In other words, you don't start out deciding to do a survey, then looking for appropriate research questions to ask and attendant philosophies and theories. For a simple breakdown of how philosophy of knowledge, theory of knowledge, and research style combine to produce research programs, see our 2010 article "Thinking about Knowing"- this could help you write the dreaded "philosophy and theory" sections of a master's thesis, for example.
Another useful resource is this overview of qualitative research in conservation by Moon et al. 2016, including a helpful list of questions in Table 5 that will help you avoid many common pitfalls.
2. What are the research questions you want to answer?
It is essential that you have a manageable number (i.e., not more than three) clearly articulated research questions that guide all further research design. Coming up with a good research question is an art in itself (take a look at The Craft of Research for a whole chapter on how to do this).
In short, my view is that a good research question is grounded in a real-world problem, linked to other research and/or theory in the field, specific enough to be answerable with the resources you have available, and something you are passionate about answering.
3. Are surveys the appropriate method to answer your research questions?
Be sure to consider alternatives and convince yourself that surveys are a good way forward, that will help you answer your research question. If you can directly observe behavior or use existing data to answer your question, that will be easier than designing and carrying out a good survey. On the other hand, if you need to know how people think (attitudes) or feel (values) about something, then you need a survey to ask them directly to find out.
4. Who is your target population, and how will you sample from them?
Consider that your sampling design will affect the validity and generalizability of your conclusions (i.e., if you want to do inferential statistical analysis to draw conclusions about significance, you need a random sample that is large enough to be representative). William Trochin has a good, concise overview of sampling and social research design issues on his Research Methods Knowledge Base.
Issues to consider here include how you identify the target population (what characteristics must they have?), and how you identify them (what criteria will you use to include or exclude participants, and how many do you seek?). Write down all your criteria in your Methods section under Sample Selection.
5. How will you design your survey?
Here's a really nice, simple, clear overview of best practices for survey research design by Kelley et al. 2003. This is a great place to start.
Good tips on survey design, question construction, and administration in a book chapter from "Investigating the Social World" by Russell Schutt.
6. How will you analyze survey data?
The simplest way is using descriptive statistics- e.g., graphing the distribution of the results in a histogram.
A more complex way is inferential statistics- using statistical tests to draw conclusion about the likelihood of observations being due to chance, attributing numerical confidence to the results. This is a big topic; to get started, check out this guide from The University of Reading Statistical Service Centre on the analysis of survey data.
For descriptive data (verbal or written textual responses to open questions), the aim is not to condense them into a number, but to use them to represent and explore different views. Here the process of coding can be helped with qualitative analysis software like NVivo or Atlas.ti. You will chose (and should state in your Methods) whether you are using inductive coding (themes emerge from the data) or deductive (trying to match responses to categories from previous literature or theory), or a combination.
Whatever analysis approach you take, describe it clearly in your Methods, and explain why you chose it.
7. Research Ethics
Conducting research with human subjects requires the researcher to take responsibility for considering and minimizing the risks presented to participants, and make sure that clear, prior, informed consent is obtained from your participants (this means your participants understand that their participation in the study is voluntary, they understand any risks that may be presented, and they know that they can end their participation at any time of their choice without penalty). You need to think about how you will protect the privacy of your participants and how to handle their data fairly and with what degree of anonymity. It is good practice to obtain written, signed informed consent before enrolling a participant (this might be ticking a box on the front page of an online survey, or initialing and signing a separate consent form including a copy for them for in-person interviews).
Be aware that you are responsible for following ethical guidelines in the country where you conduct your research. In the US and Canada among other countries, completion of a formal course in research ethics and approval of a human subjects protocol is often required before conducting survey research.
Here is the human subjects protocol I submitted to Stanford University for my PhD research in 2006- there are lots of helpful questions here to consider. And here is the consent form I used with my study participants. Feel free to use these as templates for your own research. Answers to the protocol questions can go in the Methods section of your thesis, and the consent form can be modified and used with your study participants.
In Sweden, research should follow the Swedish Research Council Vetenskapsrådet's guide to Good Research Practice, with many useful areas to consider. Research conducted for a master's thesis generally does not require a formal application to the regional review board (meets monthly, costs 5000-16000 SEK to review applications, which must be in Swedish). Here is the Swedish law regarding research ethics, for research conducted in Sweden, that presents a sufficient risk or sensitive data.
However, researchers still have a responsibility to follow good practices. Please be aware that you may be required to demonstrate good practices were followed and informed consent was obtained in order to publish your research results in a peer-reviewed journal. See below for two examples of text published in peer-reviewed articles describing how informed consent was obtained in Sweden.
Text about informed consent: "Although the study scope exempted it from Swedish requirements for formal ethical review by an institutional review board, all procedures performed in this study involving human participants were in accordance with the ethical standards of the institution, and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Prior, written informed consent was obtained from all individual participants included in the study (See consent form in S1).”
--Source: Wynes and Nicholas, in second review Feb 2019, PLOS ONE
"According to the Swedish regulations for conducting research on humans, there was no need to seek ethical clearance for this study. All participation was voluntary after informed consent, and the participants could withdraw from the study at any time. In order to protect the participants’ identity, revealing information has been removed from the results such as specific faculty, discipline, gender, ethnicity, and age. Furthermore, before data was analysed, all participants were invited to confirm their own interview transcripts and remove quotations that they did not want to share with others."
--Source: Brodin & Avery, in press
I hope these tips are helpful- happy research!
Last October, I gave a Sunday morning talk to a group of early-career researchers attending the Earth Systems Governance conference. It was a day-long program on "Developing a career in earth system governance: opening up science." I enjoyed the chance to gather my thoughts and pass along some good advice I've been given (and some earned through experience!).
Thanks very much to Ina Möller, who made a podcast from our conversation. You can have a listen here.
Here's a condensed list and links to resources I've found helpful. Hope they're useful to others!
Here’s a short summary I co-wrote with Céline Fernandez compiling the highlights of our November meeting for the LuPOD professional development program.
Tools for digital learning:
Best practices for films for educational purposes:
Issue regarding available learning material online: often no quality control. Therefore, make sure to review material before assigning, point out to students to be critical when using information online, and consider making your own material.
References and further reading:
Here are some tips I compiled with my friend and colleague Josh Goldstein when we were both finishing our PhDs and tackling the job market. Good luck, job seekers!
Here are some of my top tips, and a suggested template of a letter to contact a potential advisor at the end:
I came across your work through X (conference/paper/my professor X’s recommendation). I am very interested in your approach to (topic) using (method/specific thing they do that interests you).
I’m writing to inquire if you are accepting PhD students in your lab for fall 2016?
I am aiming to pursue a PhD dissertation focusing on X topic, specifically looking at the question of X puzzle in X case for X purpose.
My background is in X bachelors from X university/Y masters from Y university. My research to date has focused on X, which I investigated most recently in my master’s thesis on X topic in X place, finding that X key highlight (please see attached thesis FYI). I have written about this work in a popular science blog here (link) and am currently preparing the manuscript for submission to the journal of X with my supervisor X.
I also have experience in X business/NGO/policy/government, where I accomplished X. For more details, please see my attached CV.
Regarding funding, I am currently applying for funding from X, X and X scholarship agencies to support the project I’ve outlined above. If there are any additional funding sources that you think might be a good fit to support this work, I would really appreciate any tips. And of course, if you have any currently funded projects that could support a PhD student, I would love to hear about them.
Thank you very much for your time and I look forward to hearing from you.
If you don’t hear back from them in 2 weeks, resend your message with a polite note at the top: “Dear Dr. X, I thought my earlier message may have caught you at a busy time. I’m still very keen to pursue a PhD in your lab, and hoping you can let me know if you are accepting new students next fall. Thank you, X”
If there’s someone you’re really excited about who hasn’t gotten back to you, consider picking up the phone to call them. Bold move, but it just might work! People get a bajillion emails, but few phone calls nowadays. Practice a shortened version of the text above to ask them.
It’s smart to apply to at least several schools (I just checked and I applied to seven PhD programs, which now seems excessive- but at least three is good). You don’t know where you’ll get in, and you’ll have to consider personal factors about where you want to live, so it’s good to have options.
Most advisors will be contacted by many students and will have many applicants for each open position in their lab. So, you both are on the lookout for the person who will be the best fit! Most advisors will understand this (some may even ask where else you are applying).
That said, if you apply for external fellowships, you may have to specify your top choice for where you want to go and who you want to work with (depending on the fellowship). In this case, it’s important to share your plans with your potential advisor and get their agreement to support your application. In any case, if you really click with a potential advisor, consider asking them to give you feedback on your external fellowship application (for something like NSF GRFP, not for their own university applications where they would have a conflict of interest). Hopefully their comments can help you strengthen your proposal.
In any case, good communication with a potential advisor is important, so ask them questions to clarify expectations or any points of confusion.
Hope this helps- let me know if you have any comments!