This is a tough and thankless job, but science depends on it! Here are a few principles I keep in mind when suggesting (to journals that ask for them) or soliciting (when I'm an editor) peer reviewers. 

When identifying reviewers for a particular paper, I try to find a balance of: 

• expertise (all reviewers need appropriate expertise, but this can range between e.g., specific topic/research question, methods, theory, implications, study region, ...)
  
• regional location of institution (e.g., Global North/South)
• gender balance
  
• career stage (I find that earlier careers, PhDs through postdocs, often write the best reviews, and getting started with peer reviews is helpful). 

Where to find reviewers? 

• It's often good to try to find someone who has published in your target journal (or a journal with similar reach). 
  
• The Journal/Author Name Editor, JANE, can be a good resource for finding reviewers (leans towards medicine). You enter title and/or abstract and it finds similar papers, authors. 

(Along those lines- if you’re publishing make sure you’re giving back to the community by serving as a peer reviewer and/or editor yourself! Read my guide to writing a solid peer review  or how to get started, and register as a potential reviewer with journals in your field). 

Ethics: 

• Suggested reviewers must avoid conflicts of interest, that is, they should not have a personal or professional relationship with any authors that would prevent impartial scientific judgment. Definite conflicts of interest are co-published authors, people at the same institution, former or current academic mentors/advisees.  
  
• I avoid suggesting personal friends even if they have relevant expertise. (I try to put myself in the reverse situation and think, if I were asked to review them, do I start with a positive predisposition just because I know they're a nice person/ in general I think well of them/ it would be awkward to reject them/etc? My goal is to have reviewers who are able to focus on the quality of the work alone, independent from the qualities of the people who produced it, insofar as this is possible in a small community of human beings!) 
• Technically the editors should also screen for conflicts, but this is a time-consuming and imperfectly accurate process done by busy volunteers, so when authors are asked to provide reviewers it's our responsibility to meet all the guidelines.)
  
• Some specific guidelines on conflict of interest below. 

See guidelines for picking reviewers: https://methodsblog.wordpress.com/2015/10/15/preferred-reviewers/ 


For the PNAS guidelines see here: http://m.pnas.org/site/authors/coi.xhtml
Springer, Conflict of Interest: http://www.springer.com/authors/manuscript+guidelines?SGWID=0-40162-6-795522-0 
Article on COI in medicine: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2246405/ 

I remember being confused about what was expected of scientific authorship in grad school. My mentor Pam Matson had a helpful rule of thumb: there are three things you can do to contribute to a scientific paper: (1) have the idea, (2) get the money, and (3) do the work. At least two of these three are required for authorship. (Thus, under this model, a PI who has an idea and gets funding to support a PhD student on that theme would be expected to be a coauthor on all resulting papers.) 

I appreciate having clear guidelines and expectations for authorship, so I was glad to come across the authorship guidelines from the Vancouver Convention. Basically, they recommend 4 criteria for authorship (all four criteria must be met for authorship):

1. Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND

2. Drafting the work or revising it critically for important intellectual content; AND

3. Final approval of the version to be published; AND

4. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

This is the model I aim to follow in my collaborations. Thus, I expect myself and all authors to make a substantial intellectual contribution (#1) and contribute to writing and editing the manuscript (#2). 

I interpret #3 above as the lead (first) author has responsibility to solicit and integrate input from all authors in making revisions, and obtain their approval before sending to the journal. I interpret this responsibility as applying at three stages: 

1.  During drafting of a manuscript, until all authors approve the MS being submitted to the journal; 

2. During peer review, when the lead author takes primary responsibility for addressing comments from peer review, with input from all authors, and gets approval from all authors for the version to re-submit to the journal (this stage repeated as necessary if there is more than one round of peer review); and 

3. During copyediting, when the lead author shares the typeset and corrected final proof with all authors for their approval before submitting for processing and publication. 

I think all three of these stages are important in order to ensure that the last round (approval before publication) is sufficiently met, so that all authors are in a position to take ethical responsibility for the work (#4).  

(See my tips on how to work with revisions suggested by reviewers here.) 

When working on revisions, and especially with large and diffuse author groups, the lead author has to herd the cats and balance between giving everyone opportunity for input, and making decisions about the most appropriate direction for the paper (especially when coauthors or reviewers may have contradictory suggestions). After giving all authors a chance for input, during revisions the lead author might send around a version that incorporates changes suggested and say something like,

“Thanks for all your comments, which have been incorporated in the attached version. I had to balance between suggestions X and Y, which I did by Z; I hope everyone is satisfied with this approach. I would like to submit on X date (eg 1 week in the future). Please reply with either (a) any critical changes needed for accuracy or (b) your approval to submit. Thanks!”

It's especially essential to receive positive affirmation (i.e., a verbal or written OK to submit) from each author for the final version to be published. 

It's academic job hunt season! 

In case it's useful to folks preparing their own applications, here's the application I submitted 10 years ago (!) to get my current job. 

See also my advice (with Josh Goldstein) on the academic job hunt process. 

Good luck!! 

Whoohoo, you wrote a thesis, congrats! 

Now it's time to present it to the world, and defend it to your academic colleagues! This is exciting, and also stressful. That's normal. Do what you can to prepare (give a practice talk at least several days in advance to some supportive friends and incorporate their feedback for improvements; run through the talk at least several times until you are confident you have the timing down). After that, don't worry about being nervous. My experience with giving talks is that I am always nervous, and that's ok; I can be nervous and still do a good job.  

Here's my advice on structuring your presentation. 

For the session with your opponent, be prepared for both big picture and detail questions, on both your written thesis and oral presentation. The following are by no means exhaustive, but just a sample of some kinds of questions that have been asked in my experience (of course your own experience may vary).

It's OK to take a moment to consider your answer, or to ask for clarification about what the opponent is looking for. 

Here are some questions you can prepare to answer (out loud on your own, or at the end of your practice talk with friends): 

1. Why did you choose this study system?
   
   
2. OK, you work on carbon sequestration in grasslands. What % of the global carbon budget are grasslands? Do they sequester more or less than forests? Similar question from a friend’s defense: why is it that the Ross Sea is so productive compared with other global oceans? (Put your work in larger context.)  
   
   
3. What is the contribution of your dissertation? [to X field]?
   
   
4. How do you know that XYZ assumption you made was appropriate? Why was this method (regression analysis, interviews, etc.) appropriate for the question you wanted to answer?  
   
   
5. How would you demonstrate X? (Might be a logical next step/extension of the work you did, or might be a totally off the wall thing that makes no sense, partly testing you on whether you recognize this.)
   
   
6. What is the mechanism that drives the result you found? Does it apply in other systems? How?
   
   
7. What does the literature say about your findings/this work? Who are the people at the cutting edge of this field? Who are the people who would disagree with your findings? Related: So-and-so just published a Science paper/gave a talk on X (may be closely or not so closely related). How does this relate to your work? Do your findings agree or disagree with theirs?
   
   
8. You should have measured/done XYZ instead. (This is the kind of thing that reviewers say all the time so I think this is to prepare you for that. Talk about how that would be interesting to answer a different question/with more resources, but for your question, your approach was solid.)  
   
   
9. Explain the two-year reproductive cycle of the grapevine, draw the global patterns of nitrogen availability, temperature & C3 vs. C4 grasses (... various specific things with right and wrong answers... hard to "study" for these as they could be about anything even tangentially related... perhaps spend 30 minutes glancing over a grad-level textbook most closely related to your field to jog your memory, but most importantly stay calm, use logic, draw on the vast amount of trivia you’ve been filling your brain with for the last 5-10 years, but admit when you don't know an answer and say where you would go (or what you would do experimentally) to find it out.)
   
   
10. What are your next steps? (Both in terms of this research, and professionally, including where do you want to be in 5-10 years.)  
    
    
11. What are the management or policy implications of this work? How have/will you communicate these to managers/policymakers?
    
    
12. What are the theoretical implications/contributions of this work? (Put in context of broad concepts of the field, not just literature but bigger picture, throw some terms around that relate to theory like “vulnerability analysis,” “mass balance,” whatever).
     
    
13. In one sentence, what was the aim of your thesis? 
    
    
14. What do you mean by [term used in thesis]?
    
    
15. How would your results have been different if you had [interviewed X/done the work in X case/...] instead? 
    
    
16. What literature did you include? For example, was it only from Europe and North America? How does this affect your conclusions? 
    
    
17. Who could potentially benefit from your thesis? How will you communicate your results to them? 
    ​
    
18. What's the potential your proposed course of action will work? What barriers exist? Is there anywhere this has already worked? Where and why? 
    
    

I did not do this… but it occurs to me that it would be helpful to write out a list of questions that you would ask a student presenting your work. Include the most gnarly questions you can think of, the ones you hope they don’t ask that expose the weaknesses or shortcomings. Then get a trusted friend to ask them to you, and practice your answers. I think this would make you feel a lot better, and will probably be much harder than the actual defense. If you're feeling brave you can even incorporate these points in your slides (i.e., acknowledge and justify shortcomings or what you'd do differently).

​Good luck! You'll do great! 

In other words, you don't start out deciding to do a survey, then looking for appropriate research questions to ask and attendant philosophies and theories. For a simple breakdown of how philosophy of knowledge, theory of knowledge, and research style combine to produce research programs, see our 2010 article "Thinking about Knowing"- this could help you write the dreaded "philosophy and theory" sections of a master's thesis, for example.  

Another useful resource is this overview of qualitative research in conservation by Moon et al. 2016, including a helpful list of questions in Table 5 that will help you avoid many common pitfalls. 

2. What are the research questions you want to answer? 

It is essential that you have a manageable number (i.e., not more than three) clearly articulated research questions that guide all further research design. Coming up with a good research question is an art in itself (take a look at The Craft of Research for a whole chapter on how to do this).

In short, my view is that a good research question is grounded in a real-world problem, linked to other research and/or theory in the field, specific enough to be answerable with the resources you have available, and something you are passionate about answering.
 
3. Are surveys the appropriate method to answer your research questions? 

Be sure to consider alternatives and convince yourself that surveys are a good way forward, that will help you answer your research question. If you can directly observe behavior or use existing data to answer your question, that will be easier than designing and carrying out a good survey. On the other hand, if you need to know how people think (attitudes) or feel (values) about something, then you need a survey to ask them directly to find out. 

4. Who is your target population, and how will you sample from them? 

Consider that your sampling design will affect the validity and generalizability of your conclusions (i.e., if you want to do inferential statistical analysis to draw conclusions about significance, you need a random sample that is large enough to be representative). William Trochin has a good, concise overview of sampling and social research design issues on his Research Methods Knowledge Base. 

Issues to consider here include how you identify the target population (what characteristics must they have?), and how you identify them (what criteria will you use to include or exclude participants, and how many do you seek?). Write down all your criteria in your Methods section under Sample Selection. 

5. How will you design your survey? 

Here's a really nice, simple, clear overview of best practices for survey research design by Kelley et al. 2003. This is a great place to start.  

Good tips on survey design, question construction, and administration in a book chapter from "Investigating the Social World" by Russell Schutt. 

6. How will you analyze survey data?

The simplest way is using descriptive statistics- e.g., graphing the distribution of the results in a histogram.

​A more complex way is inferential statistics- using statistical tests to draw conclusion about the likelihood of observations being due to chance, attributing numerical confidence to the results. This is a big topic; to get started, check out this guide from The University of Reading Statistical Service Centre on the analysis of survey data. 

For descriptive data (verbal or written textual responses to open questions), the aim is not to condense them into a number, but to use them to represent and explore different views. Here the process of coding can be helped with qualitative analysis software like NVivo or Atlas.ti. You will chose (and should state in your Methods) whether you are using inductive coding (themes emerge from the data) or deductive (trying to match responses to categories from previous literature or theory), or a combination. 

Whatever analysis approach you take, describe it clearly in your Methods, and explain why you chose it. 

7. Research Ethics

Conducting research with human subjects requires the researcher to take responsibility for considering and minimizing the risks presented to participants, and make sure that clear, prior, informed consent is obtained from your participants (this means your participants understand that their participation in the study is voluntary, they understand any risks that may be presented, and they know that they can end their participation at any time of their choice without penalty). You need to think about how you will protect the privacy of your participants and how to handle their data fairly and with what degree of anonymity. It is good practice to obtain written, signed informed consent before enrolling a participant (this might be ticking a box on the front page of an online survey, or initialing and signing a separate consent form including a copy for them for in-person interviews). 

Be aware that you are responsible for following ethical guidelines in the country where you conduct your research. In the US and Canada among other countries, completion of a formal course in research ethics and approval of a human subjects protocol is often required before conducting survey research.

Here is the human subjects protocol I submitted to Stanford University for my PhD research in 2006- there are lots of helpful questions here to consider. And here is the consent form I used with my study participants.  Feel free to use these as templates for your own research. Answers to the protocol questions can go in the Methods section of your thesis, and the consent form can be modified and used with your study participants. 

In Sweden, research should follow the Swedish Research Council Vetenskapsrådet's  guide to Good Research Practice, with many useful areas to consider. Research conducted for a master's thesis generally does not require a formal application to the regional review board (meets monthly, costs 5000-16000 SEK to review applications, which must be in Swedish). Here is the Swedish law regarding research ethics, for research conducted in Sweden, that presents a sufficient risk or sensitive data.   

However, researchers still have a responsibility to follow good practices. Please be aware that you may be required to demonstrate good practices were followed and informed consent was obtained in order to publish your research results in a peer-reviewed journal. See below for two examples of text published in peer-reviewed articles describing how informed consent was obtained in Sweden. 

Text about informed consent: "Although the study scope exempted it from Swedish requirements for formal ethical review by an institutional review board, all procedures performed in this study involving human participants were in accordance with the ethical standards of the institution, and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Prior, written informed consent was obtained from all individual participants included in the study (See consent form in S1).”​
--Source: Wynes and Nicholas, in second review Feb 2019, PLOS ONE

"According to the Swedish regulations for conducting research on humans, there was no need to seek ethical clearance for this study. All participation was voluntary after informed consent, and the participants could withdraw from the study at any time. In order to protect the participants’ identity, revealing information has been removed from the results such as specific faculty, discipline, gender, ethnicity, and age. Furthermore, before data was analysed, all participants were invited to confirm their own interview transcripts and remove quotations that they did not want to share with others."
--Source: Brodin & Avery, in press

I hope these tips are helpful- happy research! 

Hooray, at least one editor and one or more experts in the field have given their valuable time and attention to read and comment on your hard work, and you have a long-awaited response from the journal! 

Here are some tips to help you navigate this process: 

1. First, take a deep breath! Remember the goal of peer review is to improve the final manuscript (and research shows that papers that have been through several rounds of peer review get cited more than those that were more immediately accepted!). Keep your reader in mind and work to make the changes that will make the text more accurate, clear and valuable for them. As a baseline, assume that editors and reviewers also share this goal and try to take their comments in that spirit. 
2. Read through the reviews carefully, paying special attention to the Editor's instructions - this should guide your prioritization for revision. Look for common themes across reviewers- areas of agreement are important to clarify in the manuscript. Consider the type of comment- does it relate to the overall argument/soundness of the work in supporting claims with evidence, is it more about the structure of the writing, or a small detail to clarify in wording? Getting the big picture helps you get an idea about where to focus your revisions. 
3. As first author, you have primary responsibility for leading revisions to address reviewer's comments, and for keeping all of your coauthors informed- including asking for their help with revisions where needed, and getting input and consent from all coauthors on your suggested response to reviews and the revised manuscript before resubmitting it to the journal. It is important to share the reviews received with your coauthors right away and keep them informed of due dates and progress. You should suggest a timeline for making the revisions needed, that will give all coauthors time to read and respond to your suggested revisions. Be sure to ask for what you need in terms of support- i.e., do you want a brainstorming Skype session to help formulate your overall response strategy? Are you unclear what certain reviewers' comments mean, or what they are looking for? Are there parts of reanalysis or writing that you need coauthors to lead?   
4. Save a new draft of the manuscript where you'll work in Track Changes to share with your coauthors. Consider annotating this with some of the key, overarching comments from reviewers (i.e., "Insert Comment" in Word at the relevant section where the reviewer's comment applies- see example above left from a recently accepted paper led by Klara Winkler). This can help the manuscript read more coherently and coauthors will understand your motivation for the changes you've made, and help assess if these changes have fully addressed the reviewer's concern.
5. Make a table with the Editor's and Reviewer's responses in the left column, labeling the source of each comment (e.g., first all comments from the Editor, then from Reviewer 1, etc.). Give each main idea its own row. Be sure to include any in-text comments that were received here as well. In the right column, you will write your response to address this comment (see example above right from Winkler et al., accepted). The final version will answer the reviewer's question (e.g., "Yes you're right and we did this on line X," quoting the relevant manuscript text here if needed). For a first draft, this column can contain notes to your coauthors, e.g., "I don't understand what the reviewer means here" or "Kim, can you please answer this one?", or some initial bullet points about your planned response ("Rewrite this para to make link with research question 2 explicit").  

I hope that's helpful to get started!

For more in depth tips, I suggest the excellent article "How to reply to referees' comments when submitting manuscripts for publication." Peer review would go so much more quickly and smoothly if everyone followed Hywel Williams' three Golden Rules: Answer completely, answer politely, answer with evidence. See also his phrasings for suggested responses to reviewers, and tips for how to deal with common scenarios like reviewers who disagree, or who are rude. 

Another good source is "10 simple rules for writing a response to reviewers," by William Stafford Noble in PLOS. 

Happy revising!  

Last October, I gave a Sunday morning talk to a group of early-career researchers attending the Earth Systems Governance conference. It was a day-long program on "Developing a career in earth system governance: opening up science." I enjoyed the chance to gather my thoughts and pass along some good advice I've been given (and some earned through experience!).

Thanks very much to Ina Möller, who made a podcast from our conversation. You can have a listen here.

Here's a condensed list and links to resources I've found helpful. Hope they're useful to others! 

Here’s a short summary I co-wrote with Céline Fernandez compiling the highlights of our November meeting for the LuPOD professional development program. ​

Best practices for films for educational purposes:

• Max length: 5 min
• Availability: no more than 2 clicks to access
• Sound quality is important to the viewer
• Think about the voice melody of the speakers (no monotonous tone)
• Lifetime: since the film will most probably remain on the web for a long time, use a layout that is likely to age well   

 
Issue regarding available learning material online: often no quality control. Therefore, make sure to review material before assigning, point out to students to be critical when using information online, and consider making your own material.
 
 
Teaching portfolios

• Why you need one: it helps you document and reflect on your teaching, and may be required for application to positions (e.g., lecturer or docent) or for merits like the Pedagogical Academy/Excellent Teaching Practitioner.
• Get a model: Applications for positions (such as Docent) at Lund University are public documents and should be available on request. These are valuable to study when working on your own application. Also ask colleagues to share their successful applications for inspiration.
• Pedagogical philosophy: Get beyond buzzwords from LATHE courses and show how you expressed pedagogical ideas in practice.
• Pedagogical reflection: Give both good and bad examples from your teaching practice to demonstrate your own reflection and development.

• Scholarship of teaching and learning: taking an investigative approach to teaching and learning, applying an iterative cycle of reflection and improvement.

 
References and further reading:

• Application for Excellent Teaching Practitioner at Lund University, including instructions for the teaching portfolio required, are listed by faculty. Here are links to the Social Science Faculty and LTH, which were easy to find in English on the Lund website. :) 

Science should be open. Duh. 

Easier said than done.

There are many reasons why this fails.

​Personally, I think mostly it's well-intentioned, busy people who don't follow best practices from the beginning, then don't want to spend the time cleaning up their Rube Goldberg-esque Excel sheets later.

I've definitely been there. But I'm trying to get better, and trying to help those in my lab establish good habits. 

For my last lab meeting, we discussed best practices in open data, based on this paper by my friend and colleague Lizzie Wolkovich. 

Everyone in my lab is now working on curating our own data for a current project, including making a diagram of our workflow, and organizing our data cleanly, with good meta-data. 

This is a work in progress, but here are a few resources I've found helpful: 

1. Lizzie's paper, "Advances in global change research require open science by individual researchers," gives a great motivation for why open data matters, what stands in its way, and how to design research to be open. 

2. Ten simple tips for how to design a clean spreadsheet, by @robinhouston and @SeanClarke (hint: commas, asterisks, and color-coding don't belong). 

3. Thirteen more elaborate but still simple and important tips for effective data management (what you wish you knew at the beginning of your research career, instead of learning the hard way, like always using full, consistent format for dates). 

4.  Lizzie's "Ze Template" for organizing project meta-data (what the study is about, what files it involves, where they're located), under Creative Commons license. Yay open science, and yay Lizzie! 


We're going to keep talking about this in my lab, including issues with open data in qualitative research (confidentiality, subjectivity of observations)- if you have any references on this, please let me know! 

This week was the first teaching retreat for teachers in USV, Lund University's division for faculty-free centers ranging from the Centre for Middle Eastern Studies to my own department of LUCSUS, the centre for sustainability studies. About 25 teachers stayed at a former church in the town of Höör, 25 minutes by train from Lund. 

Over two days, we learned new tools and approaches for teaching, and had time to share our ideas and learn from each other. Some of the main tools I learned about: 

1. Live voting using smartphones in class to check student understanding
   
   The Swedish company Mentimeter has designed a simple, free interface where teachers can set up questions (multiple choice or open answer), and students vote online or with their smartphones. You can see the anonymous votes accumulate in real time. Useful to check student understanding after discussion, keep engagement high, correct misconceptions. 
   ​
2. The "flipped" classroom 
   
   "Flipped" classrooms use student time at home to read, watch video lectures, and do quizzes; and then use class time for active tasks like discussion and group activities (rather than passive absorption of lecture material). This model changes the role of the teacher from "sage on the stage" to "guide on the side".  We were encouraged to try this model and shown some data on how effective it can be for student learning. 
   
   
3. Filming lectures for online & MOOCs 
   
   Online lectures are increasingly popular for MOOCs (massive online open courses), or as part of a traditional or flipped classroom. We were advised that a good video lecture for student learning is short (5-9 minutes), where the teacher addresses students directly (not students in another classroom), shows passion, and underlines important ideas. The video/audio quality isn't critical for student learning (as long as you can be clearly seen and heard), so don't worry about fancy technology- the webcam on your computer is good enough! 
   
   Lund University has a new tool, LU Play, for recording screencasts of lectures. We played around with it a bit- quite easy to use to film slides and your presentation, and you can include interactive quizzes. One disadvantage though is that it's only available internally to LU students logged in with a Stil-account, and not easy to upload directly to YouTube or other public platforms (you have to download to your own computer first), which made me wonder about archiving and long-term access. 
   
   I found a good video from UBC professor Rosie Redfield on why and how she records her lectures herself, with encouraging tips and helpful explanation of the technology she uses. Also it seems that the free open-source Open Broadcaster Software is a good option for recording screencasts (though a little less intuitive than LU Play at first). 
   
   
4. Syllabus design 
   We discussed the purpose and role of a course syllabus with Katarina Mårtensson Lund University Division for Higher Education Development (AHU). She pointed out all the different roles that a syllabus can play, including: forcing the teacher to focus and prioritize, making the learning outcomes visible, as a communication tool between teachers and students, between teachers to focus on overall program coherency, and to help new teachers who inherit a course, or have to take over on short notice. 
   
   We reviewed a syllabus from a colleague from another department and gave feedback on questions we would have as a student. This made me realize the importance of being familiar with similar courses to highlight what distinguishes my course (so I should do a syllabus review when designing a course, similar to a literature review for writing an academic paper).