5 Semester 1 Week 9
5.1 Collecting data, developing evaluation for the introduction
In this session, I tend to cover data collection (i.e. trying to pre-empt any issues before they happen) and also developing evaluation of the Introduction section. I usually have both quant and qual projects so cover both types of data collection - I would only cover one though, if students were only doing one type. We then cover evaluation in Intros using some resources that Wil and I developed for our Research Methods 2 course. (Discussions are covered in Semester 2 Week 5)
Although I cover data collection in Week 7 (if I have time), that was an initial discussion to put it at the forefront of their minds and this is more in-depth, as they should be close to having ethics approved if they don’t already.
5.1.1 Expected progress
- By now, ethics approvals should be imminent or through already
- Some students may have already started data collection, or might be getting materials together in anticipation of starting
- If doing secondary data analysis, the timeline is a bit less pressured
- Students should be working on introductions
5.1.2 Data Collection (Quant projects)
(N.B. I only supervise survey-based quant projects, so my notes refer to this only)
- How is data going to be collected (e.g. Qualtrics, experimentum?). Student should use university email, not personal email for account.
- Even if ethics isn’t through yet, can still familiarise themselves with it.
- Are students clear on how to present their study on whichever system they are using?
- Test, test and test again
- It’s important to make sure that scales are presented correctly. I’ve had one student have to start from scratch before because they put Strongly Agree - Disagree - neither agree nor disagree - Agree - Strongly Disagree and only noticed mid-way through data collection
- We also had one project where the participants saw a picture and it wasn’t clear from the data which picture was presented (and it was the between subjects condition) and she had to start data collection again.
- I emphasise to students that they should not start data collection without extensive testing and I reiterate to let me know when it is ready for me to look over. Other supervisors may differ on this, but it’s caused issues in the past (see examples above).
- Students need Emily’s extended debrief if going to recruit Level 1 students
- SONA is used to recruit Level 1 students (students should see Emily’s sway on how to do this)
5.1.3 Data Collection (Qual projects)
- How are students collecting data?
- If online, must use GDPR-approved system (e.g. Zoom or Teams)
- If in person, will need a) appropriate recording equipment, b) a quiet space
- Some supervisors differ on their opinions on this, but I ask students to send out questions in advance for all participants, as it can be more accessible for certain people (e.g. those who are neurodivergent or who don’t speak English as a first language)
- I ask students how they are feeling about collecting data. Typically, they can be a bit nervous
- Important to practise setting things up/recording etc.
- Good to pilot things (e.g. with a friend). I’m also happy to help in office hours etc.
- Can practise with each other (i.e. others in the group) to help get comfortable
- Can be helpful to write a list of everything need to cover at the start and end so that they don’t forget anything
5.1.4 Evaluation in introductions
In RM2, Wil and I have made some excerpts from intros and discussions for our labs. In today’s session, I only cover intros and do discussions in a later session.
Here, I show the students the poor and stronger versions and we then have a discussion about it (slides with notes).
This resource has ‘not so good’ examples, then ‘stronger’ examples and then notes for things to look out for and finally some tips.
The three things we look at are:
- Synthesising evidence
- Focused criticism
- Why is your study worth doing?
5.2 Internal resources
Guidance from Dr Emily Nordmann on extended debriefs for Level 1 students and how to recruit on SONA
Excerpts plus notes and tipes on Developing Evaluation in Introductions by Dr Wil Toivo and I