Appendix B — Research skills sessions
Ashley has recorded optional research skills sessions on different aspects of qualitative research. These sessions take a deeper dive into some of the topics and may be particularly useful if you are planning on doing a qualitative dissertation next year. Each session has its own section below, with a link to the recording and the activities that go with it.
B.1 Developing questions
B.1.1 Overall structure
It’s important to think about the overall structure of an interview or focus group schedule. A good structure to use is: A warm up question, the main questions you want to ask, and an exit question. It is likely particularly important to use a warm up/ice-breaker question for a focus group, especially if participants don’t know each other particularly well.
Warm up Question
The aim is to allow participants to introduce themselves, ensure everyone understands the procedure, and to encourage people to feel comfortable talking. Previous projects have used photos, cartoons, and media, or asked participants to generate meanings of the terms used in the discussion (e.g., “poor sleep”, “well-being”) to get people talking. What is your idea for an ice-breaker that might lead naturally into the main discussion?
What makes a good/poor ice-breaker?
Thinking about the points above: What do you think of the following ice-breakers?
- An FG on lived experience during COVID-19 of international students: What is your favourite meal from your home country?
- An FG on coping strategies in students: What do you think is the best way to reduce stress for students?
- An FG on feelings of belonging in a work context: What do you think about the graphic below? Does it resonate in any way?
Main Questions
Bear in mind that the discussion may wander off track or focus on one aspect. Consider which part of the research question you think is most important to get people to answer. Start with this and be prepared to come back to it at the end if you need to. If you have more sensitive questions in your research schedule, they should be included later on, so people answer them when they are more comfortable.
What should the questions be like? Avoid asking questions that ask two or three things in one sentence (e.g., about behaviour and impact on mood). Break it down into simpler questions. Limit the number of direct questions that elicit yes/no answers. These can be useful to gather information but are less helpful in generating a discussion. Add in probes/prompts to elicit further information if required.
Try to:
Frame questions in an open way that will facilitate discussion (e.g. ‘do you use Facebook?’ is closed (can be answered as yes/no), whereas ‘can you tell me about your experiences with social media?’ is a bit more open - you could then follow up with prompts to specifically ask about types if you want to get that information).
Frame questions in a way that is not leading. For example, ‘how did lockdown negatively impact on your ability to study?’ assumes that it did, when this may not be the case. A better question would be “Do you think that lockdown restrictions impacted on students’ experiences of studying? If so, how? If not, why not?”
If you have sensitive questions, try to frame questions in a way that is not personal (i.e. you give people a chance to answer related to themselves, but they can also choose not to). This approach also invites people who may not have lived experience to respond, e.g. “How do you take care of yourself when you are anxious?” would be better phrased as “How might students support themselves when they are feeling anxious?” This is more important for focus groups, as people might feel put on the spot and go beyond their comfort levels.
Avoid asking lots of different things within the one question, e.g. “Can you tell us about your experiences of studying in Semester 1, how it made you feel and how that affects your studying in Semester 2?” You could ask about experiences of studying generally, and then could follow up with specific sub-questions asking them about semester 1 and 2, if that is something that was important to you.
Closing question
The aim here is to give people an opportunity to add anything that might have been missed in the discussion/to wrap things up.
Examples:
- Thank you very much for all the great points you had today! Before we wrap things up, do you have any final points that you would like to make, that perhaps weren’t covered in the interview?
- Today we covered X, Y, and Z in the focus group. Is there anyone who has any final thoughts about what we covered here today that they’d like to share?
- That’s us finished with the main part of the focus group. We’d like to end by giving you all an opportunity to add anything that you think is important but has not been covered in the focus group so far. If you’d like to add something, please press raise hand on Teams and we can make sure we go round those people before we finish up!
B.1.2 Braun & Clarke (2013) activity
Here, we have an adapted version of an interview schedule that is included in the Braun & Clarke (2013) book, on the topic of Partner Relationships. This is purposefully poor. What problems can you spot?
- If you are not already married, are you planning to get married within the next few years?
- I love going to the cinema with my partner, do you?
- Do you have a good time with your partner?
- Would you go to Relate if you had problems in your relationship?
- When and how did you meet your partner?
- Do you earn more money than your partner?
B.1.3 Further things to consider with qualitative surveys
- No opportunity for clarification, so wording has to be clear
- Similar points to above in terms of not asking too many different things, having a sensible structure
- No rapport, unlikely to be an ‘ice-breaker’ (although might have an intro question)
- People might be more comfortable answering sensitive questions as they can be anonymous
- Try not to have lots of questions as your participants will likely lose patience - what are the key things you want to know?
- Can use branching e.g. in Qualtrics
B.2 Data collection
B.2.1 General tips
- Be organised - get ethics in early etc., keep track of potential participants
- Think through all the different steps involved as you will need to refer to this in your ethics application - how do you plan to recruit, collect the data, store the data? Before you start collecting data, it would be good to have a checklist with everything you need to consider
- Only collect the demographics that you need - if you don’t need to know how many siblings they have or what time they go to bed, then don’t ask for that information
- Be sensitive to who your participants are - you might ask different questions in a focus group (where people are in front of others) than you would in an online questionnaire (where it is easier to withdraw if uncomfortable).
- Think about how you keep that data safe/anonymous
- Know in advance how many participants you need
- Know how you plan to analyse the data
- Consider what you can do to make the experience as comfortable as possible for participants (e.g. sending out questions in advance is particularly helpful for neurodivergent people/people who do not speak English as a first language).
- For interviews and focus groups - reassert consent before you start & always remember to press record! Also it’s normal to feel a bit nervous - try to enjoy the process!
- For all, pilot the questions you ask - make sure that people take from them what you think
- When recruiting, think about who your target group is and it’s good to justify why you are looking for a particular demographic. For example, do you definitely require someone to have a particular diagnosis?
B.2.2 Practicalities of running focus groups
- Establish ground rules - esp. confidentiality
- Set expectations - ask participants to respond to each other rather than each person individually responding to the question (if possible, this is not always the case)
- Moderator has to manage the group as best they can
- Wrapping things up.
- Can be helpful to have a notetaker to note down anything interesting/if doing it in person, to note down order that people speak in. Allows the moderator to focus on discussion
Tricky situations
- How do you develop rapport/make people feel more comfortable?
- How do you manage tricky dynamics within a group?
- What if no-one speaks/one person dominates?
- What if people go off topic?
- How to keep people to time?
- Giving everyone who wants it an opportunity to speak - if online could ask people to raise their hand
B.2.3 Practicalities of running interviews
Some of the things that we discussed in focus groups are relevant here, but specifically when thinking about interviews:
- Try to develop rapport and make the participant feel comfortable
- Try to focus on the person rather than taking notes - are you giving them encouragement to keep speaking (e.g. nods etc.)?
- If doing a semi-structured interview, try to notice if the participant brings something up that could/should be followed up with an additional question
- It’s usual for some interviews to be shorter than others - some people are very concise in how they answer whereas others talk a lot more (and this may be on topic or not)
- Be aware of how someone is presenting - this is less relevant for coursework and more for future projects - if you are doing quite a sensitive topic, it’s good practice to reassert consent and to ask people if they need a break for a little while/if they are okay to continue.
- Try to end on a positive question if you can - it can help to lift the mood for the participant and leave them in a good place
B.2.4 Practicalities of running qualitative surveys
- Think about whether there would be any possibility for the questions to be misinterpreted - you won’t have a chance to correct people if they interpret something wrong (this is where doing a pilot can be useful)
- Try not to have too many questions - what are the key things you want to know? People will be less likely to respond with long, in depth answers if there are a lot of questions to answer.
- It might be good to give an indication of how many questions they have to go through.
- Easier to keep data anonymous than it is with other forms of data collection above - however still be sensitive to locations, names, personal experiences as these will have to be anonymised.
B.3 Preparing for data analysis
B.3.1 Thinking about reflexivity
We ask you to think about your own position in the research as part of your RM2 report (there is a section in the methods!).
This can be quite tricky to do, but is very important for qualitative research. Some suggestions to get you started:
- Think about the topic you are doing for your research and your position in it. Where would you place yourself in relation to it? What are your own views on the topic? Do you feel quite strongly about it or not?
- Write as you reflect. Writing – be it using pen or pencil on paper, typing on a keyboard, dictating to some voice capture device, or some other mode – is an important tool for developing reflexive depth. It is good to do this at various stages of the research process.
- Start with your own experiences, understandings and views, but then also try to interrogate those (i.e. why do you have these particular understandings, views etc.?)
Some questions that you might ask yourself:
- What do I expect to come up in the data? How does this relate to what we found?
- What broader experiences have I had that might influence how I’m thinking about this?
- What are my values (i.e. what is important to me)? What about my beliefs? How might these have influenced my thinking?
- How might all of these be connected to my identities and the communities I am a part of?
B.3.2 Activity 1: Read this extract and take familiarisation notes
Read this extract and take some familiarisation notes for yourself (these are just for you and won’t be shared). We will then go through what Braun & Clarke say about it.
Some context:
This activity is based on a data extract from an interview study exploring LGBT students’ experiences of university life. The participant is a gay man – a mature student, who spoke English as a second language. The extract comes from early in the interview and in response to a question about whether he is ‘out’ as a gay man at university – whether he is openly gay.
In this activity (5 mins):
- First, reflect on your assumptions and expectations about this topic, and your positioning in relation to this topic (are you an insider researcher? Outsider?).
- Second, read the data extract and make some familiarisation notes.
- Braun and Clarke’s familiarisation notes follow – reflect on similarities and differences in your and their analytic observations and ‘take’ on the data.
Andreas: …I sometimes try to erm not conceal it that’s not the right word but erm let’s say I’m in a in a seminar and somebody- a a man says to me ‘oh look at her’ (Int: mm) I’m not going ‘oh actually I’m gay’ (Int: mm [laughter]) I’ll just go like ‘oh yeah’ (Int: mmhm) you know I won’t fall into the other one and say ‘oh yeah’ (Int: yep) ‘she looks really brilliant’ (Int: yep) but I sorta then and after them you hate myself for it because I I don’t know how this person would react because that person might then either not talk to me anymore or erm might sort of yeah (Int: yep) or next time we met not not sit next to me or that sort of thing (Int: yep) so I think these this back to this question are you out yes but I think wherever you go you always have to start afresh (Int: yep) this sort of li- lifelong process of being courageous in a way or not …
B.3.3 Activity 2: A code or a theme?
In this activity, we’ll go through a handout that asks you whether something is a code or a theme.
For each label, decide whether it is a code or a theme.
- Keep in mind that although there is no absolute distinction between codes and themes, codes tend to have a single facet and capture one insight or observation about the data, whereas themes are ideally multifaceted and capture several insights and observations. Themes should also capture a pattern of shared meaning – this can be shared meaning on the data surface, or shared implicit or underlying meaning.
- Reflect on why you have decided that each code/theme is a code or theme. What features of the data, the label/name, and brief explanation of the scope and focus of each code/theme influenced your judgement?
Show me Braun & Clarke’s solution
Three themes were presented in the analysis: “Two themes capture contradictory ideas: that men’s body hair is natural, and that men’s body hair is unpleasant. A third theme introduces the concept of ‘excess’ hair, which allowed sense-making of this contradiction, mandating men’s grooming of ‘excessive’ hair.” (Terry & Braun, 2016, page 14)
B.3.4 Some common problems with thematic analysis
- Thematic analysis that fails to address the research question.
- Unconvincing or under-developed analysis: 1) Too many or too few themes 2) Too much overlap between themes or themes are unrelated 3) Themes are vague or not internally consistent or coherent
- Little or no analytic work has been carried out (for example, using data collection questions as ‘themes’).
- Topic summaries being presented as themes
- Mismatch between data and analytic claims (e.g. going beyond what your participants say)
- Too few or too many data extracts (and little or no analytic commentary).
- Paraphrasing rather than analysing and interpreting data
- Arguing with the data – pointing out the ‘errors’ in a participant’s account.
B.3.5 Strategies for ensuring quality in your analysis
Reflexive journalling
Keep reflecting, and writing, throughout the whole research process, not just at the end (see Thinking about reflexivity).
Talk about your data with others
- Discuss your analysis with other people – it helps to clarify your analytic insights
- Do phases 1-2 in pairs if you want
- Discuss your findings in groups
- Come along to office hours
- Present your analysis (even if it’s preliminary), this can help solidify where we are going with it
Give yourself time
- Try not to leave the analysis until the last minute
- Give yourself time to step away and breathe
- Perhaps work on other sections of your report at the same time, so that you have a mental break from analysis
Make sure themes are themes (and be careful when naming them)
Connelly and Peltzer (2016, p.55): “When a researcher designated a 1-word theme, such as ‘collaboration’, what does that mean in relationship to the experiences of the informants as interpreted by the researcher? Using only 1 word as a theme, there is no way of knowing, for example if the experiences of collaboration were positive or negative, or whether collaboration is important to the nurses. One-word themes do not convey what the researcher found out about collaboration”