Tag: data generation

Obtaining the data you really need: on conducting qualitative interviews

Leave a comment

I have been planning a new qualitative research project, and reading draft proposals and draft methodology chapters for students I am coaching, so I have been thinking about qualitative data lately; particularly how to get the right kinds of data from participants when we are conducting interviews.

There are three main forms of interview that are discussed in methodology texts and guides: structured, semi-structured and unstructured. Generally, when embarking on a qualitative project, students tend to opt for semi-structured interviews. This enables them to have a set series of questions, hopefully well connected to their theory and literature, but also to create space for participants, or interviewees, to add views and insights that may not strictly follow the questions. It’s sort of a best-of-both-worlds scenario. Unstructured interviews are hard to manage, especially for postgraduate students, many of who are doing this form of fieldwork for the first time. And fully structured interviews can veer into not allowing any space for additional, unpredicted insights and information – they can, in other words, limit the conversation, and also the kind of data collected. *Caveat: you do need to choose the right tool for your project, regardless of whether it is hard to do or not.

But, semi-structured interviews, while they seem to be a dominant preference for many students doing qualitative research via interviews, are not easy to do well. A key issue I am thinking about in relation to my new project, and that I assist other researchers with, is this: how do I conduct the actual interview so that I get the data I really need? What tools or techniques so I need to be aware of? I have listed a few points here that have helped me, and that are also the product of my own mistakes and learning in conducting qualitative interviews.

interview 1

Photo by nappy from Pexels

  • The point of the first interview is to get a second interview. 

I didn’t know this during my own PhD, which was also my first major research project. I did one set of interviews, and I do recall, on reading the transcripts, wishing that I could go back and ask some of the questions differently or find out more about specific issues. But I didn’t really know I could (and I didn’t plan for this so I ran out of time). But, now it makes complete sense. Unless you are interviewing someone you already know well, chances are you and the interviewee will be strangers to one another. Establishing a rapport, and trust, can take time – certainly more time than one interview allows for. So, you need to make the goal of the first interview establishing a connection that will enable you to go back, and talk to that person again. You may get the basic data you need in the first interview, but chances are that you will need more that interview #1 can yield. You will want richer, or deeper responses to some of the questions, or will see that certain questions could be asked differently, to yield slightly better, or more relevant, responses considering your research questions and aims.

Plan enough time for at least two interviews, and ensure that you check with the interviewee that you could approach them again if needed. Listen to the recording of the interview as soon as you can, then, make notes, and think about what the data is saying in relation to those research questions. Then work out whether you have enough of the rights kinds of data to include in your analysis, where any gaps are, and plan interview #2 accordingly.

  • Record your interviews and take notes.

You always want to audio-record the interviews, so that your transcription, analysis and reporting will be accurate, and rich. But, you also want to take notes – not so many that you are unable to maintain eye contact and engagement in the interview, but have a notebook and pen to record things that the audio may not capture. Perhaps body language is a factor to consider in terms of the kind of data you are generating – if participants appear nervous, or distracted, this may be something you want to consider in the analysis alongside their words. Perhaps what their words say, and what their body language says are two different things, and this could be important as a possible finding. Perhaps they mention names of other people you could consider interviewing as well, or names of websites, documents and other sources of information you need to follow up on. All of this can be captured in a few short notes, and can add to the richness of your overall findings and analysis later on.

  • Plan for a pilot interview.

This has been a big point of learning for me. I tend to get over-eager in interviews, and I just plain talk too much. I find myself listening to the audio and cringing, and wishing I could just be quiet, and let the interviewee talk! There are two benefits, here, to a pilot interview: the first is that you can practise being an interviewer. You can work out how to record and take notes, how to talk less and listen more, how to direct and redirect the conversation as needed, and how to manage your own facial expressions and body language, to remain as neutral as possible so as not to bias or shift the interview in unhelpful directions (like looking shocked or disapproving at something an interviewee says to you). The second is that you can figure out whether the style you are using and questions you have as a guide are yielding the answers you need. Are there too many questions? Are some of the questions too long and ‘wordy’? Do any of the questions yield answers that seem a bit off topic, or that are repetitive? Make notes and adjust the interview plan before the real interviews begin. You need to do a pilot with someone who represents the interviewee demographic, and preferably with enough time to make changes and adjustments before you start meeting with the study’s group of participants.

interview 2

Photo by nappy from Pexels

  • Plan for a conversation, rather than a Q&A session.

This has been a really big point of learning for me. In a qualitative study, regardless of the topic, you need rich, detailed data. Qualitative research is about depth, rather than breadth, in simple terms. Thus, you need depth in the data, to achieve depth in your analysis, findings and conclusions. Thus, what you want to do with conducting the interview is create a conversational space, rather than a stiff, Q&A session where you ask a long list of questions, and the interviewee responds. The best way, I find, is to start off by asking the interviewees to tell me a story, related to the research I am doing and my interview questions. For example, if I am doing a project on postgraduate supervision, and my interviewees are students and supervisors, I don’t want to start off with something like: “Do you enjoy supervising students, and why or why not?” (Typical kind of question you find in interview schedules). This will not help me to get a richer sense of what the supervisor does as a supervisor, and what aspects they do and do not enjoy. So, what I could rather do is ask the question like this: “Can you start off by telling me a bit about your current supervision situation?” And then just listen. In that response, then, I listen for what aspects they seem to like, or not like, and in the follow up, I could ask: “You seem to find supervision a struggle, rather than a joy, right now. Would you say that is accurate? Could you say a bit more about that?” And then listen. And so on. So my questions become a guide, but not a determinant to the structure of the interview.

The main thing in doing qualitative interviews, I am learning, is for an interviewer to have empathy. Trying to be in the moment, and create a sense for the interviewee that their stories are valuable, and worth sharing and hearing, enables the creation of a rapport that can lead to follow-on interviews, and that can encourage interviewees to see me as an ally, and someone who will share their stories responsibly, ethically and with care. This is crucial when you are working with people who trust you enough to give you truthful stories, experiences and accounts. Ultimately, you need to listen closely, and record accurately, so that what you share in your study is able to meaningfully shape knowledge, research and practice in your field.

Developing well-constructed data gathering tools, or methods, for your study

1 Comment

I spent the better part of last week working with emerging researchers who are at the stage of their PhD work where they are either working out what data they will need and how to get it, or sitting with all their data and working out how to make sense of it. So, we are talking theory, literature, methodology, analysis, meaning making, and also planning. In this post I want to focus on planning your data gathering phase, specifically developing ‘instruments’, such as questionnaires, interview schedules and so on.

tools

Whether your proposed study is quantitative, qualitative or mixed methods,  you will need some kind of data to base your thesis argument on. Examples may include data gathered from documents in the media, in archives, or from official sources; interviews and/or focus groups; statistical datasets; or surveys. Whatever data your research question tells you to generate, so as to find an answer, you need to think very carefully about how your theory and literature can be drawn into developing the instruments you will use to generate or gather this data.

In a lot of the postgraduate writing I have read and given feedback on, there are two main trends I have noticed in the development of research methods. The first is what I considered ‘too much theory’, and the other ‘not quite enough’. In the first instance, this is seen in researchers putting technical or conceptual terms into their interview questions, and actually asking the research questions in the survey form or interview schedule. For example: ‘Do you think that X political party believes in principle of non-racialism?’ Firstly, this was the overall research question, more or less. Secondly, this researcher wanted to interview students on campus, and needed to seriously think about whether this question would yield any useful data  – would her participants know what she meant by ‘the principle of non-racialism’ as she understood it theoretically, or even have the relevant contextual knowledge? Let’s unpack this a bit, before moving on to trend #2.

The first issue here is that you are not a reporter, you are a researcher. This means you are theorising and abstracting from your data to find an answer that has significance beyond your case study or examples. Your research questions are thus developed out of a deep engagement with relevant research and theory in your field that enables you to see both the ‘bigger picture’ as well as your specific piece of it. If you ask people to answer your research question, without a shared understanding of the technical/conceptual/theoretical terms and their meanings, you may well end up conflating their versions of these with your own, reporting on what they say as being a kind of ‘truth’, rather than trying to elicit, through theorising, valid, robust and substantiated answers to your research questions, using their input.

This connects to the second issue: it is your job to answer your research question, and it is your participants’ job to tell you what they know about relevant or related issues that reference your research question. For example, if you want to know what kinds of knowledge need to be part of an inclusive curriculum, you don’t ask this exact question in interviews with lecturers. Rather, you need to try and find out the answer by asking them to share their curriculum design process with you, talk you through how they decide what to include and exclude, ask them about their views on student learning, and university culture, and the role of the curriculum, and knowledge, in education. This rich data will give you far more with which to find an answer to that question than asking it right out could. You ask around your research questions, using theory and literature to help you devise sensible, accessible and research-relevant questions. This also goes for criteria for selecting and collating documents to research, should you be doing a study that does not involve people directly.

analysis of data

Photo by Startup Stock Photos from Pexels

The second trend is ‘not enough theory’. This tends to take the form of having theory that indicates a certain approach to generating data, yet not using or evidencing this theory in your research instruments.  For example structuralist theories would require you finding out what kinds of structures lie beneath the surface of everyday life and events, and also perhaps how they shape people, events and so on. An example of disconnected interview questions could be asking people whether they enjoy working in their university, and whether there are any issues they feel could be addressed and why, and what their ideal job conditions would be, etc., rather than using the theoretical insights to focus, for example, on how they experience doing research and teaching, and what kinds of support they get from their department, and what kinds of support they feel they need and where that does and should come from, etc. You need to come back to using the theory to make sense of your data, through analysis, so you need to ensure that you use the theory to help you create clear, unambiguous, focused questions that will get your participants, or documents, talking to you about what matters to your study. Disconnecting the research instruments from your theory, and from the point of the research, may lead to a frustrating analysis process where the data will be too ‘thin’ or off point to really enable a rich analysis.

Data gathering tools, or methods for getting the data you need to answer your research questions, is a crucial part of a postgraduate research study. Our data gives us a slice of the bigger research body we are connecting our study to, and enables us to say something about a larger phenomenon or set of meanings that can push collective knowledge forward, or challenge existing knowledge. This is where we make a significant part of our overall contribution to knowledge, so it is really important to see these instruments, or methods, not as technical or arbitrary requirements for some ethics committee. Rather, we need to conceptualise them as tools for putting our methodology into action, informed and guided by both the literature our study is situated within as well as what counts as our theoretical or principled knowledge. Taking the time to do this step well will ensure that your golden thread is more clearly pulled through the earlier sections of your argument, through your data and into your analysis and findings.

 

 

‘Retrofitting’ your PhD: when you get your data before your theory

2 Comments

I gave a workshop recently to two different groups of students at the same university on building a theoretical framework for a PhD. The two groups of students comprised scholars at very different points in their PhDs, some just starting to think about theory, some sitting with data and trying to get the theory to talk to the data, and others trying to rethink the theory after having analysed their data. One interesting question emerged: what if you have your data before you really have a theoretical framework in place? How do you build a theoretical framework in that case?

I started my PhD with theory, and spent a year working out what my ‘gaze’ was. I believed, and was told, that this was the best way to go about it: to get my gaze and then get my data. In my field, and with my study, this really seemed like the only way to progress. All I had starting out was my own anecdotal issues, problems and questions I wanted answers to, and I needed to try and understand not just what the rest of my field had already done to try and find answers, but what I could do to find my own answers. I needed to have a sense of what kinds of research were possible and what these might entail. I had no idea what data to generate or what to do with it, and could not have started there with my PhD. So I moved from reading the field, to reading the theory, to building an internal language of description, to generating data, to organising and analysing it using the theory to guide me, to reaching conclusions that spoke back to the theory and the field – a closed circle if you will. This seems, to me certainly, the most logical way to do a PhD.

But, I have colleagues and friends who haven’t necessarily followed this path. In their line of work, they have had opportunities to amass small mountains of data: interview transcripts, documents, observation field notes, student essays, exam transcripts and so forth. They have gathered and collected all of these data, and have then tried to find a PhD in the midst of all of it. They are, in other words, trying to ‘retrofit’ a PhD by looking to the data to suggest a question or questions and through these, a path towards a theoryology.

Many people start their doctoral study in my field – education studies – to find answers to very practical or practice-based questions. Like: ‘What kinds of teaching practice would better enable students to learn cumulatively?’ (a version of my own research question) Or: ‘What kinds of feedback practices better enable students to grow as writers in the Sciences?’ And so on. If you are working as a lecturer, facilitator, tutor, writing-respondent, staff advisor or similar, you may have many opportunities to generate or gather data: workshop inputs, feedback questionnaires, your own field notes and reports, student essays and exam submissions, and so on. After a while, you may look at this mountain of data and wonder: ‘Could there be a thesis in all of this? Maybe I need to start thinking about making some order and sense out of all of this’. You may then register for a PhD, searching for and finding a research question in your data, and then begin the process of retrofitting your PhD with substantive theory and a theoryology to help you work back again towards the data so as to tell its story in a coherent way that adds something to your field’s understanding or knowledge of the issues you are concerned with.

The question that emerged in these workshops was: ‘Can you create a theoretical framework if you have worked so far like this, and if so, how?’ I think the answer must be ‘yes’, but the how is the challenging thing. How do you ask your data the right kinds of questions? A good starting point might be to map out your data in some kind of order. Create mind-maps or visual pictures of what data you have and what interests you in that data. Do a basic thematic analysis – what keeps coming up or emerging for you that is a ‘conceptual itch’ or something you really feel you want or need to answer or explore further? Follow this ‘itch’ – can you formulate a question that could be honed into a research question? Once you have a basic research question, you can then move towards reading: what research is being or has been done on this one issue that you have pulled from your data? What methodologies and what theory are the authors doing this research using? What tools have they found helpful? Then, much as you would in a more ‘traditional’ way, you can begin to move from more substantive research and theory towards an ontological or more meta-theoretical level that will enable you to build a holding structure and fit lenses to your theory glasses, such that you have a way of looking at your data and questions that will enable you to see possible answers.

Then you can go back to your data, with a fresh pair of eyes using their theory glasses and re-look at your data, finding perhaps things you expect to see, but also hopefully being surprised and seeing new things that you missed or overlooked before you had the additional dimension or gaze offered by your theoretical or conceptual framing. But working in this ‘retrofitted’ way is potentially tricky: if you have been looking and looking at this data without a firm(ish) theoretically-informed or shaped gaze, can you be surprised by it? Can you approach your research with the curious, tentative ‘I don’t know the answers, but let’s explore this issue to find out’ kind of attitude that a PhD requires? I think, if you do decide to do or are doing a PhD in what I would regard as a middle-to-front sort of way, with data at the middle, then you need to be aware of your own already-established ideas of what is or isn’t ‘real’ or ‘true’, and your own biases informed by your own experience and immersion in your field and your data. You may need to work harder at pulling yourself back, so that you can look at your data afresh, and consider things you may be been blind to, or overlooked before; so that you can create a useful and illuminating conversation between your data and your theory that contributes something to your field.

Retrofitting a PhD is not impossible – there is usually more than one path to take in reaching a goal (especially if you are a social scientist!) – but I would posit that this way has challenges that need to be carefully considered, not least in terms of the extra time the PhD may take, and the additional need to create critical distance from data and ‘findings’ you may already be very attached to.

Fieldwork: to participate or not to participate…

Leave a comment

This is my last post about fieldwork. This final one is about observations, and whether and how to participate or not participate in what you are observing. In my case I was observing classroom teaching, but I think these comments could also apply to tutorials, meetings, workshops – any kind of encounter where there is an opportunity for you to be present, watching and taking notes, and in some cases also participating.

I have read a little bit about participant and non-participant observations, and the relative pros and cons of each. I chose non-participant observation, and in the spirit of this blog, I want to add my own voice on what it was actually like to sit in lectures for 14 weeks and not participate very much at all, what eventually tipped me from being totally quiet to venturing a little participation, and why I think non-participant observation can be a challenge.

I decided not to participate in the classes I was sitting in on for one big reason and a couple of smaller ones. The big reason was that I had majored in the one subject I was including in my study (Politics), and I have worked for 4 years with lecturers teaching the other particular course  (Law) so I have come to know a fair bit about the knowledge and I find it very interesting. I was worried, in short, that if I participated I would ask too many questions or make comments that would in some way silence the voices of legitimate students or perhaps lead to the lecturer and I engaging in a conversation or debate in class that might exclude students. I have been in higher education as a student and tutor for a long time and these students are by-and-large in their first year of study. I felt I had no right, really, to come into their classroom and take up their time with their lecturers. 

One smaller reasons were that I thought I would be able to capture more accurate and objective fieldnotes if I was not too involved in the course. The more you participate, I reasoned, the more you perhaps want to agree with the lecturer, or the less you want to make a note of things that could be negative or less flattering, so your fieldnote data can be skewed or incomplete. I think that this ties in with my first post on fieldwork, where I talked about the Trowler and Williams’ articles on doing research in your ‘backyard’ and the possibility of finding out knowledge that can put you as a researcher in a tricky position in relation to your participants or your university/organisation. I felt that participating might tip my own personal scales in a too-subjective direction. I can’t here go into a full conversation about whether research like mine can be called fully objective (suffice to say it can’t be because there is always some researcher bias in qualitative studies like mine), but I will say that I was trying to record, as verbatim and as faithfully as possible everything that went on in the lectures without trying to pre-judge or pre-organise my data into categories or decide what had to stay and what could go, and I felt that being too involved in the lectures would hinder and further bias this process.

Another smaller reason was a little more vain: I simply wanted to be invisible. I didn’t want to call attention to myself because  after all my years of studying and teaching, I still get palpitations when I have to speak up in class or ask a question in a meeting where people will look at me. So I liked the quietness of non-participant observation, even though I had introduced myself to the whole class at the start of lectures and they knew who I was and why I was there.

However, being that quiet, especially when I really had a question to ask or an answer to a question posed by the lecturer, was really difficult. At times my notes record this frustration: ‘I really want to join in the discussion. So hard not to comment’. I felt, especially in Politics, that I had some useful thoughts to share, but I resisted the urge to call out answers because I felt it was unfair. I did this course when I was an undergrad so answering would have felt a bit like cheating on a test. Right at the end of both courses, though, I gave up resisting and I asked a couple of questions in Law lectures, and at least raised my hand to vote on issues in Politics although I did not ask questions there. I was nervous about doing that, but the lecturers included me as a student and did not offer any special treatment which allayed at least my worries about taking over a student space.

This year, I am participating a little – as a very-semi-participant observer – in my post-doc data gathering. I am doing it partly to try out a different way of doing observations like this, and partly because I have learned that limited and careful participation does not necessarily lead to the issues I was concerned about, like skewing my data or distracting the lecturer or muscling in on students’ space. But I do think if you are going to be a participant observer you have to be careful and keep a record of your participation in your fieldnotes. You need at all times to be the researcher first and the participant second. You need to check with those you are observing if it is okay, and to what extent you could or should participate.

It is in many ways easier and less fraught to stay silent in the background and just watch and make notes, but participating can be more fun even if it brings possible complications with it. It’s up to the researcher considering the situations in which data are being gathered to decide what will work best. Be pragmatic, take careful notes and be open, and don’t forget to tell your readers why you made the decisions you did when you get to your methodology chapter!

Fieldwork: making and transcribing field notes

Leave a comment

This is the second post on fieldwork: this one is specifically about field notes – some thoughts on how to write them and how to transcribe them. I am still working this out, so it’s a thinking process in progress.

For my PhD, I gathered data largely from sitting in on lecturers’ classes and watching them teach, scribbling furiously during each one hour lecture. As you can imagine, over 14 weeks in two courses this ends up being a rather thick pile of notes. In my case it amounted to 5 and a half notebooks full of notes (over 500 A5 pages). These all had to be transcribed and organised so as to make sense of of them, and so that I could put them into NVivo10 to analyse them as part of my larger data set. Also, they needed to be typed up so I could copy and paste relevant pieces into my chapters as needed. I procrastinated a lot about doing the transcription. It’s not my favourite activity as a researcher. But I couldn’t have someone else do it because these notes were something I really needed to read several times, understand and sift through. I think, in hindsight and in agreement with Pat Thomson, that tedious as it is all researchers should try and do as much of their own transcription as possible because of how involved it enables you to become with your data. It makes the analysis process more enjoyable and productive too.

I thought what I would do in this post is list some of the things I did, why and what I learnt along the way in the hopes that you may find it useful if you, too, are gathering some of your data this way.

1. I handwrote my notes for two reasons: the first is that in the one class students were not allowed to use laptops because the lecturers wanted them paying attention as part of their training for the profession they will eventually enter, and the second is because I write faster than I type and writing is quieter. I tend to bash my keyboard a bit, and I did not want to distract other students or stand out too much by using equipment they were not allowed to use. A further plus with handwriting that I learned along the way was that I could easily copy diagrams lecturers put up on slides or drew on the boards, and I could represent what they were saying pictorially or non-linearly as well, which was often quicker and easier and made my notes feel more authentic to me.

IMG00075-20130612-0930

2. Pat Thomson wrote in a blog post a while back that she takes lots of field notes, and that she tries to capture verbatim what is going on as much as she can. I tried to do that too, and often was able to succeed in bits in one of the two courses because both lecturers talked fairly slowly and deliberately and paused for students to ask questions or take notes, so I could keep up. This was harder in the  other course where the lecturers spoke quite fast. I am having this issue now, too, in my post-doc research where one lecturer in particular is a very fast talker. However, I am much more comfortable with my theoretical framework now, so I can note particular kinds of phrases or instructions or comments that he says because I know I will use them in my analysis later on. However, I want to avoid being too selective in my hearing, because I don’t want to pre-empt the data or tell it what to tell me. I want to be surprised by it too, and find things I am not necessarily anticipating. Thus, I try now as I did last year to write as much as I can of what is going on in the moment, and can then sift the notes later and reorganise them during transcription.

3. I developed a shorthand: lecturers’ initials for the lecturers involved, like CA or BM (not their real initials). I also used S for student and tried to keep track of students’ questions or inputs where I could hear them clearly (the classes were large and often noisy). So I would have S1, S2 and so on engaging the lecturer in conversation or debate.

4. I didn’t transcribe everything. By being such a procrastinator about the transcription of the field notes, I ended up transcribing them while I was also beginning to analyse the data, so in a way this worked out well because it meant I had a sense of what I needed and what was just additional information that was unimportant, like comments made about admin issues, or comments I wrote about the lecturer telling the students off for not coming to tutorials the week before. Thus, I did not need to transcribe every word I wrote in my notes, and what I did was to read through my notes first, quickly, to remind myself of what I had written, and noted things I could excise. I learnt that it is okay to cut bits of your notes out and just not transcribe them. You won’t analyse ALL your data.

5. I had to do some reorganising when I transcribed my notes. Field notes are very in the moment – you are just trying to keep up and get it all down as faithfully and fully as possible, and you don’t really have time to think. When you go back to transcribe, though, you do have time and you can see how you can transcribe your sometimes chaotic and messy notes to impose a little more order, often needed in data analysis, and also how you can represent your pictures and scribbles in words so that you know what you mean, and can show readers what you mean if you use those examples in your chapters. I think that you need to be careful with reorganising, though, because you don’t want to rewrite history and make things that were chaotic seem simple, or things that were challenging seem easy. You would be skewing or tampering too much with your data and this would be unethical. It may also rob you of some potentially interesting findings. However, a little reorganisation that makes the notes easier to read and easier to represent to a reader, while staying true to the original scribbles, may sometimes be necessary.

I think the biggest thing I learned, and am still learning, is that it is an ongoing process of learning how to write these notes well, and how to collect rich and interesting data in ways that will be usable and make sense to me later. Stop every few notes and look back – reflect on what is working and what is not, and try to use that reflection to improve your taking of field notes. Capturing them can be tedious but field notes can also give you many-faceted and rich data for later probing and analysis.