I spent the better part of last week working with emerging researchers who are at the stage of their PhD work where they are either working out what data they will need and how to get it, or sitting with all their data and working out how to make sense of it. So, we are talking theory, literature, methodology, analysis, meaning making, and also planning. In this post I want to focus on planning your data gathering phase, specifically developing ‘instruments’, such as questionnaires, interview schedules and so on.
Whether your proposed study is quantitative, qualitative or mixed methods, you will need some kind of data to base your thesis argument on. Examples may include data gathered from documents in the media, in archives, or from official sources; interviews and/or focus groups; statistical datasets; or surveys. Whatever data your research question tells you to generate, so as to find an answer, you need to think very carefully about how your theory and literature can be drawn into developing the instruments you will use to generate or gather this data.
In a lot of the postgraduate writing I have read and given feedback on, there are two main trends I have noticed in the development of research methods. The first is what I considered ‘too much theory’, and the other ‘not quite enough’. In the first instance, this is seen in researchers putting technical or conceptual terms into their interview questions, and actually asking the research questions in the survey form or interview schedule. For example: ‘Do you think that X political party believes in principle of non-racialism?’ Firstly, this was the overall research question, more or less. Secondly, this researcher wanted to interview students on campus, and needed to seriously think about whether this question would yield any useful data – would her participants know what she meant by ‘the principle of non-racialism’ as she understood it theoretically, or even have the relevant contextual knowledge? Let’s unpack this a bit, before moving on to trend #2.
The first issue here is that you are not a reporter, you are a researcher. This means you are theorising and abstracting from your data to find an answer that has significance beyond your case study or examples. Your research questions are thus developed out of a deep engagement with relevant research and theory in your field that enables you to see both the ‘bigger picture’ as well as your specific piece of it. If you ask people to answer your research question, without a shared understanding of the technical/conceptual/theoretical terms and their meanings, you may well end up conflating their versions of these with your own, reporting on what they say as being a kind of ‘truth’, rather than trying to elicit, through theorising, valid, robust and substantiated answers to your research questions, using their input.
This connects to the second issue: it is your job to answer your research question, and it is your participants’ job to tell you what they know about relevant or related issues that reference your research question. For example, if you want to know what kinds of knowledge need to be part of an inclusive curriculum, you don’t ask this exact question in interviews with lecturers. Rather, you need to try and find out the answer by asking them to share their curriculum design process with you, talk you through how they decide what to include and exclude, ask them about their views on student learning, and university culture, and the role of the curriculum, and knowledge, in education. This rich data will give you far more with which to find an answer to that question than asking it right out could. You ask around your research questions, using theory and literature to help you devise sensible, accessible and research-relevant questions. This also goes for criteria for selecting and collating documents to research, should you be doing a study that does not involve people directly.
The second trend is ‘not enough theory’. This tends to take the form of having theory that indicates a certain approach to generating data, yet not using or evidencing this theory in your research instruments. For example structuralist theories would require you finding out what kinds of structures lie beneath the surface of everyday life and events, and also perhaps how they shape people, events and so on. An example of disconnected interview questions could be asking people whether they enjoy working in their university, and whether there are any issues they feel could be addressed and why, and what their ideal job conditions would be, etc., rather than using the theoretical insights to focus, for example, on how they experience doing research and teaching, and what kinds of support they get from their department, and what kinds of support they feel they need and where that does and should come from, etc. You need to come back to using the theory to make sense of your data, through analysis, so you need to ensure that you use the theory to help you create clear, unambiguous, focused questions that will get your participants, or documents, talking to you about what matters to your study. Disconnecting the research instruments from your theory, and from the point of the research, may lead to a frustrating analysis process where the data will be too ‘thin’ or off point to really enable a rich analysis.
Data gathering tools, or methods for getting the data you need to answer your research questions, is a crucial part of a postgraduate research study. Our data gives us a slice of the bigger research body we are connecting our study to, and enables us to say something about a larger phenomenon or set of meanings that can push collective knowledge forward, or challenge existing knowledge. This is where we make a significant part of our overall contribution to knowledge, so it is really important to see these instruments, or methods, not as technical or arbitrary requirements for some ethics committee. Rather, we need to conceptualise them as tools for putting our methodology into action, informed and guided by both the literature our study is situated within as well as what counts as our theoretical or principled knowledge. Taking the time to do this step well will ensure that your golden thread is more clearly pulled through the earlier sections of your argument, through your data and into your analysis and findings.