Developing well-constructed data gathering tools, or methods, for your study

I spent the better part of last week working with emerging researchers who are at the stage of their PhD work where they are either working out what data they will need and how to get it, or sitting with all their data and working out how to make sense of it. So, we are talking theory, literature, methodology, analysis, meaning making, and also planning. In this post I want to focus on planning your data gathering phase, specifically developing ‘instruments’, such as questionnaires, interview schedules and so on.

tools

Whether your proposed study is quantitative, qualitative or mixed methods,  you will need some kind of data to base your thesis argument on. Examples may include data gathered from documents in the media, in archives, or from official sources; interviews and/or focus groups; statistical datasets; or surveys. Whatever data your research question tells you to generate, so as to find an answer, you need to think very carefully about how your theory and literature can be drawn into developing the instruments you will use to generate or gather this data.

In a lot of the postgraduate writing I have read and given feedback on, there are two main trends I have noticed in the development of research methods. The first is what I considered ‘too much theory’, and the other ‘not quite enough’. In the first instance, this is seen in researchers putting technical or conceptual terms into their interview questions, and actually asking the research questions in the survey form or interview schedule. For example: ‘Do you think that X political party believes in principle of non-racialism?’ Firstly, this was the overall research question, more or less. Secondly, this researcher wanted to interview students on campus, and needed to seriously think about whether this question would yield any useful data  – would her participants know what she meant by ‘the principle of non-racialism’ as she understood it theoretically, or even have the relevant contextual knowledge? Let’s unpack this a bit, before moving on to trend #2.

The first issue here is that you are not a reporter, you are a researcher. This means you are theorising and abstracting from your data to find an answer that has significance beyond your case study or examples. Your research questions are thus developed out of a deep engagement with relevant research and theory in your field that enables you to see both the ‘bigger picture’ as well as your specific piece of it. If you ask people to answer your research question, without a shared understanding of the technical/conceptual/theoretical terms and their meanings, you may well end up conflating their versions of these with your own, reporting on what they say as being a kind of ‘truth’, rather than trying to elicit, through theorising, valid, robust and substantiated answers to your research questions, using their input.

This connects to the second issue: it is your job to answer your research question, and it is your participants’ job to tell you what they know about relevant or related issues that reference your research question. For example, if you want to know what kinds of knowledge need to be part of an inclusive curriculum, you don’t ask this exact question in interviews with lecturers. Rather, you need to try and find out the answer by asking them to share their curriculum design process with you, talk you through how they decide what to include and exclude, ask them about their views on student learning, and university culture, and the role of the curriculum, and knowledge, in education. This rich data will give you far more with which to find an answer to that question than asking it right out could. You ask around your research questions, using theory and literature to help you devise sensible, accessible and research-relevant questions. This also goes for criteria for selecting and collating documents to research, should you be doing a study that does not involve people directly.

analysis of data

Photo by Startup Stock Photos from Pexels

The second trend is ‘not enough theory’. This tends to take the form of having theory that indicates a certain approach to generating data, yet not using or evidencing this theory in your research instruments.  For example structuralist theories would require you finding out what kinds of structures lie beneath the surface of everyday life and events, and also perhaps how they shape people, events and so on. An example of disconnected interview questions could be asking people whether they enjoy working in their university, and whether there are any issues they feel could be addressed and why, and what their ideal job conditions would be, etc., rather than using the theoretical insights to focus, for example, on how they experience doing research and teaching, and what kinds of support they get from their department, and what kinds of support they feel they need and where that does and should come from, etc. You need to come back to using the theory to make sense of your data, through analysis, so you need to ensure that you use the theory to help you create clear, unambiguous, focused questions that will get your participants, or documents, talking to you about what matters to your study. Disconnecting the research instruments from your theory, and from the point of the research, may lead to a frustrating analysis process where the data will be too ‘thin’ or off point to really enable a rich analysis.

Data gathering tools, or methods for getting the data you need to answer your research questions, is a crucial part of a postgraduate research study. Our data gives us a slice of the bigger research body we are connecting our study to, and enables us to say something about a larger phenomenon or set of meanings that can push collective knowledge forward, or challenge existing knowledge. This is where we make a significant part of our overall contribution to knowledge, so it is really important to see these instruments, or methods, not as technical or arbitrary requirements for some ethics committee. Rather, we need to conceptualise them as tools for putting our methodology into action, informed and guided by both the literature our study is situated within as well as what counts as our theoretical or principled knowledge. Taking the time to do this step well will ensure that your golden thread is more clearly pulled through the earlier sections of your argument, through your data and into your analysis and findings.

 

 

PhD workout: warming up your writing muscles

So, I am writing a book. I have been sort-of-kind-of writing a book for a long time now. We have an on and off relationship, my book and I. But, a proposal is being reviewed, and the hope is that the feedback will be a green light, so I have to get writing. And soon. But, I am a bit out of practice. I wrote a fair bit last year – 3 book chapters (a few drafts each) as well as part of a paper with colleagues. But this is a different beast altogether – as long and as complex as a PhD thesis. I am finding I am out of shape here.

This is not an unfamiliar feeling. I wrote here and here about moving from one year of PhD or post-doc into the next, after having a break and getting a bit flabby around the writing middle, so to speak. I know, therefore, that I have felt unfit before, and have made myself fitter and gotten the writing work done. But, this is – like actual fitness – hard work and requires a level of emotional and psychic energy that can be hard to find sometimes. I have decided, therefore, that I am going to start with gentle warm-ups, rather than jumping straight into the whole thing (Thank you, Roger Federer :-)).

rfed warm up

The first thing I am doing is starting with something manageable, that I could want to do every day – or at least 4-5 times a week. If I want to do it, and it feels manageable, it is very likely I will actually do it (and enjoy the experience). Instead of doing what I too often do, and writing ‘Chapter 1 draft’ on one day of my calendar, I am writing ‘one pomodoro’ every other day. I can do this. It’s 30 minutes of writing. I can then tick this off, and actually add days as a I go, or keep it every other day and work up to 2 pomodoros at least. If I can do it, I won’t fail, and if I don’t fail, I can keep enjoying this writing time and make it productive. Too often I set myself overly lofty goals, in life and writing, and set myself up to fail rather than succeed. Last week I wrote my first blog post in over 4 months, scheduled this post, and also managed about 1000 words on my book. HUGE success I say. All in these little manageable chunks.

The second thing I am going to do is keep it steady. Rather than having a good week, and thinking I can now escalate to high levels of writing productivity, I am going to keep going at this pace for now. Probably, realistically, this will be the pace for the year, with bursts of higher productivity around deadlines and when I have excess time and energy. As one of my writing students said to me last year: ‘Eat the elephant one bite at a time’. Apologies to elephant lovers – I am one too – but this is a good metaphor for taking it steady with life and writing. One task, one pomodoro, one idea at a time. This way, things actually do get done as opposed to being menacing, un-ticked-off tasks on your to-do list.

Finally, for now anyway, I am going to get me some writing buddies. Face-to-face if I can, but virtually if not. I am always thinking I should join a Twitter shut-up-and-write group, or create my own writing group. And then work, and kids, and life, and my writing gets pushed down (with me attached) to the bottom of my list. My writing time is also time for me – it’s personal as well as professional. So, I have to actually value it, and myself. As a working mother I am too often too far down my list. And so is my writing. I am hopeful, that with positive peer encouragement, we can collectively make our writing more present each week in the to-do lists, and make appreciable progress on our projects.

group yoga

Warming up these tired writing muscles to fuller strength will take some time – what do people say?If it’s too easy you’re not doing it right? Maybe so. I don’t think writing should always be hard, but good writing should take effort and time. Maybe you are in this spot too, coming back to work and PhD and research writing, and working out how to begin your “elephant meal”. Hopefully some of these steps to warming up your writing muscles will help you, too.

If you have other ideas, please share in the comments. All the best for 2019!

Having theory and ‘theorising’: reclaiming the verb

Jeepers, I have been away a while! But, rather than dwelling on all the Things I let get in the way of my writing and creativity last year, I will look to a new year, and new ideas for the blog instead. In a conference workshop in November, I had a really interesting discussion with colleagues about theory – where theory comes from, and whether its origins are relevant to its current use (for a later post). Part of that discussion focused on how we use theory. It is this I would like to focus on in this post, to kick off the new year.

Theory is key part of research – particularly at postgraduate and doctoral level. In the social sciences we tend to really spell out the theory we use – critical social theory, behavioural theory, post-structural theory and so on, whereas is the natural sciences there is less obvious mention of theories although they are most certainly there (I am told by scientist friends it is usually some version of positivism, but I stand to be corrected). In essence, theory is always present when we are trying to make sense of a smaller part of the bigger picture – connecting the specifics of our study with a more general or wider phenomenon. But, the way we use theory to do this is often inadequately talked about or grappled with in postgraduate spaces. Do we ‘have theory’ or do we ‘theorise’? 

question mark

Patrick White, in an excellent book on research writing, argues that to be called by such a name, theories need to be abstract, explanatory and testable. In other words, they need to be able to apply across more than one field; they need to explain, rather than just name, the part of the world we are researching; and they need to be able to be tested, to see whether and how they apply to the problem we are researching. But, in many projects, especially those where the theory needs to shape choices made in the research design, methods, and analysis of data, theory can be under-utilised in the act of making meaning and creating new knowledge. In other words, I am arguing that there are projects that ‘have theory’, but this is not adequately used in the act of ‘theorising’ new understandings and explanations of the problems we research. 

So what, then, do I mean in my distinction between ‘having theory’ and ‘theorising’?

Let’s start with theorising. This is an act – something we have to do, usually in an iterative manner. This is the act of bringing theory and data into conversation with one another, in the process seeking meanings that will help us both explain the nature of the problem we are researching, and test the viability of the theory we have chosen in making sense of that problem. We are also locating the problem, through theory, literature and methodology, within the bigger picture it connects with. For example, using critical social theory, such as Margaret Archer’s morphogenetic cycles to critique and understand processes of change or stasis in one university’s leadership environment (understood within the broader context of neoliberalism and marketisation of higher education). To theorise, we need to make the theory do work. The work of making sense of the world – through a chosen and always partial lens – but making sense nonetheless.

When we use theory to theorise – to seek, debate and construct meanings so as to create and add to knowledge about the part of the world we are researching – we need to acknowledge that the theory we have chosen is but one of probably a number of theories that could offer an explanation. Research is about looking for truth – a form of truth that enables us to see, know and act with greater knowledge and insight. Thus, the theory we choose, following White, must be able to offer the kinds of explanations that enable this. Your research question – the problem you want to solve – must guide the choices you make here: what theoretical tools, frameworks, explanations will enable you to solve this problem in the clearest, most useful way at this point in time? We choose, and build, theoretical frameworks, or theoryologies as I have called them, based on the problems we are trying to solve, or the truths we seek. They are not just found, fully formed, ready to be placed into the right part of the thesis. 

constructing theory

Photo by David McBee from Pexels

This brings me to dissertation work that has theory, but does not fully theorise. There may be a chapter entitled ‘Theoretical framework’ or some version of that, and the theory is laid out and explained. (In the sciences the theory may be implied in the earlier parts of the thesis rather than explained in its own chapter.) Yet, when the analysis and data sections are reached, the theory is oddly silent, or under-utilised. Rather, you may find a thematically organised discussion of the data that recounts what the researcher has found, and makes suppositions and suggestions of what it could mean without fully engaging the powerful theory to really make meanings that can, following White again, show that the theory has been tested, and is able to explain the data in ways that enable the researcher to create knowledge that adds to what we already know. These findings can then be built on by other researchers as part of the abstracted explanations they may offer for why different parts of the truth, and new solutions to current problems still need to be sought, and found. 

Theory is powerful. Or, it can be when used to actively theorise – to make meanings that are new, or additional to those we already have access to. This process is not simple, or linear. It requires immersion in your data, it requires you to suspend some of your assumptions or beliefs about what is truth, and what your data is saying, so you can allow the theory to act as a lens with which to look at your data with more ‘naïve’ or unassuming eyes. Theorising is an iterative, at times frustrating process, that is both intellectually and personally challenging. But, skimping this process to get the thesis done belies a misunderstanding of what the doctorate – or research – is. It is not (just) the thesis, or the paper. It is the process of engaging with current truths and meanings, findings under-explored problems and questions, and working to make meanings that will add to our knowledge about the world, and how we live in it (and that will transform your own understandings and knowledge).

In these uncertain, and challenging times in which we all live, we all need to embrace the difficult act of theorising – we need to reclaim the verb, over the noun. We need to embrace the research process and the learning therein – both personal and professional. The product that emerges in the form of that thesis or paper – and its author and readers – will be the richer for it.