Tag: working with theory and data

Developing well-constructed data gathering tools, or methods, for your study

1 Comment

I spent the better part of last week working with emerging researchers who are at the stage of their PhD work where they are either working out what data they will need and how to get it, or sitting with all their data and working out how to make sense of it. So, we are talking theory, literature, methodology, analysis, meaning making, and also planning. In this post I want to focus on planning your data gathering phase, specifically developing ‘instruments’, such as questionnaires, interview schedules and so on.

tools

Whether your proposed study is quantitative, qualitative or mixed methods,  you will need some kind of data to base your thesis argument on. Examples may include data gathered from documents in the media, in archives, or from official sources; interviews and/or focus groups; statistical datasets; or surveys. Whatever data your research question tells you to generate, so as to find an answer, you need to think very carefully about how your theory and literature can be drawn into developing the instruments you will use to generate or gather this data.

In a lot of the postgraduate writing I have read and given feedback on, there are two main trends I have noticed in the development of research methods. The first is what I considered ‘too much theory’, and the other ‘not quite enough’. In the first instance, this is seen in researchers putting technical or conceptual terms into their interview questions, and actually asking the research questions in the survey form or interview schedule. For example: ‘Do you think that X political party believes in principle of non-racialism?’ Firstly, this was the overall research question, more or less. Secondly, this researcher wanted to interview students on campus, and needed to seriously think about whether this question would yield any useful data  – would her participants know what she meant by ‘the principle of non-racialism’ as she understood it theoretically, or even have the relevant contextual knowledge? Let’s unpack this a bit, before moving on to trend #2.

The first issue here is that you are not a reporter, you are a researcher. This means you are theorising and abstracting from your data to find an answer that has significance beyond your case study or examples. Your research questions are thus developed out of a deep engagement with relevant research and theory in your field that enables you to see both the ‘bigger picture’ as well as your specific piece of it. If you ask people to answer your research question, without a shared understanding of the technical/conceptual/theoretical terms and their meanings, you may well end up conflating their versions of these with your own, reporting on what they say as being a kind of ‘truth’, rather than trying to elicit, through theorising, valid, robust and substantiated answers to your research questions, using their input.

This connects to the second issue: it is your job to answer your research question, and it is your participants’ job to tell you what they know about relevant or related issues that reference your research question. For example, if you want to know what kinds of knowledge need to be part of an inclusive curriculum, you don’t ask this exact question in interviews with lecturers. Rather, you need to try and find out the answer by asking them to share their curriculum design process with you, talk you through how they decide what to include and exclude, ask them about their views on student learning, and university culture, and the role of the curriculum, and knowledge, in education. This rich data will give you far more with which to find an answer to that question than asking it right out could. You ask around your research questions, using theory and literature to help you devise sensible, accessible and research-relevant questions. This also goes for criteria for selecting and collating documents to research, should you be doing a study that does not involve people directly.

analysis of data

Photo by Startup Stock Photos from Pexels

The second trend is ‘not enough theory’. This tends to take the form of having theory that indicates a certain approach to generating data, yet not using or evidencing this theory in your research instruments.  For example structuralist theories would require you finding out what kinds of structures lie beneath the surface of everyday life and events, and also perhaps how they shape people, events and so on. An example of disconnected interview questions could be asking people whether they enjoy working in their university, and whether there are any issues they feel could be addressed and why, and what their ideal job conditions would be, etc., rather than using the theoretical insights to focus, for example, on how they experience doing research and teaching, and what kinds of support they get from their department, and what kinds of support they feel they need and where that does and should come from, etc. You need to come back to using the theory to make sense of your data, through analysis, so you need to ensure that you use the theory to help you create clear, unambiguous, focused questions that will get your participants, or documents, talking to you about what matters to your study. Disconnecting the research instruments from your theory, and from the point of the research, may lead to a frustrating analysis process where the data will be too ‘thin’ or off point to really enable a rich analysis.

Data gathering tools, or methods for getting the data you need to answer your research questions, is a crucial part of a postgraduate research study. Our data gives us a slice of the bigger research body we are connecting our study to, and enables us to say something about a larger phenomenon or set of meanings that can push collective knowledge forward, or challenge existing knowledge. This is where we make a significant part of our overall contribution to knowledge, so it is really important to see these instruments, or methods, not as technical or arbitrary requirements for some ethics committee. Rather, we need to conceptualise them as tools for putting our methodology into action, informed and guided by both the literature our study is situated within as well as what counts as our theoretical or principled knowledge. Taking the time to do this step well will ensure that your golden thread is more clearly pulled through the earlier sections of your argument, through your data and into your analysis and findings.

 

 

Why is theory such a big deal in postgraduate research?

1 Comment

I am working with a new student. Long story short, I am not his first supervisor, and this his not his first attempt at his PG research project. He’s had a tough time thus far: significantly with theory as his first supervisor did not seem to feel he needed any. Quite understandably, then, one of his first questions to me was ‘why are we making such a big deal about theory [when my research is narrative]?’ In answering this question, I have been pondering a bit more about why theory is such a big deal in research, especially at PG level.

The best way to begin is with an overview of what postgraduate research (any academic research perhaps) is for: to make a novel, valuable and needed contribution to knowledge in your field or study and/or practice. Often, particularly in the social sciences, we are taking a known problem and trying to solve it with a new approach, or we are critiquing the work of others from a particular perspective to extend knowledge further, or we are introducing a new problem, solvable with established approaches in ways that extend or consolidate knowledge and practice. To achieve this contribution to knowledge, we focus on small slice of the known world – our data – and we analyse this in ways that connect our findings to broader understandings/knowledge/phenomena so that what we are contributing clearly fits within the bigger picture in our field. 

If this, then, is basically why we do research, then how do we actually achieve this goal of saying something new and fitting the new into the established knowledge in our field? This is, in many instances, where theory really does its best work.

leaving star trek GIF-downsized_large

When we do academic research, any research, we are trying to find an answer to a question that needs one. We start with a research problem, and we read around that, becoming increasingly focused until we have read enough to locate a gap in the field that we can contribute to filling with our research. We then narrow down a research question, the answers to which will fill (part of) this gap. At this point, we have a sense of what data we are going to generate and how (research design and method) and we may even (from reading) have a basic sense of what we may find. But, what we need is a framework within with to understand what we may find, and tools to use to make meaning from this data. We need to ensure that we move beyond purely descriptive meanings, even in descriptive studies. If all we are doing is describing or narrating our small slice of the world, it may be interesting, but perhaps only to a tiny group of potential readers who understand the specifics well enough to extract meanings of their own. This falls short of the kinds of contribution to knowledge expected of postgraduate scholars and publishing academics.

The potentially frustrating and difficult issue of finding the right framework for your research is that you can’t really ‘find’ one and just put it into your project, where it will do its own thing. Doing this would be akin to writing a ‘theory’ chapter or section, and then doing nothing with that theory in the analysis to connect your study to the field. Rather, you have to build and use your theoretical framework to make sense of your study, and its contribution to the field. This means you need to find theory that fits with your research problem and questions, that can help you understand this problem in helpful ways. Then, you need to select the relevant parts of the whole theory (you don’t necessarily, for example, need to include everything Pierre Bourdieu ever wrote in your thesis if all you really need to focus on is the interplay between capital and habitus in the structuring of a field). This selected theory then needs to be explained, exemplified in relation to your study, and connected into a coherent structure, or framework. 

scrabble mess

Once you have what Bernstein called the ‘internal language of description’ for your study – your study’s own account of the theory it will be using and why this theory is the most appropriate choice for this study – you can generate, or analyse generated, data. This is where theory becomes the big deal that it is. Theory is transformed when it is brought into contact with data. It stops being quite so abstract, and becomes more alive and real. It actually helps you to say something about why you see what you do in your data, and what the things you see actually could mean, connected to the larger picture. It helps you create an ‘external’ language of description – a translation device as Maton puts it – which transforms theory in the abstract into an analytical language that can describe and make meaning of data. Other researchers can draw on, adapt, and add to this in their own studies, further amplifying the value of your research.

For example, several students have told you that no one will assist them with supervisor issues. rather than saying that this is just an unsupportive environment, you can use theory that gives you insight into power and university cultures around autonomy. With this insight, you could postulate that the environment is structured so as to give administrators and supervisors way more power than students, and with that power they can maintain an unsupportive status quo. Perhaps this unsupportive environment is created and maintained with the (misguided) notion that students need to be autonomous and independent, but you can now critique this with your data and theory to show why this doesn’t actually work. And you could back up this postulation with reference to other studies that have made similar or related arguments.

Instead of just a small story about your data, and why you think it is interesting, you now have a potentially powerful analysis of the data that says what is means, why this meaning is important to pay attention to, and how this meaning connects with other meanings, thus making a contribution to research in your field.

research-2

Theory isn’t just an odd requirement that has to be met in postgraduate research. It also is not some sort of relic of an ‘elitist’ version of higher education (one criticism I have heard a few times now). It’s a tool: it helps us really say something important and valuable about the world around us. We need to be doing research that connects us to other people, other research, other meanings, so that all of these meanings and arguments can build on one another cumulatively, amplifying our findings and voices. If what we want is better understanding of problems, new solutions to old problems and powerful change, then we need to harness the power theory offers us as researchers and use it to help us achieve these goals.

Get your ‘gaze’ then get your data

6 Comments

As promised, a post on something a little less personal and a little more ‘academic’. The topic for this week’s post is theoretical frameworks, and why it’s a very good idea to have one fairly clearly in place before you start collecting and analysing your data.

Right. When I started my PhD in 2010, I spent the whole of that year reading about academic literacies and writing centres, thinking that I wanted to do my PhD on a topic related to the role writing centres play in developing students’ academic literacies in the disciplines. I just couldn’t get into it, though. I had a whole lot of substantive theory, but nothing that looked or felt like a framework for a study that would guide my data gathering or my analysis. After a long and helpful meeting with my supervisor at the end of that year, I found my way to Basil Bernstein’s work and from there to Pierre Bourdieu’s and to Legitimation Code Theory. I had, much to my joy, found my framework.  I had the beginnings of a ‘gaze’ to use Bernstein’s term, or a set of specific lenses with which to scratch at my ‘conceptual itch’ to use Lis Lange’s term (well, it was her I first heard it from) – the thing that I really needed to find out or the question I really wanted an answer to. I could now rethink my topic, and begin to form this gaze so that I would know what data I needed to collect and what I should do with it to get at the answers and scratch the itch.

So, I spent a lot of time in 2011 and 2012 writing and rewriting what I suppose some might call a literature review, but which my supervisor and I called a conceptual or theoretical framework. I had to lay very firm foundations for this study because the theory I was reading and the concepts I used were so new to me, and I really was starting from scratch in many ways in this field. I often felt frustrated at how much time I was spending on this one chapter out of six that had to be written, and I felt lost a great deal of the time in the mess of reading and thinking and connecting of dots. It was a hard couple of years, but in that time I got my proposal through and wrote a pretty good draft of chapter two. I found myself, at the end of 2012, very nervous about leaving Theoryland, as I thought of it, up on its pretty, fluffy clouds. I felt safe there, because I had pretty much worked out what Bernstein called the ‘internal language of description’ for my study. In other words, I could tell you how the different parts of the theory and concepts I chose to use cohered, and why they fitted together like that and what sense to make of them in the context of my study. But that was all I could do at this point. So I had to move on.

I was encouraged (read shoved) off my clouds and into the mires and muck of field work and data gathering. And this is where things started to get really exciting for me (and then boring later and then exciting again). I observed lecturers teaching and read their course documents, as well as conducted in-depth interviews. In the first week of lectures, I found I could see and hear the theory – that internal language – being spoken in the teaching and coming to life in front of me. It was so exciting and also so affirming. This was the right data to be gathering and my study was moving in the right direction, given my focus and my questions. I do not think that this would have happened quite so clearly had I tried to shortcut the first stage of laying those foundations and working out, as carefully and fully as I could, that internal language of description. I needed to have my gaze in place, as nebulous and fuzzy as it felt, before I gathered by data and started to think about what to do with it in analysis.

I won’t be glib and say that the data gathering was easy – it was often really tedious and the semester felt so long; recording detailed fieldnotes in 8 lectures per week of an hour long each for the better part of 14 weeks was tough going for me. But at no point did I really doubt that I was gathering the right data for my study, and even when things got very fuzzy and I couldn’t hear the theory so clearly or see my way, I could count on the fact that my gaze was there and that it would deepen and develop as I really started to get into my data more systematically. It was very reassuring on the whole, and while it didn’t make the field work stage a breeze, it did take some of the anxiety and doubt away, and kept me moving towards the next step.

I would, based on my experience, argue that even if you feel frustrated and feel like you are taking way too long working out your own internal language of description, it is worth doing as well as you can. The firmer the foundation going forward, the less the likelihood (I think) of having to rush back and get different or more data because you don’t have enough or the right kinds of data. It reduces the stress and anxiety that inevitably come with gathering data and starting to think about analysing it, because having your gaze in place can assure you that you are moving in the right kind of direction. However, a word of caution: it can be really lovely and safe and warm up in Theoryland, but staying there will never get your PhD finished. You will need to find the bridge down to the Data Swamps so that you can move between the two as you scratch your own itch and answer the questions that drive you onwards.

How can I get PhD if I still don’t understand the theory?

2 Comments

So I am revising chapter two, the ‘theory’ chapter, where I explain to my readers what ‘lenses’ I will be casting on my data are and why I need them. So, yesterday I got to the last section of the chapter which is too short and not detailed enough because it’s using conceptual tools that I need but that were only written about in draft form when I wrote the chapter last year. I decided I needed to do some more reading before I could finish it, and the drafts I was reading then have been updated and published in a book, and other papers applying these concepts have also been recently published, which is great. This morning I read two of the chapters. The first chapter I read was helpful – I have been misusing a concept slightly but can see how I can correct it quite simply. So, useful. The next chapter, introducing the next conceptual tool, induced a freak-out of large proportions. I don’t understand what the author is talking about in this chapter beyond explaining the concept I am using. There are so many big words and complex terms that it made me feel a bit dizzy. I actually stopped reading halfway through, put the book down and went to fold the laundry. I told lovely husband rather petulantly and with not a small amount of panic that this PhD is never going to be finished and I am giving up now because I cannot possibly be awarded this degree when I don’t even understand the theory.

Of course, now that lovely husband has very patiently talked me off the ledge, and worked through my misunderstandings and panic with me, I can see that I do understand the theory I am actually using and need, and that the bits that are freaking me out may not be necessary at this late stage of the game. Just because the theory is there does not mean I have to use it all. But I have to confess I am a bit lost in this chapter. I need to add these missing details and pieces that I can now read about, but I feel like I have way too much ‘theory’ and I worry that I actually don’t really understand it all; that my examiners and readers will see that and I will be found out as someone who only sort of knows what she is talking about. I would like to actually know what I am talking about at the end of all this hard work.

The theory was clearer in my head before I gathered and analysed the data. It was lovely and abstract and it made sense. Then I gathered data. I organised it and coded it and reorganised it and analysed it and started writing about it. And I could ‘see’ the theory but the data has also changed it. It’s not just abstract anymore, it’s applied now. The data is speaking back to the theory, challenging it and changing it. This is great, because it means I can actually make a contribution to knowledge in my field. I can add to the research others are doing and I can say something of value. But man, it’s hard work. Hard thinking work. Hard writing work. Looking at the two data chapters again makes me feel like I don’t understand the theory the way I thought I did. It makes me doubt myself, and I feel again that anxiety that I am getting this all wrong and that my examiners will be scathing in their critique and I will have so much more work to do when their reports come back next year. It’s a horrible, and sadly familiar feeling. On the good days when the writing goes well and the ideas seem strong and linked to the theory, I feel this will indeed be a good thesis and it will be finished by December. Today was not one of these days. I feel like I have lost the theory, lost my grip on it, and it does not make full sense to me. I don’t want to read anymore, but I also don’t want to have an incomplete chapter, or write a thesis that looks good on the surface but is theoretically or analytically shallow and weak.

But I need some perspective so I am not going to read anymore tonight. I will start again tomorrow with the published papers that report on have empirical research because these are easier to make sense of. I will finish these revisions, so that I can move onto the next set of revisions. I will remember that this is not my life’s work. It is a project, a thesis, a very big exam, and I am using this project to show my examiners that I can do the things that will mark me as having met the requirements set for me, mark me as being a researcher and scholar of a more experienced and more able kind. I will keep breathing, and writing and thinking and remind myself that I do understand the theory, really. Today was just a tough day.