Putting your theory to work in analysis

You now have generated data – in some form, whether primary or secondary – and now you need to code and make sense of it; you need to put it to the task of answering your research question(s). In other words: analysis. This was the toughest part of my own PhD: I had a mountain of data – how to choose the right pieces? What to say about them? How to make sense of them in relation to my research questions?

This is where theory and concepts come into their own in a PhD or MA. You will have some form of theoretical or conceptual framework (for clarity on theory and concepts, how they differ and work together, please watch this short video). Where students often go off track, though, is not using these concepts or theory to do the work in analysis. The theoretical or conceptual framework ends up standing alone, and some form of thematic description of the data is made, with a rather thin version of analysis. In this situation, it may be difficult to offer a credible answer to your research question.

Analysis is, in essence, an act of sense-making. It requires you to move beyond a common sense, everyday understanding of the world, and your data – the level of the descriptive – to a theorised, non-common sense understanding – the level of the analytical (and critical). Analysis means connecting the specific (your study and its data) with the general (a phenomenon, theory, concept, way of looking at the world) that can help to explain how the specific fits in with, or challenges, or exemplifies the general. If you do not make this move, all you may end up with is a set of data that describe a tiny piece of the world, but with little or no relevance to anyone else’s research except perhaps the few other people researching the same thing you are.

theory specs 2

So, how might you ‘do’ analysis?

Imagine you are doing a study on the role of reflective learning in building students’ capacity to critique and create professional knowledge that encourages ongoing learning and problem-solving. ‘Reflection’, or ‘reflective practice’ would be a key concept, as would ‘professional knowledge’, ‘problem-solving’, and ‘learning’. These have generalised, or conceptual meanings that could apply in a range of ways, depending of the parameters and questions of a specific study. Thus, they can do analytical work, helping you to theorise as you answer your research questions.

Then imagine your data set is assessment tasks completed by students in social work and accounting, as two professional disciplines which require adaptive, ongoing learning and problem-solving. You now need a way of employing your key concepts in analysis. You could look at the intentions of the task questions – how they do, or do not, explicitly or actively enable or encourage problem-solving and reflective thinking and learning, and then look at students’ responses and see the extent to which the desired forms of learning are visible or not. This would yield useful findings to feed back to these disciplines in using assessment more effectively.

To reach theorised findings that go beyond describing what the tasks and the student writing said, and conjecture about what the tasks and written responses mean in relation to your study’s understanding of professional knowledge, learning, problem-solving and reflection, you need to start with questions.

theory giphy

For example: these tasks seem to be using direction words such as ‘name’, ‘list’, ‘describe’, ‘mention’, which require mainly memorising, or learning the notes in a rote manner. What kind of learning would this encourage? What impact would this have on students’ ability to move on to more analytical tasks? Is there a progression from ‘memorisation’ towards ‘problem-solving’ or using knowledge to reflect on and learn from case studies etc? What kind of progression is there? Is it sensible, or not, and how could this affect students learning? And so on.

You could then present the data: e.g., this is the task, and this is when students work on this task in the semester or progression of the course, and this is the task that follows (show us what these look like by copying them out, or including photographs). This part of the analysis is quite descriptive. But then you pose and answer relevant questions guided by your overall research objectives: if these two disciplines – social work and accounting – require professional learning and knowledge that is built through reflection, and the capacity to USE rather than just KNOW the knowledge in the field so that professional can adapt, continue learning, and solve complex problems, what kinds of assessment tasks are needed in higher education? Do the tasks students are doing in the courses I am studying here do these kinds of tasks? If yes, how are they working to build the rights kinds of knowledge, skills and aptitudes? If no, what might be the outcome for these students when they graduate and move into the professions? You then have to use the concepts you have pulled together to create a theorised understanding of professional reflective learning to pose credible answers, that are substantiated with your data (as evidence). This is the act of analysis.

analysis

In both qualitative and quantitative studies, the theory or concepts you choose, and the data you generate, are informed by your research aims and objectives. And in both kinds of studies, analysis requires moving beyond description to say something useful about what your data means in relation to the general phenomenon you are connecting with, and that informs your theorisation (student learning, climate change, democratic governance, etc). Thus, you need to work – iteratively and in incremental stages – to bring your theory to your data, to make sense of the data in relation to the theory so that your study can make a contribution that speaks both to those within your research space, and those beyond it who can draw useful conclusions and lessons even if their data come from somewhere else.

 

Spinning the ‘golden thread’ that can sew your PhD together

When I was doing my PhD, someone at some stage asked me (probably in response to my ramblings about what my PhD was about): ‘what is your “golden thread”?’ This stumped me. My what? I hadn’t really heard that term before, although my supervisor has talked about it since, as have other colleagues who all supervise students – it seems to be a fairly common notion then, this notion of a ‘golden thread’ with which you can ‘sew’ your PhD thesis together. But what, indeed, is a golden thread, where do you get one, and how do you work out how to sew your PhD together?

To begin with what it is: the golden thread is, for want of a better explanation, the central argument that pulls through your whole thesis, and creates coherence across the literature review, the research questions, the theoretical and conceptual framework, the methodology, and finally the analysis and organisation of the data and the conclusions you are able to draw (on the basis of that argument you set out to make). It sounds quite straightforward when it is put like this, but in my experience (and in the experience of many other PhD students) it is really difficult to find and hold onto over the long course of researching and writing a PhD thesis. Another way of thinking about it would be to keep reminding yourself about what the point of your PhD is. What is it actually about – what are you trying to say here? A friend of mine types her main research question into the header of each page she works on in each of her chapters, so that she is not tempted to go off track in her writing and thinking; another friend wrote a haiku about the main point her PhD was making, and stuck it in a place she could see it when she was writing; another wrote her research questions on several sticky notes and put them above her desk at work and her desk at home, so that she had them in front of her whenever she was working on the thesis. I kept a fairly faithful research journal, and re-read it often, to remind myself what I was actually making my argument about.

So, how do you get one? Sadly, you cannot go to PhDarguments.com and order one; you have to make or build one, and this takes time and is really challenging. I think of it a bit like Rumpelstiltskin turning all the straw into golden thread (except without all the creepiness). What you have when you start a PhD is straw – ideas, concepts, theory, methods, questions, literature you have read – and you have to pull the right pieces of straw together to make a strong, shiny length of golden thread that you can then use to sew a beautifully coherent and persuasive PhD thesis. Like theoretical frameworks, analytical frameworks, literature reviews, an argument is built part by part and always in relation to the main question it is being made to answer. There are key parts of the thesis that you need to put into place as you go to help you create strong and coherent sub-arguments that build towards the overall, central argument your PhD will make.

You need to scope your field well, and find a gap into which your research could fit – this helps you to start asking more refined questions, which can turn into research questions. You need to move from this reading into tougher theoretical and conceptual territory – you need to find your theoryology, and with it, further refinement and focus of your research questions. You need then to consider how you will answer these questions: what data will you need? How will you find it? What will you do with it in order to make sense out of it, and select what is relevant to analyse in relation to your research questions? Then you need to further consider the research questions you are trying to answer as you connect the theory with the data in the process of analysing it, and using it to tell the story that answers your questions, and explains why both the questions and the answers are important to your readers, and your research community or field. Following a logical and coherent process, and pulling each part of the process through with you into the subsequent stage or part of the process, really helps. In other words, don’t leave all your theory and research questions behind when you plan out your methodology and generate your data. Don’t forget the scoping of the field you have done, the research questions you are asking, and your theoretical framework and conceptual tools when you organise and begin to analyse that data in order to build your strong, shiny argument.

Image from uklpf.co.uk

Image from uklpf.co.uk

The argument, in the end, is the thing with the PhD. You cannot have your readers get to the end of it wondering: ‘So what? Why did I just read all of that? What was the point?’ The golden thread is just that: the answer to the ‘so what’ question; the point of the research; the central argument you have made on the basis of the research you have done. Without it you don’t have a PhD thesis; you have parts of a whole that has not been realised or pulled together. In order to sew those parts into something that represents what Trafford and Leshem have termed ‘doctorateness’, you need to channel Rumpelstiltskin, and start turning all your straw into your own golden thread, so that you can sew the parts of your research into a coherent, persuasive, strong PhD thesis.

Two more scary -ologies

This post follows on from last week’s post about a couple of -ologies I have found a little (and a lot) intimidating over the course of my PhD. These -ologies are often stumbling blocks for me, and for other students I know. It’s almost as if your brain hears them and immediately puts its fingers in its ears and starts saying ‘la la la’ really loudly until you stop asking it to think about these concepts. I suggested last week finding ways to best these -ologies by finding examples and stories to explain them in the context of your own research, or in the context of more familiar ideas. This week I have two more -ologies I have done this with. The first is a very familiar one – Methodology – and the second is my own one – Theoryology. I’ll start with the made-up one.

When I started my PhD in 2010, I had no idea what a conceptual or theoretical framework was. My MA research project was a smaller one – 25000 words – and my supervisor was in Canada while I was in South Africa writing it and she was not a lover of email or sustained feedback, so I did not get a great deal of help with the thinking. I did alright in the end, but it was not a very conscious writing process. I felt like I was simply following her basic instructions to get it done rather than crafting my own project. So, starting the PhD was properly scary because I had never done a research project like this, and I was intimidated and very unsure of myself. And, on top of that I had to start with theory.

Now, I like theory, and I am pretty good at putting pieces of a puzzle together to create a nice, coherent whole. So, I write really good literature reviews. But a theoretical framework is much more than a literature review. It’s just that: a framework; a holding structure; something that makes you study intelligible to readers and that guides the study as it progresses. A literature review is really more of a guide to all the other research you are drawing on to make sense of what your study is about, what it thinks it can contribute to the field, and where it is located in the field. Starting out, my supervisor talked more about me reading my way into finding a framework, rather than a literature review. I felt overwhelmed by the sheer volume of reading I needed to do, and battled to find the time but more than that the headpsace to do all the reading and thinking. I read a lot and made a mountain of notes, mostly copied quotes from these texts, but at the end of the year I had no framework. I had a great draft of a literature review, but nothing that could really shape and guide my study. I had no Theoryology – no strategy to guide the selection and coherent melding of theoretical tools into a framework capable of shaping and guiding a research study.

I took a break for a while in 2011, and when I started reading again I came to the literature trying to look for something different – trying to find this sort of strategy. I found my way to social and critical realism, and these two fields began to give me what I was looking for – theoryology. I could slowly begin to see the shape of my project, and the scope of the questions I could ask and answer. I think the key difference was that I started to see the theory I was reading as more than just ideas; I started to see tools: tools that could become analytical and a lens with which to look for and look at data; tools that could do different things in different parts of my study. There was other theory that I used as well to build the framework for my study – substantive theory that was more ideas than tools. You need both, but I learned that different kinds of theories can do different things. Some have greater explanatory and conceptual power than others, and you need to find the overarching conceptual or theoretical framework – the theoryology – as well as the substantive theory that helps you to understand and explain parts of your study and that helps you to locate your study in relation to other relevant research.

The other -ology I took a while to get a handle on was Methodology. I took ages to get going on this chapter, mostly because I had no idea what I was really doing with it. What exactly is a ‘methodology’? I did some reading, but what I found was a lot of stuff on methods, which is not a methodology. For ages, though, I thought it was. I think lots of researchers, especially less experienced ones, might use these terms to mean the same thing. I did initially, but that was because I did not see the crucial difference. A methodology is another kind of strategy, linked to your theoryology. You use both to give a more abstract picture of your study – one at a remove from the nitty gritty of the data and the details. The bigger picture stuff, if you like. So, if the theoryology is the framework or holding structure for your study, then the methodology is the strategy you employ for putting the study into action. What data will you collect? Why will you collect this data and not other kinds of data? What will you do with this data once you have it? How will you look at it, question it, analyse it? How will this data connect to the theory and the literature? These are some critical methodological questions. When you get out into the field, and interview, survey, observe, and then come home again and transcribe, draw, sift etc – these are your methods shaped into a coherent form by your methodology – your overall strategy – and held in place further by your theoryology or the theoretical (and eventually analytical) framework you will use to guide your whole study to its conclusions.

These don’t seem like such scary -ologies now, but I think they are potentially intimidating and also confusing for many scholars. Again, finding ways to make them make sense to you and to your study is the best way to gain control over them and out them to work for you.

Get your ‘gaze’ then get your data

As promised, a post on something a little less personal and a little more ‘academic’. The topic for this week’s post is theoretical frameworks, and why it’s a very good idea to have one fairly clearly in place before you start collecting and analysing your data.

Right. When I started my PhD in 2010, I spent the whole of that year reading about academic literacies and writing centres, thinking that I wanted to do my PhD on a topic related to the role writing centres play in developing students’ academic literacies in the disciplines. I just couldn’t get into it, though. I had a whole lot of substantive theory, but nothing that looked or felt like a framework for a study that would guide my data gathering or my analysis. After a long and helpful meeting with my supervisor at the end of that year, I found my way to Basil Bernstein’s work and from there to Pierre Bourdieu’s and to Legitimation Code Theory. I had, much to my joy, found my framework.  I had the beginnings of a ‘gaze’ to use Bernstein’s term, or a set of specific lenses with which to scratch at my ‘conceptual itch’ to use Lis Lange’s term (well, it was her I first heard it from) – the thing that I really needed to find out or the question I really wanted an answer to. I could now rethink my topic, and begin to form this gaze so that I would know what data I needed to collect and what I should do with it to get at the answers and scratch the itch.

So, I spent a lot of time in 2011 and 2012 writing and rewriting what I suppose some might call a literature review, but which my supervisor and I called a conceptual or theoretical framework. I had to lay very firm foundations for this study because the theory I was reading and the concepts I used were so new to me, and I really was starting from scratch in many ways in this field. I often felt frustrated at how much time I was spending on this one chapter out of six that had to be written, and I felt lost a great deal of the time in the mess of reading and thinking and connecting of dots. It was a hard couple of years, but in that time I got my proposal through and wrote a pretty good draft of chapter two. I found myself, at the end of 2012, very nervous about leaving Theoryland, as I thought of it, up on its pretty, fluffy clouds. I felt safe there, because I had pretty much worked out what Bernstein called the ‘internal language of description’ for my study. In other words, I could tell you how the different parts of the theory and concepts I chose to use cohered, and why they fitted together like that and what sense to make of them in the context of my study. But that was all I could do at this point. So I had to move on.

I was encouraged (read shoved) off my clouds and into the mires and muck of field work and data gathering. And this is where things started to get really exciting for me (and then boring later and then exciting again). I observed lecturers teaching and read their course documents, as well as conducted in-depth interviews. In the first week of lectures, I found I could see and hear the theory – that internal language – being spoken in the teaching and coming to life in front of me. It was so exciting and also so affirming. This was the right data to be gathering and my study was moving in the right direction, given my focus and my questions. I do not think that this would have happened quite so clearly had I tried to shortcut the first stage of laying those foundations and working out, as carefully and fully as I could, that internal language of description. I needed to have my gaze in place, as nebulous and fuzzy as it felt, before I gathered by data and started to think about what to do with it in analysis.

I won’t be glib and say that the data gathering was easy – it was often really tedious and the semester felt so long; recording detailed fieldnotes in 8 lectures per week of an hour long each for the better part of 14 weeks was tough going for me. But at no point did I really doubt that I was gathering the right data for my study, and even when things got very fuzzy and I couldn’t hear the theory so clearly or see my way, I could count on the fact that my gaze was there and that it would deepen and develop as I really started to get into my data more systematically. It was very reassuring on the whole, and while it didn’t make the field work stage a breeze, it did take some of the anxiety and doubt away, and kept me moving towards the next step.

I would, based on my experience, argue that even if you feel frustrated and feel like you are taking way too long working out your own internal language of description, it is worth doing as well as you can. The firmer the foundation going forward, the less the likelihood (I think) of having to rush back and get different or more data because you don’t have enough or the right kinds of data. It reduces the stress and anxiety that inevitably come with gathering data and starting to think about analysing it, because having your gaze in place can assure you that you are moving in the right kind of direction. However, a word of caution: it can be really lovely and safe and warm up in Theoryland, but staying there will never get your PhD finished. You will need to find the bridge down to the Data Swamps so that you can move between the two as you scratch your own itch and answer the questions that drive you onwards.