‘Retrofitting’ your PhD: when you get your data before your theory

I gave a workshop recently to two different groups of students at the same university on building a theoretical framework for a PhD. The two groups of students comprised scholars at very different points in their PhDs, some just starting to think about theory, some sitting with data and trying to get the theory to talk to the data, and others trying to rethink the theory after having analysed their data. One interesting question emerged: what if you have your data before you really have a theoretical framework in place? How do you build a theoretical framework in that case?

I started my PhD with theory, and spent a year working out what my ‘gaze’ was. I believed, and was told, that this was the best way to go about it: to get my gaze and then get my data. In my field, and with my study, this really seemed like the only way to progress. All I had starting out was my own anecdotal issues, problems and questions I wanted answers to, and I needed to try and understand not just what the rest of my field had already done to try and find answers, but what I could do to find my own answers. I needed to have a sense of what kinds of research were possible and what these might entail. I had no idea what data to generate or what to do with it, and could not have started there with my PhD. So I moved from reading the field, to reading the theory, to building an internal language of description, to generating data, to organising and analysing it using the theory to guide me, to reaching conclusions that spoke back to the theory and the field – a closed circle if you will. This seems, to me certainly, the most logical way to do a PhD.

But, I have colleagues and friends who haven’t necessarily followed this path. In their line of work, they have had opportunities to amass small mountains of data: interview transcripts, documents, observation field notes, student essays, exam transcripts and so forth. They have gathered and collected all of these data, and have then tried to find a PhD in the midst of all of it. They are, in other words, trying to ‘retrofit’ a PhD by looking to the data to suggest a question or questions and through these, a path towards a theoryology.

Many people start their doctoral study in my field – education studies – to find answers to very practical or practice-based questions. Like: ‘What kinds of teaching practice would better enable students to learn cumulatively?’ (a version of my own research question) Or: ‘What kinds of feedback practices better enable students to grow as writers in the Sciences?’ And so on. If you are working as a lecturer, facilitator, tutor, writing-respondent, staff advisor or similar, you may have many opportunities to generate or gather data: workshop inputs, feedback questionnaires, your own field notes and reports, student essays and exam submissions, and so on. After a while, you may look at this mountain of data and wonder: ‘Could there be a thesis in all of this? Maybe I need to start thinking about making some order and sense out of all of this’. You may then register for a PhD, searching for and finding a research question in your data, and then begin the process of retrofitting your PhD with substantive theory and a theoryology to help you work back again towards the data so as to tell its story in a coherent way that adds something to your field’s understanding or knowledge of the issues you are concerned with.

The question that emerged in these workshops was: ‘Can you create a theoretical framework if you have worked so far like this, and if so, how?’ I think the answer must be ‘yes’, but the how is the challenging thing. How do you ask your data the right kinds of questions? A good starting point might be to map out your data in some kind of order. Create mind-maps or visual pictures of what data you have and what interests you in that data. Do a basic thematic analysis – what keeps coming up or emerging for you that is a ‘conceptual itch’ or something you really feel you want or need to answer or explore further? Follow this ‘itch’ – can you formulate a question that could be honed into a research question? Once you have a basic research question, you can then move towards reading: what research is being or has been done on this one issue that you have pulled from your data? What methodologies and what theory are the authors doing this research using? What tools have they found helpful? Then, much as you would in a more ‘traditional’ way, you can begin to move from more substantive research and theory towards an ontological or more meta-theoretical level that will enable you to build a holding structure and fit lenses to your theory glasses, such that you have a way of looking at your data and questions that will enable you to see possible answers.

Then you can go back to your data, with a fresh pair of eyes using their theory glasses and re-look at your data, finding perhaps things you expect to see, but also hopefully being surprised and seeing new things that you missed or overlooked before you had the additional dimension or gaze offered by your theoretical or conceptual framing. But working in this ‘retrofitted’ way is potentially tricky: if you have been looking and looking at this data without a firm(ish) theoretically-informed or shaped gaze, can you be surprised by it? Can you approach your research with the curious, tentative ‘I don’t know the answers, but let’s explore this issue to find out’ kind of attitude that a PhD requires? I think, if you do decide to do or are doing a PhD in what I would regard as a middle-to-front sort of way, with data at the middle, then you need to be aware of your own already-established ideas of what is or isn’t ‘real’ or ‘true’, and your own biases informed by your own experience and immersion in your field and your data. You may need to work harder at pulling yourself back, so that you can look at your data afresh, and consider things you may be been blind to, or overlooked before; so that you can create a useful and illuminating conversation between your data and your theory that contributes something to your field.

Retrofitting a PhD is not impossible – there is usually more than one path to take in reaching a goal (especially if you are a social scientist!) – but I would posit that this way has challenges that need to be carefully considered, not least in terms of the extra time the PhD may take, and the additional need to create critical distance from data and ‘findings’ you may already be very attached to.

Building a theoretical framework for your study

I am presenting a seminar tomorrow to PhD students on how I developed my PhD’s theoretical framework. When I agreed to do this workshop last year, I thought this would be fairly easy to do. However, I am finding it difficult to articulate the differences between substantive theory and what I think of as ‘framework theory’, and how to use both in different ways to contextualise your study and build a framework for it. This was, for me, the first and biggest ‘threshold’ (to use Meyer and Land’s term) that I crossed in my own PhD, and it’s a very important one.

There are two main places you tend to use theory in a social sciences PhD (I’d be really interested to hear about the differences between these and PhDs in the natural sciences): in your literature review where you contextualise your study and the rationale for it, and in your theoryology, as I call it, which works at a more ‘meta’ level to discuss how you understand the questions you are researching, how you plan to approach the research, and how you will move from theory to methodology and methods, and on to analysis. You come back to the theory in your conclusion, connecting it to your findings, but it needs to be laid out ahead of this in the earlier chapters.

I have written about writing a literature review, and using the research in the field in a more substantive way to build the context and rationale for your own study, and situate it within your field, so I won’t focus too much on that in this post. Here, I would rather focus on the more difficult-to-articulate issue: the theoretical framework. The reading PhD students almost always start with is the substantive ‘theory’ and research. You need to know what research has been done in your field, what the key issues and questions are, where the field is heading, and where you, coming into that field, can place your own research. But you may well find, if you spend long enough reading the substantive theory and research, that you start going in circles, confirming what you have already found out and slowly getting stuck. This is when you need to take a break. If you know what the lie of the land is, and you have a grasp of the trends, issues and research problems that are being or have been researched, and where the gaps you, you now need to find your question, your project. You should find the topic you want to research, and the focus, from your substantive reading, but you need to hone this to turn it into a lightning rod for all the other research and reading you will do to cling to. In my (albeit limited) experience, moving on to the ‘meta’ stuff at this point helped me to do this.

What do I mean by ‘meta’ stuff? The Free Dictionary defines metatheory as ‘a formal system that describes the structure of some other system, and ‘a theory devised to analyse theoretical systems’. Slightly obscure definitions on their own – theories that can analyse theories sounds like more going round in circles. But I’m going to concentrate on the words ‘formal system’, ‘structure’, and ‘analyse’ because this, for me, is what a theoryology hinges on. What comes after you  have focused on one research question your study can explore and answer? You need to design a methodology, and then use appropriate research methods to generate data, and then employ an analytical framework to organise that data. Thereafter, you need tools of analysis to make sense of the data in relation to the theoretical structure you have put together, and the field in which your research is located. All of this is nigh on impossible to do in any coherent, structured and organised way without an organising structure that can hold the project in place and on course. You need, then, a formal system, a structure, that can inform the methods you use, your data generation plans and strategies, and the ways in which you then organise and analyse the data to answer your research question. This formal system is your theoretical or conceptual framework (your theoryology).

But how do you build one? What kind of theory is theoryology-worthy? Theory is a weird word here, and it is often misused. I am sure I misuse it all the time. Often, when we say ‘theory’ we mean ‘other research that has been done in the field that informs how I think about my own research and/or practice’. By metatheory is meant something that, while derived from empirical research and data, is raised a bit further above the empirical to create a more generalised or abstract set of principles that can be applied as a formal organising or structural system in a range of studies. Theoretical frameworks are built out of this kind of theory. You need an abstract, generalised set of principles that you can adapt and apply to your own study, and that will inform what data you choose to generate, and how you then can organise that data and decide on methods of analysis. A helpful way of finding your way to this kind of theory that might be right for your study is to look at what kinds of frameworks or conceptual tools other researchers are using in your field or allied fields.

Finding a framework for your study is essential in the social sciences, certainly. Without an overarching metatheory to organise and analyse your project you may end up generating a mountain of data that you just get lost in because you’re not sure what is important or what is not; you may end up with the wrong kinds of data; or you may write a ‘findings’ chapter that describes rather than analyses the data in relation to the field itself, and the question your are answering, which would leave you in a precarious position, given the pressure on PhDs to make new contributions to knowledge. Don’t be too worried if this process of developing your own framework takes time (I spent most of my second year on mine). In my experience, if you have a solid framework that is chosen well and clear to you, what follows is generally more organised and less fraught on the whole*.

*Of course, all sorts of things can and do go wrong in a PhD, and this is just my experience, but colleagues who have followed this route have found similarly that the data generation and analysis process has been less stressful if they have a clearer sense of what they are looking for, why, and how to look at it.

Disciple or pathmaker? Critiquing your theoretical ‘gurus’

I recently enjoyed an amazing week at the University of Sydney, where I spent time talking about my work with the researcher who developed the conceptual tools I used to frame my theoryology and methodology, as well as with PhD scholars and researchers who are also using these tools in their own research. Two of my friends asked me whether the week with my ‘guru’ was a good one, and these comments coupled with a recent seminar I attended on data and theory gave me pause for thought. I do not think of this person as my ‘guru’, although I find the tools truly brilliant, and I am so enjoying working with them in my research and practice. I think (I hope) I am capable of putting aside my admiration for both him and this conceptual toolkit to offer critique and questions, and to find ways in which I can contribute to this field of research and practice with my own work. But, when you are new to a particular theoretical or conceptual approach, or new to a significant piece of research like a PhD which requires so much deep and extended engagement with theory, it can be really difficult to do this. I certainly found it difficult to be critical of the concepts when I was working with them during the PhD. This ability I now feel I have to critique the tools and the theory is very much a post-PhD thing.

Why are we so tempted to make ‘gurus’ out of our favourite theorists? Why is it hard to be critical of the theories we have chosen to use to provide the foundation and analytical tools for our research, especially in a PhD? I think, for me, the answer was quite simple: because I needed them to be right. I really wanted to have answers to my questions, and I really needed the theory and the tools to provide me with those answers. I suppose, too, I was afraid that pointing out gaps, holes and areas where the theory was still fuzzy would ultimately weaken my stance and my arguments. I opted, without quite realising it at the time, for being a bit of a theory disciple. This was in big part due to what I have just said, but it was also due to the fact that I was so excited about the ways in which the theory opened my eyes to things in the environments I was researching that I had not been able to see otherwise, or in quite those ways, that I got a bit carried away by just how thrilling my research actually was for me when I got into the data  and started finding tentative answers to my questions.

One of my thesis examiners suggested to me in the report I received that I should exercise caution in getting too excited and too carried away. Part of the role of a good researcher is to be able to stand back a bit from the thing we are looking at, or the lenses we are looking through or with, and wonder if we are seeing the right sorts of things, or asking the right sorts of questions. We need to be able to see gaps, holes, inconsistencies, not just to avoid being accused of having a weak argument and having no defence in your viva or examination, but more importantly to show that you are clear enough on what theory you are using, as well as why and how you are using it, that you can show that although there may be questions you cannot yet answer, or things you cannot yet see, you know that the answers you have are good ones, even though they are always partial and fallible. You aren’t answering all the questions in your PhD – you are just answering one – but you need to be able to show not just all the reasons why your theoretical framework and tools are right for answering this question; you also need to be able to be critical and careful, so that you anticipate and can stand up to critique of your own work and answer back to the critics.

Rather than being theory disciples, I think we should be aiming to be research pathmakers. Theory on its own is a bit pointless. Your research will bring any of the theory you use to life in a range of ways depending on how you draw the framework, and also what data you choose to generate, analyse and interpret using the theory. You can be a pathmaker, even if the bit of path you are chipping out for others to walk on is small or short. But, this is not easy. I think you have to be well-read and brave to be critical, and you also need to know your theoryology well. It is almost impossible to offer a sound and useful (but not too damning) critique of your own theoretical framework unless you really know it well. In chipping out your piece of path, you will be following the paths of those who have gone there before you, and you’ll hopefully be either extending the paths they have created, or branching off in slightly new and unexplored directions, rather than simply smoothing out their already-trod path or pruning the bushes on either side of it :). It’s hard to be this kind of pathmaking researcher if you are not going to be brave enough to take your ‘gurus’ off the pedestals you could tend to place them on when you start your PhDs, and offer thoughtful, relevant and useful critique that shows your readers just how well you do know your own field, and that adds depth and credibility to your own researcher’s voice.

I have learned that I don’t need the theory or my answers to be ‘right’ (if they even can be); but I do need them to be credible, productive and interesting, and being able to both believe in and offer measured critique of the theoryology that serves my research well will certainly help me to find my ways to the kinds of answers I am seeking.


Two more scary -ologies

This post follows on from last week’s post about a couple of -ologies I have found a little (and a lot) intimidating over the course of my PhD. These -ologies are often stumbling blocks for me, and for other students I know. It’s almost as if your brain hears them and immediately puts its fingers in its ears and starts saying ‘la la la’ really loudly until you stop asking it to think about these concepts. I suggested last week finding ways to best these -ologies by finding examples and stories to explain them in the context of your own research, or in the context of more familiar ideas. This week I have two more -ologies I have done this with. The first is a very familiar one – Methodology – and the second is my own one – Theoryology. I’ll start with the made-up one.

When I started my PhD in 2010, I had no idea what a conceptual or theoretical framework was. My MA research project was a smaller one – 25000 words – and my supervisor was in Canada while I was in South Africa writing it and she was not a lover of email or sustained feedback, so I did not get a great deal of help with the thinking. I did alright in the end, but it was not a very conscious writing process. I felt like I was simply following her basic instructions to get it done rather than crafting my own project. So, starting the PhD was properly scary because I had never done a research project like this, and I was intimidated and very unsure of myself. And, on top of that I had to start with theory.

Now, I like theory, and I am pretty good at putting pieces of a puzzle together to create a nice, coherent whole. So, I write really good literature reviews. But a theoretical framework is much more than a literature review. It’s just that: a framework; a holding structure; something that makes you study intelligible to readers and that guides the study as it progresses. A literature review is really more of a guide to all the other research you are drawing on to make sense of what your study is about, what it thinks it can contribute to the field, and where it is located in the field. Starting out, my supervisor talked more about me reading my way into finding a framework, rather than a literature review. I felt overwhelmed by the sheer volume of reading I needed to do, and battled to find the time but more than that the headpsace to do all the reading and thinking. I read a lot and made a mountain of notes, mostly copied quotes from these texts, but at the end of the year I had no framework. I had a great draft of a literature review, but nothing that could really shape and guide my study. I had no Theoryology – no strategy to guide the selection and coherent melding of theoretical tools into a framework capable of shaping and guiding a research study.

I took a break for a while in 2011, and when I started reading again I came to the literature trying to look for something different – trying to find this sort of strategy. I found my way to social and critical realism, and these two fields began to give me what I was looking for – theoryology. I could slowly begin to see the shape of my project, and the scope of the questions I could ask and answer. I think the key difference was that I started to see the theory I was reading as more than just ideas; I started to see tools: tools that could become analytical and a lens with which to look for and look at data; tools that could do different things in different parts of my study. There was other theory that I used as well to build the framework for my study – substantive theory that was more ideas than tools. You need both, but I learned that different kinds of theories can do different things. Some have greater explanatory and conceptual power than others, and you need to find the overarching conceptual or theoretical framework – the theoryology – as well as the substantive theory that helps you to understand and explain parts of your study and that helps you to locate your study in relation to other relevant research.

The other -ology I took a while to get a handle on was Methodology. I took ages to get going on this chapter, mostly because I had no idea what I was really doing with it. What exactly is a ‘methodology’? I did some reading, but what I found was a lot of stuff on methods, which is not a methodology. For ages, though, I thought it was. I think lots of researchers, especially less experienced ones, might use these terms to mean the same thing. I did initially, but that was because I did not see the crucial difference. A methodology is another kind of strategy, linked to your theoryology. You use both to give a more abstract picture of your study – one at a remove from the nitty gritty of the data and the details. The bigger picture stuff, if you like. So, if the theoryology is the framework or holding structure for your study, then the methodology is the strategy you employ for putting the study into action. What data will you collect? Why will you collect this data and not other kinds of data? What will you do with this data once you have it? How will you look at it, question it, analyse it? How will this data connect to the theory and the literature? These are some critical methodological questions. When you get out into the field, and interview, survey, observe, and then come home again and transcribe, draw, sift etc – these are your methods shaped into a coherent form by your methodology – your overall strategy – and held in place further by your theoryology or the theoretical (and eventually analytical) framework you will use to guide your whole study to its conclusions.

These don’t seem like such scary -ologies now, but I think they are potentially intimidating and also confusing for many scholars. Again, finding ways to make them make sense to you and to your study is the best way to gain control over them and out them to work for you.

Scary -ologies

I think all PhD students – all academics – have been there: to the place where they are confronted with an -ology and asked to explain it only to find that while it makes sense (sort of) in their heads, it doesn’t make any sense in actual words. I’ve been there, so many times. I’m still there with some of these -ologies. I call them my ‘scary -ologies’.

There are the big ones – ‘epistemology’; ‘ontology’ – I still can’t really explain these in small, clear soundbites (or even longer, less clear phrases) without confusing myself and others. This is frustrating because in the quietness of my mind I do feel like I sort of know what they mean. But please don’t ask me to use them is a sentence. I can tell you, probably not completely correctly, that epistemology has to do with knowledge and knowing, and ontology has to do with being. But that’s not very helpful if you are new to these terms or trying to get a handle on them yourself. Sorry. I could cite some online dictionary definitions (some of which are actually quite helpful) but the best way to get to grips with any -ology or tough concept is to find examples you can use to explain them, or ways to break them down into simpler, easier terms. I thought I’d share my examples and self-explanations on these two bigger and more scary -ologies.

Epistemology, at its most simple, is the study of knowledge – its scope and also its limits and validity. It is also defined as a branch of philosophy that concerns itself with the origins, nature and so forth of knowledge – it is often called a ‘theory of knowledge’.  I think this is where I usually come unstuck in my own understanding of what it is and how to explain it to myself in less dictionaried terms. I’ll come back to this though. Ontology is defined, generally, as a ‘theory of being’, or or theory about the nature of being and what kinds of things exist and why.  I tend to get stuck when people start referring to these things as ‘theories’ about other things. [Theoryology is another scary -ology for me (so scary I made it into an ology).]

I have tried, then, not to be put off my the word ‘theory’ which in my mind signals other notions like ‘grand’, ‘impenetrable’, ‘will take months of reading to work out’ and so on. I have tried to look rather at what these ‘ologies’ are about. Epistemology is about knowledge, and what we can know and how we can know it and what the limits of that knowledge and knowing are or could be. As an example: we can ask whether something we know, like the world being roughly spherical in shape,  is justified. We can say ‘yes, it is justified to know that’. We can also ask how or why that knowledge is justified or true, and in response we could posit evidence drawn from physics, geography, astronomy and so on. You can ask these questions about the nature of many different kinds of knowledge, and that for me (right now anyway) is epistemology – this kind of inquiry and thinking process around what I think I know and why I think I can know it or not.

Ontology is about being and this -ology is more tricky for me to get my head around. I try to understand it in relation to my work or my research. For example, I research pedagogy and what lecturers and students do in relationship in classrooms in order to teach and learn and come to know and so on. A big part of teaching and learning is a process of becoming – you need to become, for example, a lecturer or teacher; it’s not something you just are. It’s a process. In the same way, students need to become lawyers or accountants or physicists – these are ontological as well as epistemological processes because they involve not just knowledge and knowing but also personal and metaphysical traits as well. So, ontology, then, would involve questioning these processes, and the nature of them and what is happening and what that means. I’m not sure I’m right about this – -ologies can make even the most sure-footed researcher doubtful (and I’m not all that sure-footed anyway). But I am very willing to be corrected.

There are other scary -ologies I will get to in my next post – tune in next week – but for now my advice would be to work on the concepts that scare you by finding simple examples or stories to try and contextualise and explain them, all the better if you can do so in the context of your own research. Turning them into stories and examples takes some of their power to scare you and render you speechless in the face of ‘Do you know what X means?’-type questions. And half the battle with concepts and -ologies is getting them under our control rather than being too much under theirs :-).


Notes: the Stanford Encyclopedia of Philosophy online has a very useful entry on epistemology.