What do you think you ‘know’ about your research problem?

*You can listen to this post on the podcast player.

I am currently supervising several students at PhD and MA level, and a few of them are starting out with reading, writing, constructing their gap and setting out the shape and size of their research problems. This is an exciting and also at times frustrating process for students and supervisors. It is often characterised by steps forward and then backward or perhaps sideways, and sometimes what feels a lot like going around in circles. I’m not focusing on this whole process here, but rather one dimension of this part of doing research: the things we think we “know” before we start our research, the assumptions we bring into our reading and writing that we may not always see clearly, and why we need to think reflexively about these issues to do more ethical and credible research.

We often approach research, perhaps more at doctoral than any other level, from the starting point of our own interests, passions, questions: what do we want to know more about? What do we want to use this research project to find out or change or do? This means that we may be coming into this research with existing knowledge about the topic we are researching. And, crucially, our approach to the new research may be shaped or influenced by certain assumptions about what we think the project will be about, or how it might unfold, or what we might discover or find out. This is all really normal: when I started my PhD I had been working as a lecturer and tutor for almost 10 years and I had a lot of ideas about what we should be doing better or what we might be doing poorly and what could be improved and how. Many of these were founded in what might be termed a form of experiential knowledge or practice wisdom*, and some of them were probably quite common-sense understandings of teaching, learning and higher education.

Photo by fauxels on Pexels.com

This is knowledge we all have, whatever field we are working in, I think. This may be more pronounced in some fields than in others, especially when you come from a professional space you have been working in to the doctoral research space. Some of this knowledge may come from overt forms of learning, such as reading research or attending seminars and making deliberate notes and working to make sense or meaning of all of this input in relation to our work and research. But, much of it can also be gained more tacitly, through time, experience, being in the environment and working things out as you go. This can feel like “common-sense” knowledge: this is the way we do things, it works, it’s fine. But we cannot rely on common-sense knowledge as a basis for research because it is too partial, too biased, and perhaps not necessarily supported by peer-reviewed, theorised research in our field of study.

I’ll offer an example from my own research journey. I work in higher education, as many of you may know by now. Specifically, I am a writing and academic development specialist and practitioner. I started my PhD a year after I started managing a university writing centre, and about 6 years after I has been consistently involved in teaching academic writing to undergraduate students at three different universities in different faculties. So, I had a lot of ideas about the “problems” with student writing itself, with students’ “motivation” for writing, and with the teaching that was happening. These ideas were shaped by colleagues I had worked with, the departmental ethos and course structures and materials I worked within and with, and my own experiences with students. I had not read very much of the scholarly research because I was not asked to or required to, and thus it didn’t seem relevant. It was pretty much all I could do most weeks to keep up with the teaching and marking. When I started my PhD and wrote the first few drafts for my supervisor, I was really challenged on a lot of my assumptions – assumptions I did not realise I had – that revealed positions and ideas that were not supported by the more critical parts of the conversation the field was having about literacy development, student writing, and effective teaching and learning. I wanted – needed – to join this critical conversation and this meant that I had some difficult thinking, reading, and reflecting work ahead. Specifically, I had to unlearn some of what I had learned about curriculum design and teaching, and notions of student “deficits” and discourses that put individual development and change over and above deeper and more sustainable systemic development and change.

This was hard work: not just intellectually, but also emotionally because it challenged who I thought I was as a teacher at that time. I always thought I was helpful, kind, made tough things like writing essays easier for my students. I think I was that person for some of my students, but the reading I did and the related thinking and writing work at PhD level made me realise that I was probably not that person for all my students because I had blind spots around questions of knowledge, access, success, systemic dis/advantage and how university opens spaces for some students to succeed and closes places for others, for a range of reasons that need to be made visible, named and addressed.

My early research questions and writing reflected my early blind spots and biases – in the language I was using (“In what ways can we better allow students to succeed”; “How do we deliver the curriculum more effectively”). My verbs, perhaps more than anything else, gave me away: they revealed my assumptions and bias. These needed to be challenged and made visible for two key reasons, I think. The first is that if we cannot or do not see that we have all sorts of assumptions and ideas coming into a project, we may well end up with a significant mismatch between the work we want to do and the work we will probably end up doing. We may say, for example, we want a participatory research design but end up doing something far more directive and hierarchical if we have an assumption that we know what the answers are and all we need to do is work to find them, or that the researcher needs to be an authority in their project, for example. This may extend to disconnects between theoretical and conceptual frameworks and the language you use in writing the thesis, or between the theoretical and methodological frameworks.

The second reason is that these assumptions and biases, if they remain unseen or unchallenged, may affect the way we obtain and make sense of our data. If you think you “know” that student motivation is the issue in whether or not students are successful, even if your reading of the literatures challenges this, you may construct data gathering instruments, such as survey items or interview questions, that are leading – that try to get participants to confirm these assumptions or ideas. You may also only seek out literatures that confirm these assumptions or ideas. In all research, we need to be quite aware of and cautious around confirming our biases, rather than constructing a research design and data gathering process that reflects the literatures we have included and aligned our study with, our theoretical frameworks, and that is open to discovery and analysis, rather than seen as “The answer”. “The answer” may be too coloured by our common-sense knowledge and what we think we “know” to really push us forward in our thinking, challenge some of the more problematic aspects of our work, create new knowledge in our fields of study and practice. In my own study I used Nvivo to code my data and in one of the training videos there was a phrase that stayed with me: “We need to be open to being surprised by our data”. Scientists may tell you: “Believe what you see; Don’t see what you believe”.

Photo by fauxels on Pexels.com

There are many different research paradigms, or worldviews, that shape what we think the world is like, what we think counts as valid or credible knowledge or “truth”, and how we can reliably come to know that knowledge or verify those “truths”. Our worldviews are not always visible to us because we may come to them quite tacitly, over long periods of time through immersion in particular fields or disciplines of research and practice, through prolonged engagement in particular social and political environments, and so on. Surfacing our assumptions about what we “know” and how we know it and why we think it is valid knowledge is a powerful step in making our research paradigm clear enough to ourselves to explain it to readers and examiners. And this is important because it lends further credibility to our choices of research design, analytical framework and methods, and our eventual findings. On what basis do you make claims to “truth” or knowledge? How must others evaluate and make sense of your claims? How do you show the different ways in which you are making your contribution to knowledge and thereby persuade the reader that they matter and are valid?

This is not always easy work and it happens over the course of the project rather than all up front in the proposal. You need to begin there, at the beginning, reading and thinking with the scholarly and additional relevant literatures in your field, keeping a reading journal to write to yourself about how the reading and your own thinking and purposes/plans are connecting and speaking to one another. Speak to others about your research, including but not limited to your supervisors. Share your thoughts as openly as you can with people you know will listen and think with you, and not shut you down. Consider having a critical friend to regularly debrief with about different stages of your research process, so that you have help in spotting and revealing blind spots and biases that may derail your study or undermine the credibility of your arguments. The key thing, I found, was to find as many different useful spaces as I could to be brave enough to talk about my work and my thinking, and be open to listening to the critique and doing the work it demanded. It was not easy, but I do think that my research itself and myself as a researcher are better for it.

*This a term I reference to Veronica Bamber and her keynote at HECU7 in 2016.

Researching your own ‘backyard’: on bias and ethical dilemmas

This is a post particularly for those in the social sciences and humanities who may be doing a form of ethnographic research within the context in which they work or study – in other words, doing ‘insider research’ to use Paul Trowler’s term. Researching a context with which one is intimately familiar and in which one has a vested interest can create possible bias and ethical dilemmas which need to be considered by researchers in these situations. The last thing you want, in presenting your completed research, is for your findings to be called into question or invalidated because you have not accounted clearly enough for issues of insider bias, and your own vested interests.

Insider bias and vested interests

In the article cited in this post, Trowler considers issues of bias in data generation. Bias in research can be defined as having only part of the ‘truth’ in your data but treating that part as a whole, ignoring other possibilities or answers because you are prejudiced towards the ones that best represent your interests or investment. If you are working in a context with which you are familiar, especially your own department or faculty, or an organisation in which you have worked or do work, you will have a vested interest in that context. Either you want everyone and everything to look amazing, or perhaps you are unhappy about certain aspects of the ways in which they work and you want your research to show problems and struggles so you have a basis for your unhappiness. Either way, you have to acknowledge going in that you cannot be anything but biased about this research.

bias blindspot

However, acknowledging that you are biased, and detailing what that bias might entail for readers and examiners, does not undermine your position as researcher. By making yourself aware of potential blindspots in your research design – for example the participants you have chosen, or the cases you are including and excluding from your dataset (and why) – you can better head off possible challenges to the validity of your data later on, and you can strengthen your research design choices. Be honest with yourself: there is a balance to strike here between being pragmatic and strategic in choosing research participants, sites, or cases that will be accessible and that will yield the data you need to make your argument, and between choosing too neatly and risking one-sided or myopic data generation. Why these participants, these cases, these sites? Are there others that you know less well that you could include to balance out the familiarity, and increase the validity of your eventual findings? If not, how might you maintain awareness of your ‘insiderness’ and account for this in analysis and discussion later on?

You need to account for these decisions and questions in your methodology, and discuss what it means for your study that you are doing insider research, and that this does imply particular forms of bias. I don’t think you can get away from being biased in these cases, but you can think through how this may affect your data generation processes, and your analysis as well, and share this thinking with your readers frankly and reflexively.

Insider bias and ‘intuitive analysis’

Another point Trowler makes concerns insider ‘intuition’ when analysing the data you have generated and selected for your study. You may be analysing a policy process you were part of, or meetings you sat in on, or projects you were involved in. You have insider knowledge of what was said, the tone of the conversations, background knowledge (and perhaps even gossip) about participants – in other words, you have a kind of cultivated ‘intuition’ about your data set that you reader will not be privy too. Accounting for bias here is crucial, because if you cannot see it, you may rely too much on this insider intuition in analysing your data, and too much of the language of description you are using to convey your theorised findings will be tacit and hidden from the reader. They will then struggle to understand fully on what basis you are claiming that X is an example of poor management, or that Y means that the department is doing well in these particular areas.

ideas

It is thus vital that you get feedback here on whether it is clear to your reader why you are making particular claims, and whether they can see and understand the basis on which you are making such claims. Do they understand your ‘external language of description’ or ‘translation device’ to use Bernstein’s and Maton’s terms respectively? If they do not, you may be relying too much on your insider view of your case or participants, and may need to find a way to step back, and try to see the data you are looking at as more strange and less familiar. Getting help from a supervisor or critical friend who can ask you questions, and expose and critique possible points of bias is a useful way to re-interrogate your data with fresher eyes.

Ethical dilemmas

An ethical dilemma is defined as ‘a choice between two options, both of which will bring a negative result based on society and personal guidelines’. In research, this definition could be nuanced to suggest that an ethical dilemma presents itself when you have to make a decision to protect the interests of your research or the interests of your participants or study site. For example, in an interview with a senior manager you learn information that may be better off staying private and confidential, yet would also add an important and insightful dimension to your findings. What do you do? A participant in your study asks you for help, but to help might be to prejudice that participant’s responses in a later survey or interview, possibly skewing your data. Yet it is your job to help them. Study first, or job first? These are the kinds of dilemmas that can arise when you do research in the same spaces in which you work, and with people you work with and have other responsibilities to outside of your research.

Cheating-clients,-ethical-dilemmas

As researchers we have a duty to be as truthful and ethical in our research as possible. We are working to create and add to knowledge, not to simply maintain the status quo. In your study this may mean being carefully but resolutely critical, reflective and challenging, rather than only saying the palatable or easy things to say. This work is always going to present difficulties and dilemmas, but accounting as far as possible for your own bias and vested interests, and for your own relevant insider knowledge, can create space in your study for the development of your own reflexivity as a researcher, and can bolster rather than undermine the validity and veracity of your findings.

Trowler, P. (2011) Researching your own institution: Higher Education, British Educational Research Association online resource. Available online at [http://www.bera.ac.uk/files/2011/06/researching_your_own_institution_higher_education.pdf]