Researching your own ‘backyard’: on bias and ethical dilemmas

This is a post particularly for those in the social sciences and humanities who may be doing a form of ethnographic research within the context in which they work or study – in other words, doing ‘insider research’ to use Paul Trowler’s term. Researching a context with which one is intimately familiar and in which one has a vested interest can create possible bias and ethical dilemmas which need to be considered by researchers in these situations. The last thing you want, in presenting your completed research, is for your findings to be called into question or invalidated because you have not accounted clearly enough for issues of insider bias, and your own vested interests.

Insider bias and vested interests

In the article cited in this post, Trowler considers issues of bias in data generation. Bias in research can be defined as having only part of the ‘truth’ in your data but treating that part as a whole, ignoring other possibilities or answers because you are prejudiced towards the ones that best represent your interests or investment. If you are working in a context with which you are familiar, especially your own department or faculty, or an organisation in which you have worked or do work, you will have a vested interest in that context. Either you want everyone and everything to look amazing, or perhaps you are unhappy about certain aspects of the ways in which they work and you want your research to show problems and struggles so you have a basis for your unhappiness. Either way, you have to acknowledge going in that you cannot be anything but biased about this research.

bias blindspot

However, acknowledging that you are biased, and detailing what that bias might entail for readers and examiners, does not undermine your position as researcher. By making yourself aware of potential blindspots in your research design – for example the participants you have chosen, or the cases you are including and excluding from your dataset (and why) – you can better head off possible challenges to the validity of your data later on, and you can strengthen your research design choices. Be honest with yourself: there is a balance to strike here between being pragmatic and strategic in choosing research participants, sites, or cases that will be accessible and that will yield the data you need to make your argument, and between choosing too neatly and risking one-sided or myopic data generation. Why these participants, these cases, these sites? Are there others that you know less well that you could include to balance out the familiarity, and increase the validity of your eventual findings? If not, how might you maintain awareness of your ‘insiderness’ and account for this in analysis and discussion later on?

You need to account for these decisions and questions in your methodology, and discuss what it means for your study that you are doing insider research, and that this does imply particular forms of bias. I don’t think you can get away from being biased in these cases, but you can think through how this may affect your data generation processes, and your analysis as well, and share this thinking with your readers frankly and reflexively.

Insider bias and ‘intuitive analysis’

Another point Trowler makes concerns insider ‘intuition’ when analysing the data you have generated and selected for your study. You may be analysing a policy process you were part of, or meetings you sat in on, or projects you were involved in. You have insider knowledge of what was said, the tone of the conversations, background knowledge (and perhaps even gossip) about participants – in other words, you have a kind of cultivated ‘intuition’ about your data set that you reader will not be privy too. Accounting for bias here is crucial, because if you cannot see it, you may rely too much on this insider intuition in analysing your data, and too much of the language of description you are using to convey your theorised findings will be tacit and hidden from the reader. They will then struggle to understand fully on what basis you are claiming that X is an example of poor management, or that Y means that the department is doing well in these particular areas.

ideas

It is thus vital that you get feedback here on whether it is clear to your reader why you are making particular claims, and whether they can see and understand the basis on which you are making such claims. Do they understand your ‘external language of description’ or ‘translation device’ to use Bernstein’s and Maton’s terms respectively? If they do not, you may be relying too much on your insider view of your case or participants, and may need to find a way to step back, and try to see the data you are looking at as more strange and less familiar. Getting help from a supervisor or critical friend who can ask you questions, and expose and critique possible points of bias is a useful way to re-interrogate your data with fresher eyes.

Ethical dilemmas

An ethical dilemma is defined as ‘a choice between two options, both of which will bring a negative result based on society and personal guidelines’. In research, this definition could be nuanced to suggest that an ethical dilemma presents itself when you have to make a decision to protect the interests of your research or the interests of your participants or study site. For example, in an interview with a senior manager you learn information that may be better off staying private and confidential, yet would also add an important and insightful dimension to your findings. What do you do? A participant in your study asks you for help, but to help might be to prejudice that participant’s responses in a later survey or interview, possibly skewing your data. Yet it is your job to help them. Study first, or job first? These are the kinds of dilemmas that can arise when you do research in the same spaces in which you work, and with people you work with and have other responsibilities to outside of your research.

Cheating-clients,-ethical-dilemmas

As researchers we have a duty to be as truthful and ethical in our research as possible. We are working to create and add to knowledge, not to simply maintain the status quo. In your study this may mean being carefully but resolutely critical, reflective and challenging, rather than only saying the palatable or easy things to say. This work is always going to present difficulties and dilemmas, but accounting as far as possible for your own bias and vested interests, and for your own relevant insider knowledge, can create space in your study for the development of your own reflexivity as a researcher, and can bolster rather than undermine the validity and veracity of your findings.

Trowler, P. (2011) Researching your own institution: Higher Education, British Educational Research Association online resource. Available online at [http://www.bera.ac.uk/files/2011/06/researching_your_own_institution_higher_education.pdf]

Fieldwork: to participate or not to participate…

This is my last post about fieldwork. This final one is about observations, and whether and how to participate or not participate in what you are observing. In my case I was observing classroom teaching, but I think these comments could also apply to tutorials, meetings, workshops – any kind of encounter where there is an opportunity for you to be present, watching and taking notes, and in some cases also participating.

I have read a little bit about participant and non-participant observations, and the relative pros and cons of each. I chose non-participant observation, and in the spirit of this blog, I want to add my own voice on what it was actually like to sit in lectures for 14 weeks and not participate very much at all, what eventually tipped me from being totally quiet to venturing a little participation, and why I think non-participant observation can be a challenge.

I decided not to participate in the classes I was sitting in on for one big reason and a couple of smaller ones. The big reason was that I had majored in the one subject I was including in my study (Politics), and I have worked for 4 years with lecturers teaching the other particular course  (Law) so I have come to know a fair bit about the knowledge and I find it very interesting. I was worried, in short, that if I participated I would ask too many questions or make comments that would in some way silence the voices of legitimate students or perhaps lead to the lecturer and I engaging in a conversation or debate in class that might exclude students. I have been in higher education as a student and tutor for a long time and these students are by-and-large in their first year of study. I felt I had no right, really, to come into their classroom and take up their time with their lecturers. 

One smaller reasons were that I thought I would be able to capture more accurate and objective fieldnotes if I was not too involved in the course. The more you participate, I reasoned, the more you perhaps want to agree with the lecturer, or the less you want to make a note of things that could be negative or less flattering, so your fieldnote data can be skewed or incomplete. I think that this ties in with my first post on fieldwork, where I talked about the Trowler and Williams’ articles on doing research in your ‘backyard’ and the possibility of finding out knowledge that can put you as a researcher in a tricky position in relation to your participants or your university/organisation. I felt that participating might tip my own personal scales in a too-subjective direction. I can’t here go into a full conversation about whether research like mine can be called fully objective (suffice to say it can’t be because there is always some researcher bias in qualitative studies like mine), but I will say that I was trying to record, as verbatim and as faithfully as possible everything that went on in the lectures without trying to pre-judge or pre-organise my data into categories or decide what had to stay and what could go, and I felt that being too involved in the lectures would hinder and further bias this process.

Another smaller reason was a little more vain: I simply wanted to be invisible. I didn’t want to call attention to myself because  after all my years of studying and teaching, I still get palpitations when I have to speak up in class or ask a question in a meeting where people will look at me. So I liked the quietness of non-participant observation, even though I had introduced myself to the whole class at the start of lectures and they knew who I was and why I was there.

However, being that quiet, especially when I really had a question to ask or an answer to a question posed by the lecturer, was really difficult. At times my notes record this frustration: ‘I really want to join in the discussion. So hard not to comment’. I felt, especially in Politics, that I had some useful thoughts to share, but I resisted the urge to call out answers because I felt it was unfair. I did this course when I was an undergrad so answering would have felt a bit like cheating on a test. Right at the end of both courses, though, I gave up resisting and I asked a couple of questions in Law lectures, and at least raised my hand to vote on issues in Politics although I did not ask questions there. I was nervous about doing that, but the lecturers included me as a student and did not offer any special treatment which allayed at least my worries about taking over a student space.

This year, I am participating a little – as a very-semi-participant observer – in my post-doc data gathering. I am doing it partly to try out a different way of doing observations like this, and partly because I have learned that limited and careful participation does not necessarily lead to the issues I was concerned about, like skewing my data or distracting the lecturer or muscling in on students’ space. But I do think if you are going to be a participant observer you have to be careful and keep a record of your participation in your fieldnotes. You need at all times to be the researcher first and the participant second. You need to check with those you are observing if it is okay, and to what extent you could or should participate.

It is in many ways easier and less fraught to stay silent in the background and just watch and make notes, but participating can be more fun even if it brings possible complications with it. It’s up to the researcher considering the situations in which data are being gathered to decide what will work best. Be pragmatic, take careful notes and be open, and don’t forget to tell your readers why you made the decisions you did when you get to your methodology chapter!

Fieldwork: making and transcribing field notes

This is the second post on fieldwork: this one is specifically about field notes – some thoughts on how to write them and how to transcribe them. I am still working this out, so it’s a thinking process in progress.

For my PhD, I gathered data largely from sitting in on lecturers’ classes and watching them teach, scribbling furiously during each one hour lecture. As you can imagine, over 14 weeks in two courses this ends up being a rather thick pile of notes. In my case it amounted to 5 and a half notebooks full of notes (over 500 A5 pages). These all had to be transcribed and organised so as to make sense of of them, and so that I could put them into NVivo10 to analyse them as part of my larger data set. Also, they needed to be typed up so I could copy and paste relevant pieces into my chapters as needed. I procrastinated a lot about doing the transcription. It’s not my favourite activity as a researcher. But I couldn’t have someone else do it because these notes were something I really needed to read several times, understand and sift through. I think, in hindsight and in agreement with Pat Thomson, that tedious as it is all researchers should try and do as much of their own transcription as possible because of how involved it enables you to become with your data. It makes the analysis process more enjoyable and productive too.

I thought what I would do in this post is list some of the things I did, why and what I learnt along the way in the hopes that you may find it useful if you, too, are gathering some of your data this way.

1. I handwrote my notes for two reasons: the first is that in the one class students were not allowed to use laptops because the lecturers wanted them paying attention as part of their training for the profession they will eventually enter, and the second is because I write faster than I type and writing is quieter. I tend to bash my keyboard a bit, and I did not want to distract other students or stand out too much by using equipment they were not allowed to use. A further plus with handwriting that I learned along the way was that I could easily copy diagrams lecturers put up on slides or drew on the boards, and I could represent what they were saying pictorially or non-linearly as well, which was often quicker and easier and made my notes feel more authentic to me.

IMG00075-20130612-0930

2. Pat Thomson wrote in a blog post a while back that she takes lots of field notes, and that she tries to capture verbatim what is going on as much as she can. I tried to do that too, and often was able to succeed in bits in one of the two courses because both lecturers talked fairly slowly and deliberately and paused for students to ask questions or take notes, so I could keep up. This was harder in the  other course where the lecturers spoke quite fast. I am having this issue now, too, in my post-doc research where one lecturer in particular is a very fast talker. However, I am much more comfortable with my theoretical framework now, so I can note particular kinds of phrases or instructions or comments that he says because I know I will use them in my analysis later on. However, I want to avoid being too selective in my hearing, because I don’t want to pre-empt the data or tell it what to tell me. I want to be surprised by it too, and find things I am not necessarily anticipating. Thus, I try now as I did last year to write as much as I can of what is going on in the moment, and can then sift the notes later and reorganise them during transcription.

3. I developed a shorthand: lecturers’ initials for the lecturers involved, like CA or BM (not their real initials). I also used S for student and tried to keep track of students’ questions or inputs where I could hear them clearly (the classes were large and often noisy). So I would have S1, S2 and so on engaging the lecturer in conversation or debate.

4. I didn’t transcribe everything. By being such a procrastinator about the transcription of the field notes, I ended up transcribing them while I was also beginning to analyse the data, so in a way this worked out well because it meant I had a sense of what I needed and what was just additional information that was unimportant, like comments made about admin issues, or comments I wrote about the lecturer telling the students off for not coming to tutorials the week before. Thus, I did not need to transcribe every word I wrote in my notes, and what I did was to read through my notes first, quickly, to remind myself of what I had written, and noted things I could excise. I learnt that it is okay to cut bits of your notes out and just not transcribe them. You won’t analyse ALL your data.

5. I had to do some reorganising when I transcribed my notes. Field notes are very in the moment – you are just trying to keep up and get it all down as faithfully and fully as possible, and you don’t really have time to think. When you go back to transcribe, though, you do have time and you can see how you can transcribe your sometimes chaotic and messy notes to impose a little more order, often needed in data analysis, and also how you can represent your pictures and scribbles in words so that you know what you mean, and can show readers what you mean if you use those examples in your chapters. I think that you need to be careful with reorganising, though, because you don’t want to rewrite history and make things that were chaotic seem simple, or things that were challenging seem easy. You would be skewing or tampering too much with your data and this would be unethical. It may also rob you of some potentially interesting findings. However, a little reorganisation that makes the notes easier to read and easier to represent to a reader, while staying true to the original scribbles, may sometimes be necessary.

I think the biggest thing I learned, and am still learning, is that it is an ongoing process of learning how to write these notes well, and how to collect rich and interesting data in ways that will be usable and make sense to me later. Stop every few notes and look back – reflect on what is working and what is not, and try to use that reflection to improve your taking of field notes. Capturing them can be tedious but field notes can also give you many-faceted and rich data for later probing and analysis.

Fieldwork: custom, character and questions of ethics

I have been thinking about fieldwork a lot lately, and how to improve on what I did with it during my PhD because I am doing it all again, post-doctorally. I have started a new, connected research project which I will probably write about later on, and I am wondering if the way I am doing my fieldwork is the best way. I am not really doing anything too different yet, and I’m working in the same two departments although with different lecturers. This will probably be one of three posts thinking through different aspects of doing fieldwork, so I’d like to start with considering the question of ethics, and the ethical behaviour of researchers in ‘the field’.

Fieldwork is generally defined as ‘an investigation or search for material, data, etc, made in the field as opposed to the classroom, laboratory, or official headquarters’ (http://www.thefreedictionary.com/field+work).  I gather data in classrooms, and in conversations with lecturers, and from documents. This year I am also adding student voices to the mix. This may not fit this definition but for the sake of using one word instead of two, and because it tends to be a ‘catch-all’ term for this phase of a research project, I am going to use the term fieldwork to talk about the phase of gathering different kinds of data from different sources, even is a classroom or lecture hall.

The lecturers I worked with last year and the ones I am working with this year are colleagues. I know them and they know me, and we respect and like one another professionally and personally as well. So it is very important that we can work together well, and that I am ethical in the way I behave as a researcher because that will affect the way I am received as a colleague. There are several benefits to working like this: access to data is much less complicated – I have been welcomed with open minds and arms, and they are interested in what I am doing; I can ask questions after class and even in class if I want to, so I feel like part of the environment rather than a detached outsider (more about this in another post perhaps); I am really enjoying this work a great deal because these departments are so interested and welcoming.

However, there are also drawbacks. The biggest, for me, stemmed quite specifically from the kind of data gathering I was (and am still) doing – what Paul Trowler calls doing research in your own ‘backyard’ and learning what Kevin Williams has called ‘guilty knowledge’. I work at the same university I gather my data in, and I do other kinds of work from time to time with the lecturers who are talking to me and letting me into their classrooms. So, I am not a detached outsider. I am part of this environment, and it is thus a challenge to try to be more objective about what I am seeing and thinking, and not get too emotionally involved with the courses or the lecturers and students and therefore end up skewing the representations of my data, or omitting important observations because they may not paint the lecturers or students in a good light.  Williams especially talks about this in his paper – he argues that doing research with colleagues in your own university, in a place in which you have invested part of yourself, is difficult because sometimes you learn things you are not sure you should disclose, or dig into deeper. This can leave you and your research in a tricky place, as your professional identity as a staff member can conflict with your identity as a researcher. Choices may have to be made, and this is where custom can clash with character.

Essentially, the literature on research ethics talks about ‘custom’ as being chiefly about the forms you fill in and the ethical protocols you agree to abide by. These are the standard ethical rules to live by in your field. ‘Character’, on the other hand, is to do with how you behave as a researcher when confronted with guilty or difficult knowledge or situations that present you with ethical dilemmas. This is an important distinction. I filled in forms, and got ethical clearance and promised, quite truthfully, to abide by the ethical rules laid down by my university. But when I got into the field, I was confronted by a couple of dilemmas that those forms and rules did not necessarily help me to solve. I had to call more in my character as a researcher, reflect very carefully on the dilemma, and speak to my participants openly about the problem yet without compromising myself or my research in that process. I had to rely on character, rather than on custom, to get me through and to keep the integrity of my research project intact.

This was not easy, but through this part of my fieldwork phase, I realised that while the rules and protocols are there for a reason and need to be observed diligently, there are also things they cannot account for. It is when these unexpected twists and turns arise that you need to call on your own character as a person and as a researcher. You need to cultivate relationships, as far as you can, with your participants that are open, so that when difficulties arise that include or affect them you can share these and reach an understanding, solution or compromise as needed. Share with them, if it’s helpful, pieces of what you are writing and get their feedback. Show them your classroom or interview transcripts, and ask for their input and whether they would like anything omitted. Discuss their requests for omissions or changes with them openly, especially if they may compromise your research. Talk about this in your methodology, so your reader knows what happened too.

It’s important to actually be ethical, and not just to say you will be, and it’s important to realise that things don’t always go according to plan in the field, so having an ethical and upfront character and approach to your research will stand you in good stead in case the unexpected is part of your journey too.