It is a well established fact that IT is reshaping just about every single field of human activity. But one thing that has remained surprisingly stable is research methodology in social sciences. Social science scholars still today focus on doing interviews and distributing surveys, as well as conducting the occasional observation study. Collecting primary data has not yet been disrupted in any way, despite its tremendously labour-intensive and fuzzy characteristics. Quantitative researchers still struggle with low response rates on surveys, meticulous item development and non-respondent analysis. Qualitative researchers still struggle with tedious interviews, labor-intensive transcriptions and subjective coding of interview data. Shouldn’t there be a better way to conduct rigorous data collection in our modern digitized society?
Reinvention of social science?
Digitalization in any human activity often follows a pattern of going from small adaptations, to larger transformations and finally to end up in complete reinvention. If we look at how research is conducted, we are arguably still at the adaptation stage. When distributing surveys we use a web-based survey engine sent through email. When conducting interviews we record what is being said by using a convenient recording app on our tablet or smartphone. When observing human activity we record it on video using a camera or perhaps a smartphone. But we are not even near a transformation of how data is collected, and the reinvention of social science research is nowhere in sight.
In my work I have stumbled upon a completely new way to collect primary data for research purposes. It’s not an interview method, but you can still ask follow-up questions to respondents in a synchronous way and take context into account. It’s not an observation method, but you can still observe what is going on between key social actors such as between a teacher and her students. It’s not a survey method, but you can still distribute it easily on large amounts of people and readily quantify their experiences. So, what is it? Well, that is the question I am currently struggling with. I am now writing an article on the topic, and this blog post is a way to get started writing text.
The data comes to me
What we are talking about here is an IT system that is possible to deploy at a large number of people in a context I want to study, such as an entire school or an entire incubator for entrepreneurs. It resembles social media, but it’s more private. It resembles learning platforms, but it’s way less administrative. It is similar to a messaging app, but it has more structure and also a few mandatory quantification steps in each transmission of text. So, while it is similar to many things, it is equal to none I have seen so far. The system we have built goes under the name LoopMe (see here), and we have tried to categorize it as “Social Learning Media”. We believe this is a new category of IT systems. I’ve done a video about it that you can watch here.
For me as a social science researcher it is really a dream come true. Once the system is deployed, the data comes to me. I can sit at a distance and observe what’s going on, and whenever I see something of interest I can intervene and ask follow-up questions in real-time. All the generated data is saved on a central database. I can download the entire dataset in an Excel sheet, and analyze it however I see fit. I can make really nice Pivot diagrams showing causation in terms of which activity led to which kind of perceived outcome. This does not preclude offline real-life encounters, but can actually empower such meetings. I have used Loopme as a sampling strategy by choosing who to interview based on information online. When I meet a respondent, I often prepare a semi-structured interview template where questions are tailored to the particular respondent. I can skip contextual questions, and go directly to the core issues relating to the research question I am investigating.
Martin’s 1000 eyes or just unrealistic hype?
The real challenge comes when I want to explain to other researchers what this method is about, and why it’s good. A research colleague commended the method and called it “Martin’s 1000 eyes”, since each user of the IT system becomes a participant observer working for me as a researcher, telling me whenever something significant happens in their natural setting. Another research colleague was more hostile, saying “Here you come with your little app, and everything will solve itself – that is an arrogant rejection of established educational research methods”.
While I have been using it now in my own research since 2012, I have not been able to get other researchers to use it in their data collection procedures. I really wonder what is stopping them, and I suspect it is my low ability to explain what is going on here, coupled with new unorthodox ways of working that scare people off. So I hope this text will turn into an article that makes sense and clarifies some things (also to me). A first step I’ve done is to try to categorize common methods, see image below. As far as I can see, LoopMe spans the entire field here, from the subjective nonverbal social communication you would capture in observation studies, through written accounts you get from transcribed interviews, to the objective individual quantified responses you would expect from a survey.
Win-win for all involved
The main reason it works so well is because it delivers a very tangible and appreciated value for the practitioners using it, which contrasts to surveys and many interviews that are done primarily for the researcher’s benefit. The Loopme system connects people in ways they have not experienced before, and this helps them a lot in their daily life. This is what gives the new method its observational quality – participants use it not for me as a researcher, but for their own benefit in their natural setting. This makes it even more odd that it is new – how could we come up with a new social media tool at the same time as we constructed a new kind of research tool? Well, maybe precisely because of this. When optimizing social media for research purposes, we ended up with some truly novel functionality. There is no equal solution for practice, resulting in the obvious conclusion that using such a solution for research purposes is something that most likely has not happened before.
Capturing the “flow” of experience
The most similar development I’ve seen within research is a couple of IT systems that try to facilitate for researchers working with a method called ‘experience sampling’. Experience sampling is when short surveys are used to capture respondents’ experiences directly in their natural environment, attempting to capture the “flow” of everyday experience. There are a few companies out there developing such tools, such as Metricwire and MovisensXS. But there seems to be no thriving community of MethodTech companies, pushing the boundaries of data collection, and redefining what it means to be a researcher. That is a missed opportunity in my opinion.
I’ve also found a small community of ethnographic researchers discussing how to use Social Networking Sites (SNSs) for research purposes (see for example here, here and here). Facebook is the most common platform. Some ethnographers have viewed Facebook posts as empirical data. Others have used Facebook for snowball sampling in order to find respondents that otherwise would have been very difficult to find. I have however not found any example of an ethnographic research team building their own SNS tailored to the needs of such research. This leaves ethnographers with a number of unsolved dilemmas, such as how to protect participants’ privacy, and other fully legitimate requirements from ethical committees.
Towards a MethodTech community?
In our increasingly digital world there are endless opportunities to collect data in novel ways. But established solutions used by practitioners for communication purposes, such as Facebook and others, are currently almost never optimized for research purposes. While we have EdTech, FinTech, AdTech, GreenTech and WhateverTech, we do not yet seem to have MethodTech. We scholars will have to create it ourselves. Who wants to join me here? The first step is to be able to explain myself. So if you get what I’m writing about here, let me know. And if you don’t get it, let me know too! I write this blog post in hope of getting some help in explaining myself and growing this non-existent MethodTech community from one (?) single researcher. Anyone else out there?
