“We completed our quantitative survey, but we don’t understand why people responded in the way that they did.” I hear bridgebuilding colleagues express this frustration regularly. It does not mean there is something wrong with the survey. It means there is a limit to what numbers alone can tell us. There are some questions that quantitative data just isn’t designed to answer.
In bridging, and the broader work of nurturing belonging, we talk a lot about data for evaluation and learning. Practitioners and funders need to know whether and how their programs are contributing to meaningful change. As a field, we are currently seeking many of our answers in the form of numbers: how many people participated, how they rated the experience, and the extent to which attitudes toward difference have shifted. When we rely primarily on numbers, we can miss important information about what is actually happening within our communities—such as the experiences underlying our survey data.
That matters because bridging takes place in dynamic, context-specific environments. What happens in one program or place may not look the same in another. Change may take forms we didn’t anticipate, and the factors shaping change may themselves be shifting. Understanding these changes involves qualitative data. Ultimately, our choice of methods should depend not on our preferences, but on the learning questions we are asking. Many of the questions that are central to bridging cannot be answered solely by numbers. We need more qualitative data, alongside and integrated with our quantitative methods.
This post kicks off a multi-author series to introduce a range of qualitative approaches suitable for bridging and belonging work in the USA. I’m Michelle Garred, and I’ll anchor the series, drawing on more than two decades of experience innovating qualitative approaches in contexts of intergroup tension, polarization, and marginalization, both domestically and internationally. Along the way, we’ll hear from exceptional co-authors Abbie Haug, Ella Duncan, Allison K. Ralph and many more.
Broadening your data toolkit
Quant data can tell you how far and how fast. Qual data can help you understand the terrain, clarify your direction of travel, and navigate the trip.
We are fortunate to have a wealth of quantitative (“quant”) tools to inform the work of bridging! Quant data shows up in the form of numbers and statistics. Like all approaches, it has both strengths and limitations. Quant data is particularly useful for identifying patterns at scale and estimating their magnitude; for example:
- How much / how many instances of a particular type of change have happened within this project?
- Which types of people experienced that change (and which did not)? Under which circumstances?
- To what extent might these patterns also apply beyond the people or settings included in the study?
The numbers provided by quant data can feel reassuringly clear. However, while numbers can tell us how much of something is happening, someone has to first decide what that “something” is. Qualitative (“qual”) data, in the form of words, narratives, observations and images, can be particularly useful for identifying the “something.” Quant and qual data can complement each other.
Let’s think of your program as a journey in a vehicle, say an EV or a bus. Quant data can fill your dashboard with important numerical measures, including how far you have travelled and at what rate of speed. However, you will also need qual data to help you understand the terrain, clarify your direction of travel, and navigate the trip. Qual data is less familiar to many of us, so let’s explore it further.
Why add qual data?
Qual data brings at least four key contributions to our work.
Qual data can identify what is changing as a result of your program. For example, are participants changing their community engagement patterns? Are organizations developing new forms of collaboration across lines of difference? In what specific ways?
This information can shape decisions about which types of changes are worth measuring at scale using a quant survey. Indeed, a major limitation of qual data is its weakness in answering the questions of how many or how much, especially on a large scale.
For example, I am currently supporting a qual exercise within a program focused on changing narratives about gender diversity. The qual data will be essential for telling us what narrative shifts are emerging, and how. If we later need to know how many people across the state have experienced those shifts, we will follow up with a quant survey to measure the changes identified through qual research.
Qual data on what’s changing also makes it possible to:
- Identify behavioral shifts, which are an essential pathway to social change.
- Identify changes that may be unexpected and sometimes undesirable.
- Make adjustments to fine-tune the effectiveness of your program.
Qual data can reveal why and how change happens. When qual data includes numerous accounts of change, it can be analyzed for patterns in how your program contributes to change. It is possible to learn which elements of your program appear to be making a difference, or explore the consistent characteristics of how change unfolds. This is essential information for refining your program strategy.
Qual data can illuminate what change means in context. It allows us to unpack how participants experience and interpret change in different contexts. For example: Which changes matter most to participants, and why? Could the same type of change look and feel different depending on participants’ identities and experiences? What aspects of the change process are context-specific? Which aspects of the broader context appear to be shifting? And what might those shifts mean for our program planning?
Qual data also helps us navigate contexts that are complex. I use “complex” here as a technical term. It refers to socio-political systems in which many actors and factors are interconnected and continually responding and adapting to one another. Change is often multicausal and nonlinear, and new patterns can emerge without anyone intending to create them. In complex contexts, it can be difficult to predict how the system will respond to an intervention or evolve over time. We experience such uncertainty right here in the USA.
And yet, you still need information to navigate decision-making. In a complex context, it can be risky to rely exclusively on assumptions derived from theory, research, or other programs. Those are important sources of insight – but there is no guarantee that what was true in another place and time will hold true in your own circumstances. Locally collected qual data can “reality check” those assumptions by revealing what is actually happening within your own current context. Qual data can also help us identify changes in the context that our original measurement framework did not anticipate.
Qual data can make equity possible through listening. Quant surveys require your participants and partners to choose between preset options in order to express themselves. Speaking for myself, when responding to a question, I often end up choosing “none of the above” or “does not apply.” When the response options don’t align with my experience, I don’t feel heard. The greater the difference in culture or life experience, the more likely the mismatch.
On the other hand, qual approaches tend to ask open-ended questions that create space for people to share their thoughts, ideas and concerns more completely. This allows people to exercise their own voice. Equity has many facets, and listening alone is not enough to make an evaluation equitable. But creating space for people to express themselves in their own words is an important starting point. Qual approaches can make that kind of listening possible—and that is a possibility we can build on.
But what about…?
At this point in the conversation, perhaps you are getting more interested in qual data, and wondering what is feasible. Let’s consider a few of the toughest questions right now.
Which is more credible: quant or qual research? Neither is inherently more credible. Both can fall anywhere along a spectrum from low to high credibility. In both cases, credibility depends on researcher decisions: deciding what to ask or measure, posing the questions appropriately, analyzing the data transparently, and interpreting findings in ways that acknowledge limitations and alternative explanations.
Which is easier: quant or qual research? Neither. Both disciplines require training, practice, and skill, at a level proportionate to the goals of the research. It takes time and effort for someone skilled in quant to develop qual skills, or vice versa.
Quant research sometimes offers the option of using “off-the-shelf” surveys or measures pre-designed and tested by experts. This is an important resource! At the same time, those measures aren’t necessarily the best fit for your particular context. And adapting them to better fit your context may have unintended effects on what—and how credibly—you are measuring.
Does qual research take longer? Sometimes. Qual can require significant time for data collection and analysis, especially when you are collecting in-depth information or performing a multi-step analysis. But quant can also be time-intensive, particularly when the sample size is large, or the data collection and cleaning become complicated. Neither approach is consistently faster; the timing depends on the research design.
The Takeaway
I want to encourage you to think about how qual and quant data can work together – to embark on your metaphorical programmatic journey with a map and compass, in addition to an odometer and speedometer. Quant data tells us how far and how fast; qual data helps us understand where we are and how to navigate.
Adding a data stream can feel like more work, which is tough when you’re busy. So it helps to recognize the multi-purpose nature of qual data. Qual increases the value of your quant efforts by identifying what’s worth measuring and revealing the “why” and the “how” behind the measurements. It also enables you to navigate your own complex context, adapting your program accordingly along the way, while building in listening as an equity practice.
Finally, qual data can also surface meaningful stories and testimonials for sharing with funders or potential future partners. To be clear, evaluation, fundraising and communications should remain separate functions – but, with appropriate consent protocols in place, they may draw on the same underlying data source. For many of us, making this a reality will involve learning more about qual data.
Reflections for funders:
- Are we asking our grantees to measure pre-identified changes—or giving them space to discover what meaningful change looks like in their own unique context?
- Are we supporting our grantees to make the most of both qual and quant data? Or are we unintentionally constraining their choices?
A qual-focused blog series
Starting next month, this multi-author series will introduce a range of qual approaches that hold promise for informing the work of bridging and belonging. The range and diversity of qual approaches may be greater than you ever thought possible. So let’s skip qual interviewing, an important bread-and-butter approach that many of us already know, and dive right into what’s potentially new and different.
Upcoming posts will include the following qual approaches:
- Outcome Harvesting
- Most Significant Change
- Ripple Effect Mapping
- The Qualitative Impact Protocol (QuIP)
- Deepening Contribution Analysis
- Participatory Action Research
- Participant Feedback Mechanisms & Loops
In each post, I will team up with a highly experienced colleague to unpack a particular qual approach, including an overview of its strengths and limitations, insights on when to use it (or not), in-depth real-life examples, and tips for funders. I anticipate many interesting conversations along the way – and I’d love to hear from you at any time.
Gratitude: Ella Duncan co-shaped this post through wise feedback. Vivian Mendonça coined the title. Daria Seriakova made it all look beautiful.