Tagging your way to success: How to make sense of your qualitative research

User research often leads us down long and windy paths of enquiry. Unlike quantitative data that can be neatly categorized, ranked and measured, qualitative data from interviews and user tests can quickly become hard to manage.

Whether you’re a notebook, Post-it or audio recording kind of person, collecting and making sense of your qualitative research can be a challenge. Insights can easily become buried in stacks of notes and observations, making the process onerous and time consuming.

How do I make sense of it all?!

When you’re strapped for time, qualitative analysis can quickly become unwieldy. 

The good news is that with a bit of upfront planning, digging into your data can become a lot more manageable (and enjoyable) later on.

Coding, categorizing, normalizing, bucketing — turning your data into a more digestible format has many names. The end goal is to facilitate analysis by making your data easier to filter and manipulate, allowing you to quickly identify things that may be causing ‘delight’ on your ‘homepage’ or ‘frustration’ at ‘checkout’.

It also gives you the benefit of quantifying your qualitative findings, which grants you both enormous satisfaction and superhero status among your peers.

“Tags” are added to observations, and are essentially criteria that allow you to filter and organize your data later on. Coming up with the right tags and related definitions for your project can save you a lot of time and stress, and as we all know: happy researcher, happy life (or something like that…).

What tags are most useful?

Since the launch of the Reframer beta back in 2015, researchers from all over the world have been using it to collect and make sense of their findings from user interviews, usability tests and more.

What better place to go to understand what tags researchers find most useful in analyzing data? I enlisted Mila — our bright and shiny new data scientist — to look for any patterns I could find.

When looking at what tags researchers rely on most frequently, I started by looking at the usage of one of Reframer’s newest features — the ability to create tag groups. I first wanted to identify what tag group names were most popular.

Most popular tag group names

Unsurprisingly, sentiment reigned supreme. Sentiment tags, such as ‘frustration’ or ‘delight’ help in identifying positive and negative moments in our research, and make it easier to pinpoint areas where improvements can (and should) be made.

The second most popular group, ‘tasks’, helps to identify what it is that we’re testing.

Whether it’s a task, an activity, a screen in your prototype or a specific question you want to explore, tags help in linking observations back to your research objectives. I then looked at what tags were most popular.

Most commonly used tags

As for tag names themselves: apart from sentiment tags such as ‘positive’, ‘negative’, ‘painpoint’ or ‘confusion’, the most popular tags are those that refer to specific elements of the pages and workflows we are testing. From ‘search’, ‘navigation’ or ‘content’, through to devices and touchpoints such as ‘mobile’ or ‘email’ — there was a lot of variety.

In summary, tags are most often used to identify:

1. How the participant feels about their experience (sentiment),
2. What the observation is referring to (for example task, screen, workflow, question)
3. Devices or page elements specific to your product or service (for example content, navigation, pricing, mobile, email).

So what should you consider when creating a tagging strategy for your next project?

Setting yourself up for success

1. Keep analysis in mind

Think of your tags as variables that you’d like to use to filter your data later with in the process and work backwards. While you can’t predict the outcomes of your research, you can anticipate some of the core themes you’ll want to explore based on your objectives.

Consider some of the standard elements you capture in your sessions — whether it’s sentiment, significance ratings, touchpoints, devices used or pages visited.

By carefully considering your strategy once, you’ll likely find use for it in future projects, too.

2. For usability tests, predefine your tags at the start of your project

For task-based research (such as usability testing), you can predict some of the tags you’ll find most useful for your project based on your objectives. A group of tags that relate to sentiment is a safe bet. Using tags to identify the tasks or screens you’re testing, or the activities you’re asking your participants to complete is also a good idea.

By filtering by task and sentiment, you’ll quickly be able to identify the most common usability issues or frustrations encountered. Including a significance or severity rating can also help in surfacing critical issues.

Creating tags on the fly while note-taking can be a big distraction. Determining your tagging strategy at the start will help to keep things consistent and make it easier to make sense of your data later on. By combining related tags into groups, you’ll also give your tag list most structure and make it easier to navigate.  

3. For interviews, determine your approach once your data rolls in

In the case of more open-ended research such as interviews or contextual enquiries, predefining tags beforehand may lead to unintentional bias.

While you may add some relevant tags before you begin, refrain from settling on your tagging structure until you’ve collected all of your data. Taking some time to immerse yourself in what you’ve heard will help develop a set of tags best suited to the data you’ve collected.

4. Less is more

It can be tempting to extend the list of tags you’re using as you go, amalgamating tags such as ‘homepage’ and ‘confusion’ into ‘homepage_confusing’.  Convoluted tag combinations won’t make analysis easier, and will only slow you down.  

Consolidating your tags is a great way to tidy and refine your tagging system, and can easily be done using the ‘merge’ feature in Reframer. Creating groups for related tags (for example ‘sentiment’, ‘tasks’ or ‘devices’) will also help keep things organized. 

A poorly-thought through set of tags will double the time it takes to make sense of your data.

Create distinct tags that will be useful and informative in analysis. What is your participant doing? What device are they using? What is their goal? How do they feel about it? After all, there’s no point in tagging something that could easily be identified through a simple keyword search. 

How many tags should you create? It will vary depending on your project. With researchers creating an average of 3 groups consisting of 7-8 tags each using Reframer, 15-25 tags is a healthy number to aim for.

5. Make use of session tags

While tags can help you make sense of your individual observations, they’re also useful for identifying session level information, such as participant demographics, personas, user groups or devices used.

Session tags in Reframer allow you to automatically add tags to all observations captured within session, giving you more flexibility for segmenting your data across sessions later on.

Keep it simple, stupid

Whether you predefine your tags, create them on the fly or wait until you’ve collected all your data, simplicity is your friend. 

Not having a well-defined tagging structure leads to over complication and time lost. Create tags that are meaningful enough for you to easily make sense of them, without being too specific.

Making use of tag groups is a great way to help organize your tags, while reducing complexity.

It’s important to have a clear understanding what level of granularity you’re working with – sometimes more detail is needed, sometimes less detail is sufficient — a clear structure should ensure that you’re focussing on the right level of granularity for your project.

It may take a few projects to arrive at a tagging strategy that works best for you, but with time,  the process of discovering patterns in your data will become a lot more satisfying. 

Reflect, iterate and repeat.

Sounds familiar, doesn’t it?