This post builds on my recent article “Global Macro Trends Impacting the World of Insights and CX” by drilling down into what is happening in the world of insights and research.

The Participant Crisis

The number of people willing to answer surveys or take part in qualitative research has been falling for years, but it has moved from being a nuisance, to a problem, and now to a crisis. The two key drivers of this decline and crisis are:

1. People are being asked too often to take part in surveys, especially with the rise in the use of CX. This is simply a function of more companies wanting to hear from customers and from a growth in the collection of ‘in the moment’ information.

2. The number of long, boring, and badly-designed surveys.

A second part of the participant crisis is the continued growth of rogue respondents. If your study uses an online panel and/or if it uses incentives, then there are people out there trying to take the survey as many times as they can in order to collect the rewards. Studies by Coalition for Advancing Survey Excellence (CASE) and reported by the Insights Association, suggest that some of these rogue participants are doing more than 100 surveys a day and that 30% to 40% of some data is suspect. This problem is likely to be worst when using cheaper sample sources and when using black box solutions, where the user is not selecting the sample source and does not have options to clean the sample themselves.

The key remedies being worked on include: work by the panel companies to combat fraud, more research being shifted to online communities (where clients know who the participants are), and moves to make CX projects more engaging (shorter and less boring), more targeted, and more relevant (e.g. by providing more feedback about the outcomes of their responses). The movement to focus on Human Experience is part of the solution. If we look at the survey-taking process from the view of the customer, rather than from the perspective of the company, then we would not ask too many surveys, with too many questions, badly scripted and targeted.

Democratization of Insights

More people are accessing insights and more people are creating and conducting research. This reflects the growth in customer-centricity and the increased use of evidence-based decision making. The change has been enabled by the massive growth in the number of platforms, variously referred to as DIY or self-serve. Studies by ESOMAR have suggested that more than 50% of all research projects conducted globally are being conducted internally by end-clients, rather than by research agencies. Further, the proportion of research conducted internally is growing and it is growing through projects being conducted by people without insight and research experience and training.

Whilst there are some who are trying to fight this trend, the simple truth is that this trend will continue. The role for insight teams is to shape this process, to be the coaches, to be the subject matter experts, and to help determine which tools are used and how they are used. The role of agencies is to focus on those things that cannot readily be run internally – for example, large projects, complex techniques, specialist samples and such.

The Growth in the Need to Show the ROI of Insights

Research and insights have been too slow in showing the ROI of research. This seems to have been based on a position that says “Since we can’t do it perfectly, we can’t do it”. However, the trend is towards insight having to prove its ROI, and the industry has been brought dragging along.

There are a wide variety of approaches that can be taken to establish the ROI, as I spell out in another post “How do we measure the ROI of Insights”. Insight teams and agencies need to get on board with this trend, or the measuring of the ROI will be conducted by third-parties.

A Move to Simpler Techniques

Every year sees new developments in research techniques, including biometrics, neuroscience, automated semiotics, etc. However, the key trend is towards an ever-larger percentage of the research total being conducted via simple - often very simple - techniques. This shift towards simpler research is due in large part to the democratization of research. More people are conducting research, they have little training, so they gravitate sensibly towards simple techniques.

At its best, simpler research can be used as an iterative tool in conjunction with agile product development to help create great outcomes. At its worst, simple research consists of asking the wrong questions, to the wrong people and believing the answers to be 100% useful.

By contrast, the trend within agencies is to develop the more complex solutions that are not so easily brought in-house – a sensible response to the democratization of insights mentioned above.

The Consolidation of the Industry

Over the last year, there has been a massive number of mergers and acquisitions in the insights and research industry. This has been fueled by large amounts of money entering the industry. If you check MRWeb, you will see new announcements of mergers and acquisitions almost every day, and often multiple changes on the same day.

The drivers of this change are mainly 1) investor money with no better places to be used, 2) the focus on democratization and platforms, 3) the need to create integrated solutions.

This rise in consolidation has several implications. Suppliers and agencies are becoming more focused on valuations, rather than traditional metrics such as profit. Companies are operating with a heavier burden of debt – making it harder to make good profits. More people with no research knowledge are making decisions at board level.

So What?

Here are four suggestions to the points raised in this post, for four different groups of people.

For client-side insight teams

Become the coaches and managers of the democratization process. Don't fight it, it is going to happen. Make it better, make it good, make it useful.

For agencies

Don’t compete with the self-serve platforms, you can’t make your staff work ever harder for ever lower wages and expect to beat an algorithm. Focus on doing what the platforms can’t do. Your job is to help clients make better decisions, your job is not necessarily to conduct surveys or facilitate online diaries and discussions.

For suppliers to the research industry

Most of you will not succeed, the nature of innovation and competition is that many seedlings are established so that some will prosper. In terms of business advice, most of you should make your system as easy to use as possible – power users are a rarity.

For researchers

In general, learn how to use the platforms so that you become the user rather than the victim when the platforms succeed. Understand how research helps businesses make better decisions. Finally, learn one or more specialty, for example, qualitative analysis, choice modelling, ethnography, storytelling, videography or another useful skill.

‍