Most tracking studies, most pre/post tests, and most CX projects are sub-optimal, because they use cross-sectional analyses, instead of a longitudinal approach. Here is an example of why a longitudinal approach is much better, taken from my experience as an elected councillor in the UK.

A local neighbourhood scored badly on a number of deprivation indices (income, health, education, crime etc). Over the course of ten years, a number of programmes were put in place and money was spent to improve the neighbourhood. During the ten years, the indices were measured and very little seemed to be improving.

However, when the community was investigated properly (longitudinally), it was found that large numbers of people had been helped by the scheme and one of the first things they did was move to a ‘nicer area’. As these people moved out, they were replaced by other people who faced the same sorts of problems they used to face.

Measuring the figures for the area, but not following individual people, masked the message. The positive impact of the schemes was visible when using a longitudinal approach, but not when taking cross-sectional measurements.

‍

Consider NPS

‍From the world of insights, consider the case of NPS, the widely used and often disparaged Net Promoter Score. Most NPS studies are cross-sectional; they take a sample of people and measure that sample’s likelihood to recommend a product or service. The people in one wave of the study are different from the people in another wave. That is what we mean by cross-sectional. If an NPS score falls, the assumption is that it is caused by some people moving from being high scorers to being lower scoring. The implication is that these unhappy customers are likely to take their business elsewhere i.e. churn.

Similarly, when the NPS score remains the same from one period to another, the NPS data suggests that there has been no switching between those who are Detractors or Promoters. However, when people conduct longitudinal analysis of NPS scores at the individual level, the picture is more nuanced.

For example, the analysis of Australian telco Telstra (presented at the AMSRS Annual Conference in 2018 by Matt Howley & Michelle Rogers) reported that a) there were times when no movement at the aggregate level masked some Promoters becoming Detractors and some Detractors becoming Promoters, and b) that over a period of years, a customer can move from detractor to promoter and back to detractor. The same phenomena are discussed in a white paper from MirrorWave, which reported in 2018 that, in their longitudinal studies, it was not unusual for an unchanged NPS score to mask 25% of customers becoming less favourable while 25% became more favourable.

‍

How Brands Grow

‍Another strong case for the use of longitudinal analysis (and of the weakness of cross-sectional analysis) is shown in Byron Sharp’s book How brands grow. Sharp uses longitudinal data to show how an individual’s patterns of consumption vary over time and that this information is an essential part of understanding the aggregate picture. The key data that Sharp draws on come from longitudinal panels, where it is possible to look at actual purchases over time. For example, Sharp highlights that a bank whose customer base is growing rapidly will still have defections; what matters is the ratio of acquisitions and defections. As Sharp notes “much defection occurs for reasons beyond the firm’s control e.g. moving house, no longer needing the service, being directed by head office, etc.” All of these effects are invisible when only using cross-sectional analysis.

‍

Avoiding Survivor Bias

‍One of the best-known biases in research is the survivor bias. A researcher wants to know why some dieters are successful, so the successful dieters are researched to find out what they have in common, and this becomes the recommendation. However, this research ignores all of the people who did the same thing and did not lose weight. To measure the true drivers of an outcome you need to follow a group of people from the outset, measuring those who drop out as well as those who achieve their goal. Only by comparing those who met the goal with those who started with the same aim but who did not meet the goal, will you understand the true drivers of success. This has implications for path to purchase research and for studies which try to find the drivers of an action. This sort of research requires a longitudinal approach.

‍

Why is so much research cross sectional, when there are other, more powerful alternatives?

‍There are three key reasons why most research uses cross-sectional analysis:

1. It is easy to conduct cross-sectional research
If you want to track the views of 1000 customers per month (for example as you roll out a new product and its marketing), you can simply commission 1000 interviews a month from an access panel. By using fresh sample each time, we can use all of those statistical tools we learned at school - which depend on independent samples – without having to worry about issues like ‘learning’ and ‘sensitising’.
2. Assuming that averages are sufficient
As the old statistical joke goes, “His head was in the freezer, his feet were in the oven, on average he was fine!” Averages have been the go-to tool for researchers since the dawn of research, but this habit has masked many important findings. A focus on people as people requires a move away from averages.
3. Assuming that people will not want to take part in longitudinal research.
The evidence from communities, from academic projects following people for decades, and from MirrorWave, shows that when it is done properly, people will stay engaged for years.

Times have changed and these arguments are no longer valid. There is now a variety of ways to conduct longitudinal research, including communities and bespoke tools such as MirrorWave (which has recently partnered with Potentiate). The use of averages was always an oversimplification, but now that brands have moved to customer-centricity and are evolving towards a more human-centric view, the ‘average customer’ concept is a myth that will hold insights back. Research needs to be at the individual level, i.e. longitudinal.

‍

‍Communities and Longitudinal Research

‍As we explained in ‘Communities – the next ten years’, the future of communities is all about long-term communities, and these communities will become larger and larger as their role becomes ever more central. Of course, communities will be used for wide range of one-off, tactical projects, such as message testing and concept development. However, they are also being used for a wide range of longitudinal approaches such as:

• Looking at key measures over time e.g. the NPS research mentioned above
• Changes in preferences over time, for example how do people start eating a different range of meals, drinking red instead of white wine, or changing their mode of shopping?
• Paths to purchase, following individuals as they move from one product choice to another, and avoiding the problems of survivor bias
• Looking at how changes in life stage impact needs and behaviour – for example marriage, children, retirement etc.
• Sparking and following conversations about nominated topics of mutual interest
• Following individual experiences through complex processes

Communities are the best way to measure relationships or experiences over time. If you are running a conventional brand tracker, you may be getting a good guide on brand recall, but you are unlikely to be optimising your relationships with customers.

‍

Want to find out more?

‍For a general introduction to where communities are headed, please see my previous post in this series, Communities – The Next Ten Years.

If you’d like to know more about how you can leverage the longitudinal power of communities, please contact us today and ask about our HX Communities solution.

‍