AI in market research: your questions answered

Artificial intelligence is a branch of computer science that aims to enable machines to make highly complex decisions – autonomously.

Although Artificial intelligence systems are not replicating human intelligence, they can make similar decisions to humans (in the right situation) much faster e.g. While a person can brilliantly recognise the contents of a picture or the meaning of a sentence, they cannot process thousands per minute (or even second)!

Our exploration has shown that the bar is not set as high or as ‘technical’ as you might imagine when starting out on your AI journey… which is why it’s becoming increasingly common to hear people shouting about their “AI” solutions. Even simple macros or decision trees with clever applications can be classified as AI, so the ability to probe a little deeper to understand solutions is key.

There are two main forms of Artificial Intelligence:

1. Expert systems are based on a system of rules giving guidance on reaching specific outcomes. While they are based on a great deal of past data and experience, they tend to be fairly rigid in nature (in practice varying from simple algorithms to VERY complex ones e.g. the decisions made by a self-driving car, or autopilot in a plane)
2. Machine learning is a form of Artificial Intelligence where data is used to formulate and then improve this decision-making process. Instead of being based just on hard and fast rules, the computer can formulate its own decisions based on patterns it uncovers (e.g. Speech recognition used by Alexa, Google Assistant etc.)

Machine learning in market research often utilizes image content analysis. Here, a system is given a large number of coded images to predict future images.

Brands are under increasing pressure to get to market quickly

• This is vital to tap into dynamic consumer trends and maintain competitive advantage Consumers are demanding more!
• Driven by Gen Z, but increasingly the case across all age cohorts, expectations from brands and the desired product experience are upshifting
• Skipping consumer insights is not an option Whilst undoubtedly a way to save time in innovation, exclude hearing from consumers at your peril!

The benefits to CPG companies of embracing AI are being heavily lauded; “We found that by using AI and advanced analytics at scale, CPG companies can generate more than 10% revenue growth through more predictive demand forecasting, more relevant local assortments, personalized consumer services and experiences, optimized marketing and promotion ROI, and faster innovation cycles” – from The Boston Consulting Group following joint research was undertaken with Google found here.

At the same time, CPG companies also have access to more data than ever and this is only set to continue so businesses are increasingly looking for ways that AI and ML can make sense of this information, improve business processes and efficiency and ultimately develop more compelling and competitive products.

Based on our experience to date (both self-funded and work with our clients), the biggest opportunities come from harnessing AI to:

• Provide broader, more holistic context via access to more comprehensive inputs
• Access the ‘voice of the consumer’ at scale
• Facilitate a more open, exploratory approach and conversation with consumers (without needing to predefine themes or questions)
• Improve the survey participants experience (giving better data quality) via tailoring and personalisation
• Ask fewer ‘questions’ but get a better understanding (smarter inputs and predictive analysis)
• Process large data sets efficiently and quickly
• Benefit from overall cost and time savings (‘more bang for your buck’)

We’re capitalising on in house data sciences expertise and working with exciting tech suppliers to develop exciting new solutions across these areas.

For simplicity, we tend to split AI into the following simple buckets when investigating new tools and adopting elements into our own toolkit:

1. INPUTS – gathering consumer and broader inputs more efficiently / effectively
2. OUTPUTS – making more of the data and information available

Often these solutions will go hand-in-hand, but it helps keep us focused on what the tech or AI solution needs to deliver and not get carried away by the latest cool capability.

Our in-house tech innovation function, Nova, invest heavily in experimentation. They spend time trawling opportunities & suppliers and pressure testing new approaches – so you don’t have to!

See them as an ‘honest broker’, looking for solutions with clear added value and avoiding tech for tech’s sake. We keep a close eye on developments and are actively exploring a host of additional AI capabilities to understand their potential application in product & packaging development research.

AI can only learn from or react to the data that it is given. Therefore, obtaining the right source of information for these methods is absolutely crucial. If the algorithms are trained on data containing biases, then these algorithms will mirror them in the decisions they make.

It is important that input data is representative of the purpose the algorithm will predict for:

• Elements like research method, product set and regional differences can influence research results
• Care must be taken sourcing data from social media where views can be extreme and not representative of the population as a whole

More data is not necessarily better!

• It can be tempting to include as much data as possible
• However, including unrepresentative data increases the likelihood of finding spurious links
• Use of advanced methods and processing power still requires careful consideration of data inputs and examination of the data
• Having expert category knowledge is an important complimentary skill to apply on top of these methods
• We recommend a deeper interrogation of AI systems to pressure test the data inputs and the code/logic behind them when selecting solutions.
• We’ve had impressive and disappointing experiences with suppliers when we’ve ‘lifted the hood’

Whilst AI undoubtedly offers huge advantages to our industry, we must also be mindful that it cannot understand or sense check content in the same way as humans (and can make mistakes). Many elements of the research process can be greatly enhanced with the use of AI. However, it is important to understand how decisions are made in an algorithm, and not rely upon mysterious ‘black boxes’.

• Algorithms do not understand the content of an image or piece of text. They are based on numeric pattern matching and so they can be distracted, ‘confused’ or focus on spurious elements
• Proper diagnostics, and an understanding of the algorithm’s decision-making process, are important for the results to be trusted
• Regular assessment of AI results and underlying diagnostics will ensure algorithms are functioning well, giving you safe and reliable predictions
• Whilst an attractive proposition, we’re yet to be convinced that AI can fully negate the need for consumer opinions, particularly when it comes to product optimisation

AI and Machine Learning undoubtedly offer exciting opportunities to better engage with consumers, drive the scale of input (in terms of depth, breadth and type) and generate robust and targeted outputs. With a range of approaches already capitalising on AI and more in the pipeline, we’d love to work with you to identify the benefits to your innovation process.