Research News Live

Automation, machine learning & AI in market research: the basics

We love to throw around terminology for new technologies in the market research industry, but do we really know what these terms mean? Saida Eldon and Horst Feldhaeuser do some defining.

Today’s most popular discussions centre around automation, machine learning and artificial intelligence (AI). It’s important for us to understand not only a high-level definition of what these terms mean, but also how they may affect our jobs as researchers.

By far the most widely used of these three technologies in the market research industry, automation involves setting up a process that does not involve any human thinking to run. In its simplest definition, automation is just a set of guidelines that machines follow to execute and deliver an output. In automation, all the environmental parameters are well known at the time of design and are not expected to change during the process.

This makes good sense for our industry, as automation can achieve compliance and accuracy, all while quickly executing tasks like data processing, sampling, and charting. It also increases productivity and decreases processing time – which can be a boon when trying to turn around consumer insights in a fast-paced environment. In short, automation frees us up from repetitive, time-consuming tasks and allows time to really dig into the data and think about its meaning for our clients.

Machine learning
One segment that is driving the growth of AI is machine learning, which is the ability of software to learn things. For us, it means that each time we interact with our data – applying our own tweaks, rules and exceptions – the machine learns something new and then automatically applies relevant preferences next time. Eventually the machine helps us to generate insights rather than just processing data.

One good example of machine learning within the market research industry is text analytics. The most advanced providers of text analytics automatically code and classify large samples of open-ended survey questions into themes and sub-themes, allowing for any level of detail. They can also score the overall sentiment into a gradual scale, rather than just ‘good’ and ‘bad’. By using machine learning in this process, data can be processed very quickly and consistently, unlike manual coding. Of course, there is a need for initial human input, to familiarise a machine with the terminology used in the client’s business. However, from that point, the machine then applies this learning next time it works on additional data.

Artificial intelligence
There are many different definitions of AI, even among the experts. Our preferred definition of this technology is: ‘AI is the simulation of human intelligence processes by machines, especially computer systems. These processes include learning (the acquisition of information and rules for using the information), reasoning (using the rules to reach approximate or definite conclusions), and self-correction.’ In short, AI tries to make machines or software mimic human behaviour, reasoning and intelligence.

One use of AI that is relevant to our industry is IBM Watson Analytics, an AI application that is used in various fields such as finance and healthcare to help users make decisions. Like Siri, Watson understands natural human language and comes up with an answer. In the healthcare field for example, it will suggest various types of treatment to doctors based on knowledge of things like a patient’s genetic profile and medical history.

Thinking time
There’s no doubt that these technologies have huge implications for our industry, well beyond the very real benefits of saving time and money. They can help us improve accuracy in processing, analysing, visualising and reporting, and finally give us the luxury of valuable thinking time by freeing us from process-driven workloads. These tools will also allow us to interrogate larger quantities of data more efficiently, which will lead more people and businesses to utilise the cheaper, faster and more reliable market research options that are available.

One application that Infotools is working on is ‘automated machine report writing’. At a basic level, the ‘machine’ will be tasked with updating, analysing and commenting on results, using a system that automatically updates when new waves of data are added. It can create trend graphs and other reports, either with the user’s specified list of items or automatically, using algorithmic data. From identifying all levels of change within the data, all the way to creating an audio-visual reporting experience and commentary, this technology promises to change the way we generate reports.

As for our jobs as researchers, our roles will start to shift toward the innately human skill sets like empathy, creativity and the desire for connection that are the cornerstones of strategic thinking and cannot be automated.

Automation, machine learning and AI mean good news for both agencies and end-users. These are exciting times to be in the insights industry – if we are ready to embrace the opportunities.

Authors: Saida Eldon, Client Services Engagement Manager, & Horst Feldhaeuser, Group Services Director, Infotools

This article also appears in the October-November 2018 edition of AMSRS publication, Research News – Research without researchers


Photo by Randall Bruder on Unsplash

About The Research Society 1063 Articles
The Research Society is the peak body for research, insights and analytics professionals in Australia. It has a diverse membership of individuals at all levels of experience and seniority within agencies, consultancies, client-side organisations, the non-profit and government sectors, support services as well as institutions and the academic community. As well as over 2,000 individual members, the Research Society has 125+ company and client partners, with the number continuing to grow. The Research Society research professionals and company partners commit to and are regulated by the Research Society Code of Professional Behaviour.