How Reliable Are Neuromarketers’

Measures of Advertising Effectiveness?

The background: “Neuromarketing” refers to the measurement of the neurological as well as physiological signals
like blood flow across the brain, brain cell activity, eye tracking, pupil dilation, facial expression coding, heart rate,
respiration rate and skin conductivity to observe the response of the customer on various aspects of marketing like
product development, advertising, pricing etc. Such measurements help understand the customer motivation,
preferences and decisions when exposed to various product details and help creative advertisers design better
advertisements as well as help marketers take better decisions on product positioning, pricing, product
development etc.

The excitement: In the mid-2000s, when the researchers began to demonstrate that advertising, pricing, branding
and other marketing aspects can have measurable impact on brain, it aroused a lot of interest from the marketers.

The challenges: However, off late, there has been an increasing pessimism about the field. The reasons are
multiple – high cost of the tools involved, a growing belief among marketers that neuroscience tells them what
they already know, or it tells them about something which they really don’t care! Many advertisers complained
that the vendors providing the neuro data using Neuro 1 specifications often used proprietary methods which
were not available to them to understand the results. With Neuro 2 specifications, the issue of lack of transparency
was attempted to be addressed with specifications clearly stating what methods to be followed for collecting data.
However, when the advertisers approached the vendors for data, the vendors expressed that they did not have the
in-house capability to use the predictive measures as identified and prescribed by Neuro 2. For example: fMRI was
a method which required fMRI machine which costed around $5 million plus additional overheads of running and
maintaining. Also, vendors argued that the methods prescribed by Neuro 2 did not apply to them and that they
used proprietary methods instead of those prescribed in Neuro 2.

More challenges: Along with these issues, there was confusion among marketers on whether to use the traditional
method or the “new” neuro method for their marketing needs. The ARF’s Neuro 1 report, however, clearly stated
that just because neuro measures were new, there was no reason to discard the traditional data, which was based
on parameters like sample size, sample location, composition, segmentation etc. The report also mentioned that
the buyers of the data should establish the credentials of the people taking the measurements, the reliability of
the machines being used, credentials of the interpreter of the data etc. It also recommended buyers to use more
than one vendor for sourcing the data, thus further making it hard for the buyers to adopt neuro for their
marketing decisions.

Search for a solution: A study “How Reliable Are Neuromarketers’ Measures of Advertising Effectiveness?” by
Duane Varan of Murdoch University and Annie Lane of Indiana University, published in Journal of Advertising
Research in June 2015, showed how buyers can compare the validity and reliability of the vendors’ approach. The
study provides new recommendations for the vendors as well as the buyers to address some of the ambiguities in
the current process.

Methodology: The most important question for the buyers was which neuro measure offers reliable and valid
answers to their marketing questions. Varan and Lane believed that Neuro 1 had this answer. They took eight
vendors and gave same eight 30-second commercials to each of them. Vendors used their standard methods on a
US population aged between 18 and 49 years in November 2010 to measure 5 Neuro 1 measures – EEG, fMRI,
Facial coding, EMG and Biometrics. The vendors measured these measures along with some traditional self-report
measures like eye tracking etc. The neuro data was processed by vendors to report attention, positive emotion and
arousal. Their research was assessed by a team of 12 subject matter experts coming from different countries and
six members in a senior review panel.
Findings: The team found that the Vendors rarely provided any information on controlling the lag factor (time
delay when the simulation was provided and the effect was seen in the neuro measure), their recommendations
depended on subjective interpretation of the results, sample sizes were too small, location was also a matter of
concern. Each of the vendors had some of the proprietary methods which they used to help arrive at conclusion as
well as reduce noise in the data. However, in spite of their claim of having proprietary methods, there were
similarities across many of their methods and their constructs.

One of the main benefits of neuro measure was that it provided the continuous measures of consumer response.
Study however revealed that no two vendors’ data matched in this area creating suspicion on the claim that a key
second or a key frame can be found out in a commercial when one of the psychological constructs (attention,
positive emotions or arousal) were at peak.

Recommendations: Authors observe that if the buyer purchased measure of same concept from different vendors,
that buyer should be able to compare the results to see whether they agree. In general, it is a good practice to use
multiple measures to assess same psychological construct.

Study also recommended that before purchasing a neuro measure, the buyer should ask the vendors to provide
the description of the concepts they measure. Study found out that each of the vendors had their own description
of the measure. Study recommended to take help of independent consultants to understand some of these
descriptions.

Further analysis: It is important to understand that in spite of the scientific nature of the neuro method, the
results from any two vendors were not consistent. It is more to do with the stage at which the technology is and
less to do with the vendors method of obtaining the measure and reporting it. Also, the nature of human brain is
such that there is no ‘buy button’. It is the accumulation of the validations confirmed through various neuro
measures that a customer may go for a positive buy decision.