Journal of the American Society of Brewing Chemists

The Science of Beer

ISSN: 0361-0470 (Print) 1943-7854 (Online) Journal homepage: https://www.tandfonline.com/loi/ujbc20

Validating the Sensitivity of the Beer Tetrad Test as

Compared with the Beer Triangle Test (A Follow-Up
Study): An American Society of Brewing Chemists
Technical Committee Report

D. Bissmeyer & ASBC Technical Committee

To cite this article: D. Bissmeyer & ASBC Technical Committee (2019) Validating the Sensitivity of
the Beer Tetrad Test as Compared with the Beer Triangle Test (A Follow-Up Study): An American
Society of Brewing Chemists Technical Committee Report, Journal of the American Society of
Brewing Chemists, 77:3, 222-223, DOI: 10.1080/03610470.2019.1619323

To link to this article: https://doi.org/10.1080/03610470.2019.1619323

Published online: 21 Jun 2019.

Submit your article to this journal

Article views: 18

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=ujbc20
JOURNAL OF THE AMERICAN SOCIETY OF BREWING CHEMISTS
2019, VOL. 77, NO. 3, 222–223
https://doi.org/10.1080/03610470.2019.1619323

REPORT

Validating the Sensitivity of the Beer Tetrad Test as Compared with the Beer
Triangle Test (A Follow-Up Study): An American Society of Brewing Chemists
Technical Committee Report
D. Bissmeyer and ASBC Technical Committee

ABSTRACT KEYWORDS
Previous testing by the sensory subcommittee confirmed that the Tetrad Test was more sensitive Beer; discrimination tests;
than the Triangle Test on beer samples with higher Effect Sizes. The subcommittee also wanted to sensory; Tetrad Test;
confirm that the Tetrad Test was equally effective for samples with lower Effect Sizes; therefore, a Triangle Test
new study was designed to validate the sensitivity of the Tetrad Test for smaller d’ values. The
results of the second study show that for samples with smaller d’ values, the Tetrad Test resulted
in a 15% reduction in d’ compared with the Triangle Test. This is less than the theoretical 1/3
reduction, indicating that the Tetrad Test is more sensitive. The Tetrad Test also resulted in a
higher proportion of correct responses, though the difference was not significant at the 95% confi-
dence interval. The subcommittee recommended that the Tetrad Test method be included in the
ASBC Methods of Analysis.

Introduction
Difference test methods are widely used in the food and
beverage industry to determine if a sensory difference
between samples can be detected by a panel or by consum-
ers. An important consideration of these tests is their statis-
tical power or the capacity of the test to reliably detect a
difference between products. The Tetrad Test is a relatively
new type of discrimination testing that is theoretically more
powerful than the Triangle Test and has the added advan-
tage of using about one-third the number of panelists.
This was the second year of the subcommittee’s existence.
The subcommittee was formed based on the recommenda-
tion of the American Society of Brewing Chemists (ASBC)
Sensory Subcommittee to determine if the Tetrad Test is as
sensitive as the Triangle Test for samples with smaller d’
values. For the Tetrad Test, panelists are simultaneously pre-
sented with a set of four coded samples, comprised of two
sets of identical samples. They are instructed to group the
four samples into two pairs based on the level of similarity
between samples and if they cannot determine a difference,
they must guess.[1]
Procedure
Two production lots of Goose Island IPA and New
Belgium’s Fat Tire were received at the Wolf Group.
Samples were immediately coded and refrigerated and
moved to the Wolf Group’s controlled refrigerated storage
room (4
C). Only the Wolf Group’s trained and validated
panel participated in the testing.
Production samples and panelists
Cold Stored New Belgium Fat Tire lot 1 (4
C)
Cold Stored New Belgium Fat Tire lot 2 (4
C)
Cold Stored Goose Island IPA lot 1 (4
C)
Cold Stored Goose Island IPA lot 2 (4
C)
Each test had a minimum of 18 panelists and 2 repeti-
tions for a total of 36 data points.
Results and discussion
The proportions of panelists who correctly identified the odd
sample in the Tetrad and Triangle Tests for each sample are
provided in Table 1. For the New Belgium Fat Tire Sample,
the proportion of correct answers for both the Triangle and
Tetrad was 33%, which is the same as the guessing probabil-
ity. This portion of the test was therefore inconclusive. For
the Goose Island IPA sample, the Tetrad resulted in 53% cor-
rect responses compared with 47% for the Triangle. This dif-
ference is not significant at the 95% confidence interval.
Effect Size (d0 ) is an estimate of the amount of perceived
difference between the samples and can be thought of as a
signal-to-noise ratio. Theoretically, switching from Triangle
to Tetrad should result in a decrease in Effect Size of 1/3
because of the increase in noise from the addition of a sam-
ple; therefore, to be considered more powerful, the reduction

CONTACT D. Bissmeyer asbc@scisoc.org

Subcommittee Members: D. Bissmeyer, Chair; C. Lakenburges, Co-chair; T. Teras, Co-chair; Wolf Group’s Trained Panel; and L. Barr (ex officio).
ß 2019 American Society of Brewing Chemists, Inc.

Table 1. Comparison of proportion correct for Tetrad and Triangle methods.
Tetrad
New Belgium Fat Tire 33%
Goose Island IPA 53%
Table 2. Summary of d0 results for Goose Island IPA.
% d0 reduction
Test Standard from Triangle % d0 reduction
Method d0 deviation of d0 to Tetrad
Goose Island IPA Triangle 1.32 0.455 15%
Tetrad 1.11 0.289
in Effect Size from Triangle to Tetrad must be <1/3. The
results for d0 are shown in Table 2.
The reduction in d0 from Triangle to Tetrad was 15%
for the Goose Island IPA test. This is less than the theoret-
ical 1/3 (33.3%) reduction, indicating that the Tetrad should
be more powerful. In addition, the standard deviation of d0
was lower for the Tetrad, indicating that the Tetrad is
more precise.
Hummer[2] laid out the business risk associated with
Effect Size: Too small to be noticed (d0 < 0.61), threshold
(0.88  d0  1.10), very small (1.29  d0  1.47), and
noticeable (d0 > 1.64). Previous testing was conducted on
samples with larger d’ values. This follow-up study
was designed to validate the sensitivity of the Tetrad with
samples that were similar. Because the d’ of the study was in
the “threshold” and “very small” range, this objective
was achieved.
Conclusions
The Tetrad Test is a suitable replacement for the Triangle
Test. Previous testing performed by this committee indicated
that the Tetrad Test was more sensitive than the Triangle
JOURNAL OF THE AMERICAN SOCIETY OF BREWING CHEMISTS 223
Test for samples with larger d’ values. This study was
Triangle designed to validate the sensitivity of the Tetrad Test for
33% smaller d’ values. For samples with smaller d’ values, the
47%
Tetrad resulted in a 15% reduction in d’ compared with the
Triangle Test. This is less than the theoretical 33.3% reduc-
tion, indicating that the Tetrad Test is more sensitive. The
Tetrad Test also resulted in a higher proportion of correct
< 33.3%? responses, though the difference was not significant at the
Yes 95% confidence interval.
Recommendations
1. The subcommittee recommends that the Tetrad Test
method be included in the Methods of Analysis.
2. Samples with strong or lingering flavors may have too
much carryover for the Tetrad Test to be effective. This
was not a part of the current study but should be a con-
sideration in test selection.
Note
The ASBC Methods of Analysis is available online only
(http://methods.asbcnet.org/), with free access included as
part of the personal ASBC membership package or access
can be purchased by company subscription.
Literature cited
[1] Ennis, J. M. Guiding the Switch from Triangle to Tetrad
Testing. J. Sens. Stud. 2012, 27, 223–231. DOI: 10.1111/j.1745-
459X.2012.00386.x.
[2] Hummer, S. Business Risk Assessment From the Application of
Discrimination Testing. In SSP Sensometrics Joint Workshop:
Discrimination Testing in Practice, 2016, Society of Sensory
Professionals Conference, October 26–28, Atlanta, GA, 2016.