LWT - Food Science and Technology 125 (2020) 109216

Contents lists available at ScienceDirect

LWT - Food Science and Technology

journal homepage: www.elsevier.com/locate/lwt

Intelligent evaluation of black tea fermentation degree by FT-NIR and T

computer vision based on data fusion strategy
Ge Jin, Yujie Wang, Luqing Li, Shanshan Shen, Wei-Wei Deng, Zhengzhu Zhang, Jingming Ning∗
State Key Laboratory of Tea Plant Biology and Utilization, School of Tea and Food Science & Technology, Anhui Agricultural University, Hefei, Anhui, 230036, PR China

A R T I C LE I N FO A B S T R A C T

Keywords: Fermentation is the most crucial step in black tea processing. In this study, we used a combination of Fourier
Black tea fermentation time transform near-infrared spectroscopy (FT-NIR) and computer vision system (CVS) for comprehensively evalu-
Computer vision ating the black tea fermentation degree. 110 samples of black tea at different fermentation degrees were col-
FT-NIR lected in this research. The tea polyphenols (TP) content was measured using a UV–Vis spectrometer, and TP
Mid-level fusion
degradation during black tea fermentation was measured for classifying the degree of fermentation. Two mid-
Tea polyphenols
level strategies were used to analyze the fusion signals of FT-NIR and CVS. K-nearest neighbors, linear dis-
criminant analysis, and a support vector machine (SVM) were applied for classification modeling. The ad-
vantages of FT-NIR and CVS were integrated based on mid-level fusion, and the results obtained were better than
those obtained using independent methods for evaluating black tea fermentation. The mid-level fusion strategy
that combined FT-NIR and CVS for principal component analysis feature extraction was the most effective in the
SVM model. Classification accuracies of calibration and prediction sets were each 100%. Therefore, in this study,
it was demonstrated that a combination of FT-NIR and CVS at mid-level fusion strategy can be used as a method
to rapidly evaluate the black tea fermentation degree.

1. Introduction
Black tea is one of the most enjoyable and popular beverages in the
world (Gallardo, Alegret, & Valle, 2005; Mao et al., 2018) and accounts
for 78% of tea consumption worldwide (Kraujalytė, Pelvan, &
Alasalvar, 2016). Black tea is made from fresh tea leaves in five steps
(Lin & Zhu, 2004), namely withering, rolling, fermentation, drying, and
sorting. Fermentation is considered the most critical step in black tea
processing. The essence of fermentation is the oxidation reaction of tea
polyphenols (TP). As fermentation duration increases, TP content de-
creases. At this point, a series of chemical reactions occur, including the
formation of theaflavins (TFs) and thearubigins (TRs) from catechins
under the action of polyphenol oxidase (PPO) and peroxidase (PO)
(Marie-Claude, Shengmin, Yang, Chi-Tang, & Rosen, 2004; Samanta
et al., 2015). Lipids, amino acids, carotenoids, and terpenoids form
some volatile organic compounds. These compounds together impart
black tea its distinctive flavor (Gill, Kumar, & Agarwal, 2011). How-
ever, insufficient or excessive fermentation affect black tea quality
(Jolvis Pou, 2016). Therefore, identifying the most appropriate amount
of black tea fermentation is essential. Exploring TP degradation is
conducive to achieve the most appropriate fermentation degree.
Traditionally, artisans evaluate the fermentation degree based on
changes in the color and smell of tea leaves (Zhu et al., 2019); however,
experience, psychological state, environment, and other factors affect
the evaluation process, making it highly subjective. To objectively
evaluate black tea fermentation, some nondestructive and rapid de-
tection techniques have been applied. An electronic nose device
(Bhattacharyya et al., 2007; Sharma et al., 2015) and an electronic
tongue (Ghosh et al., 2015) were developed to evaluate black tea fer-
mentation. However, the aforementioned methods are complex, greatly
affected by the environment, and costly. Therefore, evaluating black tea
fermentation in a simple, quick, and accurate manner remains a pro-
blem to be solved urgently.
Near-infrared spectroscopy (NIR) is advantageous because it allows
rapid and nondestructive evaluation of tea leaf quality (Chen, Zhao,
Huang, Zhang, & Liu, 2006; Li, Jin, Sun, Ye, & Liu, 2019). Favorable
results were achieved when NIR spectroscopy was used to monitor the
ratio of TFs, TRs (Dong, Li, et al., 2018), and phenolic compounds
(Zareef et al., 2019) in black tea fermentation. However, it only mon-
itored chemical changes during fermentation and overlooked color-re-
lated information. The black tea fermentation evaluation method is
relatively single-faceted. A previous study used computer vision system
(CVS) to evaluate black tea fermentation (Dong, Liang, et al., 2018);
however, the study only analyzed the correlation between color

∗
Corresponding author.
E-mail address: ningjm1998009@163.com (J. Ning).

https://doi.org/10.1016/j.lwt.2020.109216
Received 17 November 2019; Received in revised form 17 February 2020; Accepted 27 February 2020
Available online 03 March 2020
0023-6438/ © 2020 Elsevier Ltd. All rights reserved.
G. Jin, et al.
characteristics and fermentation quality indexes. Further research is
required to comprehensively evaluate the relationship among fermen-
tation color, chemical composition change, and black tea fermentation
degree. A single sensor is insufficient for evaluating the high complexity
of food, and data fusion can improve food quality evaluation (Rosa,
Leone, Cheli, & Chiofalo, 2017). Using NIR spectroscopy and CVS to
evaluate fish freshness (Huang et al., 2016; Wu, Yang, Zhou, Lai, &
Zhong, 2019) is more effective than using a single sensor.
This study established a quick, simple, and objective method for
determining the black tea fermentation degree; we combined Fourier
transform near-infrared spectroscopy (FT-NIR) and CVS. No study has
reported the application of data fusion technology to evaluate black tea
fermentation. The objectives of this study were (1) to investigate TP
content at different fermentation time; (2) to compare the performances
of FT-NIR and CVS in evaluating black tea fermentation; and (3) to
compare and analyze the feasibility and advantages of principal com-
ponent analysis (PCA) and successive projections algorithm
(SPA) + Pearson correlation analysis (Pearson) feature extraction data
fusion methods.
2. Materials and methods
2.1. Sample preparation
Black tea samples at different fermentation time were collected from
the Jindong tea factory (Huangshan, Anhui) on April 19, 2019. The
samples were produced using traditional manufacturing processes, such
as tea leaf collection (one bud with two leaves), withering, rolling (with
a rolling machine diameter = 90 mm), and fermentation at different
time intervals. The finely crushed tea leaves were placed in a fermen-
tation room where the temperature was controlled at 30 ± 3 °C and
humidity was 90 ± 5%. The fermentation time was 300 min, and 10
samples were collected every 30 min, that is, a total of 110 samples
were collected. During sample acquisition, computer vision experi-
ments were performed. The 110 collected samples were placed in a
storage box with dry ice during transportation to the laboratory.
Finally, they were stored in a refrigerator at −80 °C until further ex-
perimentation.
The experimental process is presented in Fig. 1. In addition, before
modeling, The 110 fermented samples were numbered consecutively
Fig. 1. Flowchart of the primary step in data fusion analysis for evaluating black tea fermentation.
TP: Tea polyphenols; PCA: Principal component analysis; SPA: Successive projections algorithm; Pearson: Pearson correlation analysis.
2
LWT - Food Science and Technology 125 (2020) 109216
and divided into calibration set and prediction set according to a ratio
of 2:1. The calibration set was used for calibrating model and the
prediction set that used to test the robustness of the model. Eventually,
the calibration set contained 73 samples and the prediction set con-
tained 37 samples. To examine the performance of the model, correct
classification rate (CCR) were used:
NRight
CCR =
N
where NRight is the number of correct classified samples in calibration
set or prediction set, N is the total number of samples in calibration set
or prediction set.
Considering color and spectral characteristics as independent vari-
ables and degradation results of four groups of TP as dependent vari-
ables, a fermentation degree discrimination model was developed.
2.2. Image acquisition
The CVS, comprising a lighting room and digital camera, was used
to obtain image information during black tea fermentation. To ensure
light source stability, a 40 × 40 × 40 cm3 black box was equipped with
two fluorescent lamps. The light was turned on 30 min before image
collection. In addition, a digital camera (ESO 700D, Canon) with a re-
solution of 2592 × 1728 pixels was installed, with the distance be-
tween the camera and the sample fixed at 40 cm. Images were acquired
using the following camera settings: manual exposure with a shutter
speed of 1/100 s (zoom and flash functions off) and an ISO number of
400. All the images were captured using the same camera settings. A
previous method (Huang et al., 2016) was used as reference to extract
color features (R, G, B, H, V, S, L*, a*, and b*) from the images.
2.3. Spectra acquisition
The black tea fermentation samples were removed from the −80 °C
refrigerator. Then recover to room temperature in the drying vessel.
The samples were scanned in a diffuse reflection mode using an FT-NIR
spectrometer (Bruker Optics Inc, Ettlingen, Germany). Each spectrum
was the average of 32 scans recorded between 800 and 2500 nm, with a
resolution of 8 cm−1. The samples were analyzed in triplicate with a
rotation of 120° after each step. Spectra were collected at an ambient
G. Jin, et al.
Fig. 2. Scheme of data fusion approaches used to combine the CVS and FT-NIR.
temperature of 25 ± 3 °C (Li, Wei, Ning, & Zhang, 2015).
2.4. Extraction and determination of TP
TP content was determined using a UV–Vis spectrophotometer assay
according to the Chinese standard method (GB/T8313-2018). In a 10-
mL centrifuge tube, 0.2 g of the black tea fermentation sample and
5.0 mL of 70% methanol were mixed. The mixture was extracted at
70 °C for 10 min, cooled to room temperature, and then centrifuged at
3500 rpm for 10 min. The supernatant was transferred to a 10-mL
volumetric flask. The aforementioned steps were repeated to re-extract
the supernatant in the next step. The supernatants obtained in each step
were mixed in a 10-mL centrifuge tube, and deionized water was added
to obtain a 10 mL solution. Then, 1 mL TP extract was added to a 100-
mL volumetric flask, and deionized water was added to obtain a 10 mL
solution.
Gallic acid solution (10, 20, 30, 40, and 50 μg/mL), water (1 mL),
and polyphenol extract (1 mL) were added to a 10-mL volumetric flask,
and Folin Ciocalteu was added (5 mL, 10.0%). After a 5-min reaction,
4 mL of 7.5% Na2CO3 was added and allowed to remain for 1 h at room
temperature. A UV–Vis spectrophotometer was used to determine the
absorption value of the test solution at a wavelength of 765 mm.
2.5. Multivariate data analysis
2.5.1. Principal component analysis
PCA is the most commonly used dimension reduction technology
that converts original variables into new irrelevant variables, which is
called principal components (Bro & Smilde, 2014). The purpose is to
improve the data processing speed and saves considerable time and cost
3
LWT - Food Science and Technology 125 (2020) 109216
(Gallardo et al., 2005).
2.5.2. K-nearest neighbors
K-nearest neighbors (KNN) is a classification method based on the
close proximity of similar samples in a pattern space. It calculates the
distance between K known samples in the nearest field and the position
of the samples to be judged (Berrueta, Alonso-Salces, & Héberger,
2007). The optimization analysis of parameter k is necessary to improve
the performance of the KNN classifier (Chen, Zhao, Chen, Lin, & Zhao,
2011). If k is too large, the model becomes simpler and big classes will
overwhelm small ones; if k is too small, the model becomes complex
and prone to overfitting. According to the principle of cross validation
of highest classification accuracy, k value was screened and optimized
to obtain the best model results.
2.5.3. Linear discriminant analysis
LDA was one of the most popular supervised methods, which at-
tempts to find linear combinations of features to find the best linear fit
for separating two or more sets of samples (Chen, Ke, Cai, Zhu, & Chen,
2015).
2.5.4. Support vector machine
The main idea of the support vector machine (SVM) is to establish a
classification hyperplane as the decision surface. It is advantageous
because of its nonlinear and small sample sets (Min & Lee, 2005). In this
study, SVM was built using RBF as kernel function. RBF kernel function
parameters, namely c and gamma (g), are crucial for designing effective
SVM models. The parameter c minimizes the fitting error and the
complexity of the model, while g defines the nonlinear mapping from
the input space to a high-dimensional feature space. In order to obtain
G. Jin, et al. LWT - Food Science and Technology 125 (2020) 109216

the best generalization performance, the grid-search program searches black tea fermentation was moderate. After 3.5 h, the TP continued to
for the optimal combination of (c, g) through 5-fold cross validation degrade, and the lowest degradation rate of TP was 28.63%, which
and the ranges of c and g were both 2−10-210 (Li, Xie, Ning, Chen, & resulted in excessive fermentation. The decline trend of TP content was
Zhang, 2019). similar to the results of the study (Muthumani & Kumar, 2007).
Therefore, the black tea fermentation samples could be classified ac-
2.5.5. Data fusion strategy cording to the TP degradation rate as preliminary, insufficient, mod-
The data can be combined basically at three levels, namely low-, erate, and excessive fermentation.
mid-, and high-level fusion. In mid-level fusion, informative features of
the raw data from each instrument are separately extracted using sev- 3.2. Computer vision data analysis
eral types of variable selections or feature extraction algorithms and are
then aligned and concatenated into a single matrix that is used for With an increase in the fermentation duration, the color of black tea
multivariate analysis (Borràs et al., 2015). Fig. 2 summarizes that in changed to some extent. During fermentation, PPO oxidized TP to form
this study, two mid-level fusion methods combine two different feature o-quinones (a highly unstable form), which then formed TFs and TRs. In
extraction information blocks. The mid-level fusion of PCA feature ex- addition, chlorophyll degraded to form pheophytin (Gill et al., 2011).
traction consisted of merging variables that had been previously se- During fermentation, these compounds led to the change in the black
lected by applying the PCA dimension reduction technology. Char- tea color. Black tea images at different fermentation time are presented
acteristic variables with cumulative variance > 99% were extracted. in Fig. 3. During black tea fermentation, the color changed from green
FT-NIR and CVS both generated three new variables, respectively. For to deep copper red. To accurately classify black tea fermentation ac-
the mid-level fusion of SPA and Pearson feature extraction, three cording to color characteristics, multivariate analysis techniques were
variables extracted using FT-NIR were selected for SPA, and four vari- used, which include different algorithm types. Three classical modeling
ables extracted using Pearson correlation (> 0.6) were selected for CVS methods, namely KNN, LDA, and SVM, were used to model the black
(Supplementary Table S1). tea fermentation quality index. The main motivation for testing clas-
sification algorithms with different properties (linear/non-linear) was
2.5.6. Software to solve complex pattern classification problems by selecting the op-
The algorithms were executed in MATLAB 2014b operating on timal decision. As presented in Table 2, the KNN, LDA, and SVM models
Windows 7, and mapping was performed using Origin 2017 (Origin correctly classified the samples, with classification calibration set ac-
Lab. Corp., Northampton, MA). curacies of 65.75%, 91.78%, and 89.04%, respectively, and prediction
set accuracies of 89.19%, 89.19%, and 94.59%, respectively. The effect
3. Results and discussion of KNN model calibration set was very poor, which might because the
sample of black tea fermentation degree was unbalanced, and KNN is
3.1. Data analysis of TP not good during training. Although LDA and SVM model achieved good
results, revealed that 9 characteristic values of R, G, B, H, S, V, L*, a*,
The changes in TP values during fermentation are presented in and b* based on computer vision were not sufficiently effective in
Table 1. During black tea fermentation, PPO and PO cause enzymatic evaluating black tea fermentation. This might due to black tea fer-
oxidation and polymerization of TP to form compounds that impart mentation involves chemical changes in addition to color changes.
flavor and special tastes to black tea. As presented in Table 1, during
preliminary fermentation, the average TP content of black tea was the 3.3. FT-NIR spectra information analysis
highest at 140.46 mg/g. After fermentation for 0.5–2.0 h, the TP con-
tent decreased significantly, which implies that TP were rapidly oxi- FT-NIR was used to assess the chemical information of the black tea
dized. At higher concentrations of TP, the enzyme reaction rate reduced fermentation process. Fig. 4 presents the raw spectra of the black tea
to maintain a maximum enzyme reaction rate (Muthumani & Kumar, fermentation samples, which recorded the FT-NIR spectra curve for
2007). After fermentation for 2.5–3.0 h, the degradation rate of TP was wavenumbers in the range 800 and 2500 nm. The spectrogram of the
in the range 21.35%–21.39% and the TP rarely degraded. At this time, four fermentation degrees depicts slight differences, which indicates
that during fermentation, the chemical components and the changes in
Table 1 surface color affected the FT-NIR spectral response characteristics of the
Effect of fermentation duration on TP content during black tea fermentation. samples (Porep, Kammerer, & Carle, 2015). In this study, standard
normal variate transformation (SNV) was applied for spectra pre-
Fermentation TP (mg/g) Average Standard TP Group
processing, eliminating the effect of uneven sample size on light pro-
time (h) (mg/g) deviation decline
(mg/g) rate pagation. Because of the redundancy in spectral data, PCA was used to
(%) reduce data dimensionality and to visualize the distribution of samples
during black tea fermentation. Fig. 5a shows the scores for the first two
0.0 126.47–153.78 140.46 7.45 0.00% Preliminary
PCs. The total variance explained by the first two components was
fermentation
0.5 126.47–162.18 138.28 12.58 1.56% Insufficient 97.14% (PC1 = 88.97% and PC2 = 8.17%). It can be seen that the four
1.0 118.49–158.40 127.82 11.88 9.00% fermentation fermentation stages are almost all separate. This indicates that FT-NIR
1.5 114.85–129.13 123.35 3.97 12.18% combined with PCA can effectively determine the degree of black tea
2.0 110.50–136.13 120.04 8.39 14.54% fermentation. The main spectral contributions observed in the loading
2.5 100.84–121.43 110.50 6.47 21.35% Moderate
3.0 97.48–126.05 110.42 10.29 21.39% fermentation
are shown in Fig. 5b. PC1 obtains information bands: 1350–1375 nm
3.5 97.48–111.76 104.20 4.94 25.82% Excessive (C–H group first-order frequency doubling) and 1800–1900 nm (C]O
4.0 93.28–109.66 100.42 4.79 28.51% fermentation bonds). As for PC2, the main contribution comes from the following
4.5 90.76–113.03 100.25 6.13 28.63% band 1380–1420 and 1900–2000 nm (O–H combination bands). During
5.0 87.39–105.46 100.88 5.34 28.25%
the fermentation of black tea, some important chemical changes, such
TP: Tea polyphenols. as catechins, amino acids, and tea pigments, are associated with the
Preliminary fermentation: 0 h. absorption peaks of the principal component load figure.
Insufficient fermentation: 0.5–2.0 h. The PCA-processed FT-NIR datasets were severally used as input
Moderate fermentation: 2.5–3.0 h. matrices for the KNN, LDA, and SVM models. The identification results
Excessive fermentation: 3.5–5.0 h. of the calibration and prediction sets are presented in Table 2. For the

4
G. Jin, et al. LWT - Food Science and Technology 125 (2020) 109216

Fig. 3. Black tea images at different fermentation time: (a) preliminary fermentation, (b) insufficient fermentation, (c) moderate fermentation, and (d) excessive
fermentation.

Table 2
Comparison of discrimination accuracies obtained using KNN, LDA, and SVM
methods on different fermentation samples.
Model Data Parameters Correct classification rate
source
Calibration set Prediction set

KNN NIR k=3 90.41% 89.19%

CVS k=3 65.75% 89.19%
LDA NIR – 89.19% 86.30%
CVS – 91.78% 89.19%
SVM NIR c = 64, g = 2.2974 97.26% 89.19%
CVS c = 21.1121, 89.04% 94.59%
g = 0.143587

g: gamma.

KNN, LDA, and SVM models, classification calibration set accuracies

were 90.41%, 89.19%, and 97.26%, respectively, and prediction set
accuracies were 89.19%, 86.30%, and 89.19%, respectively. Compared
with the model of image information, the prediction set of SVM model
(non-linear) and LDA (linear) model correct classification rate is re-
duced. But, in the KNN model, the performance of FT-NIR was better
than that of CVS. This result indicating that the chemical information of Fig. 4. Raw spectral data of fermented black tea.
black tea fermentation is a very complex change, and the determination
of fermentation degree by a single sensor needs to be improved. sensor data to evaluate food quality is insufficient, and a combination of
multisensor data technology is necessary (https://www.sciencedirect.
com/science/article/pii/S0260877417301796 Borràs et al., 2015).
3.4. Data fusion based on FT-NIR and CVS signals
Therefore, in this study, two different mid-level fusion feature extrac-
tion methods were used to evaluate the black tea fermentation degree.
Black tea fermentation is complex and involves a change in color,
SPA was used to extract three characteristic wavelengths: 802, 945, and
aroma, and chemical composition; therefore, acquisition of single-

5
G. Jin, et al. LWT - Food Science and Technology 125 (2020) 109216

Fig. 5. PCA of spectral data of fermented black tea. (a) Score plot; (b) Loading plot.

Table 3 was very beneficial to data fusion. For the SVM model, two methods of
Comparison of results of KNN, LDA, and SVM models of classification for dif- mid-level fusion feature extraction were compared. PCA feature ex-
ferent fermentation samples using different feature extraction methods at mid- traction is better than SPA and Pearson because the feature variables
level fusion. extracted using SPA and Pearson retain the features of most related
Model Feature extraction Parameters Correct classification rate variables; however, some variables that were not significantly corre-
lated were overlooked. More chemical information was characterized
Calibration set Prediction set by fusing spectral and image data collected during black tea fermen-
KNN SPA + Pearson k=1 80.82% 75.68%
tation. Spectral information explains the change in catechins (Dong
PCA k=1 87.67% 91.89% et al., 2017), and image information corresponds to generation of TFs
LDA SPA + Pearson – 98.63% 91.89% and TRs (Dong, Liang, et al., 2018), which are formed by TP degrada-
PCA – 95.89% 94.59% tion. Therefore, the black tea fermentation process can be compre-
SVM SPA + Pearson c = 512, 100.00% 94.59%
hensively evaluated by using FT-NIR and CVS.
g = 0.353553
PCA c = 6.9644, 100.00% 100.00% A low-level fusion method requires numerous sensors and extensive
g = 1.31951 data processing (Kiani, Minaei, & Ghasemi-Varnamkhasti, 2016), which
is time-consuming and difficult to apply in black tea fermentation.
However, because of the potential modern industrial applications of
1661 nm (Supplementary Fig. S1). Absorption at 802, 1661, and black tea, data must be processed and analyzed in real time. The mid-
945 nm can be attributed to N–H stretching (Wang et al., 2018), C–H level fusion method efficiently addresses this challenge because using it
and O–H stretching (Wu, Meng, et al., 2019), and Ar-OH stretching for feature extraction significantly reduces the data dimension. Mid-
(Leme et al., 2018), respectively. These characteristic wavelengths were level fusion methods achieved favorable results in the evaluation of
related to TP. A Pearson correlation analysis was used to extract cor- black tea fermentation because the fusion of data from complementary
relation variables H, S, a*, and b*. Data from these two different sensors sensors responded to different characteristic phenomena and increased
were combined into a single matrix for a multivariate analysis. As the correct classification rate (Rosa et al., 2017). Real-time monitoring
presented in Table 3, in the SPA and Pearson feature extraction model, of black tea fermentation should consider the possibility of rapid data
the calibration set accuracies of KNN, LDA, and SVM were 80.82%, processing and automated online monitoring of industrial-scale opera-
98.63%, and 100%, respectively, and the prediction set accuracies were tions. The data fusion method offers a viable solution, and con-
75.68%, 91.89%, and 94.59%, respectively. Adulterated olive oil (Li, temporarily, the results are encouraging; however, urgently designing a
Xiong, & Min, 2019) for SPA extraction of characteristic spectral in- portable NIR and a low-cost camera is necessary because of their por-
formation fusion did not improve the model. However, in this study, table applications.
SPA spectral information feature extraction and integrated image in-
formation improved model performance. It may be that integrating
multiple different types of sensors more effective than a single spectral 4. Conclusion
data fusion. In PCA feature extraction modeling, calibration set ac-
curacies of KNN and LDA were 87.67% and 95.89%, and prediction set In this paper, we demonstrated the feasibility of applying FT-NIR
accuracies were 91.89% and 94.59%, respectively. PCA of SVM was the and CVS technologies for evaluating black tea fermentation. The results
most effective with calibration and prediction set accuracies of 100% present that the mid-level fusion method is better in evaluating the
each. After the fusion of multiple-information, compared with FT-NIR black tea fermentation degree than using a single feature, implying that
data, the performance of the KNN model has not improved, suggesting information obtained from an individual sensor is insufficient. In ad-
that the model after data fusion may not necessarily improve. For the dition, the PCA and SPA and Pearson mid-level fusion feature extraction
LDA model, the prediction set results are improved in both middle level methods were compared. The mid-level fusion SVM model based on
data fusion. In addition, the SVM model obtained the best classification PCA feature extraction had the highest classification accuracy, and the
results. SVM is recognized as the best method for solving classification calibration and prediction set accuracies were 100% each. The results
problems by projecting data into high dimensions, and constantly of this study provided a theoretical basis to automate the black tea
seeking optimal classification (Ouyang, Zhao, & Chen, 2014), which fermentation process. In the future, a portable NIR device and a simpler
CVS at the mid-level need to be developed to automate the monitoring

6
G. Jin, et al.
of black tea fermentation processes.
Declaration of competing interestCOI
Ge Jin declares that he has no conflict of interest. Yujie Wang de-
clares that he has no conflict of interest. Luqing Li declares that she has
no conflict of interest. Shanshan Shen declares that she has no conflict
of interest. Weiwei Deng declares that she has no conflict of interest.
Zhengzhu Zhang declares that he has no conflict of interest. Jingming
Ning declares that he has no conflict of interest.
CRediT authorship contribution statement
Ge Jin: Conceptualization, Methodology, Software, Investigation,
Data curation, Writing - original draft. Yujie Wang: Methodology,
Formal analysis, Software. Luqing Li: Validation, Formal analysis,
Investigation. Shanshan Shen: Investigation, Data curation. Wei-Wei
Deng: Resources, Writing - review & editing, Supervision. Zhengzhu
Zhang: Resources, Writing - review & editing, Supervision. Jingming
Ning: Resources, Supervision, Writing - review & editing, Funding ac-
quisition, Project administration.
Acknowledgments
This work was financially supported by the National Key Research
and Development Plan (No. 2017YFD0400800) and The University
Synergy Innovation Program of Anhui Province (GXXT-2019-012).
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://
doi.org/10.1016/j.lwt.2020.109216.
References
Berrueta, L. A., Alonso-Salces, R. M., & Héberger, K. (2007). Supervised pattern re-
cognition in food analysis. Journal of Chromatography A, 1158(1), 196–214. https://
doi.org/10.1016/j.chroma.2007.05.024.
Bhattacharyya, N., Seth, S., Tudu, B., Tamuly, P., Jana, A., Ghosh, D., ... Bhuyan, M.
(2007). Monitoring of black tea fermentation process using electronic nose. Journal of
Food Engineering, 80(4), 1146–1156. https://doi.org/10.1016/j.jfoodeng.2006.09.
006.
Borràs, E., Ferré, J., Boqué, R., Mestres, M., Aceña, L., & Busto, O. (2015). Data fusion
methodologies for food and beverage authentication and quality assessment – a re-
view. Analytica Chimica Acta, 891, 1–14. https://doi.org/10.1016/j.aca.2015.04.042.
Bro, R., & Smilde, A. K. (2014). Principal component analysis. Anal. Methods, 6(9),
2812–2831. https://doi.org/10.1039/c3ay41907j.
Chen, X., Ke, L., Cai, J., Zhu, D., & Chen, H. (2015). Identification of heavy metal-con-
taminated Tegillarca granosa using infrared spectroscopy. Analytical Methods, 7(5),
2172–2181. https://doi.org/10.1039/C4AY02396J.
Chen, Q., Zhao, J., Chen, Z., Lin, H., & Zhao, D. A. (2011). Discrimination of green tea
quality using the electronic nose technique and the human panel test, comparison of
linear and nonlinear classification tools. Sensors and Actuators B: Chemical, 159(1),
294–300. https://doi.org/10.1016/j.snb.2011.07.009.
Chen, Q., Zhao, J., Huang, X., Zhang, H., & Liu, M. (2006). Simultaneous determination of
total polyphenols and caffeine contents of green tea by near-infrared reflectance
spectroscopy. Microchemical Journal, 83(1), 42–47. https://doi.org/10.1016/j.
microc.2006.01.023.
Dong, C., Liang, G., Hu, B., Yuan, H., Jiang, Y., Zhu, H., et al. (2018). Prediction of congou
black tea fermentation quality indices from color features using non-linear regression
methods. Scientific Reports, 8(1), 10535. https://doi.org/10.1038/s41598-018-
28767-2.
Dong, C., Li, J., Wang, J., Liang, G., Jiang, Y., Yuan, H., ... Meng, H. (2018). Rapid de-
termination by near infrared spectroscopy of theaflavins-to-thearubigins ratio during
Congou black tea fermentation process. Spectrochimica Acta Part A: Molecular and
Biomolecular Spectroscopy, 205, 227–234. https://doi.org/10.1016/j.saa.2018.07.
029.
Dong, C., Zhu, H., Wang, J., Yuan, H., Zhao, J., & Chen, Q. (2017). Prediction of black tea
fermentation quality indices using NIRS and nonlinear tools. Food Science and
Biotechnology, 26(4), 853–860. https://doi.org/10.1007/s10068-017-0119-x.
Gallardo, J., Alegret, S., & Valle, M. D. (2005). Application of a potentiometric electronic
tongue as a classification tool in food analysis. Talanta, 66(5), 1303–1309. https://
doi.org/10.1016/j.talanta.2005.01.049.
Ghosh, A., Bag, A. K., Sharma, P., Tudu, B., Sabhapondit, S., Baruah, B. D., ...
Bandyopadhyay, R. (2015). Monitoring the fermentation process and detection of
optimum fermentation time of black tea using an electronic tongue. IEEE Sensors
Journal, 15(11), 6255–6262. https://doi.org/10.1109/JSEN.2015.2455535.
LWT - Food Science and Technology 125 (2020) 109216
Gill, G. S., Kumar, A., & Agarwal, R. (2011). Monitoring and grading of tea by computer
vision – a review. Journal of Food Engineering, 106(1), 13–19. https://doi.org/10.
1016/j.jfoodeng.2011.04.013.
Huang, X., Xu, H., Wu, L., Dai, H., Yao, L., & Han, F. (2016). A data fusion detection
method for fish freshness based on computer vision and near-infrared spectroscopy.
Analytical Methods, 8(14), 2929–2935. https://doi.org/10.1039/c5ay03005f.
Jolvis Pou, K. R. (2016). Fermentation: The key step in the processing of black tea. Journal
of Biosystems Engineering, 41(2), 85–92. https://doi.org/10.5307/jbe.2016.41.2.085.
Kiani, S., Minaei, S., & Ghasemi-Varnamkhasti, M. (2016). Fusion of artificial senses as a
robust approach to food quality assessment. Journal of Food Engineering, 171,
230–239. https://doi.org/10.1016/j.jfoodeng.2015.10.007 2016.
Kraujalytė, V., Pelvan, E., & Alasalvar, C. (2016). Volatile compounds and sensory
characteristics of various instant teas produced from black tea. Food Chemistry, 194,
864–872. https://doi.org/10.1016/j.foodchem.2015.08.051.
Leme, L. M., Rocha Montenegro, H., da Rocha dos Santos, L., Sereia, M. J., Valderrama,
P., & Março, P. H. (2018). Relation between near-infrared spectroscopy and physi-
cochemical parameters for discrimination of honey samples from Jatai weyrauchi and
Jatai angustula Bees. Food Analytical Methods, 11(7), 1944–1950. https://doi.org/10.
1007/s12161-018-1148-7.
Li, X., Jin, J., Sun, C., Ye, D., & Liu, Y. (2019). Simultaneous determination of six main
types of lipid-soluble pigments in green tea by visible and near-infrared spectroscopy.
Food Chemistry, 270, 236–242. https://doi.org/10.1016/j.foodchem.2018.07.039.
Lin, D., & Zhu, L. (2004). Polycyclic aromatic hydrocarbons: Pollution and source analysis
of a black tea. Journal of Agricultural and Food Chemistry, 52(26), 8268–8271. https://
doi.org/10.1021/jf048636n.
Li, L., Wei, L., Ning, J., & Zhang, Z. (2015). Detection and quantification of sugar and
glucose syrup in roasted green tea using near infrared spectroscopy. Journal of Near
Infrared Spectroscopy, 23(5), 317–325. https://doi.org/10.1255/jnirs.1178.
Li, L., Xie, S., Ning, J., Chen, Q., & Zhang, Z. (2019). Evaluating green tea quality based on
multisensor data fusion combining hyperspectral imaging and olfactory visualization
systems. Journal of the Science of Food and Agriculture, 99(4), 1787–1794. https://doi.
org/10.1002/jsfa.9371.
Li, Y., Xiong, Y., & Min, S. (2019). Data fusion strategy in quantitative analysis of spec-
troscopy relevant to olive oil adulteration. Vibrational Spectroscopy, 101, 20–27.
https://doi.org/10.1016/j.vibspec.2018.12.009.
Mao, S., Lu, C., Li, M., Ye, Y., Wei, X., & Tong, H. (2018). Identification of key aromatic
compounds in Congou black tea by partial least‐square regression with variable im-
portance of projection scores and gas chromatography–mass spectrometry/gas
chromatography–olfactometry. Journal of the Science of Food and Agriculture. https://
doi.org/10.1002/jsfa.9066.
Marie-Claude, M., Shengmin, S., Yang, C. S., Chi-Tang, H., & Rosen, R. T. (2004). Analysis
of theaflavins and thearubigins from black tea extract by MALDI-TOF mass spectro-
metry. Journal of Agricultural and Food Chemistry, 52(9), 2455–2461. https://doi.org/
10.1021/jf035427e.
Min, J. H., & Lee, Y. C. (2005). Bankruptcy prediction using support vector machine with
optimal choice of kernel function parameters. Expert Systems with Applications, 28(4),
603–614. https://doi.org/10.1016/j.eswa.2004.12.008.
Muthumani, T., & Kumar, R. S. S. (2007). Influence of fermentation time on the devel-
opment of compounds responsible for quality in black tea. Food Chemistry, 101(1),
98–102. https://doi.org/10.1016/j.foodchem.2006.01.008.
Ouyang, Q., Zhao, J., & Chen, Q. (2014). Instrumental intelligent test of food sensory
quality as mimic of human panel test combining multiple cross-perception sensors
and data fusion. Analytica Chimica Acta, 841(23), 68–76. https://doi.org/10.1016/j.
aca.2014.06.001.
Porep, J. U., Kammerer, D. R., & Carle, R. (2015). On-line application of near infrared
(NIR) spectroscopy in food production. Trends in Food Science & Technology, 46(2),
211–230. https://doi.org/10.1016/j.tifs.2015.10.002.
Rosa, A. R. D., Leone, F., Cheli, F., & Chiofalo, V. (2017). Fusion of electronic nose,
electronic tongue and computer vision for animal source food authentication and
quality assessment – a review. Journal of Food Engineering, 210, 62–75. https://doi.
org/10.1016/j.jfoodeng.2017.04.024.
Samanta, T., Cheeni, V., Das, S., Roy, A. B., Ghosh, B. C., & Mitra, A. (2015). Assessing
biochemical changes during standardization of fermentation time and temperature
for manufacturing quality black tea. Journal of Food Science & Technology, 52(4),
2387–2393. https://doi.org/10.1007/s13197-013-1230-5.
Sharma, P., Ghosh, A., Tudu, B., Sabhapondit, S., Baruah, B. D., Tamuly, P., ...
Bandyopadhyay, R. (2015). Monitoring the fermentation process of black tea using
QCM sensor based electronic nose. Sensors and Actuators B: Chemical, 219, 146–157.
https://doi.org/10.1016/j.snb.2015.05.013.
Wang, X., Esquerre, C., Downey, G., Henihan, L., O'Callaghan, D., & O'Donnell, C. (2018).
Assessment of infant formula quality and composition using Vis-NIR, MIR and Raman
process analytical technologies. Talanta, 183, 320–328. https://doi.org/10.1016/j.
talanta.2018.02.080.
Wu, D., Meng, L., Yang, L., Wang, J., Fu, X., Du, X., ... Huang, L. (2019). Feasibility of
laser-induced breakdown spectroscopy and hyperspectral imaging for rapid detection
of thiophanate-methyl residue on mulberry fruit. International Journal of Molecular
Sciences, 20(8), https://doi.org/10.3390/ijms20082017.
Wu, T., Yang, L., Zhou, J., Lai, D. C., & Zhong, N. (2019). An improved nondestructive
measurement method for salmon freshness based on spectral and image information
fusion. Computers and Electronics in Agriculture, 158, 11–19. https://doi.org/10.1016/
j.compag.2019.01.039.
Zareef, M., Chen, Q., Ouyang, Q., Arslan, M., Hassan, M. M., Ahmad, W., ... Ancheng, W.
(2019). Rapid screening of phenolic compounds in congou black tea (Camellia si-
nensis) during in vitro fermentation process using portable spectral analytical system
coupled chemometrics. Journal of Food Processing and Preservation, 43(7), https://doi.
org/10.1111/jfpp.13996.
Zhu, H., Liu, F., Ye, Y., Chen, L., Liu, J., Gui, A., ... Dong, C. (2019). Application of
machine learning algorithms in quality assurance of fermentation process of black
tea– based on electrical properties. Journal of Food Engineering, 263, 165–172.
https://doi.org/10.1016/j.jfoodeng.2019.06.009.