Machine Learning Facilitated Multiscale Imaging For Energy Materials

ll
OPEN ACCESS
Review
Machine learning-facilitated multiscale
imaging for energy materials
Guo-Xu Zhang,1,2 Yajie Song,1,2 Wei Zhao,1,2 Hanwen An,1,2 and Jiajun Wang1,2,3,*
SUMMARY
The relationship between the structure of a material and its proper-
ties, the so-called structure-property correlations, is at the center of
materials science. The microstructure of a material is an essential
feature for the optimization of physicochemical properties with
improved performance. However, it remains a challenge to recog-
nize and extract all relevant information from microscopic images.
Machine learning (ML) has entered the field of multiscale character-
ization and visualization in energy materials. The aim of this review is
to provide concise tutorials on multiscale imaging techniques and
ML methods, showing how they can be incorporated for solving
physical and chemical problems. With a particular focus on image
segmentation, we discuss noteworthy applications of ML in X-ray
and electron microscopy imaging for rechargeable batteries, solar
cells, and fuel cells and discuss how ML can facilitate identifying mi-
crostructures, enhancing image quality, and tracking dynamic pro-
cesses occurring both inside and at the materials interfaces.
INTRODUCTION
Today we are facing global challenges, including the ever increasing energy need
and climate change. This calls for the inevitable urge to search for better energy
materials and to develop energy conversion and storage technologies.1–3 A long-
standing and central task in molecules and materials remains to unfold structure-
property correlations for a given purpose, while most advances in physical and
chemical sciences are still based on the traditional trial-and-error technique, which
is time consuming and costly. To accelerate compliance with climate and sustainable
energy goals, breakthroughs are no doubt needed in understanding both energy
materials and complex processes taking place inside materials.4,5
Along with entering the fourth paradigm of scientific research, the so-called data-
intensive scientific discovery initially proposed by Jim Grey,6 data-driven materials
science has emerged as a state-of-the-art discipline for materials design and discov-
ery, often in association with big data, open source, data management, machine
learning (ML) and artificial intelligence (AI).7 Machine learning is a subset of AI,
and it is often defined as the study of computer algorithms that improve automati- 1MIIT Key Laboratory of Critical Materials
cally with experience.8 Within the fast development of efficient graphics processing Technology for New Energy Conversion and
unit (GPU) parallelization,9,10 ML has entered the field of microstructure characteriza- Storage, School of Chemistry and Chemical
Engineering, Harbin Institute of Technology
tion and visualization in energy materials. The microstructure of a material (e.g., (HIT), Harbin 150001, China
grains, grain boundaries, defects, impurities) is an essential feature that determines 2Chongqing Research Institute of Harbin Institute
its chemical and physical properties such as hardness and roughness; it provides of Technology (HIT), Chongqing 401120, China
insights into the so-called processing-structure-property links.11,12 To reveal such 3Lead contact
information, imaging techniques typically used are light microscopy, scanning elec- *Correspondence: jiajunhit@hit.edu.cn
tron microscopy (SEM), transmission electron microscopy (TEM), X-ray tomography https://doi.org/10.1016/j.xcrp.2022.101008
Cell Reports Physical Science 3, 101008, September 21, 2022 ª 2022 The Author(s). 1
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
ll
OPEN ACCESS
Review
(XRT), electron backscatter diffraction (EBSD), and others; their combination allows
multiscale analysis of material behavior (at the micro- and nanoscales). Although
there have long been studies using some of the above techniques, how to best
recognize and represent the microstructure using rich information from images is still
among the most fundamental research subjects.
Among various imaging tools, X-ray microscopy has become the method of choice
for nondestructively determining internal features of materials, with its spatial reso-
lution located between those of light and electron microscopy. An X-ray is a form of
high-energy electromagnetic radiation with much shorter wavelengths compared
with visible light, thus making it well suited for penetrating inside thick samples.
The basic working principle of such characterization lies in the fact that characteris-
tics of X-ray interactions distinguish structural, chemical, and morphological proper-
ties of the measured material, corresponding to different adsorptions of X-ray.8
Generally speaking, X-ray microscopy can refer to several imaging techniques for
materials characterization by using X-ray light sources.13 Hard X-rays are distin-
guished from soft X-rays by their higher photon energies, and their high penetrating
ability enables them to image inside of samples in a non-invasive way. It is note-
worthy to mention that different parts of X-rays are suitable for different
applications, such as, medical computed tomography (CT), X-ray crystallography,
and security checks, to name just a few. For an extended discussion, the reader is
referred to comprehensive review articles in this regard.14–17
The concept of X-ray microscopy can be traced back to 1896, shortly after the dis-
covery of X-rays by Roentgen, and the first X-ray microscope was proposed by Kirk-
patrick18 in the late 1940s after many pioneers. However, X-ray imaging techniques
were hampered for decades, mainly by the challenge of obtaining X-ray sources with
sufficient intensity. Until the 1970s, high-performance X-ray microscopy had cele-
brated its success, accompanied by the developments of synchrotron light sources,
computing, and data storage power.19 Nowadays X-ray imaging systems have
gained wide popularity in medical imaging, security check, earth science, cosmo-
chemistry, materials and surface science, to name just a few. Images generated
from X-rays (called radiographs or skiagraphs) contain abundant information about
the interior features of a substance. Thus, a critical task for X-ray imaging is to under-
stand how X-rays interact with matter when they pass through it. Generally, this can
be achieved by measuring the amount of energy loss (which is material dependent),
and a certain contrast can be used to detect an X-ray image.20
X-ray tomography microscopy represents an advanced X-ray imaging method, and it

can provide structural and compositional information in three dimensions and even
four dimensions (under time-resolved in situ conditions) through cross-sectional
images inside a sample. Such a technique makes it an ideal tool for characterizing
a material’s integrity and reconstructing three-dimensional (3D) images.21–23 In
particular, the use of synchrotron light sources for X-ray tomography has offered
new opportunities for visualizing the 3D interior features of real objects nondestruc-
tively with high spatial resolution.24,25 This allows the complex microstructural
analysis of most types of materials with sub-micro resolution. For example, X-ray to-
mography has shown its powerful role in studying battery operation and degrada-
tion of lithium-ion batteries (LIBs) over the past decade.26,27
Similar to XRT, electron tomography is another imaging technique that enables us to

reveal detailed 3D structures but with higher resolution. Specifically, electron tomo-
graphic images make it possible to obtain rich information about structure and
2 Cell Reports Physical Science 3, 101008, September 21, 2022

ll
OPEN ACCESS
Review
dynamics mostly by in situ TEM or SEM imaging. TEM is used to characterize ultra-
thin samples by electrons passing through them; thus, it provides detailed informa-
tion about interior structures as small as individual atoms, at the highest possible
spatial resolution. SEM studies images by scanning the surface with a focused elec-
tron beam, yielding information about surface morphology (secondary electron
signal) or compositions (backscattered electron signal). Scanning TEM (STEM), liter-
ally speaking, is a combination of TEM and SEM which uses a focused probe similar
to a scanning electron microscope but in transmission mode; it thus enables chem-
ical analysis at high spatial resolution by energy-dispersive X-ray spectroscopy, elec-
tron energy-loss spectroscopy, and annular dark-field imaging.28
Nowadays multiscale imaging techniques have gained continuous attention in mod-

ern materials science, although we are still facing many challenges in quantitatively
understanding detailed information contained in microscopic (and spectroscopic)
images,29 as illustrated in Current challenges. To unfold rich information about ma-
terials, more effort is needed by combining different image analysis tools. As an
emerging and fast growing field, machine learning methods have been adopted
for accelerating materials design and discovery on the basis of (physics-inspired)
theoretical simulations and experiments. All ML studies rely on data; it is thus inev-
itable that computational methods are particularly suited for solving ML problems,
as they can generate high-quality data that ML algorithms require. Because of the
complexity of solving the many-body problem of quantum mechanics, various
encouraging computational methods have been developed for approximately treat-
ing realistic systems for a given purpose across time and length scales. It should be
noted that no computational method links all scales and that many of them can be
combined for practical applications (well balanced between accuracy and effi-
ciency). Encouragingly, ML applies to all scales, thus showing great promise in
combining ML with widely used methods, such as density-functional theory, semi-
empirical methods, and empirical potentials. ML has made tremendous break-
throughs and is widespread in both the natural sciences and engineering.30–32 The
interested reader is referred to other in-depth review articles for molecular and
materials science.33–36 This review is written for new and experienced researchers
working at the intersection of materials science, microscopy imaging, and data-
driven science. As mentioned above, ML has been applied to experimental charac-
terization and interpretation. This review focuses on recent progress in machine
learning, in particular deep learning, for developing renewable energy technologies
such as energy generation, conversion, and storage. We aim to provide a critical re-
view of ML applied to multiscale imaging for energy materials, including batteries,
fuel cells, and solar cells, together with the future challenges in those domains.
CURRENT CHALLENGES
Recent advances in experimental detection and characterization have enabled us to
gain detailed structural and property insights into energy materials. Among various
techniques, microscopy imaging has proven to be a critical tool for providing real-
space information on the structure of matter. For example, X-ray computed
tomography is particularly important for nondestructive 3D imaging analysis across
multiple length scales. This imaging technique has been applied to study battery
materials during operation and degradation; the application is rather recent but it
has shown great promise in resolving structural identification and performance
prediction.15,16,37 However, many challenges remain. The first challenge is how to
recognize and extract all relevant information from an observed microscopic image.
This can be understood by the fact that every pixel of an image is a degree of
Cell Reports Physical Science 3, 101008, September 21, 2022 3

ll
OPEN ACCESS
Review
freedom, thus leading to extremely high dimensionality of the information space.38

Tackling this problem is closely related to the question of how to better represent a
microstructure image. A typical quantification of the microstructure requires image
segmentation, the process of assigning a label to every pixel in an image. Image seg-
mentation is a fundamental task in image processing and computer vision, and it is
an initial step for image analysis, image understanding and pattern recognition.
Despite the long history of research, image segmentation remains a big challenge,
without a single standard method proposed. In materials science, image segmenta-
tion is a non-trivial task of image processing that can lead to a set of segments (or
contours) to detect single particles, grains, pores, or defects, where chemical and
physical intuition are certainly required to better guide the segmenting. Thus re-
searchers working in this area should understand interdisciplinary languages, such
as materials science, microscopy imaging, and data-driven science. Second, the
signal noise and distortion of raw experimental microscopy images typically become
apparent at high magnifications and resolutions.39,40 For instance, SEM provides in-
formation on surface morphology and composition, but the noise level can be critical
at high scanning speeds. To solve these types of problems, various image restora-
tion techniques have been developed. We refer the reader to previous publica-
tions41,42 for a more extended discussion. Third, there are unavoidable drawbacks,
such as insufficient spatial and temporal resolution, as well as low contrast of these
systems.29 Last but not least, manually labeling the image data is tedious and time
consuming. In addition to the image data themselves, the quality of manual segmen-
tation by human experts typically lacks consistency because of observer variability.
For systems with increasing complexity, the labeling process can be prohibitive.
With the rapid growth of microstructure image databases, the above issues can
be tackled by combining data-driven approaches, more specifically, machine
learning, for materials design and discovery, as illustrated in Figure 1. In the
following sections, an overview of ML algorithms, in particular deep learning for im-
age analytics, together with its recent applications for energy materials will be
discussed.
MACHINE LEARNING IN A NUTSHELL

Machine learning studies algorithm design whose performance automatically im-
proves with data.8 In particular, ML algorithms build statistical models by the use
of the training data, and the trained model is used to make predictions or decisions
for a certain task without using explicit instructions.43 All ML algorithms rely on data
that can be represented as features (independent variables), generally by five di-
mensions (referred as to the 5 V’s44): volume, variety, velocity, veracity, and value.
The success of ML and AI approaches arises from the fact that computers can deal
with massive and high-dimensional data. With the rapid growth of supercomputers’
performance, ML has gained wide popularity in science and technology. Nowadays
ML makes an indispensable part of ‘‘the fourth industrial revolution’’ and ‘‘the fourth
paradigm of science,’’6 and applications of ML algorithms have become apparent in
our daily life, in smartphones, web searches, e-mail filtering, and health care.
Machine learning often incorporates statistics into its algorithm. In recent decades,
there have been decent developments in novel statistical methods. Machine
learning algorithms are typically classified into three broad types on the basis of
the characteristics of training samples, namely, supervised, unsupervised, and
reinforcement learning.45 Supervised learning is the most common, and it has
become the method of choice for the majority of using ML in physical sciences.33 Su-
pervised learning is suited to tasks of training an ML model on labeled samples

ll
OPEN ACCESS
Review
Figure 1. Machine learning integrated with materials design and discovery

Machine learning models are trained on datasets by machine learning algorithms that learn a mapping from input variables X to target variables Y.
(example-input and desired-output pairs) for unseen inputs, and the goal of this type
of learning algorithm is to learn a function for mapping new input-output pairs. A su-
pervised learning algorithm can refer to either regression or classification problems.
As opposed to supervised learning, unsupervised learning uses unlabeled input

data for training an ML model. This type of learning is widely used for discovering
hidden trends, patterns or clustering in a dataset. Two of the primary techniques
used in unsupervised learning are dimensionality reduction and cluster analysis.
Dimensionality reduction is used to identify patterns in features of high-dimensional
data, and it can facilitate visualization and understanding of the data after the
process of reducing the number of features. Cluster analysis (clustering) is an unsu-
pervised task that is used to group or segment unlabeled data, aiming to identify ho-
mogeneous subgroups.
Reinforcement learning represents a subfield of machine learning that requires ma-

chine creativity through interacting with environments. This category of ML models
is trained by finding optimal actions to solve reinforcement learning problems from a
dynamic environment that provides a sequence of rewards and penalties, and the
computer algorithm learns how to act to achieve a goal for reward maximization.46
Unlike supervised learning, there are no ‘‘correct’’ input-output pairs involved in
reinforcement learning algorithms; instead, agents decide the sequence of actions
(often known as policies) on the basis of the maximum cumulative reward. As a
fast growing ML field, reinforcement learning has shown great promise in exploring
chemical space and searching for new molecules and materials using algorithms,
such as objective-reinforced generative adversarial networks (GAN).7,47,48

ll
OPEN ACCESS
Review
Currently, ML techniques have been increasingly applied to energy materials design

and discovery, ranging from energy storage and energy conversion to energy man-
agement.49 In materials science, many novel ML approaches have been developed
to facilitate the research objectives, including materials discovery, advancing mate-
rials modeling, and understanding material phenomena.7,47,48,50 Of particular inter-
est are applications on the structural stability of energy materials,51–54 surface
adsorption, reaction barriers, bandgaps, density of states, to name just a few.
Numerous studies have shown that ML is a promising way towards accurately
predicting structures and compositions, as well as revealing novel chemical in-
sights.55–57 More recently, much attention has been paid to using ML methods to
assist experimental characterization and interpretation.58,59 This is due mainly to
the massive development of GPUs and GPU software that enable parallel computa-
tions using deep neural networks efficiently. The interested reader is referred to
recent articles for an extended discussion on experimental imaging methods such
as STEM,60–62 X-ray diffraction (XRD),63 and X-ray absorption near-edge structure
(XANES).64
In summary, the discipline of ML is growing rapidly, and the widespread use of ML

has become apparent in nearly all sciences. Deep learning (DL) belongs to a class
of machine learning algorithms based on artificial neural networks (ANNs) and can
be applied to supervised and unsupervised learning tasks. Deep learning has
received increasing attention in many research fields, and today it is opening oppor-
tunities for much ongoing work.65,66
DEEP LEARNING FOR IMAGE SEGMENTATION

A central task of materials science is to describe the structure-property relationships
of materials; the structure at the nano- and microscales typically covers atomic struc-
tures, bulk solids, surface, interfaces, morphology, and so on. Machine learning
techniques are becoming increasingly popular towards this goal. In particular,
deep learning is among the fastest growing field, although it is still in its early stage
in materials science. Combining with ML methods enables us to accelerate research
studies, as one can avoid the expense of performing useless and unfeasible exper-
imental or theoretical studies, on the basis of the knowledge contained in a well-
trained ML model. Once the structure-property links are known, novel materials
can be synthesized for novel applications. This is basically among the most funda-
mental research subjects for most materials of great interest including energy mate-
rials. Essentially, all ML techniques address the challenge of achieving both higher
accuracy and better efficiency for predictions of unknowns, with the eventual goal
of replacing more expensive and laborious experimental and theoretical studies
that require a lot of human expertise. Recent applications of deep learning have
shown the potential for accelerating materials design and discovery.67–69 Of partic-
ular interest in this article are DL-based imaging techniques for microstructure visu-
alization and characterization. Next, we will briefly describe basic concepts and
recent advances on this topic.
Key concepts of deep learning

‘‘Deep learning’’ refers to a wide class of ANN-based machine learning methods with
multiple processing layers to learn representations of data.70,71 In representation
learning, a computational model is composed of multiple hidden layers of neurons
that progressively transform the representation from the raw data into higher level
features. Figure 2A shows the schematic diagram of a typical ANN and artificial neu-
rons, consisting of an input layer, a hidden layer, and an output layer. The term

ll
OPEN ACCESS
Review
A B
Figure 2. Deep learning-based image segmentation

(A) A typical artificial neural network with one hidden layer consisting of units that transform the input information to all units in the output layer. Weights
(w) and biases (b) are learned during the training process.
(B) Commonly used image segmentation approaches.
(C) A schematic of image segmentation: an example of a regular convolutional neural network.
‘‘deep’’ refers to the use of multiple hidden layers in a neural network, that is, the
number of node layers is more than three in the pattern recognition process under
non-linear mathematical operations. In deep learning, each layer consists of multiple
neurons as input information (or features), the weighted sum of which is transformed
into a new representation. This will yield the output of the unit, which can be used as
input to the next layer. As a result, multiple layers are connected through the whole
network and a training procedure is a way of learning the weights between each unit.
In a deep learning model, computational cost becomes elaborate for networks with
more layers because of more mathematical operations involved, while generally the
more advanced the neural network, the more complex the features a DL model can
recognize. For instance, in image processing, the first layer of the network may iden-
tify edges, while higher layers enable the identification of other structural informa-
tion such as object parts (digits, letters or faces). Some widely used deep learning
architectures include deep neural networks (DNNs), recurrent neural networks
(RNNs), convolutional neural networks (CNNs), and so on. Note that CNNs are
among the most popular choices of neural networks for dealing with images and
videos (see Image segmentation).
As motivated in the previous section, deep learning is making tremendous progress

in various domains of natural science and engineering, and it is widely used in solving

ll
OPEN ACCESS
Review
problems such as speech recognition, computer vision, and drug discovery, to name
just a few. In this review, we address the current state of DL-based imaging tech-
niques for energy materials. The successes and challenges of DL-aided applications
will be discussed, providing useful insights into this exciting emerging field.
Image segmentation
Image segmentation is a computer vision technique that involves the process of par-
titioning an image into distinct segments on the basis of some criteria. Image seg-
mentation is generally used to identify objects and boundaries in two-dimensional
(2D) or 3D space of images, and it is an initial step for more advanced image process-
ing and analysis such as image reconstruction and enhancement. In practice, seg-
menting an image is carried out by assigning labels to elements of pixels in 2D
and voxels in 3D spaces of image data and distinguishing those elements with the
same label.72,73 As the first step of image analysis, the significance of performing im-
age segmentation can hardly be overestimated in computer vision; during the pro-
cess of segmentation, complex spatial information stored in an image is simplified
by a set of meaningful and distinct segments of contours extracted from the im-
age.74 Image segmentation has emerged as a fast growing research topic in mate-
rials science for characterizing microstructures.36 Some typical segmentation tasks
include identifying material phases, detecting the size and shape of particles, grains
and fibers, and so on. It is clear that care needs to be taken for ensuring the perfor-
mance of segmentation that determines further image processing and analysis of
structure-property relationships in materials.
Image segmentation is generally considered a non-trivial and challenging task in

practical applications, and it can be subjective and time consuming with manual seg-
mentations. The availability of large-scale image datasets has led to an ever
increasing need to develop ML-assisted techniques for automatically segmenting
images. In the past decade, there have been significant efforts to develop deep
learning architectures used for image segmentation; in particular, convolutional
neural networks have gained increased popularity.75 Different from image classifica-
tion or object recognition, image segmentation can be divided into two major types:
semantic segmentation and instance segmentation. Specifically, semantic segmen-
tation is the process of detecting each pixel in an image that belongs to the same
class of object. Instance segmentation is among the most difficult computer vision
tasks; it denotes the method of detecting each distinct object in an image. In a recent
review, Guo et al. summarized the field of semantic segmentation using deep neural
networks in general, and they placed the existing segmentation methods into three
categories: region-based semantic segmentation, fully convolutional network
(FCN)-based semantic segmentation, and weakly supervised segmentation. Here
we provide a brief history of well-known deep learning-based algorithms for image
analysis.
Regions with convolutional neural network features (R-CNN)76 is among pioneering

region-based methods for objection detection and semantic segmentation. R-CNN
models generate region proposals within selective search,77 compute CNN features
from each region proposal, and finally perform region classification using the ex-
tracted features with a support vector machine (SVM). R-CNN has shown significant
improvements over traditional CNN models for solving the task of object detection,
while it encounters difficulties in dealing with the task of image segmentation, such
as the compatibility of features, computational cost and data storage. Many encour-
aging approaches have been developed on the basis of FCN proposed by Long
et al.78 in 2014. The fundamental concept of FCN-based methods is that they learn

ll
OPEN ACCESS
Review
Figure 3. Workflow of using deep learning on image analytics

A typical workflow of machine learning includes five key steps: task analysis, data acquisition,
feature engineering, model building, and model application.
a mapping from pixels to pixels without extracting the region proposals.79 In fact,
the growing R-CNN family was greatly influenced by FCN, and some notable archi-
tectures include Fast R-CNN80 and Faster R-CNN.81 The Mask R-CNN algorithm was
proposed by He et al.82 in particular to improve object instance segmentation. It ex-
tends Faster R-CNN by adding a third branch that generates the segmentation mask
on each region of interest (RoI) in addition to the class label and bounding-box
offset. Moreover, Mask R-CNN replaces the traditional RoIPool module used in
Faster R-CNN for feature extraction with a more accurate one, RoIAlign, thus
enabling a pixel-to-pixel alignment between network inputs and outputs. Mask
R-CNN was shown to be very successful in multi-task segmentation models for
numerous application areas.83–85 Recently, Jiang et al.86 combined Mask R-CNN
in the study of the microstructure of lithium-ion battery cathodes.
Workflow of using deep learning on image analysis

Building a deep learning model is in principle no different from any other ML
method. A typical workflow of ML includes five key steps: task analysis, data acqui-
sition, feature engineering, model building, and model application, as depicted in
Figure 3.
Task analysis
The initial step in any ML project is task analysis, that is, understanding the problem.
Specifically, this stage is to define the learning problem, including the task category
and the goal of the ML model, for example, classification, regression, or clustering.87
Understanding the learning task is important, and often considered the most essen-
tial step for an ML project. This can be understood by the fact that the ‘‘defined’’ goal
must be potentially learnable from collected data. It is important to note that the
wrong choice of a learning goal can lead to ML models with large errors or

ll
OPEN ACCESS
Review
uncertainties.87 Thus, understanding the problem guides the choice of ML strate-

gies. For example, when solving imaging tasks (such as object recognition, segmen-
tation, or restoration), CNNs are typically the default method of choice.
Data acquisition
Data plays a crucial role in training an ML model. The more data on the algorithm, the
more accurate the model application. When generating and exploring datasets for an
ML study, care needs to be taken for ensuring the quality and the size of datasets, in
the sense of being both reliable and diverse. The ever increasing massive amount of
data from both experiments and computations enable many applications using machine
learning approaches.88–91 Data acquisition involves collecting data from available (and
reliable) sources for further operations, such as storing, cleaning, and preprocessing.
There are two types of collected data, self-generated and open-source databases.
Here we refer the reader to some of the important open-source platforms in materials
science, including Automatic-FLOW (AFLOW),92 the Materials Project’s software suite
comprising the Python Materials Genomics (pymatgen) materials analysis library,93 the
Automated Interactive Infrastructure and Database for Computational Science
(AiiDA),94 and the Novel Materials Discovery (NOMAD) repository.95
Feature engineering
A key factor to the success of all ML studies is the representation of the data. Typically,
raw data cannot be directly used for training an ML model. Therefore, extracting
informative features from the raw data and transforming them into an individual
ML algorithm will directly affect the interpretability and performance of a learning
model. This process is often referred to as feature engineering (or feature extraction),
that is, using the domain knowledge to better represent the input data for more effec-
tive ML algorithms. The selected features are often termed descriptors. Feature en-
gineering helps reduce the number of resources required by extracting useful fea-
tures from existing data, and it facilitates the subsequent learning and
generalization steps.87 There have been significant efforts to develop feature
engineering approaches in recent years.96–98 There are certain requirements for
good descriptors. For example, features in materials science are often based upon
the atomic structure and should capture sufficient information about a given system.
The calculation of the descriptor should be efficient enough. On the other hand,
feature engineering for a deep learning model is embedded in the learning algorithm
where features are extracted automatically via neural networks. Compared with other
types of ML algorithms, DL-based approaches usually need a huge training dataset.
Therefore, training a DL model is closely related to the fast growing GPU system to
reduce execution time. For the task of imaging, features (or descriptors) include prop-
erties such as corners, edges, region-of-interest points, ridges, and so on. Another
important aspect is that real-world applications typically suffer from the lack of avail-
able (labeled) data. In that sense, it is often the case that image data are collected
from different resources (e.g., shared, self-generated, simulated, synthetic).99
Although it is generally acknowledged that a larger training dataset yields better per-
formance of model application, one cannot ignore data quality (e.g., variety and ve-
racity). It is thus common to improve the data quality via image preprocessing steps,
such as data cleaning, data standardization, and data discretization. Particularly for
dealing with data from multiple sources, data standardization is an important step to-
ward scaling the data to a uniform format for further processing.
Model building
The next step is to use the prepared data for model building, more specifically,
choosing machine learning algorithms on the basis of the learning task and the

ll
OPEN ACCESS
Review
available dataset. Some typical models include ANN, SVM, decision trees, Gaussian
process, and regression analysis. The dataset size is a critical factor for training ML
models, and it even determines the available choices among different types of ML
algorithms. Approaches such as kernel ridge regression, support vector machines,
and decision trees are suitable for smaller dataset sizes, mainly because of the
complexity and computational cost that grow rapidly with the number of training
samples. On the other hand, neural network-based approaches can solve tasks
with a larger amount of data points involved, e.g., image analysis and audio recog-
nition. As illustrated in Image segmentation, CNNs are among the best choices for
image segmentation. Some recent applications of training deep learning networks
for energy materials will follow in the next section.
Model application
The final step in the machine learning workflow is the model application, which in-
volves performance evaluation and prediction of the properties or making deci-
sions/classifications for new data. This process applies to all application areas of
ML, with the aim of achieving the generalization of the model to unseen data.7 ML
models are trained on data by ML algorithms, and model performance depends
heavily on hyperparameters that can be tuned while creating the model. As for neu-
ral networks and deep learning methods, one can select hyperparameters such as
the number of hidden layers and the number of nodes in each layer. During the
model training process, hyperparameters are adjusted by iteratively minimizing
the loss function, in order to find the optimal set of configurations for the best
performing model. This procedure is often referred to as hyperparameter tuning
(or hyperparameter optimization), which will certainly affect the predictive and
computational performance of the model. It should be noted that unbiased and
robust performance evaluation remains one of the most significant challenges in ma-
chine learning development. Model performance can be evaluated by using
different metrics. For regression models, commonly used statistical quantities
include the mean absolute error (MAE), mean square error (MSE), root-mean-square
error (RMSE), determination coefficient (R2), and correlation coefficient (R). When
solving classification problems, a surrogate loss function, such as a confusion matrix,
or receiver operating characteristic (ROC) curves, the area under the curve (AUC), is
typically used. For a more detailed discussion, the reader is referred to existing text-
books on the topic.8,46
APPLICATIONS TO ENERGY MATERIALS IMAGING

There are various existing and emerging experimental methods used to detect and
characterize energy materials, and typically only their combination allows a more
comprehensive understanding of structure-property relationships. Unlike imaging
techniques such as TEM, STEM, AFM, and X-ray microscopy, ML-based methods
are at a very early stage for energy materials imaging. In this section, we review
the recent advances in using machine learning on image analytics for battery mate-
rials, solar cells, and fuel cells (see Table 1).
Battery materials
Climate change has emerged as the biggest threat facing our world today. Batteries
are arguably a key technology in tackling issues of greenhouse gas emissions and
large-scale energy storage. Among rechargeable batteries, lithium-ion batteries
are by far the dominant battery technology in most applications, including transpor-
tation and portable electronics. However, current-generation lithium-based batte-
ries are approaching their performance limits.100 Many difficulties have been
encountered in attempts to understand battery materials themselves, as well as

ll
OPEN ACCESS
Review
Table 1. A summary of the application cases using machine learning-based imaging techniques for energy materials
Characterization
Materials method Target properties Algorithms Prediction accuracy Reference
NMC X-ray nano-CT particle identification Mask R-CNN 92.2% Jiang et al.86
GSCE synthetic data 3D reconstruction CNN(3D U-net) not given Müller et al.99
and XTM
NMC X-ray nano- 3D reconstruction random forest in good agreement Nguyen et al104
holotomography with FIB/SEM data
LiCoO2 simulated data crack detection MLP; watershed 73% Finegan et al.37;
and X-ray nano-CT Petrich et al.
NMC light microscopy defect detection CNN 0.71–1 Badmos et al.107
Li | LLZO | Li X-ray micro-CT pore formation CNN outperform the Dixit et al.113
interfacial kinetics binarization process
ASSLiB X-ray nano-CT identification of low- CNN(U-net) 95% Kodama et al.114
contrast CT images
NMC FIB-EBSD grain boundary enhancement CNN (3D U-net) not given Furat et al.115
LiCoO2/LASGTP STEM-EELS Li-ion dynamics sparse coding not given Nomura et al.121
NMC hyperspectral identification of spectral MCR-ALS 94% Baliyan et al.123
Raman signals and capacity degradation
NMC; Ni/YSZ X-ray nano-CT stochastic reconstruction DC-GAN not given Gayon-Lombardo
et al.142
PV modules EL defect classification SVM; CNN 82.44%; 88.42% Deitsch et al.128
PV modules EL defect detection CNN 93.02% Akram et al130
PV modules EL defect classification SVM; CNN 72.74%; 91.58% Ahmad et al.131
PV modules EL defect classification GAN/CNN 83% Tang et al.132
PV modules EL multitype defects detection Mask R-CNN 96% Zhao et al.133
and defect screening criteria
PV modules multispectrum surface defects multispectral CNN 94.3% Chen et al.134
GDC/LSC/pores FIB-SEM phase identification DeepLabV3+ 78% Hwang et al.135
Ni/YSZ FIB-SEM phase identification DeepLabV3+ 91.9–95.7% Hwang et al.136
Ni/YSZ; Ni/ FIB-SEM phase identification and Patch-CNN >96% Sciazko et al.137
GDCLSCM/GDC 3D reconstruction
MoNi4 X-ray nano-CT 3D reconstruction SVM; R-CNN not given Topal et al.138
Pt-Co/C XANES-CT degradation unsupervised learning not given Tan et al.139
cathode
GDL/MPL/CL X-ray micro-CT monitoring PA migration random forest (Ilastik) not given Bailey et al.143
Pt NPs TEM particle size analysis CNN (U-net) 86% Colliard-Granero et al.141
CL, catalyst layer; CNN, convolutional neural network; EL, electroluminescence; GDC, gadolinium-doped ceria; GDL, gas diffusion layer; GSCE, graphite-silicon
composite electrode; LASGTP, Li1+x+yAlx(Ti,Ge)2 xSiyP3 yO12; LSC, lanthanum strontium chrome; LSCM, lanthanum strontium chrome manganese; MCR-ALS,
multiple-curve resolution; MLP, multilayer perceptron; Nano-CT, nano computed tomography; Ni/YSZ, Y2O3-stabilized ZrO2; NMC, Li(Ni1-x-yMnxCoy)O2; Pt
NPs, platinum (Pt) nanoparticles (NPs); PV, photovoltaic; R-CNN, regions with convolutional neural network features; SVM, support vector machine; TEM, trans-
mission electron microscopy; XTM, X-ray tomographic microscopy.
complex and dynamic processes at battery interfaces.83,101,102 In fact, interfaces in

batteries remain a big challenge, and it is by far the least understood aspect of bat-
tery studies. The lack of proper understanding of critical processes taking place at
interfaces, such as formations of dendrites, solid electrolyte interphase (SEI), and
cathode-electrolyte interface (CEI), prevents us from developing ultra-high-perfor-
mance batteries. Therefore, it is clear that more efforts are needed. Recently, ma-
chine learning approaches have received increasing attention as promising tools
for accelerating the discovery of battery materials.36,50,55,103 Here we discuss the lat-
est applications of using ML to image analytics, and focus on two aspects: (1) the
identification of microstructures and (2) the enhancement of the image quality.
Identification of microstructures
As discussed in Current challenges, microscopic images contain abundant informa-
tion regarding the material; thus their quantification through various imaging tech-
niques plays a crucial role in understanding the structure-property relationships.

ll
OPEN ACCESS
Review
Figure 4. Machine learning-assisted microstructure analysis of lithium-ion batteries

(A) More than 650 unique particles of different sizes, shapes, positions, and degrees of cracking were successfully identified and isolated from the
imaging data automatically. Comparison of conventional segmentation results and the machine learning-assisted segmentation results for a few
representative particles. Different colors denote different particle labels. Adapted with permission from Jiang et al. 86 Copyright 2020, Nature.
(B) Examples of the segmentation results in 2D (i) and 3D (ii) using the machine learning segmentation plugin, Trainable Weka. (iii) Visualization of the
individual NMC particle colored in red in (ii) along with the interfacial area with the other phases.
Adapted with permission from Nguyen et al. 104 Copyright 2021, John Wiley.
Deep learning approaches have been recently used to identify microstructures of

battery materials.
X-ray tomography has shown great promise in understanding lithium-ion batteries

on the basis of imaging data of porous materials in a nondestructive and possibly
quantitative way. Because of the complex composite of LIBs, it is non-trivial to reli-
ably distinguish multiple phases including active materials (AMs), carbon/binder
domain (CBD), and pore structures. Jiang et al.86 carried out a combined study of
high-resolution X-ray nano-tomography (nano-CT) characterization, numerical
modeling, and statistical learning on the particle-carbon/binder detachment in the
Ni-rich LiNi0.8Mn0.1Co0.1O2 (NMC) composite cathode, and they used the Mask
R-CNN deep neural network architecture for particle identification and segmenta-
tion. Figure 4A86,104 shows a comparison between the traditional data and the ma-
chine learning-assisted segmentation results on particles with different shapes,
sizes, positions, and degrees of cracking. In this study, they showed that more
than 650 unique particles were identified and isolated automatically and the trained
model enabled an accurate segmentation of the active particles, the carbon/binder
domain, and the pore network of the NMC composite electrodes, with accuracy of
92.2% on the validation dataset. On the basis of the ML segmentation result, the

ll
OPEN ACCESS
Review
detachment behavior of the NMC particles was further studied, including the degree
of detachment, volume, and relative intensity. The authors demonstrated the versa-
tility of combining experiment, modeling, and machine learning techniques for
understanding fundamental mechanisms of the microstructure of the composite
electrodes and their performance. Another example is a recent study by Nguyen
et al.,104 who combined the machine learning segmentation approach with X-ray
holo-tomographic data to investigate the effects of the microstructure on the elec-
trode electrochemical performance, as illustrated in Figure 4B. In this work, the
random forest algorithm was used for the segmentation via the Trainable Weka plu-
gin in ImageJ.105 Their ML model was shown to correctly identify the active particles,
CBD, and pores as well. The validation was done by comparing the segmentation
results between the X-ray holo-tomographic data using the ML algorithm and those
of high-resolution focused ion beam (FIB)-SEM data. The foregoing discussion dem-
onstrates that ML-based segmentation methods are powerful tools for the quantifi-
cation of microstructural properties of batteries.
The effect of the microstructure of electrodes on the charge and discharge perfor-
mance of batteries is an ongoing challenge. For instance, among various failure
mechanisms in battery safety, thermal runaway has no doubt become a critical fac-
tor, yielding the need to investigate how structural changes affect this issue. In
particular, cracked particles are often formed during the thermal runaway in the elec-
trode materials, and their identification is non-trivial. A study by Petrich et al.106 was
considered among the pioneering work using machine learning for automatic detec-
tion of particle cracks in lithium-ion batteries, and a schematic illustration is shown in
Figure 5. In this study, the investigators presented a classification method based on
a stochastic 3D microstructure model using simulated data. For the classification, a
multilayer perceptron (MLP; a neural network with one hidden layer) model was
developed. This ML-based segmentation approach allows to detect the relationship
between pairs of particles in tomographic image data of the post-mortem cell, that
is, whether the two particles are the result of breakage; the validation was done by
applying the algorithm to hand-labeled X-ray nano-CT data from a real LiCoO2 elec-
trode, with an overall accuracy of 73%. More specifically, three classes of particle
pairs can be categorized: (1) BROKEN: the particle pair originated from the same
particle that broke apart during the thermal runaway; (2) WATERSHEDSEP: the par-
ticle pair belonged to two neighboring particles in the tomographic image and split
by the watershed algorithm; and (3) PARTICLESEP: the particle pair consists of un-
correlated particles. It was found that the major difficulty for the proposed algorithm
lies in distinguishing particles between BROKEN and WATERSHEDSEP, thus calling
for further improvements. All in all, this study represents a step towards ML-based
segmentation of tomographic image data of LIBs; a good performance was shown
for the proposed ML algorithm and the time-consuming hand labeling is greatly
reduced.
Deep learning methods were recently applied to detect microstructural defects in

lithium-ion battery electrodes from light microscopy images.107 Certain types of de-
fects (such as particle cracks, particle agglomerations, layer deformation, and impu-
rities) can occur during the production process of electrodes, which influence the
performance and cycle life of LIBs. Thus, monitoring and detecting microstructural
defects will help understand changes in electrode morphology and develop ultra-
high performance batteries. The study by Badmos et al.107 presents CNN-based
deep learning models for evaluating the quality of LIBs from images of the elec-
trodes (cf. Figure 6). In this work, defect classification was conducted on light micro-
scopy images using six different CNN models. The image dataset contains 3,286

ll
OPEN ACCESS
Review
B C
Figure 5. Machine learning for automatic detection of particle cracks of lithium-ion batteries
(A) Overview of the model development for crack detection in lithium-ion batteries.
(B) Steps for breaking particles.
(C) Six examples of particle pairs with their true and predicted class label.
Adapted with permission from Petrich et al. 106 Copyright 2017, Elsevier.
original images in total, split into a training set (2,284 images) and a test set (1,002
images). Four types of defects were considered: layer deformation, metal particle
contaminants, non-uniform coating, and artifacts. The transfer learning method
was used as an effective way to improve the performance of CNN models. It was
shown that the proposed method is able to identify various types of defects from mi-
croscopy images of LIB cells without the requirement of manual feature extraction.
The transfer learning approach outperforms over traditional learning algorithm.
Despite the good performance of the trained model for a particular task, the
misclassification of defects could appear when applied to a large number of unla-
beled images. The comparison of CNN models results with manual inspection re-
sults and the classification of defects is still missing, calling for further improvements.
Solid-state batteries are considered the next generation rechargeable battery

because of their high energy density and safety.108–112 Tracking morphological
changes in electrodes/solid electrolytes interfaces remains a difficult task. For
example, a challenging problem one may often encounter is the identification of
Li species. Dixit et al.113 used imaging techniques to track morphological trans-
formations at Li metal solid-state batteries. Synchrotron X-ray tomography
characterization was carried out, showing evidence of pore formation in Li metal
upon tripping. Within Li metal electrodes, high-resolution imaging of Li is

ll
OPEN ACCESS
Review
Figure 6. Structure of proposed convolutional neural network (CNN) based deep learning models for evaluating the quality of lithium-ion batteries
from images of the electrodes
(A) Large image of electrode micrograph.
(B) Extracted patches.
(C) Pre-trained CNN.
(D) New classification module.
(E) Defect detection and localization.
(F) Cropped result.
Adapted with permission from Badmos et al. 107 Copyright 2020, Springer.
prohibitive, however, because of its low atomic number. The deep learning model
enables to process low-contrast images (Li metal and pores). Upon the use of the
CNN architecture (ResNet34) trained on 800 images, the segmentation of lithium
and pores was achieved from the reconstructions of in situ experiments and a sche-
matic illustration is shown in Figure 7A. This work provides physical insights into
microstructure transformations in Li metal and the solid electrolyte upon cycling.
Another example of coupling DL-based segmentation with synchrotron X-ray to-
mography is a recent work that uses deep learning with U-net for materials identifi-
cation of all-solid-state LIB from high-resolution and low-contrast nano-CT im-
ages.114 The problem to be solved in the case is the segmentation of the
overlapped active materials (the NMC particles) and solid electrolytes, where the
conventional CT values failed to distinguish and manual segmentation is in practice
almost prohibitive. By comparing the results from manual segmentation and the
extended U-net segmentation, the authors concluded the U-net architecture allows
accurate and efficient image segmentation in three dimensions, as seen in Figure 7B.
Enhancement of the image quality

Machine learning methods have been applied to denoise and enhance the quality of
microscopy images, which facilitates quantitative microscopy analysis of battery ma-
terials. An in-depth understanding of lithium ions’ intercalation and degradation
mechanisms is exceptionally challenging. Furat et al115 investigated grain orienta-
tions and morphologies of NMC particles by a combination of FIB-EBSD and ma-
chine learning techniques. A CNN architecture, 3D U-net,116,117 was used to
enhance the grain-wise segmentation of FIB-EBSD image data. In this study, there

ll
OPEN ACCESS
Review
Figure 7. Two applications using machine learning-aided imaging for solid-state batteries
(A) Pore formation and interfacial kinetics in lithium metal anodes. Adapted with permission from Dixit et al. 113 Copyright 2020, ACS.
(B) Three-dimensional structural measurement and material identification of an all-solid-state lithium-ion battery by X-ray nano-tomography and deep
learning.
Adapted with permission from Kodama et al. 114 Copyright 2021, Elsevier.
are three steps for segmentation. In the first step, stack alignment was performed for
EBSD image data using the pyStackReg package. The second step is the CNN-
based grain boundary enhancement using a 3D U-net. This imaging process helps
reduce the amount of preprocessing required for the latter segmentation. The final
segmentation is achieved by applying a marker-based watershed algorithm.118–120
The results show the use of machine learning can improve the identification and la-
beling of distinct grains of lithium-ion electrode particles, as shown in Figure 8.
The high interfacial resistance between the cathode and the solid electrolyte remains a
big challenge for solid-state LIBs, while direct visualization of the ion dynamics
enables us to understand fundamental properties in batteries. A recent study
shows lithium-ion dynamics at LiCoO2/LASGTP (referred to as Li1+x+yAlx(Ti,Ge)2 x
SiyP3 yO12) in a solid-state battery by operando STEM-EELS.121 The authors applied im-
age denoising and super-resolution via sparse coding (an unsupervised type of

ll
OPEN ACCESS
Review
Figure 8. Visual validation of the neural networks

(A) Planar 2D section through the aligned EBSD IQ image for which no labeling of grain boundaries,
grain interiors and background is available.
(B–D) The corresponding predicted grain boundaries (B), grain interiors (C) and background (D) of
the trained CNN.
Adapted with permission from Furat et al. 115 Copyright 2021, Elsevier.
representation learning)122 to tackle the issue of insufficient temporal resolution of the

STEM-EELS measurements. The prepared thin-film battery consists of 50-mm-thick
LASGTP sheet electrolyte and 230-nm-thick LiCoO2 cathode deposited on the electro-
lyte, together with Au and Pt being as current collectors and in situ formed anode.
Sparse coding (SC) is a class of unsupervised learning algorithms that aim to represent
an input vector (e.g., pixel vector) as an efficient linear combination of a set of basis vec-
tors using a large number of unlabeled input data, thus it is particularly useful for learning
tasks where labeled data are insufficient.122 In this work, SC was used for image denois-
ing and super-resolution. The training dataset contains 157 operando STEM-EELS four-
dimensional spectrum images for learning the task of visualizing the dynamic behavior of
lithium ions in the LiCoO2 cathode. The results show a significant improvement in the
spatial and temporal resolution of lithium imaging (see Figure 9), thus the combination
of operando STEM-EELS and sparse coding offers a promising way of probing the fun-
damentals of solid-state electrochemistry.

ll
OPEN ACCESS
Review
Figure 9. Dynamic imaging of Li concentration assisted by unsupervised learning

(A–D) Part of the time series of 157 (A) ADF-STEM images, (B) Li maps obtained from the original spectrum images, (C) Li maps obtained from the MLLS-
fitted spectrum images, and (D) Li maps obtained by sparse coding (SC) reconstruction. Scale bar (A), 100 nm.
(E–G) Changes in the average Li concentrations in the areas indicated in (B)–(D).
Adapted with permission from Nomura et al. 121 Copyright 2020, Nature.
Recent advances in Raman imaging combined with machine learning have allowed the
identification of multiple chemical components and inhomogeneities of LIB electrodes.
A machine learning analysis framework (MLAF) on the basis of multiple curve resolution
(MCR-ALS) was proposed in the study of Baliyan and Imai,123 aiming for automatic hy-
perspectral Raman analysis of LIB electrodes. The workflow of the MLAF includes several
steps: preprocessing the hyperspectral Raman data, extracting reliable spectral signa-
tures and assigning class labels, training neural network models, and evaluating the
applicability of the pre-trained NN on new datasets in real time. A modified principal-
component analysis was carried out to eliminate the background and cosmic noise of
raw data with almost no human aid, thus improving the quality of data processing.
The Raman data were measured for pristine LIB sample (the original cathode), 500_IN
(the LIB sample from the inner cathode region after 500 charging/discharging cycles),
and 500_OUT (the LIB sample from the outer cathode region after 500 charging/dis-
charging cycles). By comparing the results from three types of analytics (human, unsuper-
vised, and supervised intelligence) for the above three datasets, the authors demon-
strated both unsupervised and supervised intelligence analyses show a great match
with univariate results (cf. Figure 10123); the trained NN model can achieve an accuracy
as high as 94% for class labeling. The NN model has been shown to be a powerful tech-
nique for identifying multiple components, for which human analysis may often fail.
Additionally, the MLAF enables tracking of the degradation of the LIB electrode by

ll
OPEN ACCESS
Review
Figure 10. Machine-learning-based analytical framework for automatic hyperspectral Raman analysis of lithium-ion battery (LIB) electrodes
(A–C) Results from three types of analytics are compared for pristine LIB samples: (A) human, (B) unsupervised, and (C) supervised intelligence. The
univariate analysis by human intelligence identifies the existence of two components: carbon and LiMO2 (M = Ni, Mn, Co); both unsupervised and
supervised intelligence analyses show a great match with univariate results.
Adapted with permission from Baliyan and Imai. 123 Copyright 2019, Nature.
evaluating the capacity retention coefficient. All in all, the machine learning-aided
Raman imaging technique has shown great promise in accelerating an automatic, effi-
cient and accurate hyperspectral Raman analysis of LIB electrodes, and thus can be
used in a wide variety of applications, such as real-time vibrational spectroscopy,
multi-component chemical reactions, and environmental monitoring.
Solar cells
The study of solar cells is among the most urgent needs for accessing global sustain-
able development. The search for high power conversion efficiency (PCE) of photo-
voltaic (PV) materials has led to rapid progress in developing new technologies for

ll
OPEN ACCESS
Review
large-scale applications of solar energy in the past decade.124,125 Recent advance-

ments in ML methods have shown great promise in accelerating solar cell design
and discovery.126,127 Here we discuss applications for the classification of defects
on the basis of imaging methods.
Defects during the operation of solar cells have received significant attention because
of their potential harm in power loss and module degradation. Therefore, defect detec-
tions and their impacts on the performance of a PV system play a critical role in moni-
toring the quality of PV modules. Infrared (IR) and electroluminescence (EL) are the most
widely used nondestructive imaging techniques to evaluate the quality of PV modules,
providing information such as defect types and faults. IR imaging is a useful tool for the
characterization of solar cells with many advantages (e.g., being nondestructive and
time saving); most of the general electrical problems can be detected using IR thermog-
raphy. However, the downside is the relatively low resolution of IR cameras, which pre-
vents the detection of small defects such as microcracks.128 On the other hand, EL im-
aging is another powerful method to characterize different types of defects with high
resolution in the PV field.129 Although a manual inspection of EL images requires
well-trained specialists, it is tedious and time consuming. DL-based imaging techniques
are becoming evident in tackling those issues.128,130–134
The study by Deitsch et al.128 was considered among the pioneering work using deep
learning approaches for EL imaging in the field of solar cells. The authors proposed an
ML-based framework for normal/defective classification of PV modules in high-resolu-
tion EL images. Two algorithms, namely, SVM and CNN classifiers, were used, and
both were trained using a public solar cell dataset containing 1,968 mono- and poly-
crystalline EL images. The results indicated the CNN model slightly outperforms the
SVM, with an average accuracy of 88.42% for the CNN versus 82.44% for the SVM,
while both approaches show great promise in accurately monitoring defect probabil-
ities for PV cells (cf. Figure 11128). Afterward effort has been made to further identify
defect types. Various DL-based methods were proposed for more accurate and effi-
cient classification of PV cell defects. Akram et al.130 introduced a framework using
CNN, and trained on a public solar cell dataset. They showed improved results
compared with the work of Deitsch et al. on the same dataset, with an average accuracy
of 93.02%. The proposed CNN architecture was shown to be computationally efficient
and the authors demonstrated the possibility of using it for defect types detection by
increasing the training data. In another study,131 they used CNN and SVM algorithms
to achieve the classification of seven types of solar cell defects (one normal class and six
defective classes). It was shown that the proposed CNN architecture yields better per-
formance for solar cell defect classification than the SVM (with an average accuracy of
91.58% versus 72.74%). However, the image training set contains only a limited num-
ber of samples. To tackle this issue, Tang et al.132 developed a CNN-based defect clas-
sification deep learning model for PV cells by using a generative adversarial network-
based data augmentation technique for the EL images. The proposed CNN architec-
ture, trained with the dataset generated within the GAN model, was shown to yield
high accuracy and efficiency for different types of defect classification, including
defect-free, microcracks, finger interruptions and breaks in EL images. One of the re-
maining challenges is to classify multiple coexisting defects in solar cells. A recent
study by Zhao et al.133 focused on the above issue, presenting a solution of coupling
a deep learning-based approach with a large amount of EL images training data of PV
modules having diverse types of defects. They investigated the automatic detection of
multitype defects by using Mask R-CNN (ResNet-101-FPN backbone) for the EL im-
ages. A dataset containing 5,983 labeled EL images of defective solar cells was used
to classify 19 different types of defects. Then the Mask R-CNN model was trained on

ll
OPEN ACCESS
Review
Figure 11. Automatic classification of defective photovoltaic cells in electroluminescence (EL) images
(A) Various intrinsic and extrinsic defects in monocrystalline and polycrystalline solar cells.
(B) A schematic representation of the SVM classification process, consisting of masking, key-point detection, feature description, encoding, and
classification.
(C) The proposed architecture of the modified Vgg-19 network used for the prediction of defect probability in EL images of solar cells (left panel). The
performance of the CNN in each category (right panel).
Adapted with permission from Deitsch et al. 128 Copyright 2019, Elsevier.
the top 14 defects among those 19 classes. In this work, the authors discussed in detail
optimizing and selecting the ML model and the best model yielded 70.2% mAP50
(mean average precision with at least 50% localization accuracy) for the aforemen-
tioned training set. Aiming at applying the ML method to a real production line,
they performed an analysis of an object detection-based confusion matrix, set specific
screening criteria for each type of defect, and finally applied the trained model to addi-
tional 4,791 EL images of solar cells for defect classification. The results showed high
accuracy of the proposed ML-based framework for automatic defect detection, which
offers a way of using it in real production lines. One step further is to detect surface de-
fects in PV modules. Typically, two major problems need to be tackled: the segmenta-
tion of multitype defects and automatic multispectrum feature extraction and inspec-
tion of the surface. Chen et al.134 pioneered a deep learning study using a multispectral
CNN for the visual detection of surface defects of polycrystalline silicon solar cells. The
authors proposed a multispectral CNN model to distinguish various surface defects,
shapes and background noise (cf. Figure 12134). The accuracy and robustness of the

ll
OPEN ACCESS
Review
Figure 12. Solar cell surface defect inspection based on multispectral convolutional neural
network (CNN)
(A) The feature of solar cell CNN and multispectral solar cell CNN.
(B) The receiver operator characteristic (ROC) curves using the four detection methods.
Adapted with permission from Chen et al. 134 Copyright 2020, Springer.
CNN algorithm were demonstrated by using the experimental results and K-fold cross-
validation. The proposed CNN model has shown great promise in effectively inspect-
ing the solar cell surface defects, and thus can be used to facilitate the manufacturing
process of solar cells.
To sum up, machine learning-assisted defect detection in solar cells still requires
further research in terms of the types of defects to be detected, such as the detection
of more types of defects or the detection of a variety of coexisting defects.

ll
OPEN ACCESS
Review

ll
OPEN ACCESS
Review
Figure 13. Three applications using machine learning-aided imaging for fuel cells
(A) Integrated application of semantic segmentation-assisted deep learning to quantitative multi-phased microstructural analysis in composite
materials. Adapted with permission from Hwang et al. 135 Copyright 2020, Elsevier.
(B) Schematic of operando XANES-CT imaging under PEFC operating conditions and reconstructed 3D images of the cathode catalyst layer in an MEA.
Adapted with permission from Tan et al. 139 Copyright 2019, ACS.
(C) Deep-learning-based approach for the high-throughput analysis of the particle size distributions from TEM images of carbon-supported catalysts
for polymer electrolyte fuel cells. The left panel shows the TEM image, ground truth mask, and predicted mask. The right panel compares the manually
measured particle diameter with the deep learning-based computed particle diameter.
Adapted with permission of Colliard-Granero et al. 141 Copyright 2022, RSC.
Fuel cells
Deep learning-based image analysis has been recently applied to study the
morphology and surface structures of fuel cells.135–139 The search for materials with
electrocatalytic activity, long-term stability, and low cost remains an increasingly active
field of research. Here we shall discuss recent advances in two major types of fuel cells:
solid oxide fuel cells (SOFCs) and polymer electrolyte fuel cells (PEFCs).
Solid oxide fuel cells

Solid oxide fuel cells have received continuous attention because of their advan-
tages, such as high power efficiency, fuel flexibility, eco-friendly design, and rela-
tively low cost. To quantify the microstructure of composite electrodes, Hwang
et al.135 pioneered a study by combining deep learning with FIB-SEM for semantic
segmentation of SOFCs. The model system they chose is GDC/LSC/pores, where
GDC stands for Gd2O3-doped CeO2 and LSC for La0.6Sr0.4CoO3 d. The
DeepLabV3+140 architecture with atrous convolution was used, trained on a dataset
of 40 images and tested on additional 9 images, respectively, as sketched in
Figure 13A. The trained model allowed the identification of three phases in the com-
posite cathode, that is, GDC, LSC, and pores. Even though the dataset was small,
promising results were obtained. It can be concluded that the proposed strategy en-
ables effective feature extraction for the microstructural information of the SOFCs,
including size distribution, volume fraction, and phase boundaries, thus allowing
the discovery and design of high-performance fuel cell catalysts. Following this
work, the authors extended the framework to Ni/Y2O3-stabilized ZrO2 (Ni/YSZ)
anode composites for microstructural analysis.136 The deep learning-based seg-
mentation was achieved through three steps: image data preprocessing, deep
learning implantation, and deep learning post-processing analytics. The input
data for the DeepLabV3+ algorithm comprise 120 original FIB-SEM images. The
trained model using electron micrographs was shown to be able to separate the
Ni, YSZ, and pore anode components. Furthermore, the applicability of the semantic
segmentation approach was validated in comparison with the ground truth images
(after eliminating curtain and edge effects) prepared by manual processing of well-
trained specialists. In this study, the proposed framework of combining ML with elec-
tron microscopy imaging represents a promising way of enhancing the quality of
poor images originating from artifacts in electron microscopy and can be applied
to understand the microstructure of composite electrodes for SOFCs.
Sciazko et al.137 recently developed an automated deep learning segmentation al-

gorithm on the basis of the patch-CNN network. This work aims to analyze 3D micro-
structures of solid oxide cells from FIB-SEM data. Particular attention has been paid
to the feasibility of the proposed methodology for various types of electrode micro-
structures and the efficiency of image processing. In this work, the solid oxide cell
samples investigated are Ni/YSZ, nickel-gadolinium-doped ceria (Ni/GDC), and
lanthanum strontium chrome manganese-gadolinium-doped ceria (LSCM/GDC).
An effective semantic segmentation network was designed in an encoder-decoder

ll
OPEN ACCESS
Review
manner on the basis of patch-convolutional layers, and the training/validation data-

sets for CNN were collected within the conventional semi-automatic image segmen-
tation method for phase separation of the SEM images. The results of the proposed
automatic CNN algorithm show great promise in microstructure segmentation ap-
plications, with an averaged pixel-based accuracy of more than 96%. In addition,
it was found that the microstructure reconstruction time is shortened by the order
of two in magnitude when using the patch-CNN approach.
Transition metal-based materials are of particular interest because of their role as alter-
natives to precious metal-based materials in many applications including electrocataly-
sis. The work of Topal et al.138 shed light on coupling machine learning with multiscale
X-ray tomography for the study of 3D morphology of electrocatalysts. The authors
applied this systematic approach to a novel catalyst system, Mo-Ni-based alloy electro-
catalysts anchored on MoO2 cuboids aligned on Ni foam. The use of machine learning
algorithms (R-CNN and SVM) ensures one obtains highly accurate 3D reconstruction
from micro-XCT and nano-XCT images. More specifically, the imaging artifacts of
XCT such as beam hardening were corrected and experimental uncertainty such as
misalignment and motions of samples were eliminated.138 The validation of the com-
bined scheme was achieved by a comparison of virtual cross-sections on the basis of
ML-assisted nano-XCT with the real FIB-SEM cross-sections. Such an approach will be
useful for a wide range of applications, including the design of robust 3D electrocatalysts
with advanced properties for large-scale hydrogen production.
Besides the identification of microstructures, machine learning has proved to be a

very useful tool in tracking dynamic processes for SOFCs. For instance, it is note-
worthy that the electrode degradation of Ni-based SOFCs is due mainly to coars-
ening and depletion of Ni under operation and understanding the degradation
mechanism requires detecting microstructure changes. This can be achieved by per-
forming operando experiments. However, unavoidable drawbacks (such as low res-
olution and signal noise) of raw experimental microscopy images prevent us from
reliably and quantitatively analyzing electrode dynamics, highlighting the need for
better imaging techniques. A recent combined study of deep learning with SEM im-
aging was carried out to tackle the above problem.144 Two machine learning algo-
rithms (CNN and GAN) were used to facilitate microscopy imaging of the Ni/YSZ
electrode from SEM image data. In this work, a CNN-based segmentation method
developed aims for an automated structure detection of low-resolution microscopy
images; its credibility was validated by the comparison of the results with manual
segmentation of the high-resolution SEM images. Another target of this study is
to generate SEM-quality images on the basis of the machine learning technique.
In this case, the proposed super-resolution network coupling with the pix2pix
GAN architecture145 enables image-to-image translation. More specifically, high-
resolution images can be generated by the GAN model from low-resolution images.
Such an ML-aided framework represents a promising way toward a quantitative eval-
uation of the dynamics of SOFC electrodes under operation and thus will be useful
for a wide variety of applications in energy materials.
Polymer electrolyte fuel cells

Machine learning techniques have been also applied to study polymer electrolyte
fuel cells.139,141,143 A study by Tan et al.139 was considered among the pioneering
work by combining operando X-ray absorption near-edge structure-computed to-
mography (XANES-CT) and machine learning to investigate the degradation behav-
iors of a P-Co/C cathode catalyst in a PEFC. Unsupervised learning was utilized to
study the 3D XANES-CT image data including the morphology, Pt density, Co

ll
OPEN ACCESS
Review
density, Co/Pt atomic ratio, and Pt valence states of the catalyst before and after a
PEFC-accelerated degradation test. The present study of the operando XANES-
CT coupled with data mining enabled us to understand the degradation mechanism
of bimetallic PEFC cathode catalysts.
Another application is a recent work that incorporates machine-learning segmen-

tation into X-ray micro-CT images of high-temperature polymer electrolyte fuel
cells (HT-PEFCs).143 This study analyzed 14 lab-based XCT datasets (Zeiss Xradia
520 Versa; Carl Zeiss) with the aim of accurately and efficiently investigating phos-
phoric acid (PA) leaching migrations, thus allowing links between microstructure
changes and electrochemical performance degradation mechanisms. The authors
used the procedure of accelerated stress tests (ASTs)146 for investigating HT-
PEFC degradation under high load cycling, such that high-resolution semi-quanti-
tative tomographic analysis was ensured. In this work, a two-stage image segmen-
tation framework was performed: (1) machine learning-based segmentation for
most voxel allocation, including catalyst layers (CLs), pores, gas diffusion layers
(GDLs), MPL, membrane, PA, via Ilastik,147 and (2) a localized segmentation for
membrane detection from the other phases using the Avizo package.148 The
results showed that significant changes in CL, GDL, MPL, membrane, and PA
leaching can be well identified. Such a combined X-ray lab-based micro-CT with
machine learning-based segmentation enables quantification analysis of micro-
structure changes, thus providing links of microstructures to electrochemical per-
formance of HT-PEFCs.
Although the coupling of machine learning with microscopy imaging in fuel cells and
electrocatalysis is now in its early stage, it has shown its critical role in revealing the funda-
mental properties of electrodes, electrolytes, and their interfaces. Because of the ability
to process quantities of images at high efficiency and precision, this method could be
popularized to make transparent and detailed 3D images in similar multi-phase mixtures
and make contributions to optimizing the electrochemical performance of fuel cells.
OUTLOOK
A long-standing challenge in molecules and materials is to uncover the structure-
property relationships for a given purpose, and this needs to integrate interdisci-
plinary research fields including physics, chemistry, mathematics, materials science,
and engineering related to materials. Recent advances in machine learning and arti-
ficial intelligence tools have opened new opportunities toward this goal. In partic-
ular, deep learning-based approaches have entered the field of microstructure visu-
alization and characterization in materials science. Today, the combination of
experimental, computational, and machine learning methods allows new physical
and chemical insights into materials design and discovery.
This review provided recent progress in using ML methods for multiscale imaging of
energy materials. Within the fast development of efficient GPU parallelization, deep
learning has enabled us to solve tasks using a large amount of data (with high dimen-
sionality). In this review, we discussed how deep learning can be used for image an-
alytics of energy materials, including batteries, solar cells, and fuel cells. Although
still at a very early stage, ML-assisted studies have shown great promise in identi-
fying microstructures, enhancing the image quality, and tracking dynamic processes
occurring both inside and at the materials interfaces. As an emerging and fast
growing field in materials science, there are many conceptual and practical chal-
lenges for further breakthroughs, and here we enumerate some of them.

ll
OPEN ACCESS
Review
First, along with the widespread use of ML approaches, they tend to become less under-
stood, yielding the problem of lacking reliance. To overcome this issue, two important
aspects need to be addressed: (1) Data: all ML algorithms not only require high-quality
data but also need to adequately use the data (e.g., avoiding inadequate sampling of
data while without wasting them). (2) Algorithm: new ML methods should aim at
improving the loss function, on the basis of the basic physical and chemical constraints.
Note that current deep learning-based studies are available mainly for extracting struc-
tural information, such as the identification and tracking of microstructures. It is crucial to
consider physical correlations between the structure and properties to gain meaningful
insights. On the other hand, developing data collection and preprocessing techniques
remains an important direction for improving the generalization performance of DL
models with insufficient (labeled) datasets. Some popular techniques include data argu-
mentation, transfer learning, and batch normalization, to name just a few. Second, hand
labeling of data is one of the most significant challenges for microscopy imaging of
experimental observations, but we need a lot more reliable experimental data to vali-
date ML models for realistic systems. The ever increasing training datasets are currently
available from open-source platforms, based mostly upon multiscale theoretical calcu-
lations of molecules and materials. It raises the need for building experimental open-
source databases, such as microscopy images. However, carrying out traditional exper-
iments typically requires a lot of time and cost. Therefore, how to efficiently and reliably
combine experiment and theory remains a continuous effort. Alternatively, developing
unsupervised or weakly supervised ML algorithms is an important direction for future
progress towards reducing significant time and cost. Global collaborations from
different labs are certainly optimal solutions for most ongoing challenges. At last, there
is an urgent need to develop educational programs that aim at collecting contributions
of the next generation of researchers to this emerging field. To achieve such a goal,
multidisciplinary scientists (e.g., microscopists, theoretical physicists, chemists, and
computer scientists) must work together in building joint programs for developing ML
and AI tools to solve materials questions. It is clear that experts should learn interdisci-
plinary languages (e.g., synthesis, characterization, visualization, modeling, program-
ming, architecture). We stress that (online) summer and winter school programs can
bridge the gap in multidisciplinary collaboration. To conclude, machine learning and
artificial intelligence tools are very useful for materials design and discovery, and their
power is becoming more and more apparent in all research areas.
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China
(grants 22075063 and U1932205), Chinesisch-Deutsches Mobilitätspropgamm (M-
0281), the Fundamental Research Funds for the Central Universities (grant HIT.O-
CEF.2021028), Natural Science Funds of Heilongjiang Province (grants ZD201
9B001 and LH2021B010), Heilongjiang Touyan Team (grant HITTY-20190033), the
Natural Science Fund for Distinguished Young Scholars of Chongqing (grant cstc20
21jcyj-jqX0003), Young Scientist Studio of Harbin Institute of Technology (HIT), and
funds from Chongqing Research Institute of HIT.
AUTHOR CONTRIBUTIONS
Conceptualization, G.-X.Z. and J.W.; investigation, Y.S., W.Z., H.A., G.-X.Z., and
J.W.; writing – original draft, G.-X.Z. and J.W.; writing – review & editing, G.-X.Z.,
J.W., and W.Z.; funding acquisition, J.W.; resources, Y.S., W.Z., and H.A.
DECLARATION OF INTERESTS
The authors declare no competing interests.

ll
OPEN ACCESS
Review
REFERENCES
1. Dresselhaus, M.S., and Thomas, I.L. (2001). 20. Berger, M., Yang, Q., and Maier, A. (2018). X- 33. Butler, K.T., Davies, D.W., Cartwright, H.,
Materials for clean energy. Nature 414, 331. Ray imaging. In Medical Imaging Systems: An Isayev, O., and Walsh, A. (2018). Machine
Introductory Guide, A. Maier, S. Steidl, V. learning for molecular and materials science.
2. Armand, M., and Tarascon, J.-M. (2008). Christlein, and J. Hornegger, eds. (Springer Nature 559, 547–555.
Building better batteries. Nature 451, International Publishing), pp. 119–145.
625–627. 34. Noé, F., Tkatchenko, A., Müller, K.R., and
21. Maire, E., and Withers, P.J. (2014). Clementi, C. (2020). Machine learning for
3. Larcher, D., and Tarascon, J.M. (2015). Quantitative X-ray tomography. Int. Mater. molecular simulation. Annu. Rev. Phys. Chem.
Towards greener and more sustainable Rev. 59, 1–43. 71, 361–390.
batteries for electrical energy storage. Nat.
Chem. 7, 19–29. 22. Usseglio-Viretta, F.L.E., Colclasure, A., Mistry, 35. Keith, J.A., Vassilev-Galindo, V., Cheng, B.,
A.N., Claver, K.P.Y., Pouraghajan, F., Finegan, Chmiela, S., Gastegger, M., Müller, K.R., and
4. Usiskin, R., Lu, Y., Popovic, J., Law, M., Balaya, D.P., Heenan, T.M.M., Abraham, D., Tkatchenko, A. (2021). Combining machine
P., Hu, Y.S., and Maier, J. (2021). Mukherjee, P.P., Wheeler, D., et al. (2018). learning and computational chemistry for
Fundamentals, status and promise of sodium- Resolving the discrepancy in tortuosity factor predictive insights into chemical systems.
based batteries. Nat. Rev. Mater. 6, 1020– estimation for Li-ion battery electrodes Chem. Rev. 121, 9816–9872.
1035. through micro-macro modeling and
experiment. J. Electrochem. Soc. 165, A3403– 36. Lombardo, T., Duquesnoy, M., El-Bouysidy,
5. Aspuruguzik, A., and Persson, K. (2018). H., Årén, F., Gallo-Bueno, A., Jørgensen, P.B.,
Materials acceleration platform: accelerating A3426.
Bhowmik, A., Demortière, A., Ayerbe, E.,
advanced energy materials discovery by 23. Trembacki, B.L., Mistry, A.N., Noble, D.R., Alcaide, F., et al. (2022). Artificial intelligence
integrating high-throughput methods and Ferraro, M.E., Mukherjee, P.P., and Roberts, applied to battery research: hype or reality?
artificial intelligence. Mission Innovation. S.A. (2018). Choice—mesoscale Analysis of Chem. Rev. 122, 10899–10969.
6. Tony Hey, S.T., and Tolle, K. (2009). The Forth conductive binder domain morphology in
lithium-ion battery electrodes. 37. Finegan, D.P., Scheel, M., Robinson, J.B.,
Paradigm: Data Intensive Scientific Discovery.
J. Electrochem. Soc. 165, E725–E736. Tjaden, B., Di Michiel, M., Hinds, G., Brett,
7. Himanen, L., Geurts, A., Foster, A.S., and D.J.L., and Shearing, P.R. (2016). Investigating
Rinke, P. (2019). Data-driven materials 24. Baruchel, J., Bleuet, P., Bravin, A., Coan, P., lithium-ion battery materials during
science: status, challenges, and perspectives. Lima, E., Madsen, A., Ludwig, W., Pernot, P., overcharge-induced thermal runaway: an
Adv. Sci. 6, 1900808. and Susini, J. (2008). Advances in synchrotron operando and multi-scale X-ray CT study.
hard X-ray based imaging. C. R. Phys. 9, Phys. Chem. Chem. Phys. 18, 30912–30919.
8. Mitchell, T.M. (1997). Machine Learning 624–641.
(McGraw-Hill International Editions). 38. Larmuseau, M., Sluydts, M., Theuwissen, K.,
25. Westneat, M.W., Socha, J.J., and Lee, W.-K. Duprez, L., Dhaene, T., and Cottenier, S.
9. Raina, R., Madhavan, A., and Ng, A.Y. (2009). (2008). Advances in biological structure, (2020). Compact representations of
Proceedings of the 26th Annual International function, and physiology using synchrotron microstructure images using triplet networks.
Conference on Machine Learning – ICML. X-ray imaging. Annu. Rev. Physiol. 70, npj Comput. Mater. 6, 156.
119–142.
10. Harju, A., Siro, T., Canova, F.F., Hakala, S., and 39. Jones, L., and Nellist, P.D. (2013). Identifying
Rantalaiho, T. (2013). Applied Parallel and 26. Ebner, M., Marone, F., Stampanoni, M., and and correcting scan noise and drift in the
Scientific Computing, pp. 3–26. Wood, V. (2013). Visualization and scanning transmission electron microscope.
quantification of electrochemical and Microsc. Microanal. 19, 1050–1060.
11. Olson, G.B. (1997). Computational design of mechanical degradation in Li ion batteries.
hierarchically structured materials. Science Science 342, 716–720. 40. Marturi, N., Dembélé, S., and Piat, N. (2014).
277, 1237–1242. Scanning electron microscope image signal-
27. Pietsch, P., and Wood, V. (2017). X-ray to-noise ratio monitoring for micro-
12. Panchal, J.H., Kalidindi, S.R., and Mcdowell, tomography for lithium ion battery research: a nanomanipulation. Scanning 36, 419–429.
D.L. (2013). Key computational modeling practical guide. Annu. Rev. Mater. Res. 47,
issues in integrated computational materials 451–479. 41. Holm, E.A., Cohn, R., Gao, N., Kitahara, A.R.,
engineering. Comput. Aided Des. 45, 4–25. Matson, T.P., Lei, B., and Yarasi, S.R. (2020).
28. Shanks, R.A. (2014). Chapter 2 - Overview: computer vision and machine
13. Zhou, X., and Thompson, G. (2017). Reference characterization of nanostructured materials. learning for microstructural characterization
Module in Materials Science and Materials In Nanostructured Polymer Blends, S. and analysis. Metall. Mater. Trans. A 51, 5985–
Engineering (Elsevier). Thomas, R. Shanks, and S. 5999.
14. Yan, H., Voorhees, P., and Xin, H. (2020). MRS Chandrasekharakurup, eds. (William Andrew
Publishing), pp. 15–31. 42. Na, J., Kim, G., Kang, S.-H., Kim, S.-J., and
volume 45 issue 4 Cover and Front matter.
Lee, S. (2021). Deep learning-based
MRS Bull. 45, 272–276. 29. Chen, R., Liu, P., Xiao, T., and Xu, L.X. (2014). discriminative refocusing of scanning electron
15. Cao, C., Toney, M.F., Sham, T.K., Harder, R., X-ray imaging for non-destructive microscopy images for materials science.
Shearing, P.R., Xiao, X., and Wang, J. (2019). microstructure analysis at SSRF. Adv. Mater. Acta Mater. 214, 116987.
Emerging X-ray imaging technologies for 26, 7688–7691.
energy materials. Mater. Today 34, 132–147. 43. Koza, J.R., Bennett, F.H., Andre, D., and
30. Tkatchenko, A. (2020). Machine learning for Keane, M.A. (1996). Automated design of
16. Yu, Z., Wang, J., and Liu, Y. (2020). High- chemical discovery. Nat. Commun. 11, 4125. both the topology and sizing of analog
dimensional and high-resolution x-ray electrical circuits using genetic programming.
31. Samaniego, E., Anitescu, C., Goswami, S., In Artificial Intelligence in Design ’96, J.S.
tomography for energy materials science.
Nguyen-Thanh, V.M., Guo, H., Hamdia, K., Gero and F. Sudweeks, eds. (Springer
MRS Bull. 45, 283–289.
Zhuang, X., and Rabczuk, T. (2020). An energy Netherlands), pp. 151–170.
17. Gürsoy, D., and Jacobsen, C. (2020). approach to the solution of partial differential
Multimodal x-ray nanotomography. MRS Bull. equations in computational mechanics via 44. Demigha, S. (2020). Conference: 2020
45, 272–276. machine learning: concepts, implementation International Conference on Computational
and applications. Comput. Methods Appl. Science and Computational Intelligence
18. Kirkpatrick, P., and Baez, A.V. (1948). Mech. Eng. 362, 112790. (CSCI). 1395–1340.
formation of optical images by X-rays. J. Opt.
Soc. Am. 38, 766–774. 32. Li, H., Wang, Z., Zou, N., Ye, M., Xu, R., Gong, 45. Russell, S. (2021). Artificial Intelligence: A
X., Duan, W., and Xu, Y. (2022). Deep-learning Modern Approach (Pearson).
19. Kirz, J., and Jacobsen, C. (2009). The history density functional theory Hamiltonian for
and future of X-ray microscopy. J. Phys, Conf. efficient ab initio electronic-structure 46. Bishop, C.M. (2006). Pattern Recognition and
Ser. 186, 012001. calculation. Nat. Comput. Sci. 2, 367–377. Machine Learning (Springer).

ll
OPEN ACCESS
Review
47. Le, T., Epa, V.C., Burden, F.R., and Winkler, chemical identification and tracking local 76. Girshick, R., Donahue, J., Darrell, T., and
D.A. (2012). Quantitative structure-property transformations. ACS Nano 11, 12742–12752. Malik, J. (2014). In 2014 IEEE Conference on
relationship modeling of diverse materials Computer Vision and Pattern Recognition,
properties. Chem. Rev. 112, 2889–2919. 61. Maksov, A., Dyck, O., Wang, K., Xiao, K., pp. 580–587.
Geohegan, D.B., Sumpter, B.G., Vasudevan,
48. Schmidt, J., Marques, M.R.G., Botti, S., and R.K., Jesse, S., Kalinin, S.V., and Ziatdinov, M. 77. Uijlings, J.R.R., Smeulders, A.W.M., and Scha,
Marques, M.A.L. (2019). Recent advances and (2019). Deep learning analysis of defect and R.J.H. (2011). The visual extent of an object.
applications of machine learning in solid-state phase evolution during electron beam Int. J. Comput. Vis. 96, 46–63.
materials science. npj Comput. Mater. 5, 83. induced transformations in WS2. npj Comput.
Mater. 5, 12. 78. Long, J., Shelhamer, E., and Darrell, T. (2015).
49. Moses, I.A., Joshi, R.P., Ozdemir, B., Kumar, In Proceedings of the IEEE Conference on
N., Eickholt, J., and Barone, V. (2021). 62. Han, Y., Jang, J., Cha, E., Lee, J., Chung, H., Computer Vision and Pattern Recognition
Machine learning screening of metal-ion Jeong, M., Kim, T.-G., Chae, B.G., Kim, H.G., (CVPR).
battery electrode materials. ACS Appl. Mater. Jun, S., et al. (2021). Deep learning STEM-EDX
Interfaces 13, 53355–53362. tomography of nanocrystals. Nat. Mach. 79. Guo, Y., Liu, Y., Georgiou, T., and Lew, M.S.
Intell. 3, 267–274. (2018). A review of semantic segmentation
50. Chen, C., Zuo, Y., Ye, W., Li, X., Deng, Z., and using deep neural networks. Int. J. Multimed.
Ong, S.P. (2020). A critical review of machine 63. Suzuki, Y., Hino, H., Hawai, T., Saito, K., Info. Retr. 7, 87–93.
learning of energy materials. Adv. Energy Kotsugi, M., and Ono, K. (2020). Symmetry
Mater. 10, 1903242. prediction and knowledge discovery from 80. Girshick, R. (2015). In Proceedings of the IEEE
X-ray diffraction patterns using an interpret- International Conference on Computer Vision
51. Faber, F.A., Lindmaa, A., von Lilienfeld, O.A., able machine learning approach. Sci. Rep. 10, (ICCV).
and Armiento, R. (2016). Machine learning 21790.
81. Ren, S., He, K., Girshick, R., Zhang, X., and
energies of 2 million elpasolite (ABC(2)D(6))
64. Liu, Y., Marcella, N., Timoshenko, J., Halder, Sun, J. (2017). Object detection networks on
crystals. Phys. Rev. Lett. 117, 135502.
A., Yang, B., Kolipaka, L., Pellin, M.J., Seifert, convolutional feature maps. IEEE Trans.
52. Li, Z., Xu, Q., Sun, Q., Hou, Z., and Yin, W.-J. S., Vajda, S., Liu, P., and Frenkel, A.I. (2019). Pattern Anal. Mach. Intell. 39, 1476–1481.
(2019). Thermodynamic stability landscape of Mapping XANES spectra on structural de-
82. He, K., Gkioxari, G., Dollár, P., and Girshick, R.
halide double perovskites via high- scriptors of copper oxide clusters using su-
(2017). In Proceedings of the IEEE
throughput computing and machine learning. pervised machine learning. J. Chem. Phys.
International Conference on Computer
Adv. Funct. Mater. 29, 1807280. 151, 164201.
Vision, pp. 2980–2988.
53. Park, H., Mall, R., Alharbi, F.H., Sanvito, S., 65. Goh, G.B., Hodas, N.O., and Vishnu, A. (2017).
83. Abdolhoseini, M., Kluge, M.G., Walker, F.R.,
Tabet, N., Bensmail, H., and El-Mellouhi, F. Deep learning for computational chemistry.
and Johnson, S.J. (2019). Segmentation,
(2019). Exploring new approaches towards J. Comput. Chem. 38, 1291–1307.
tracing, and quantification of microglial cells
the formability of mixed-ion perovskites by from 3D image stacks. Sci. Rep. 9, 8557.
DFT and machine learning. Phys. Chem. 66. Alpaydin, E. (2020). Introduction to Machine
Chem. Phys. 21, 1078–1088. Learning (MIT).
84. Tsai, H.F., Gajda, J., Sloan, T.F., Rares, A., and
67. Dimiduk, D.M., Holm, E.A., and Niezgoda, Shen, A.Q. (2019). Usiigaci: instance-aware
54. Kim, S., Noh, J., Gu, G.H., Aspuru-Guzik, A., cell tracking in stain-free phase contrast
and Jung, Y. (2020). Generative adversarial S.R. (2018). Perspectives on the impact of
machine learning, deep learning, and artificial microscopy enabled by machine learning.
networks for crystal structure prediction. ACS SoftwareX 9, 230–237.
Cent. Sci. 6, 1412–1420. intelligence on materials, processes, and
structures engineering. Integr. Mater. Manuf. 85. Jung, H., Lodhi, B., and Kang, J. (2019). An
55. Gu, G.H., Noh, J., Kim, I., and Jung, Y. (2019). Innov. 7, 157–172. automatic nuclei segmentation method
Machine learning for renewable energy based on deep convolutional neural networks
materials. J. Mater. Chem. 7, 17096–17117. 68. Gin, S., Dillmann, P., and Birbilis, N. (2017).
Material degradation foreseen in the very for histopathology images. BMC Biomed.
long term: the case of glasses and ferrous Eng. 1, 24.
56. Das Sarma, S., Deng, D.-L., and Duan, L.-M.
(2019). Machine learning meets quantum metals. npj Mater. Degrad. 1, 10. 86. Jiang, Z., Li, J., Yang, Y., Mu, L., Wei, C., Yu, X.,
physics. Phys. Today 72, 48–54. Pianetta, P., Zhao, K., Cloetens, P., Lin, F., and
69. Nash, W., Drummond, T., and Birbilis, N.
(2018). A review of deep learning in the study Liu, Y. (2020). Machine-learning-revealed
57. Schütt, K.T., Tkatchenko, A., and Müller, K.-R. statistics of the particle-carbon/binder
(2020). Learning representations of molecules of materials degradation. npj Mater. Degrad.
2, 37. detachment in lithium-ion battery cathodes.
and materials with atomistic neural networks. Nat. Commun. 11, 2310.
In Machine Learning Meets Quantum Physics,
70. Rosenblatt, F. (1988). The perception: a
K.T. Schütt, S. Chmiela, O.A. von Lilienfeld, A. 87. Zhang, G.-X. (2021). Machine learning in X-ray
Tkatchenko, K. Tsuda, and K.-R. Müller, eds. probabilistic model for information storage
and organization in the brain. Psychol. Rev. imaging and microscopy applications. In
(Springer International Publishing), Advanced X-ray Imaging of Electrochemical
pp. 215–230. 65, 386–408.
Energy Materials and Devices, J. Wang, ed.
71. Lecun, Y., Bengio, Y., and Hinton, G. (2015). (Springer Singapore), pp. 205–221.
58. Suram, S.K., Xue, Y., Bai, J., Le Bras, R.,
Deep learning. Nature 521, 436–444.
Rappazzo, B., Bernstein, R., Bjorck, J., Zhou, 88. Chan, E.M., and Emory, M. (2015).
L., van Dover, R.B., Gomes, C.P., and 72. Hoeser, T., and Kuenzer, C. (2020). Object Combinatorial approaches for developing
Gregoire, J.M. (2017). Automated phase detection and image segmentation with deep upconverting nanomaterials: high-
mapping with AgileFD and its application to learning on earth observation data: a review- throughput screening, modeling, and
light absorber discovery in the V–Mn–Nb ox- Part I: evolution and recent trends. Rem. Sens. applications. Chem. Soc. Rev. 44, 1653–1679.
ide system. ACS Comb. Sci. 19, 37–46. 12, 1667.
89. Xiang, C., Suram, S.K., H Ab Er, J.A.,
59. Zheng, C., Mathew, K., Chen, C., Chen, Y., 73. Asgari Taghanaki, S., Abhishek, K., Cohen, Guevarra, D.W., Soedarmadji, E., Jin, J., and
Tang, H., Dozier, A., Kas, J.J., Vila, F.D., Rehr, J.P., Cohen-Adad, J., and Hamarneh, G. Gregoire, J.M. (2014). High-throughput
J.J., Piper, L.F.J., et al. (2018). Author (2020). Deep semantic segmentation of bubble screening method for combinatorial
Correction: automated generation and natural and medical images: a review. Artif. discovery of electrocatalysts for water
ensemble-learned matching of X-ray absorp- Intell. Rev. 54, 137–178. splitting. ACS Comb. Sci. 16, 47–52.
tion spectra. npj Comput. Mater. 4, 24.
74. Malik, J., Belongie, S., Leung, T., and Shi, J. 90. Szymkuc, S., Gajewska, E.P., Klucznik, T.,
60. Ziatdinov, M., Dyck, O., Maksov, A., Li, X., (2001). Contour and texture analysis for image Molga, K., Dittwald, P., Startek, M., Ba jczyk,
Sang, X., Xiao, K., Unocic, R.R., Vasudevan, R., segmentation. Int. J. Comput. Vis. 43, 7–27. M., and Grzybowski, B.A. (2016). Computer-
Jesse, S., and Kalinin, S.V. (2017). Deep assisted synthetic planning: the end of the
learning of atomically resolved scanning 75. Goodfellow, I., Bengio, Y., and Courville, A. beginning. Angew Chem. Int. Ed. Engl. 55,
transmission electron microscopy images: (2016). Deep Learning (MIT Press). 5904–5937.

ll
OPEN ACCESS
Review
91. Flores-Leonar, M.M., Mejı́a-Mendoza, L.M., Demortière, A. (2021). 3D quantification of Computer-Assisted Intervention – MICCAI
Aguilar-Granda, A., Sanchez-Lengeling, B., microstructural properties of 2015, pp. 234–241.
Tribukait, H., Amador-Bedolla, C., and LiNi0.5Mn0.3Co0.2O2 high-energy density
Aspuru-Guzik, A. (2020). Materials electrodes by X-ray holographic nano-to- 118. Lefèvre, S. (2007). Knowledge from Markers in
Acceleration Platforms: on the way to mography. Adv. Energy Mater. 11, 2003529. Watershed Segmentation (Springer Berlin
autonomous experimentation. Curr. Opin. Heidelberg), pp. 579–586.
Green Sustain. Chem. 25, 100370. 105. Arganda-Carreras, I., Kaynig, V., Rueden, C.,
Eliceiri, K.W., Schindelin, J., Cardona, A., and 119. Chen, L., Jiang, M., and Chen, J. (2009). In
92. Curtarolo, S., Setyawan, W., Hart, G.L., Sebastian Seung, H. (2017). Trainable Weka 2009 16th IEEE International Conference on
Jahnatek, M., Chepulskii, R.V., Taylor, R.H., Segmentation: a machine learning tool for Image Processing (ICIP), pp. 4033–4036.
Wang, S., Xue, J., Yang, K., Levy, O., et al. microscopy pixel classification. Bioinformatics
(2012). AFLOW: an automatic framework for 33, 2424–2426. 120. Spettl, A., Wimmer, R., Werz, T., Heinze, M.,
high-throughput materials discovery. Odenbach, S., Krill, C.E., and Schmidt, V.
Comput. Mater. Sci. 58, 218–226. 106. Petrich, L., Westhoff, D., Feinauer, J., Finegan, (2015). Stochastic 3D modeling of Ostwald
D.P., Daemi, S.R., Shearing, P.R., and Schmidt, ripening at ultra-high volume fractions of the
93. Jain, A., Ong, S.P., Hautier, G., Chen, W., V. (2017). Crack detection in lithium-ion cells coarsening phase. Model. Simulat. Mater. Sci.
Richards, W.D., Dacek, S., Cholia, S., Gunter, using machine learning. Comput. Mater. Sci. Eng. 23, 065001.
D., Skinner, D., Ceder, G., and Persson, K.A. 136, 297–305.
(2013). Commentary: the Materials Project: a 121. Nomura, Y., Yamamoto, K., Fujii, M.,
materials genome approach to accelerating 107. Badmos, O., Kopp, A., Bernthaler, T., and Hirayama, T., Igaki, E., and Saitoh, K. (2020).
materials innovation. APL Mater. 1, 011002. Schneider, G. (2019). Image-based defect Dynamic imaging of lithium in solid-state
detection in lithium-ion battery electrode batteries by operando electron energy-loss
94. Pizzi, G., Cepellotti, A., Sabatini, R., Marzari, using convolutional neural networks. J. Intell. spectroscopy with sparse coding. Nat.
N., and Kozinsky, B. (2016). AiiDA: automated Manuf. 31, 885–897. Commun. 11, 2824.
interactive infrastructure and database for
computational science. Comput. Mater. Sci. 108. Janek, J., and Zeier, W.G. (2016). A solid future 122. Olshausen, B.A., and Field, D.J. (1996).
111, 218–230. for battery development. Nat. Energy 1, Emergence of simple-cell receptive field
16141. properties by learning a sparse code for
95. https://nomad-lab.eu/repo-arch. natural images. Nature 381, 607–609.
109. Lim, H.-D., Park, J.-H., Shin, H.-J., Jeong, J.,
96. Rupp, M., Tkatchenko, A., Müller, K.R., von Kim, J.T., Nam, K.-W., Jung, H.-G., and 123. Baliyan, A., and Imai, H. (2019). Machine
Lilienfeld, O.A., Anatole, von, and Lilienfeld. Chung, K.Y. (2020). A review of challenges and learning based analytical framework for
(2012). Fast and accurate modeling of issues concerning interfaces for all-solid-state automatic hyperspectral Raman analysis of
molecular atomization energies with machine batteries. Energy Storage Mater. 25, 224–250. lithium-ion battery electrodes. Sci. Rep. 9,
learning. Phys. Rev. Lett. 108, 058301. 18241.
110. Sun, Y.-K. (2020). Promising all-solid-state
97. Ward, L., Liu, R., Krishna, A., Hegde, V.I., batteries for future electric vehicles. ACS 124. Kojima, A., Teshima, K., Shirai, Y., and
Agrawal, A., Choudhary, A., and Wolverton, Energy Lett. 5, 3221–3223. Miyasaka, T. (2009). Organometal halide
C. (2017). Including crystal structure attributes perovskites as visible-light sensitizers for
in machine learning models of formation 111. Masias, A., Marcicki, J., and Paxton, W.A. photovoltaic cells. J. Am. Chem. Soc. 131,
energies via Voronoi tessellations. Phys. Rev. (2021). Opportunities and challenges of 6050–6051.
B 96, 024104. lithium ion batteries in automotive
applications. ACS Energy Lett. 6, 621–630. 125. Jena, A.K., Kulkarni, A., and Miyasaka, T.
98. Faber, F.A., Christensen, A.S., Huang, B., and (2019). Halide perovskite photovoltaics:
von Lilienfeld, O.A. (2018). Alchemical and 112. Albertus, P., Anandan, V., Ban, C., Balsara, N., background, status, and future prospects.
structural distribution based representation Belharouak, I., Buettner-Garrett, J., Chen, Z., Chem. Rev. 119, 3036–3103.
for universal quantum machine learning. Daniel, C., Doeff, M., Dudney, N.J., et al.
J. Chem. Phys. 148, 241717. (2021). Challenges for and pathways toward 126. Tao, Q., Xu, P., Li, M., and Lu, W. (2021).
Li-Metal-Based all-solid-state batteries. ACS Machine learning for perovskite materials
99. Müller, S., Sauter, C., Shunmugasundaram, R., Energy Lett. 6, 1399–1404. design and discovery. npj Comput. Mater. 7,
Wenzler, N., De Andrade, V., De Carlo, F., 23.
Konukoglu, E., and Wood, V. (2021). Deep 113. Dixit, M.B., Verma, A., Zaman, W., Zhong, X.,
learning-based segmentation of lithium-ion Kenesei, P., Park, J.S., Almer, J., Mukherjee, 127. Yılmaz, B., and Yıldırım, R. (2021). Critical
battery microstructures enhanced by P.P., and Hatzell, K.B. (2020). Synchrotron review of machine learning applications in
artificially generated electrodes. Nat. imaging of pore formation in Li metal solid- perovskite solar research. Nano Energy 80,
Commun. 12, 6205. state batteries aided by machine learning. 105546.
ACS Appl. Energy Mater. 3, 9534–9542.
100. Goodenough, J.B., and Kim, Y. (2010). 128. Deitsch, S., Christlein, V., Berger, S., Buerhop-
Challenges for rechargeable Li-ion batteries. 114. Kodama, M., Ohashi, A., Adachi, H., Miyuki, Lutz, C., Maier, A., Gallwitz, F., and Riess, C.
Chem. Mater. 22, 587–603. T., Takeuchi, A., Yasutake, M., Uesugi, K., (2019). Automatic classification of defective
Kaburagi, T., and Hirai, S. (2021). Three- photovoltaic module cells in
101. Bhowmik, A., Castelli, I.E., Garcia-Lastra, J.M., dimensional structural measurement and electroluminescence images. Sol. Energy 185,
Jørgensen, P.B., Winther, O., and Vegge, T. material identification of an all-solid-state 455–468.
(2019). A perspective on inverse design of lithium-ion battery by X-Ray nanotomography
battery interphases using multi-scale and deep learning. J. Power Sources Adv. 8, 129. Kntges, M., Kurtz, S., Packard, C., Jahn, U.,
modelling, experiments and generative deep 100048. and Friesen, G. (2015). IEA-PVPS T13-01 2014
learning. Energy Storage Mater. 21, 446–456. Review of Failures of Photovoltaic Modules
115. Furat, O., Finegan, D.P., Diercks, D., Usseglio- Final.
102. Lou, S., Zhang, F., Fu, C., Chen, M., Ma, Y., Yin, Viretta, F., Smith, K., and Schmidt, V. (2021).
G., and Wang, J. (2021). Interface issues and Mapping the architecture of single lithium ion 130. Akram, M.W., Li, G., Jin, Y., Chen, X., Zhu, C.,
challenges in all-solid-state batteries: lithium, electrode particles in 3D, using electron Zhao, X., Khaliq, A., Faheem, M., and Ahmad,
sodium, and beyond. Adv. Mater. 33, backscatter diffraction and machine learning A. (2019). CNN based automatic detection of
2000721. segmentation. J. Power Sources 483, 229148. photovoltaic cell defects in
electroluminescence images. Energy 189,
103. Chen, X., Liu, X., Shen, X., and Zhang, Q. 116. Furat, O., Wang, M., Neumann, M., Petrich, L., 116319.
(2021). Applying machine learning to Weber, M., Krill, C.E., and Schmidt, V. (2019).
rechargeable batteries: from the microscale Machine learning techniques for the 131. Ahmad, A., Jin, Y., Zhu, C., Javed, I.,
to the macroscale. Angew Chem. Int. Ed. segmentation of tomographic image data of Maqsood, A., and Akram, M.W. (2020).
Engl. 133, 24558–24570. functional materials. Front. Mater. 6, 145. Photovoltaic cell defect classification using
convolutional neural network and support
104. Nguyen, T., Villanova, J., Su, Z., Tucoulou, R., 117. Ronneberger, O., Fischer, P., and Brox, T. vector machine. IET Renew. Power Gener. 14,
Fleutot, B., Delobel, B., Delacourt, C., and (2015). Medical Image Computing and 2693–2702.

ll
OPEN ACCESS
Review
132. Tang, W., Yang, Q., Xiong, K., and Yan, W. (2019). Segmentation of solid oxide cell with periodic boundaries. npj Comput. Mater.
(2020). Deep learning based automatic defect electrodes by patch convolutional neural 6, 82.
identification of photovoltaic module using network. J. Electrochem. Soc. 168, 044504.
electroluminescence images. Sol. Energy 201, 143. Bailey, J.J., Chen, J., Hack, J., Perez-Page, M.,
453–460. 138. Topal, E., Liao, Z., Löffler, M., Gluch, J., Zhang, Holmes, S.M., Brett, D.J., and Shearing, P.R.
J., Feng, X., and Zschech, E. (2020). Multi-scale (2021). Lab-based X-ray micro-computed to-
133. Zhao, Y., Zhan, K., Wang, Z., and Shen, W. X-ray tomography and machine learning al- mography coupled with machine-learning
(2021). Deep learning-based automatic gorithms to study MoNi4 electrocatalysts segmentation to investigate phosphoric acid
detection of multitype defects in photovoltaic anchored on MoO2 cuboids aligned on Ni leaching in high-temperature polymer elec-
modules and application in real production foam. BMC Mater. 2, 5. trolyte fuel cells. J. Power Sources 509,
line. Prog. Photovoltaics Res. Appl. 29, 230347.
471–484. 139. Tan, Y., Matsui, H., Ishiguro, N., Uruga, T.,
Nguyen, D.-N., Sekizawa, O., Sakata, T.,
134. Chen, H., Pang, Y., Hu, Q., and Liu, K. (2020). Maejima, N., Higashi, K., Dam, H.C., and 144. Yamagishi, R., Sciazko, A., Ouyang, Z.,
Solar cell surface defect inspection based on Tada, M. (2019). Pt–Co/C cathode catalyst Komatsu, Y., Katsuhiko, N., and Shikazono, N.
multispectral convolutional neural network. degradation in a polymer electrolyte fuel cell (2021). Super-resolved in-operando
J. Intell. Manuf. 31, 453–468. investigated by an infographic approach observation of SOFC pattern electrodes. ECS
combining three-dimensional Trans. 103, 2087–2098.
135. Hwang, H., Choi, S.M., Oh, J., Bae, S.-M., Lee, spectroimaging and unsupervised learning.
J.-H., Ahn, J.-P., Lee, J.-O., An, K.-S., Yoon, Y., J. Phys. Chem. C 123, 18844–18853. 145. Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A.
and Hwang, J.-H. (2020). Integrated (2017). In Proceedings of the IEEE Conference
application of semantic segmentation- 140. Chen, L., Zhu, Y., Papandreou, G., Schroff, F., on Computer Vision and Pattern Recognition
assisted deep learning to quantitative multi- and Adam, H. (2018). Computer Vision – (CVPR), pp. 1125–1134.
phased microstructural analysis in composite ECCV 2018. 833–851.
materials: case study of cathode composite 146. Schonvogel, D., Rastedt, M., Wagner, P.,
materials of solid oxide fuel cells. J. Power 141. Colliard-Granero, A., Batool, M., Jankovic, J., Wark, M., and Dyck, A. (2016). Special Issue
Sources 471, 228458. Jitsev, J., Eikerling, M.H., Malek, K., and (5th European PEFC & H2 Forum),
Eslamibidgoli, M.J. (2022). Deep learning for pp. 480–489.
136. Hwang, H., Ahn, J., Lee, H., Oh, J., Kim, J., the automation of particle analysis in catalyst
Ahn, J.-P., Kim, H.-K., Lee, J.-H., Yoon, Y., and layers for polymer electrolyte fuel cells. 147. Sommer, C., Straehle, C., Köthe, U., and
Hwang, J.-H. (2021). Deep learning-assisted Nanoscale 14, 10–18. Hamprecht, F.A. (2011). In 2011 IEEE
microstructural analysis of Ni/YSZ anode International Symposium on Biomedical
composites for solid oxide fuel cells. Mater. 142. Gayon-Lombardo, A., Mosser, L., Brandon,
N.P., and Cooper, S.J. (2020). Pores for Imaging: From Nano to Macro.
Char. 172, 110906.
thought: the use of generative adversarial
137. Sciazko, A., Komatsu, Y., Shimura, T., networks for the stochastic reconstruction of 148. (2018).Thermal Fisher Scientific Avizo
Shikazono, N., Jiao, Z., and Shikazono, N. 3D multi-phase electrode microstructures Software 9 User’s Guide.

Machine Learning Facilitated Multiscale Imaging For Energy Materials

Uploaded by

Copyright:

Available Formats

You might also like

Machine Learning Facilitated Multiscale Imaging For Energy Materials

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Machine Learning Facilitated Multiscale Imaging For Energy Materials

Uploaded by

Copyright:

Available Formats

ll

X-ray tomography microscopy represents an advanced X-ray imaging method, and it

Similar to XRT, electron tomography is another imaging technique that enables us to

2 Cell Reports Physical Science 3, 101008, September 21, 2022

Nowadays multiscale imaging techniques have gained continuous attention in mod-

Cell Reports Physical Science 3, 101008, September 21, 2022 3

freedom, thus leading to extremely high dimensionality of the information space.38

MACHINE LEARNING IN A NUTSHELL

4 Cell Reports Physical Science 3, 101008, September 21, 2022

Figure 1. Machine learning integrated with materials design and discovery

As opposed to supervised learning, unsupervised learning uses unlabeled input

Reinforcement learning represents a subfield of machine learning that requires ma-

Cell Reports Physical Science 3, 101008, September 21, 2022 5

Currently, ML techniques have been increasingly applied to energy materials design

In summary, the discipline of ML is growing rapidly, and the widespread use of ML

DEEP LEARNING FOR IMAGE SEGMENTATION

Key concepts of deep learning

6 Cell Reports Physical Science 3, 101008, September 21, 2022

Figure 2. Deep learning-based image segmentation

As motivated in the previous section, deep learning is making tremendous progress

Cell Reports Physical Science 3, 101008, September 21, 2022 7

Image segmentation is generally considered a non-trivial and challenging task in

Regions with convolutional neural network features (R-CNN)76 is among pioneering

8 Cell Reports Physical Science 3, 101008, September 21, 2022

Figure 3. Workflow of using deep learning on image analytics

Workflow of using deep learning on image analysis

Cell Reports Physical Science 3, 101008, September 21, 2022 9

uncertainties.87 Thus, understanding the problem guides the choice of ML strate-

10 Cell Reports Physical Science 3, 101008, September 21, 2022

APPLICATIONS TO ENERGY MATERIALS IMAGING

Cell Reports Physical Science 3, 101008, September 21, 2022 11

complex and dynamic processes at battery interfaces.83,101,102 In fact, interfaces in

12 Cell Reports Physical Science 3, 101008, September 21, 2022

Figure 4. Machine learning-assisted microstructure analysis of lithium-ion batteries

Deep learning approaches have been recently used to identify microstructures of

X-ray tomography has shown great promise in understanding lithium-ion batteries

Cell Reports Physical Science 3, 101008, September 21, 2022 13

Deep learning methods were recently applied to detect microstructural defects in

14 Cell Reports Physical Science 3, 101008, September 21, 2022

Solid-state batteries are considered the next generation rechargeable battery

Cell Reports Physical Science 3, 101008, September 21, 2022 15

Enhancement of the image quality

16 Cell Reports Physical Science 3, 101008, September 21, 2022

Cell Reports Physical Science 3, 101008, September 21, 2022 17

Figure 8. Visual validation of the neural networks

representation learning)122 to tackle the issue of insufficient temporal resolution of the

18 Cell Reports Physical Science 3, 101008, September 21, 2022

Figure 9. Dynamic imaging of Li concentration assisted by unsupervised learning

Cell Reports Physical Science 3, 101008, September 21, 2022 19

20 Cell Reports Physical Science 3, 101008, September 21, 2022

large-scale applications of solar energy in the past decade.124,125 Recent advance-

Cell Reports Physical Science 3, 101008, September 21, 2022 21

22 Cell Reports Physical Science 3, 101008, September 21, 2022

Cell Reports Physical Science 3, 101008, September 21, 2022 23

24 Cell Reports Physical Science 3, 101008, September 21, 2022

Solid oxide fuel cells

Sciazko et al.137 recently developed an automated deep learning segmentation al-