Journal of Cleaner Production 385 (2023) 135782

Contents lists available at ScienceDirect

Journal of Cleaner Production

journal homepage: www.elsevier.com/locate/jclepro

A digital twin-based approach for optimizing operation energy

consumption at automated container terminals
Yinping Gao a, Daofang Chang b, Chun-Hsien Chen c, *
a
School of Management, Shanghai University, 99 Shangda Road, 200444, Shanghai, China
b
Logistics Engineering College, Shanghai Maritime University, 1550 Haigang Avenue, 201306, Shanghai, China
c
School of Mechanical & Aerospace Engineering, Nanyang Technological University, 50 Nanyang Avenue, 639798, Singapore

A R T I C L E I N F O A B S T R A C T

Handling Editor: Cecilia Maria Villas Bôas de The sustainable development of port operation management is strongly related to the energy consumption of
Almeida production at automated container terminals (ACTs). This paper focuses on the production activities at a
container yard, which is the primary facility of ACTs. A digital twin-based approach is proposed to optimize the
Keywords: operation of an automatic stacking crane (ASC) handling containers in terms of energy consumption. A virtual
Sustainable development
container yard that syncs with a physical container yard in the ACT digital twin system for observation and
Energy consumption
validation is developed. A mathematical model is established to minimize the total energy consumption of
Digital twin
Automatic stacking crane scheduling completing all tasks. Then, the Q-learning algorithm is adapted to optimize a solution based on the operating
Container yard data from the ACT digital twin system. Numerical experiments are conducted to demonstrate the effectiveness of
the proposed approach by comparing it with two other solution algorithms, viz., genetic algorithm (GA) and
particle swarm optimization (PSO). The total energy consumption of two operation strategies (i.e., centralized
and decentralized) are also compared using the proposed digital twin-based approach. With digital twin, the
operational environment and energy consumption are visualized to support optimization and management of
ASCs. Managers and operators can choose an appropriate strategy according to the designated sustainable goal.

1. Introduction that sustainable performance contributed to management and opera­

With the growth of international trade and the increase in container
throughput, container ships have become larger, and containers stacked
in the container terminal have become denser (Zaerpour et al., 2019).
The liner shipping decisions have been studied to reduce the total
turnaround cost, as well as the emissions generated in sea from an
environmental perspective (Pasha et al., 2021). The shipping line of
maritime transportation has been designed and optimized based on a
new collaborative agreement to achieve sustainable and environmen­
tally friendly targets (Dulebenets, 2022). To better serve customers and
achieve the production goals of container terminals, the stacking quality
must be improved and the number of conflicts during container loading
and unloading operations must be reduced. It was pointed out that the
management and operations of container terminals tended toward green
performance, which pursued not only operational efficiencies but also
positive environmental effects (Kuo and Lin, 2020). Junior et al. (2022)
analyzed the indicators that affect the sustainability of container ter­
minals from economic, social, and environmental aspects and suggested
tions. The shipping industry must have sustainable development, and
some of the clean energy sources used by container ships were suggested
as an alternative (Ampah et al., 2021). For the sustainability of port
operations, many measures have been investigated from multiple as­
pects, including energy source replacement, transportation modes, and
equipment utilization (Budiyanto et al., 2021; Ng and Talley, 2020;
Ypsilantis and Zuidwijk, 2019).
The refurbishment and upgrade of container terminals, including the
measures mentioned above, contribute to sustainable development.
Many actions have been investigated to reduce emissions and improve
efficiency, including replacement of older equipment, alternative power
systems, and digitalization (Alamoush et al., 2020; Liu and Ge, 2018).
Currently, container terminals are moving toward automation and in­
telligence. Large investments and advanced equipment have been
introduced to container terminals. New equipment, such as automatic
stacking cranes (ASCs) and automated guided vehicles (AGVs), are used
at automated container terminals (ACTs) which are beneficial for
environmental sustainability (Tsolakis et al., 2022). ASC, an

* Corresponding author.
E-mail address: mchchen@ntu.edu.sg (C.-H. Chen).

https://doi.org/10.1016/j.jclepro.2022.135782
Received 31 August 2022; Received in revised form 17 December 2022; Accepted 27 December 2022
Available online 28 December 2022
0959-6526/© 2022 Elsevier Ltd. All rights reserved.
Y. Gao et al.
indispensable presence, is typically used to load, unload and stack
containers at a container yard, undertaking the primary activities of
terminal production. The randomness of the arrival of container ships
and container trucks (CTs), as well as the uncertainty of severe weather
and equipment failures, affect the operations of container yards. Addi­
tionally, an unbalanced load caused by an uneven distribution of all ASC
tasks makes the capacity of some ASCs unable to be effectively utilized.
Thus, to further promote sustainable development, it is necessary to
optimize container yard operations and improve equipment utilization.
Therefore, this paper proposes to improve the energy consumption of
ASCs that are used to handle containers at the container yard with re­
gard to sustainable operations.
ASC scheduling was considered as a resource optimization problem.
Shu et al. (2015) solved a multi-objective problem that focused on the
ASC loading sequence using an improved genetic algorithm. Lu and
Wang (2019) used graph theory to state the ASC scheduling problem,
and obtained the minimum operating time with a particle swarm opti­
mization (PSO) algorithm. In contrast from heuristics algorithms,
Gharehgozli et al. (2017) developed a simulation model for twin ASC
operations and discussed a yard design to improve the working effi­
ciency of ASCs. Various approaches have been used to solve ASC
scheduling problems in previous studies (Gharehgozli et al., 2015; Hu
et al., 2016). A sustainable digitalization measure was taken to monitor
the production process and energy consumption (Ozturk et al., 2018).
The proposal and development of Industry 4.0 have been derived from
many emerging technologies and concepts, such as digital twin, Internet
of Things, augmented reality, and cloud computing, where the digital
twin functions were integrated to promote industry transformation
(Aheleroff et al., 2021). Digital twin has also been applied to resource
optimization in multiple areas, including medicine quality management
(Lopes et al., 2020), shop-floor smart production(Zhuang et al., 2021)
and intelligent maintenance of machines(Xiong et al., 2021). With the
help of digital twin, the practical production or operational conditions
could be monitored in real time and be mapped in the virtual models.
Based on the synchronization and interaction of physical and virtual
objects in the digital twin, reliable decision and optimization solutions
could be obtained. Hence, this paper proposes a digital twin-based
approach to study the ASC operation problem considering energy
consumption.
The contributions of this paper are summarized as follows: (1) This
paper attempts to apply the digital twin for resource optimization with a
focus on sustainability at ACT. No research has yet proposed using
digital twin to address the energy consumption management of re­
sources in ACT production. (2) To promote sustainable development,
this paper proposes to improve the total energy consumption of ASCs
from the perspective of workload balance rather than traditionally
considering the carbon emissions of equipment. The goal is to minimize
the gap in energy consumption across all ASCs used during scheduling.
Although the total energy consumption of ASCs is not decreased, the
ratio of working and non-working energy consumption of each ASC is
balanced to improve equipment sustainability. (3) Numerical experi­
ments of different methods are conducted to verify the effectiveness of
the proposed digital twin-based approach. The total energy consump­
tion of the operating plan solved by the proposed approach is also
compared to analyze the performance of the centralized and decen­
tralized operational strategies. (4) With the support of digital twin, a
validation of solutions is performed in the virtual container yard to
compare the working and non-working energy consumption of ASCs.
The visualization of operating plan helps achieve its appropriateness for
the practical environment.
The remainder of this paper is organized as follows. Relevant liter­
ature involving sustainable research and digital twin is reviewed in
Section 2. The problem of optimizing the energy consumption of ASCs is
stated in Section 3, and the digital twin-based approach for operating
optimization is provided in Section 4. The next section, Section 5, pre­
sents the case experiments and results analysis. Conclusions and future
2
Journal of Cleaner Production 385 (2023) 135782
research are summarized in Section 6.
2. Literature review
2.1. Scheduling research at container terminals
During vessel unloading, Hu et al. (2019) focused on QC and yard
truck (YT) scheduling, and containers were allocated to YTs as groups of
clusters. They combined the heuristic algorithm and adaptive genetic
algorithm (GA) to minimize the total travel distances and the makespan,
in which the experimental results demonstrated the effectiveness of the
method. Niu et al. (2021) studied the integrated scheduling of QCs and
yard trucks (YTs) with the goal of minimizing the service time of vessels
and the operating costs, respectively. Thus, the bacterial colony opti­
mization algorithm was improved for a robust and competitive perfor­
mance. Multi-group comparison experiments were conducted to show
the efficiency of the multi-objective algorithm.
Regarding vehicle resource allocation, Luo and Wu (2015) integrated
AGV scheduling with QC and yard crane (YC) in a simultaneous loading
and unloading scenario. To minimize the berth time of a vessel, a genetic
algorithm was adapted to tackle large-size cases. Yue et al. (2021)
investigated the QC scheduling problem with AGV scheduling to mini­
mize the ineffective time and delay time by combining the GUROBI
solver and NSGA III algorithm to obtain QC and AGV scheduling
schemes. Chen et al. (2020b) studied AGV and YC scheduling using a
systematic approach, with which a time-discretized space-time network
was established to observe the impacts caused by traffic congestion and
probabilistic actions, and decomposed the integer linear programming
model with the alternating direction method of multipliers (ADMM).
The optimal space-time scheduling solutions were accordingly obtained
for AGV and YC.
YC is an essential crane that is used in the storage yard to stack and
retrieve containers. Wang et al. (2020b) used a branch and price algo­
rithm to address the assignment and scheduling of cranes in a steel en­
terprise, which minimized the total traveling distances and balanced the
workload of cranes. Liu et al. (2022) studied the block relocation
problem in the warehouse by improving the mixed integer programming
model, which could obtain the solutions in a faster and easier way. Guo
et al. (2021) studied the gantry crane scheduling problem and solved it
with a logic-based Benders decomposition (LBBD) methodology. The
starting and ending positions of moving containers were considered to
schedule gantry cranes, and computational results show the competitive
performance of the GUROBI solver. Kim et al. (2021) focused on ASC
scheduling strategies and found an optimal solution using a search al­
gorithm and simulation.
2.2. Optimization considering sustainability at container terminals
Although the abovementioned research focused more on high effi­
ciency, low energy has not been ignored. He et al. (2015) compared two
strategies in the scheduling of inter truck (IT), YC and QC, in terms of
time savings and energy savings. Experiments with GA-PSO indicated
that the container terminal operators could choose any one scheduling
strategy based on the real operating goal because no strategy performed
better in all cases. Karakas et al. (2021) introduced the carbon footprint
into berth allocation, to minimize operation time and carbon emissions
when trucks travel from berths to container blocks. Monte Carlo simu­
lation was used to calculate the truck idling time, and then, a linear
integer programming model was established to select the optimal berth
and container block. Sim (2018) considered the carbon production of a
container terminal sourced from ships, cranes, trucks, and other
equipment, and proposed a system dynamics model to calculate the
emissions during container operation. Chen et al. (2013) discussed the
relationship between truck arrival and emission reduction using
queueing theory to simulate the operating process of trucks and estimate
emissions. Then, an improved GA-based heuristic was proposed to
Y. Gao et al. Journal of Cleaner Production 385 (2023) 135782

optimize the queueing time and idling emission of trucks. Jiang et al. and a discrete simulation was conducted to evaluate the maintenance
(2019) combined heuristic knowledge and deep Q-learning to help effects. Zhang et al. (2022) proposed an improved multi-fidelity simu­
smart robots plan paths and avoid obstacles. The experience data of lation-based optimization (IMFSO) method combined with discrete
robot movements were collected to train the neural network, which was simulation, which was applied in the digital twin floor for aircraft parts
used to explore an optimal path. production.
Wan et al. (2021) focused on the energy consumption of container­ Granacher et al. (2022) proposed a digital twin supported
ships and analyzed three different emission reduction strategies by decision-making method to design energy systems that considered
simulating ship movements. They concluded that using a shore power complex requirements and subsequent evaluations. The decision-maker
system and increasing QC efficiency was a desirable way to be envi­ using the digital twin helped in finding more meaningful solutions, as
ronmentally friendly. Facchini et al. (2020) demonstrated the economic discussed in the designed case. Park et al. (2020) combined digital twin
and environmental feasibility of dry ports, which were proposed to and machine learning to optimize the scheduling of an energy storage
reduce the container terminal congestion. Both the operation cost and system, where the charging and discharging operations were well
carbon footprint estimation were used to evaluate the container controlled to minimize the energy cost. Using a digital twin, a practical
handling strategies between the port and the dry port. A simulation was microgrid was implemented in a virtual space to obtain the complexity
performed based on a non-linear programming model and a computa­ and uncertain data for further effective optimization. You et al. (2022)
tional algorithm. Li et al. (2021) simulated the primary container ac­ proposed a deep learning and digital twin-based scheduling method in
tivities at the U-type layout container terminal and then analyzed the an integrated energy system (IES) to address uncertainties. Real-time
performance, energy consumption, and operational costs. The case data from the digital twin IES were input to predict the day-ahead
analysis demonstrated the superiority of the new design of automated scheduling using a deep neural network.
container terminals regardless of efficiency, environmental and eco­
nomic levels. Chen and Zeng (2021) optimized the scheduling of electric
2.4. Literature summary
rubber-tyred gantry cranes (ERTGs) from a carbon efficiency perspec­
tive. In the ERTG scheduling problem, both carbon emissions and task
To summarize, the most related studies of scheduling and digital
delay were considered to minimize the total cost of task groups. Two
twin applications can be classified as shown in Table 1. Sustainability
scheduling strategies were compared to obtain the results that adjusting
research of container terminals has been studied from various types of
the schedule of ERTGs based on real-time changes was better than
energy-saving actions. On the one hand, the transformation of terminal
keeping a carbon-oriented strategy.
operations was accomplished by improving the energy system and the
operating mode to reduce carbon emissions. On the other hand, some
2.3. Digital twin research
emerging designs appeared as a reform of container terminals, such as
the use of new energy equipment and the design of new layouts. These
Digital twin has been widely used in production scheduling, main­
ideas were proposed and demonstrated to save energy and reduce
tenance strategies and decision-making support in various industries.
emissions, in which the handling equipment undertook the greater part
Wang et al. (2020a) introduced digital twin into the human-robot
of production and operations. Thus, the scheduling of the handling
interactive welding, where virtual reality was used to develop the dig­
equipment should be well optimized for energy saving.
ital twin system and a data-driven method was designed to analyze the
Studies of scheduling at container terminals typically attempted to
welder behaviors generated from trained welders and unskilled welders
achieve the least time and cost required to complete all tasks and the
without any training. Fang et al. (2019) proposed a scheduling method
heuristic algorithms, such as GA and PSO, were used to obtain the so­
based on digital twin and used it in job shop production, which reduced
lutions. The digital twin-based approach was proposed in workshop
the deviation caused by some uncertainties and disturbances. Zhang
scheduling to address the uncertainties that occur in the real
et al. (2021) also developed a digital twin-enhanced scheduling method
in a job shop, which involved the prediction of machine availability,
Table 1
detection of uncertain disturbances, and evaluation of scheduling per­
Classification of related research issues in the literature.
formances. Wang et al. (2021) used the digital twin framework to
monitor machine conditions. Then, the order scheduling with these Authors Problems Approach
machines was also displayed in a dashboard to provide decision-making Guo et al. (2021) Gantry crane scheduling GUROBI Solver
support for operators and managers. considered moving positions
Negri et al. (2021) proposed a digital twin-based simheuristics Kim et al. (2021) Comparison of ASC scheduling Simulation
strategies
framework for robust scheduling that included an input data module, He et al. (2015) Scheduling of IT, YC and QC to Heuristic algorithm
digital twin module and optimization module. The real-time data of save time or energy
industrial equipment were used to synchronize and simulate the pro­ Karakas et al. (2021) Berth allocation considered Linear programming
duction in the flowshop, and then GA was used to optimize the job carbon footprint and simulation
Chen et al. (2013) Truck operating considered Heuristic algorithm
scheduling with uncertainty to obtain the minimum makespan. Zhao
emission reduction
et al. (2022) discussed the optimization of cranes in the workshop, Chen and Zeng ERTG scheduling to minimize Branch-and-bound
including how to schedule multiple cranes, and how to determine the (2021) the total cost algorithm
number of cranes with the lowest energy consumption. A digital Fang et al. (2019), Job shop scheduling Digital twin-based
twin-driven scheduling approach was developed, in which GA and the Zhang et al. framework
(2021)
simulation in the virtual crane system were used to evaluate the energy Negri et al. (2021) Uncertain scheduling in Digital twin-based GA
consumption of multi-crane operation. flowshop to minimize the
Yan et al. (2022) integrated a digital twin and a Q-learning algorithm makespan
to achieve dynamic scheduling and preventive maintenance in a job Zhao et al. (2022) Crane scheduling considering Digital twin-based GA
the lowest energy consumption
shop. In the virtual job shop, Q-learning was used to respond to the
Yan et al. (2022) Dynamic scheduling and Digital twin and Q-
operation environment and evaluate the action selections, which pro­ preventive maintenance learning algorithm
vided the best solution for decision-makers. To optimize maintenance Park et al. (2020), Uncertain scheduling of the Digital twin and
strategies in the mining industry, Savolainen and Urbani (2021) You et al. (2022) energy system machine learning
designed a modeling approach based on digital twin, including two This paper ASC scheduling to minimize the Digital twin and Q-
energy consumption learning
stages. A virtual model was developed to reflect the mine environment,

3
Y. Gao et al.
environment. Simulations could well reflect the real-time production
process, which can be followed by using heuristic algorithms or machine
learning algorithms to optimize scheduling. Thus, this paper investigates
the operating optimization of the container yard with a digital twin-
based approach, which involves using simulation and Q-learning to
minimize the total energy consumption of ASCs.
3. Problem statement
Containers, including imported or exported containers, are typically
stacked in a container yard before being transported to trucks or vessels.
ASCs are the main handling equipment at the container yard of an
automated container terminal. CTs transport exported containers to the
terminal, and then, containers are unloaded by ASCs to be stacked in the
container yard. Imported containers being unloaded from vessels are
first stored in the yard and wait for ASCs to handle and load on the CTs.
The ASCs, which handle two types of containers, operate independently
in the container yard, as shown in Fig. 1. However, the arrival time of
CTs is not fixed and may change randomly due to unexpected condi­
tions. For example, congestion during rush hours or busy roads directly
affects the time that CTs arrive at the ACT. Some other conditions may
cause uncertainties. Then, the ASC operation of handling containers
associated with CTs becomes passive. When the CT enters the storage
yard, the ASC will be scheduled to load or unload the container. The
container tasks of CTs are handled with ASCs, involving unloading
exported containers from vehicles and loading imported containers from
the yard. The allocation of tasks determines the workload of ASCs and
thus affects the total energy consumption of equipment.
The energy consumption of ASCs includes the consumption when
working and waiting. When loading one container, the ASC lifts the
container from the container yard, moves to the truck exchange area,
places the container on the truck, and then releases it. In contrast, the
ASC lifts the container and moves from the exchange area to the
container yard when unloading one container. ASCs keep working to
perform container tasks that consist of two primary actions, i.e., moving
and lifting. Therefore, the energy consumption of working is related to
the moving distance and lifting height of ASCs, and their non-working
energy consumption occurs when there are no current container tasks
that depend on waiting time. The energy consumption calculation of
ASC can thus be described in the following equations. The involved
notations are listed in Table 2.The total energy consumption of ASC
consists of the working and non-working energy consumption, as shown
in Eq. (1). When the ASC loads a container task onto the CT, the crane
needs to move from the starting position to the target position and return
Fig. 1. Layout of the container yard.
4
Journal of Cleaner Production 385 (2023) 135782
Table 2
Implications of notations related to energy consumption.
ECim = ECimw + ECimnw (1)
ECimw = 2Dmi ⋅ m μ + 2Him ⋅ m
ρ (2)
ECimnw = Tiim′ ⋅γ m
(3)
Notations Implications
ECm Total energy consumption when ASC m handles container task i.
i
ECmw
i Working energy consumption when ASC m handles container task i.
ECmnw
i Non-working consumption when ASC m handles container task i.
Dm
i Moving distances of the crane.
m
Hi Lifting height of the crane trolley.
μm Unit energy consumption of the crane required to work.
ρm Unit energy consumption of the crane trolley required to work.
Tiim′ Non-working time of ASC waiting for the next task i .
′
γm Unit energy consumption of the ASC required for non-working time.
to interact with the CT. Meanwhile, the crane trolley is lowered to pick
up the container task and is then raised to its original height. The process
of unloading a container task is similar. Thus, the energy consumption of
ASC working is denoted by Eq. (2). When the ASC waits for the next
container task i , the non-working energy consumption generated from
′
the waiting time is formulated in Eq. (3).
To calculate the total energy consumption of ASCs, the storage lo­
cations of container tasks, including the stacked bay and tier, are
required. The storage yard is composed of container blocks that are
divided into many rows, bays and tiers, as shown in Fig. 2. The arrival of
both loaded and unloaded CTs requires different container tasks to be
completed by ASCs. The ASC typically moves from the initial position,
bay zero, to the target bay of the container task and lowers to the target
tier. If no container task is allocated to the ASC, it expends its non-
working energy consumption. When the CT arrives at the container
yard later than the expected time, there is an impact on the operation of
the ASC, i.e., increasing the waiting time. An ASC working or waiting for
a long time is not conducive to equipment maintenance and sustainable
development. Thus, ASCs should be scheduled to handle container tasks
effectively, balancing the workload, which affects the energy con­
sumption of ASCs.
This paper proposes the problem of operating optimization at ACT,
which includes dealing with the uncertain factors that affect the
arrangement and execution of production plans and allocating the re­
sources of the container yard. The allocation of container tasks and
Y. Gao et al.
Fig. 2. ASC handing the container in the container block.
scheduling of ASCs are studied to improve the sustainable development
of the container yard. The optimization goal is to minimize the total
energy consumption of ASCs as much as possible without reducing
loading and unloading efficiency.
4. Approach design
This paper proposes a digital twin-based approach for operating
optimization at the container yard. The practical operation environment
of ACT is dynamic and time-varying because some factors have an
impact on the production plan including equipment operation and task
allocation. The presented ASC scheduling problem is a kind of decision
and optimization problem. Some studies addressed the hard decision
problems with intelligent algorithms in different domains. Pasha et al.
(2022) developed a multi-objective hybrid metaheuristic algorithm to
address the vehicle routing problem which minimized the total cost from
depot to customers. Rabbani et al. (2022) compared non-dominated
sorting genetic algorithm II (NSGA-II) and multi-objective particle
swarm optimization (MOPSO) in addressing the routing of ambulances,
where NSGA-II performed better. Kavoosi et al. (2019) applied an
evolutionary algorithm as a solution to the berth scheduling problem,
and the computational experiments verified its effectiveness. Dulebenets
(2018) designed a new evolutionary algorithm to solve the truck
scheduling model which improved the algorithm performance. Faced
with complex environments and unknown factors, this paper combines
digital twin and optimization algorithms to solve the proposed
Fig. 3. Digital twin-based approach for operating optimization at the container yard.
5
Journal of Cleaner Production 385 (2023) 135782
decision-making problem. As proposed by Gao et al. (2022), a digital
twin-enabled automated storage yard scheduling framework was
developed to achieve scheduling in uncertain scenarios. Based on the
digital twin, a virtual ACT system can be built. The operation changes in
the physical ACT can be synchronized in the virtual ACT with real-time
data from the physical space. The simulation can be performed in the
virtual ACT to evaluate the performance of the operating plan. When the
simulation results do not achieve the expected effect, the operating plan
needs to be optimized. The optimal scheduling after simulation valida­
tion can effectively satisfy the real operation environment and produc­
tion requirement.
As shown in Fig. 3, the ACT digital twin system mainly includes the
physical container yard, virtual container yard, and data center. The
technologies of IoT (Internet of Things), such as RFID and wireless
sensors, used in the physical container yard help collect the related
operation data. Then, data from the physical container yard, including
arrival of CTs and status of ASCs, are transmitted to the virtual container
yard so that both data sets are synchronized. The real data in physical
container yard and simulation data in virtual container yard are stored
in data center to support optimization and verification. The operating
plan is simulated in the virtual container yard, which aims to calculate
the total energy consumption. According to the simulation results, the
workload distribution of all ASCs can be determined to be balanced to
meet sustainability requirements or not. Then, mixed integer program­
ming and a machine learning algorithm are used to identify the optimal
scheduling using the digital twin-based approach.
4.1. Simulation in the virtual container yard
In the proposed digital twin-based approach, a virtual container yard
mirrors a physical container yard. The virtual space can be built based
on the models of geometry, rules and behaviors. With the data center,
the real-time information from the physical space is transmitted to the
virtual model, including the operating data and equipment status. Then,
changes in practical operation can be displayed in the virtual yard. As
shown in Fig. 4, the virtual storage yard remains in sync with the
physical yard, which is one of the advantages of the digital twin. Thus,
the simulation in the virtual storage yard can more accurately charac­
terize the real environment, whose results are more representative.
Except for presenting the current operation environment in the vir­
tual container yard, some uncertain factors can be predicted and
responded to in advance through the simulation. The operating infor­
mation in the physical container yard, such as CT arrival time and ASC
working status, is used to drive the container-handling process in the
virtual container yard. With real-time transmission, the performance of
the operating scheme is evaluated with the simulation. For example,
Y. Gao et al. Journal of Cleaner Production 385 (2023) 135782

Table 3
Definitions of notations in the mathematical model.
∑∑ ( ) ∑∑ m m ( m )
min z = xmii′ ⋅ Cim + Tim + xii′ ⋅ γ ⋅ tsi − tami (4)
m∈Mi,i′ ∈N m∈Mi,i′ ∈N

( / / )
Cim = μm ⋅ bi vce + bi vcf (5)
( / )
Tim = ρm ⋅ (H − hi ) / vte + (H − hi ) vtf (6)
∑∑
xmii′ = 1, ∀i ∈ N (7)
m∈M i′ ∈N

∑∑
(8)
′
xmii′ = 1, ∀i ∈ N
m∈M i∈N

(9)
′
xmii′ + xmi′ i ≤ 1, ∀i, i ∈ N, m ∈ M
∑ ∑
(10)
′

Fig. 4. Virtual storage yard. xmii′ = xmi′ i′′ , ∀i ∈ N, m ∈ M

′ i′′ ∈N,i′′ ∕
=i′
i∈N,i∕
=i

/ / /
some CTs do not arrive as planned, affecting ASC scheduling. It will tfim ≥ tsmi + bi vce + bi vcf + (H − hi ) / vte + (H − hi ) vtf (11)
occur that ASCs operate continuously for a long time or wait for a long ( )
(12)
′
tfim ≤ tami′ + P ⋅ 1 − xii′ , ∀i, i ∈ N, m ∈ M m
time. This unbalanced workload distribution has a direct impact on the
total energy consumption of ASCs, which is not beneficial to the main­ (
tami′ ≤ tfim + P ⋅ 1 − xii′ , ∀i, i ∈ N, m ∈ M m
) ′
(13)
tenance and sustainable use of equipment.
The virtual container yard has the same operation scenario as the tsmi ≥ tami , ∀i ∈ N, m ∈ M (14)

physical yard, so that the simulation of the container allocation and ASC tsmi ≥ ti , ∀i ∈ N, m ∈ M (15)
scheduling uses a reference for the optimization of the operating plan.
The energy consumption of ASCs depends on the workload distribution, (16)
′
xmii′ = {0, 1}, ∀i, i ∈ N, m ∈ M
which makes it necessary to perform ASC scheduling experiments. Some
tsmi , tami ≥ 0, ∀i ∈ N, m ∈ M (17)
uncertain factors disturb the operation of ASCs in the practical envi­
ronment, affecting the scheduling performance. A detailed operating
plan is input to simulate the process of ASCs handling container tasks. Notations Definitions
Through the simulation, the working time and energy consumption of
M Number of ASCs, m ∈ M
each ASC are obtained to evaluate the operating plan. Then, the total
N Number of tasks, i ∈ N
distribution of energy consumed by ASCs can be calculated as the basis vce Moving speed of the crane without loading
for optimization. When the tasks are allocated inappropriately to ASCs, vcf Moving speed of the crane with loading
and the energy consumption is unbalanced, the operating plan should be vte Lifting speed of the crane trolley without loading
adjusted to execute in the physical container yard. Therefore, the virtual vtf Lifting speed of the crane trolley with loading
H Initial height of the crane trolley
container yard not only reflects the real environment, but also provides
P A large positive integer.
operational guidance for the physical space. loci Location of task i at the container yard, bi is the distance between the
initial bay and the target bay, and li is the stack height of the container
Cm i Moving energy consumption of the crane
4.2. Mathematical model Tim Lifting energy consumption of the crane trolley
tam i Time of ASC m arriving container task i
tsm Start time of ASC m handling container task i
A mixed integer programming model is developed to optimize ASC i
tfim End time of ASC m handling container task i
scheduling using the proposed digital twin-based approach. The oper­ xm xm = 1, ASC m handles task i after the task i; xm = 0, otherwise
′

ation environment is complex and dynamic at the ACT because handling ii′ ii′ ii′

containers involves multiple actions, and requires multi-equipment in­

teractions. Unexpected activities may affect the operating plan for
loading or unloading containers at the container yard. Hence, operating
optimization is required to assist ASCs in handling container tasks with
high efficiency and low energy. In the ACT digital twin system, the task
information can be collected from the physical container yard, which
can also be synchronized in the physical container yard. The acquisition
of task data, including arrival of CTs, status of ASCs and location of
containers, is the basis to create the operating plan at the container yard.
Then, mixed integer programming is conducted to optimize ASC
scheduling considering energy consumption.
The optimization goal is to minimize the total energy consumption of
ASCs, and the mathematical model is formulated as follows. In addition,
some assumptions are provided to facilitate the calculation of energy
ASCs. The working energy consumption when the ASC loads or unloads
the container task is stated in Eq. (5). Whether loading or unloading a
container, the ASC moves twice between the initial location and the
target location of the container task, performing one trip with an empty
load and one trip with a full load. Similarly, the crane trolley is lowered
from the highest height to the target height to pick up a container, fol­
lowed by raising the container to its starting position when loading the
container from the container yard. The crane trolley also needs to be
lowered and raised once each when unloading the container from the
CT. The energy consumption of the crane trolley working is thus
described in Eq. (6). Eq. (7) and Eq. (8) ensure that each container task
can only be allocated to one ASC. Eq. (9) guarantees the handling
sequence of the ASC, which means that the ASC operates task i after the
′

consumption. The crane and crane trolley of the ASC move at a constant current task i. Eq. (10) describes the continuity of the container-handling
speed, and the working time can be obtained based on the distances and task, referring to the fact that the ASC completes task i and then begins to
speed. The ASC stays at the initial bay when there is no task operation. handle task i and the next task i′′ . Eq. (11) shows the end time of ASC
′

The container relocation problem is not considered when ASCs perform
container tasks. The parameters and variables involved in the mathe­
matical model are listed in Table 3.
The objective function provided in Eq. (4) is composed of the
working energy consumption and the waiting energy consumption of
handling the container task, which includes the processing time of the
crane moving and the crane trolley lifting to pick up or drop off one task.
Eq. (12) and Eq. (13) eliminate the subtour in the operating paths of the
ASC. The end time of the current task handled by the ASC cannot be

6
Y. Gao et al. Journal of Cleaner Production 385 (2023) 135782

earlier than the start time of the next task handled by the same ASC. The
start time of the next task is later than the end time of the previous task.
Eq. (14) means that the start time of ASC handling container is later than
the time of the ASC arriving at the container. Eq. (15) shows that the ASC
can only start to work when the container arrives at the container yard.
Eq. (16) and Eq. (17) define the ranges of variables.
The task information, such as the arrival time of the container and
the start time of ASC working, cannot be definitively determined as the
operation environment dynamically changes. Containers may not arrive
at the storage yard according to the expected time due to disturbances.
scheduling plan. In the Q-learning algorithm, the agent has candidate
selections of the action that are evaluated to obtain the best reward.
Then, the agent is designated an action and the corresponding reward.
To obtain a sustainable operating plan, the reward of an ASC and its task
actions can be calculated using Eq. (19). The allocation of the task and
ASC depends on the value of energy consumed in the handling activity.
The energy reward of agent m choosing action a at state s is determined
by the maximum Q-value that the action is rewarded.
ERm (s, a) = maxQm (s, a), 1 ≤ m ≤ M (19)
m

Moreover, ASCs will not keep waiting for the arrival of unknown tasks In the case of state s, the value obtained by each agent selecting an
and will not finish handling containers as planned. A digital twin-based action is counted with Eq. (20). After selecting and executing the task
approach is proposed in this study to address the uncertainties that may assignment with the highest energy reward value, the highest Q-value is
occur in the container yard. Based on the ACT digital twin system, the updated in each step.
operation data required in the mathematical model can be monitored
and collected from the physical container yard. Then, a machine ERmax (s, a) = maxERm (s, a) = maxQm (s, a) (20)
a m
learning algorithm is employed to obtain the solution with the minimum
energy consumption. The algorithm flow of using Q-learning to find the optimal operation
plan for ASCs is provided in Fig. 5. At the beginning, the Q-value is
initialized. Then, the agent selects an action to execute at the current
4.3. Solution with machine learning algorithm state. In other words, the container is allocated to an ASC for the next
operations according to the arrived tasks and equipment status. After
This section illustrates the machine learning algorithm, which is used performing an action, the agent obtains the reward for its selection,
to obtain the optimal solution in the proposed digital twin-based which also causes an update of the Q-value. The pursuit of the maximum
approach. The optimization objective of the abovementioned mathe­ Q-value drives the agent to learn to make a better decision. When the
matical model is to minimize the energy consumption of ASCs for all agent stays in the next state, it will take a new action that brings the best
container tasks. According to the problem statement and established reward. The Q-value is then updated with the iteration. Finally, the
model, the container task is allocated to ASC, which requires minimal maximum Q-value indicates that the agent finds the optimal strategy.
energy consumption to promote sustainable development. Hence, it is When using Q-learning algorithm to obtain a decision, the interac­
essential to calculate the energy consumed in the activities of ASC tion between agent and environment is essential to update the Q-value.
handling containers. In the proposed resource optimization problem, the With the support of digital twin, the status of ASCs, arrival information
allocation of container tasks and ASCs is directly related to the energy of tasks, and the operation environment can be observed from the
consumption of the handling equipment. The Q-learning algorithm is physical container yard and synchronized in the virtual container yard.
employed to evaluate the selections among ASCs and tasks in the ACT Accordingly, the data required in the Q-learning algorithm can be ob­
dynamic operation environment. tained to evaluate the agents and actions. For example, the operation
The main point of Q-learning is to help the agent make decisions and environment of the container yard reflects the status of the handling
gain rewards. The three primary elements of the Q-learning algorithm equipment, which assists the agent in observing the state and choosing
thus include the agent, state and action. The agent will obtain the
reward when it takes an action at the state based on the reward mech­
anism. Then, the agent moves to the next state and obtains a new reward
for the action. The action set of the agent is denoted as A. The iterative
process of determining the optimal strategy can be described in Eq. (18),
where the agent executing action a at state s will obtain reward R(s, a),
and α, γ are the learning factor and discount factor, respectively, which
range from 0 to 1. The accumulative reward value of the action is
denoted as Q(s,a). In addition, the ε-greedy strategy is used for agents to
take an action. The parameter ε denotes the probability of choosing an
optimal action, while the value of (1-ε) denotes the probability of
choosing a random action. The parameter values of Q-learning refer to
Chen et al. (2020a).
( )
Q(s, a) = (1 − α) ⋅ Q(s, a) + α⋅ R(s, a) + γ ⋅ max
′
Q(s’, a’) (18)
a ∈A

Containers are transported by CTs to the container yard and then

unloaded by ASCs. The ASC handling the container is regarded as per­
forming a task. The tasks are allocated to multiple ASCs while satisfying
the operating constraints. The final goal is to find the scheduling plan
that ASCs spend the least energy consumption completing all tasks. Q-
learning is introduced to assist ASCs, which are modeled as agents to
evaluate the allocated tasks. The agent is required to adapt to the dy­
namic environment and to learn to take an action in an appropriate way.
The ASC then expends different amounts of energy to complete various
tasks. The total energy consumption of an ASC is accumulated based on
its workload. Each ASC needs to use as little energy as possible to
complete its respective tasks for sustainable development. Therefore, the
assignment of tasks to ASCs affects the overall performance of the Fig. 5. Flow chart of Q-learning in optimizing the operation plan for ASCs.

7
Y. Gao et al. Journal of Cleaner Production 385 (2023) 135782

the action. The container task information is also related to the action set
that the agent would like to execute. Based on the ACT digital twin
system, the agent is required to perform actions with minimum energy
consumption for sustainability.
5. Case analysis
This section provides numerical experiments to demonstrate the
effectiveness of the digital twin-based approach for the operating opti­
mization at the container yard. First, the numerical case is designed to
conduct the experiment conveniently. Then, the results are obtained to
verify the feasibility of the digital twin-based approach, and solutions
with different methods are compared to discuss performance.
5.1. Case design
The data used in the instance are gathered from a port in China,
including the performance parameters of equipment and the operating
information of containers. Dedicated sensors used in the equipment and
container yard collect the status data of equipment and the container
information of tasks, which is integrated into the port operating system
and can be extracted to conduct case experiments in this paper. The
moving speed of the ASC crane without loading is 4 m/s, and its loaded
speed is approximately 1.67 m/s. The lifting speed of the crane trolley
without loading is 1.5 m/s, while the loaded speed of the trolley is 0.75
m/s. The maximum lifting height of the crane trolley is 19.75 m. In
addition, the average energy consumption of the crane per unit time is
148 Kw/h, and the average energy consumption of the crane trolley is
103 Kw/h. The average energy consumption of ASC per unit time is 80
Kw/h during a non-working period. These equipment parameters are
not unique because they vary with different equipment types.
At the container yard, the CTs transport containers to the storage
blocks. Then, ASCs are assigned to discharge containers, which are then
stacked at the yard. An empty CT can take the imported container when
leaving the container yard, and all containers that are about to arrive or
leave are handled by ASCs at the yard during a period. It is observed
from the port that container task assignments occur intensively within 4
h, which is thus set as an operating period. Port data of container tasks
are partly provided in Table 4. Task information in the experiments is
incomplete but realistic due to interest. Many storage blocks are present
in the container yard and are numbered sequentially. Exported con­
tainers are discharged to store at the block, while imported containers
are loaded onto CTs. Then, the operation type is classified into putting
down or picking up. For example, the given container task 4 stacked at
block 42 will be picked from bay 11 and tier 1 at the estimated time.
Experiments are conducted in Python with Intel Core i7-11800H @ 2.30
GHz system.
5.2. Experimental results
5.2.1. Small-scale case results
is input to solve the model. Then, the operating plan of ASCs can be
obtained, as provided in Table 5.
5.2.2. Large-scale case results
The large-scale case is designed to be solved with the proposed
digital twin-based approach. The mathematical model has been verified
in the above section. The Q-learning algorithm is used to find the
optimal solution for ASCs. Combined with the simulation in the virtual
container yard, the final solutions of the large-scale case are given in
Table 6. There are 221 container tasks and 8 ASCs for handling.
The detailed operating plan of each ASC is described in Fig. 6. The
location of the red rectangle in the figure indicates the arrival time of the
container tasks. The container tasks arrive at the scattered time during
the operating period. The gap between two adjacent red and blue rect­
angles represents the gap between the arrival time of the task and the
start working time of the ASC. When the ASC completes the current task,
it will take some time to wait for the next task. In addition, a task
allocated to an ASC cannot arrive within the estimated time, which
causes the ASC to wait. Therefore, the non-working energy consumption
of ASCs is generally higher. The width of the blue rectangle in the figure
refers to the operation time required for the ASC to handle the task, and
different tasks take their own processing time.
5.3. Comparative analysis
5.3.1. Comparison with different methods
Comparative experiments are conducted with the same case to verify
the effectiveness of the proposed digital twin-based approach using
genetic algorithm (GA) and particle swarm optimization (PSO). Both
algorithms are commonly used to solve optimization problems. In this
numerical experiment, the algorithm parameters of GA and PSO are
referred to the literature (Han et al., 2019; Lu and Wang, 2019), and six
ASCs are assigned to handle 98 container tasks. The running time of
conducting experiments with the three solutions is provided in Table 7.
The experimental results of different solution methods are given in
Table 8.
Experiments are performed ten times with GA, PSO, and the pro­
posed approach, respectively. Their running time is tracked to compare
the computational performance of different methods. Obviously, the
proposed approach obtains the solution with the least time and PSO
costs less computing time than GA. It indicates that the proposed
approach shows a better performance than the other two algorithms
when solving this case.
The total working energy consumption of all tasks is 833.17 Kw,
Table 5
Operating plan of ASCs in the small-scale case.
Task ASC Arrival Start Working energy Non-working
time of working consumption of energy
ASC time of ASC (Kw) consumption of
ASC ASC (Kw)

To validate the established mathematical model, the small-scale case 1 1 12:00:00 12:04:02 8.96 5.38
is solved with GUROBI Solver as the benchmark comparison. Five 2 2 12:00:00 12:15:21 3.63 20.47
3 1 12:08:08 13:06:09 5.47 77.36
container tasks are selected as the small-scale case. The task information
4 2 12:17:12 13:07:35 8.12 67.18
5 1 13:08:46 13:09:12 4.43 0.58
Table 4
In the small-scale case, the non-working energy consumption of ASC handling
Information on the container tasks used in the case.
task 3 and task 4 is larger mainly because the start working time is much later
Number Blocks Estimated arrival time Location Operation type than the arrival time. If the container task is not ready at the estimated time, the
1 38 12:04:02 (30,1) Putting down waiting time and thus non-working energy consumption will increase. The gap
2 40 15:16:29 (11,2) Picking up between the arrival time and the start working time of ASC is the factor that
3 41 15:09:06 (15,2) Picking up affects the non-working energy consumption. The working energy consumption
4 42 14:00:33 (11,1) Picking up of ASC depends on the workload of the task. The ASC spends more energy
5 43 13:07:35 (26,1) Putting down consumption handling the container that is stored at a far location. For task 1,
6 44 13:06:09 (15,3) Picking up the ASC crane moves to the bay that is far away from the initial location. Hence,
7 45 13:09:12 (10,3) Picking up
the working energy consumption of handling task 1 is higher than that of the
8 46 12:15:21 (7,4) Picking up
others.

8
Y. Gao et al. Journal of Cleaner Production 385 (2023) 135782

regardless of the solution methods used. However, the working energy

It can be found from Table 5 that the total energy consumption of ASC 8 is the lowest, while that of ASC 1 is the highest. ASC 8 spends the lowest working energy consumption and the highest non-working energy
consumption handling the allocated tasks. It means that the tasks are not ready for the handling equipment as the ASC waits for a long time. In contrast, ASC 1 takes the highest working energy consumption and the lowest
consumption of each ASC varies in the three solutions. For the proposed
approach, the total non-working energy consumption of ASCs is the
lowest. In addition, the total non-working energy consumption of PSO is

consumption
Total energy
lower than that of GA. It can be shown that the ASC operating plan
solved with the proposed approach has the best performance among the

591.03
581.11
568.28
543.19
543.33
523.21
523.70
522.70
three methods according to the total energy consumption.
The distribution of energy consumed by ASCs from the three
methods is drawn in Fig. 7. The bar chart shows the working energy
consumption trend of the three methods. The grey bars in Fig. 7 repre­
sent the working energy consumption of the proposed approach, and
there is a small gap in the progression. In contrast, the gap trend of the
Waiting energy
consumption

blue bars is the strongest, which are solved with GA. Hence, regarding
the working energy consumption, the difference of each ASC in the
300.00
315.67
329.24
322.11
329.42
358.78
345.73
363.53
proposed approach is the least, followed by that in PSO among the three
methods. Moreover, the line chart shows the trend of non-working en­
ergy consumption with the three methods. The yellow and blue lines,
referring to the non-working energy consumption of GA and PSO, show
a similar trend. The green line, that is, the non-working energy con­
sumption of the proposed approach, appears relatively flat, which in­
Working energy

dicates that the non-working energy consumption of ASCs does not show
consumption

a large difference from the proposed approach. Thus, considering either

291.03
265.44
239.03
221.08
213.91
164.43
177.97
159.17

the working or non-working energy consumption, container tasks in the

case can be allocated to ASCs appropriately using the proposed
approach.
1,4,11,19,22,25,34,40,48,56,62,69,75,77,84,86,88,96,106,108,116,120,124,131,137,144,146,155,163,166,173,181,190,196,204,210,212,220

5.3.2. Comparison of two strategies

In this section, a comparison of two operating modes is given with a
2,5,14,23,26,35,42,50,57,63,70,76,85,89,95,104,107,109,117,121,125,133,138,145,153,161,164,167,174,182,189,197,206,213,219

case of five ASCs. When assigning container tasks to ASCs, two opposite
strategies can be followed. In the centralized operation, containers are
allocated to the available ASCs centrally. If the ASC is idle, a task will be
3,12,20,24,27,33,39,47,55,64,71,78,87,97,103,110,118,122,126,132,139,147,154,162,168,176,185,193,198,207,214,221

scheduled for it. In this centralized operation, some pieces of equipment

will have larger workloads, while others will have smaller workloads. In
non-working energy consumption. With most tasks being allocated, ASC 1 consumes the most energy in total.

contrast, tasks can be allocated to ASCs dispersedly, using decentralized

operation. Under the circumstance, the handling equipment is scheduled
6,15,21,28,36,43,53,58,65,72,79,90,98,105,111,119,127,135,140,148,156,165,175,183,194,199,205,215

as much as possible. Then, two strategies are conducted in the case with
the proposed approach, and the energy consumption is given in Table 9.
For the decentralized operation, a penalty factor is added to calculate
7,17,29,38,45,51,59,66,73,80,91,99,112,123,134,141,149,158,169,179,187,191,200,208,216

the non-working energy consumption of ASCs. The average energy

8,13,30,37,46,52,60,67,74,81,92,100,113,128,136,150,159,170,177,184,195,203,209,217

consumption of ASC per unit time during the non-working period in­
creases by 5% if its cumulative waiting time exceeds half an hour. The
setting of the time limit can be determined by the equipment perfor­
mance. In summary, when the inactive time of the handling equipment
9,18,31,41,49,61,82,93,101,114,129,142,151,157,171,178,186,188,201,211

reaches a set value, more energy consumption is required to remain

10,16,32,44,54,68,83,94,102,115,130,143,152,160,172,180,192,202,218

active. The penalty factor of working energy consumption is also

Solutions of the large-scale case with the digital twin-based approach.

introduced, where the average energy consumption of the ASC per unit
time is increased by 10%. When the handling equipment works for a
long time, its energy consumption will grow.
The energy consumption results of ASCs are shown in Table 9. The
idle ratio refers to the rate of non-working energy consumption to
working energy consumption, which shows the distribution of the
workload. The idle ratio of the decentralized operation is lower overall
than that of the centralized operation. The total working energy con­
sumption of centralized operation is less than that of decentralized
operation, while the total non-working energy consumption presents the
opposite relationship.
The non-working energy consumption is higher in the centralized
operation, primarily when tasks are arranged centrally for some ASCs
Tasks assigned to ASC

handling and the remaining ASCs remain idle. Even with the penalty
factor added to working energy consumption, the total working energy
consumption of the centralized operation is still lower than that of the
decentralized operation. From the perspective of the total energy con­
sumption, the performance of the centralized operation is superior to
that of decentralized operation.
Table 6

The port operator can choose one strategy according to the operation
ASC

and management goals. For the long-term use and maintenance of

1
2
3
4
5
6
7
8

9
Y. Gao et al. Journal of Cleaner Production 385 (2023) 135782

Fig. 6. Gantt chart of ASC handling tasks.

virtual container yard for validation. As shown in Fig. 8, the ASC

Table 7
operation in the virtual container yard provides four views, including
Running time of experiments with the compared algorithms (seconds).
the arrival of CT and container, the action of ASC, and the overall
Number of experiments GA PSO The proposed approach conditions. At the beginning, CTs enter to deliver exported containers
1 275.27 165.98 111.81 and pick up imported containers at ACT. Then, CTs are assigned to
2 295.98 163.89 101.44 different container blocks, and ASCs are allocated to handle containers
3 292.50 161.63 100.77
according to their own task requirements. The entire process is visual­
4 266.89 168.95 100.84
5 290.45 167.08 99.25
ized in the virtual container yard.
6 284.66 163.81 112.91 Operators and managers can observe the validation in the virtual
7 274.49 164.60 99.67 container yard from operation screen. The equipment status and oper­
8 277.77 166.40 110.02 ating process are both reflected in the virtual container yard, such as the
9 285.44 165.11 112.02
task distribution of workload and the energy consumption of ASCs.
10 292.67 163.39 111.67
Average 283.61 165.08 105.40 When some ASCs fail, the replacement of equipment can respond in a
timely manner. Based on the current task distribution, a balanced allo­
cation of workload can be provided for the reasonable use of ASCs.
equipment, a decentralized operation should be used due to the lower Moreover, the operating data of the physical container yard are
idle ratio. The distribution of non-working and working time periods is collected and stored to assist in evaluating the obtained operating plan
relatively balanced in a decentralized operation, which is appropriate of ASCs. Based on the historical data on equipment usage, operators and
for sustainable development. managers have access to analyze the working and non-working energy
consumption of each ASC. When the energy consumption used by all
5.4. Validation experiment ASCs stays at a similar level, the operating plan of ASCs can be deter­
mined as appropriate because ASCs are balanced and assigned to com­
In the proposed digital twin-based approach, the solutions obtained plete all container tasks. ASCs are well scheduled to handle containers
from the optimization algorithm are required to be verified in virtual by optimizing the working and non-working energy consumption, which
storage yard. Then, the final solutions can be appropriate for the prac­ is beneficial to the sustainable use of equipment.
tical operation environment. The virtual storage yard is developed with The proposed ASC scheduling and other related optimization prob­
Unity 3D based on models of geometry, rules, and behaviors. With the lems can be solved by employing the existing approaches without
support of data from the physical container yard, the environment of applying digital twin. However, the complex and time-varying envi­
virtual container yard is synchronized, and thus the simulation results in ronment of ACT can be monitored and visualized with the support of
the virtual environment are reliable. After obtaining a solution from the digital twin. Additionally, digitalization becomes a trend in industrial
Q-learning algorithm, the operating plan of ASCs is executed in the fields due to the maturation of new technologies. This paper proposes

Table 8
Comparison results of three solutions.
GA PSO The proposed approach

Working energy Non-working energy Working energy Non-working energy Working energy Non-working energy
consumption (Kw) consumption (Kw) consumption (Kw) consumption (Kw) consumption (Kw) consumption (Kw)

ASC1 143.87 160.02 170.99 157.20 159.14 137.24

ASC2 185.30 150.22 142.76 145.49 159.77 153.98
ASC3 126.92 157.58 136.00 155.40 140.85 152.29
ASC4 164.66 147.24 142.92 156.67 125.40 146.96
ASC5 111.39 152.13 132.10 159.02 122.66 153.98
ASC6 101.04 173.84 108.40 162.47 125.36 160.42
Total 833.17 941.03 833.17 936.25 833.17 904.87

10
Y. Gao et al. Journal of Cleaner Production 385 (2023) 135782

Fig. 7. Trend of ASC energy consumption with three methods.

Table 9
Energy consumption of ASCs using two strategies.
Centralized operation Decentralized operation

Working energy Non-working energy Idle ratio (W2/ Working energy Non-working energy Idle ratio (W4/
consumption (W1) consumption (W2) W1) consumption (W3) consumption (W4) W3)

ASC1 122.27 87.93 0.72 128.41 85.13 0.66

ASC2 121.12 85.86 0.71 126.91 92.14 0.73
ASC3 114.65 90.51 0.79 119.67 84.87 0.71
ASC4 104.42 79.61 0.76 108.56 80.43 0.74
ASC5 103.55 100.89 0.97 107.22 84.02 0.78
Total 566.01 444.80 0.79 590.77 426.60 0.72

Fig. 8. Four views of ASC operating in virtual container yard.

the application of digital twin to the energy consumption management 6. Conclusion

of equipment used at ACT. The energy consumed by the ASC and
container handling can be displayed and communicated with a digital
twin-based approach that is convenient for observing and managing the
overall energy consumption. Thus, operators and managers can deter­
mine the operating plan of ASCs for sustainability.
This paper focused on ACT operating optimization while considering
energy consumption and proposed a digital twin-based approach to
facilitate decision-making. A mathematical model that minimized the
total energy consumption was established to find the ASC operating
plan. Meanwhile, this model was validated by the GUROBI solver with a
small-scale case, which obtained the scheduling of ASC handling
container tasks. Based on the digital twin, the operation information can

11
Y. Gao et al.
be observed and obtained to support the decisions of production plan. In
addition, a machine learning algorithm was used to optimize the oper­
ation of ASCs with minimum energy consumption. Experiments using
three methods verified the effectiveness of the proposed approach. The
performance of the digital twin-based approach was better than that of
PSO and GA in the large-scale case. The working and non-working en­
ergy consumption of the proposed approach was further analyzed to
compare the centralized and decentralized operation strategies. The
decentralized operation of ASCs was more suitable for equipment
sustainability.
In the proposed digital twin-based approach, a virtual container yard
was established based on the practical data from the physical container
yard, such that both were synchronized for observation and validation.
According to the collected data, the operating plan of ASCs was opti­
mized to minimize their energy consumption. The solution obtained
from Q-learning algorithm was validated in the virtual container yard
and further optimized to achieve sustainability. The working and non-
working energy consumption were compared and analyzed to lower
the idle ratio of ASCs. The detailed energy consumption of each ASC was
displayed in tables and figures to assist in allocating all tasks. A sus­
tainable solution indicated that all ASCs handled containers in a way
that balanced both working and non-working energy consumption. In
addition, the solution was certainly an alternative with the lowest total
energy consumption of ASCs, if operators and managers pay more
attention to the metrics. ACT could optimize its own operating plans
according to the observed energy consumption and actual production
goals.
Further research can be explored as follows. First, better variants
should be discussed in the future. This paper compared the proposed
approach with GA and PSO, which were classic algorithms that were
typically used in optimization problems. However, some novel and
mixed algorithms have achieved good performances in engineering
optimization problems(Bi et al., 2015, 2020; Tang et al., 2021). Other
intelligent algorithms can be included for further comparative studies,
such as simulated annealing and ant colony optimization. Second, an
integrated optimization of ASC, AGV and QC can be investigated which
would involve not only the operation at the container yard but also the
loading and unloading processes of vessels. Additionally, future studies
should investigate combining the digital twin with other optimization
algorithms. The performance of digital twin and different solution
methods can then be compared and improved.
CRediT authorship contribution statement
Yinping Gao: Conceptualization, Methodology, Software, Writing –
original draft. Daofang Chang: Visualization, Investigation, Resources.
Chun-Hsien Chen: Supervision, Methodology, Writing – review &
editing.
Declaration of competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Data availability
The authors do not have permission to share data.
Acknowledgment
This work was supported by the National Key Research and Devel­
opment Plan of China (No. 2019YFB1704403).
12
Journal of Cleaner Production 385 (2023) 135782
References
Aheleroff, S., Xu, X., Zhong, R.Y., Lu, Y., 2021. Digital twin as a service (DTaaS) in
industry 4.0: an architecture reference model. Adv. Eng. Inf. 47, 101225.
Alamoush, A.S., Ballini, F., Ölçer, A.I., 2020. Ports’ technical and operational measures
to reduce greenhouse gas emission and improve energy efficiency: a review. Mar.
Pollut. Bull. 160, 111508.
Ampah, J.D., Yusuf, A.A., Afrane, S., Jin, C., Liu, H., 2021. Reviewing two decades of
cleaner alternative marine fuels: towards IMO’s decarbonization of the maritime
transport sector. J. Clean. Prod. 320, 128871.
Bi, J., Yuan, H., Duanmu, S., Zhou, M., Abusorrah, A., 2020. Energy-optimized partial
computation offloading in mobile-edge computing with genetic simulated-
annealing-based particle swarm optimization. IEEE Internet Things J. 8, 3774–3785.
Bi, J., Yuan, H., Tan, W., Zhou, M., Fan, Y., Zhang, J., Li, J., 2015. Application-aware
dynamic fine-grained resource provisioning in a virtualized cloud data center. IEEE
Trans. Autom. Sci. Eng. 14, 1172–1184.
Budiyanto, M.A., Huzaifi, M.H., Sirait, S.J., Prayoga, P.H.N., 2021. Evaluation of CO2
emissions and energy use with different container terminal layouts. Sci. Rep. 11,
1–14.
Chen, G., Govindan, K., Golias, M.M., 2013. Reducing truck emissions at container
terminals in a low carbon economy: proposal of a queueing-based bi-objective model
for optimizing truck arrival pattern. Transport. Res. E Logist. Transport. Rev. 55,
3–22.
Chen, H., Ji, Y., Niu, L., 2020a. Reinforcement learning path planning algorithm based
on obstacle area expansion strategy. Intelligent Service Robotics 13, 289–297.
Chen, S., Zeng, Q., 2021. Carbon-efficient scheduling problem of electric rubber-tyred
gantry cranes in a container terminal. Eng. Optim. 1–19.
Chen, X., He, S., Zhang, Y., Tong, L.C., Shang, P., Zhou, X., 2020b. Yard crane and AGV
scheduling in automated container terminal: a multi-robot task allocation
framework. Transport. Res. C Emerg. Technol. 114, 241–271.
Dulebenets, M.A., 2018. A diploid evolutionary algorithm for sustainable truck
scheduling at a cross-docking facility. Sustainability 10, 1333.
Dulebenets, M.A., 2022. Multi-objective collaborative agreements amongst shipping
lines and marine terminal operators for sustainable and environmental-friendly ship
schedule design. J. Clean. Prod. 342, 130897.
Facchini, F., Digiesi, S., Mossa, G., 2020. Optimal dry port configuration for container
terminals: a non-linear model for sustainable decision making. Int. J. Prod. Econ.
219, 164–178.
Fang, Y., Peng, C., Lou, P., Zhou, Z., Hu, J., Yan, J., 2019. Digital-twin-based job shop
scheduling toward smart manufacturing. IEEE Trans. Ind. Inf. 15, 6425–6435.
Gao, Y., Chang, D., Chen, C.-H., Xu, Z., 2022. Design of digital twin applications in
automated storage yard scheduling. Adv. Eng. Inf. 51, 101477.
Gharehgozli, A.H., Laporte, G., Yu, Y., De Koster, R., 2015. Scheduling twin yard cranes
in a container block. Transport. Sci. 49, 686–705.
Gharehgozli, A.H., Vernooij, F.G., Zaerpour, N., 2017. A simulation study of the
performance of twin automated stacking cranes at a seaport container terminal. Eur.
J. Oper. Res. 261, 108–128.
Granacher, J., Nguyen, T.-V., Castro-Amoedo, R., Maréchal, F., 2022. Overcoming
decision paralysis—a digital twin for decision making in energy system design. Appl.
Energy 306, 117954.
Guo, P., He, X., Luan, Y., Wang, Y., 2021. Logic-based Benders decomposition for gantry
crane scheduling with transferring position constraints in a rail–road container
terminal. Eng. Optim. 53, 86–106.
Han, X., Wang, Q., Huang, J., 2019. Scheduling cooperative twin automated stacking
cranes in automated container terminals. Comput. Ind. Eng. 128, 553–558.
He, J., Huang, Y., Yan, W., Wang, S., 2015. Integrated internal truck, yard crane and
quay crane scheduling in a container terminal considering energy consumption.
Expert Syst. Appl. 42, 2464–2487.
Hu, X., Guo, J., Zhang, Y., 2019. Optimal strategies for the yard truck scheduling in
container terminal with the consideration of container clusters. Comput. Ind. Eng.
137, 106083.
Hu, Z.-H., Sheu, J.-B., Luo, J.X., 2016. Sequencing twin automated stacking cranes in a
block at automated container terminal. Transport. Res. C Emerg. Technol. 69,
208–227.
Jiang, L., Huang, H., Ding, Z., 2019. Path planning for intelligent robots based on deep Q-
learning with experience replay and heuristic knowledge. IEEE/CAA J. Automatica
Sinica 7, 1179–1189.
Junior, I.C.L., De Oliveira, U.R., De Almeida Guimarães, V., Ribeiro, L.G., Fernandes, V.
A., 2022. Probabilistic analysis of the sustainable performance of container
terminals. Res. Transport. Business Manag. 43, 100725.
Karakas, S., Kirmizi, M., Kocaoglu, B., 2021. Yard block assignment, internal truck
operations, and berth allocation in container terminals: introducing carbon-footprint
minimisation objectives. Marit. Econ. Logist. 23, 750–771.
Kavoosi, M., Dulebenets, M.A., Abioye, O.F., Pasha, J., Wang, H., Chi, H., 2019. An
augmented self-adaptive parameter control in evolutionary computation: a case
study for the berth scheduling problem. Adv. Eng. Inf. 42, 100972.
Kim, J., Hong, E.J., Yang, Y., Ryu, K.R., 2021. Noisy optimization of dispatching policy
for the cranes at the storage yard in an automated container terminal. Appl. Sci. 11,
6922.
Kuo, S.-Y., Lin, P.-C., 2020. Determinants of green performance in container terminal
operations: a lean management. J. Clean. Prod. 275, 123105.
Li, X., Peng, Y., Huang, J., Wang, W., Song, X., 2021. Simulation study on terminal layout
in automated container terminals from efficiency, economic and environment
perspectives. Ocean Coast Manag. 213, 105882.
Y. Gao et al.
Liu, D., Ge, Y.-E., 2018. Modeling assignment of quay cranes using queueing theory for
minimizing CO2 emission at a container terminal. Transport. Res. Transport Environ.
61, 140–151.
Liu, S., Liu, S., Lu, C., Zhou, M., Abusorrah, A., 2022. Valid inequality and variable
fixation for unrestricted block relocation problems. IEEE Trans. Intell. Transport.
Syst.
Lopes, M.R., Costigliola, A., Pinto, R., Vieira, S., Sousa, J.M.C., 2020. Pharmaceutical
quality control laboratory digital twin–A novel governance model for resource
planning and scheduling. Int. J. Prod. Res. 58, 6553–6567.
Lu, H., Wang, S., 2019. A study on multi-ASC scheduling method of automated container
terminals based on graph theory. Comput. Ind. Eng. 129, 404–416.
Luo, J., Wu, Y., 2015. Modelling of dual-cycle strategy for container storage and vehicle
scheduling problems at automated container terminals. Transport. Res. E Logist.
Transport. Rev. 79, 49–64.
Negri, E., Pandhare, V., Cattaneo, L., Singh, J., Macchi, M., Lee, J., 2021. Field-
synchronized Digital Twin framework for production scheduling with uncertainty.
J. Intell. Manuf. 32, 1207–1228.
Ng, M., Talley, W.K., 2020. Rail intermodal management at marine container terminals:
loading double stack trains. Transport. Res. C Emerg. Technol. 112, 252–259.
Niu, B., Liu, Q., Wang, Z., Tan, L., Li, L., 2021. Multi-objective bacterial colony
optimization algorithm for integrated container terminal scheduling problem. Nat.
Comput. 20, 89–104.
Ozturk, M., Jaber, M., Imran, M.A., 2018. Energy-aware smart connectivity for IoT
networks: enabling smart ports. Wireless Commun. Mobile Comput. 2018.
Park, H.-A., Byeon, G., Son, W., Jo, H.-C., Kim, J., Kim, S., 2020. Digital twin for
operation of microgrid: optimal scheduling in virtual space of digital twin. Energies
13, 5504.
Pasha, J., Dulebenets, M.A., Fathollahi-Fard, A.M., Tian, G., Lau, Y.-Y., Singh, P.,
Liang, B., 2021. An integrated optimization method for tactical-level planning in
liner shipping with heterogeneous ship fleet and environmental considerations. Adv.
Eng. Inf. 48, 101299.
Pasha, J., Nwodu, A.L., Fathollahi-Fard, A.M., Tian, G., Li, Z., Wang, H., Dulebenets, M.
A., 2022. Exact and metaheuristic algorithms for the vehicle routing problem with a
factory-in-a-box in multi-objective settings. Adv. Eng. Inf. 52, 101623.
Rabbani, M., Oladzad-Abbasabady, N., Akbarian-Saravi, N., 2022. Ambulance routing in
disaster response considering variable patient condition: NSGA-II and MOPSO
algorithms. J. Ind. Manag. Optim. 18, 1035.
Savolainen, J., Urbani, M., 2021. Maintenance optimization for a multi-unit system with
digital twin simulation. J. Intell. Manuf. 32, 1953–1973.
Shu, F., Mi, W., Li, X., Zhao, N., Mi, C., Yang, X., 2015. A double-population genetic
algorithm for asc loading sequence optimization in automated container terminals.
J. Coast Res. 64–70.
Sim, J., 2018. A carbon emission evaluation model for a container terminal. J. Clean.
Prod. 186, 526–533.
13
Journal of Cleaner Production 385 (2023) 135782
Tang, J., Liu, G., Pan, Q., 2021. A review on representative swarm intelligence
algorithms for solving optimization problems: applications and trends. IEEE/CAA J.
Automatica Sinica 8, 1627–1643.
Tsolakis, N., Zissis, D., Papaefthimiou, S., Korfiatis, N., 2022. Towards AI driven
environmental sustainability: an application of automated logistics in container port
terminals. Int. J. Prod. Res. 60, 4508–4528.
Wan, Z., Zhang, T., Sha, M., Guo, W., Jin, Y., Guo, J., Liu, Y., 2021. Evaluation of
emission reduction strategies for berthing containerships: a case study of the Shekou
Container Terminal. J. Clean. Prod. 299, 126820.
Wang, K.-J., Lee, Y.-H., Angelica, S., 2021. Digital twin design for real-time monitoring–a
case study of die cutting machine. Int. J. Prod. Res. 59, 6471–6485.
Wang, Q., Jiao, W., Wang, P., Zhang, Y., 2020a. Digital twin for human-robot interactive
welding and welder behavior analysis. IEEE/CAA J. Automatica Sinica 8, 334–343.
Wang, X., Zhou, M., Zhao, Q., Liu, S., Guo, X., Qi, L., 2020b. A branch and price
algorithm for crane assignment and scheduling in slab yard. IEEE Trans. Autom. Sci.
Eng. 18, 1122–1133.
Xiong, M., Wang, H., Fu, Q., Xu, Y., 2021. Digital twin–driven aero-engine intelligent
predictive maintenance. Int. J. Adv. Manuf. Technol. 114, 3751–3761.
Yan, Q., Wang, H., Wu, F., 2022. Digital twin-enabled dynamic scheduling with
preventive maintenance using a double-layer Q-learning algorithm. Comput. Oper.
Res. 144, 105823.
You, M., Wang, Q., Sun, H., Castro, I., Jiang, J., 2022. Digital twins based day-ahead
integrated energy system scheduling under load and renewable energy uncertainties.
Appl. Energy 305, 117899.
Ypsilantis, P., Zuidwijk, R., 2019. Collaborative fleet deployment and routing for
sustainable transport. Sustainability 11, 5666.
Yue, L., Fan, H., Ma, M., 2021. Optimizing configuration and scheduling of double 40 ft
dual-trolley quay cranes and AGVs for improving container terminal services.
J. Clean. Prod. 292, 126019.
Zaerpour, N., Gharehgozli, A., De Koster, R., 2019. Vertical expansion: a solution for
future container terminals. Transport. Sci. 53, 1235–1251.
Zhang, M., Tao, F., Nee, A.Y.C., 2021. Digital twin enhanced dynamic job-shop
scheduling. J. Manuf. Syst. 58, 146–156.
Zhang, Z., Guan, Z., Gong, Y., Luo, D., Yue, L., 2022. Improved multi-fidelity simulation-
based optimisation: application in a digital twin shop floor. Int. J. Prod. Res. 60,
1016–1035.
Zhao, N., Fu, Z., Sun, Y., Pu, X., Luo, L., 2022. Digital-twin driven energy-efficient multi-
crane scheduling and crane number selection in workshops. J. Clean. Prod. 336,
130175.
Zhuang, C., Miao, T., Liu, J., Xiong, H., 2021. The connotation of digital twin, and the
construction and application method of shop-floor digital twin. Robot. Comput.
Integrated Manuf. 68, 102075.