1810 IEEE ELECTRON DEVICE LETTERS, VOL. 42, NO.
12, DECEMBER 2021
An Image Memory Logic Unit Inspired
by Human Retina
Wen Du , Caihong Li, Ziyang Long , Yixuan Huang, Lingzhi Luo, Jihua Zou,
and Jiang Wu , Senior Member, IEEE
Abstract — Human-inspired neuromorphic computing
system breaks the limit of “von Neumann bottleneck” and
improves the computing efficiency. However, the processing
of a large amount of non-structural visual data results in
a considerable waste of computing resources. In human
visual system, visual information is preprocessed by the
retina before transmitted to the visual cortex, which effec-
tively reduces data redundancy. Therefore, multifunctional
intelligent devices which integrate sensing-memory-
processing functions are desired for the artificial visual
system. Herein, an image memory logic unit (IMLU) inspired
by human retina is proposed. The IMLU is formed by a
simple two-terminal photodetector and comparator module.
The photodetector is based on the Mox W1−x S2 alloy with
persistent photo conductivity effect. In addition, the IMLU
exhibits capabilities of in situ sensing, memory and logical
operations (“AND”, “OR”, and “XOR”). Furthermore,
the IMLU array enables image processing functions such
as dilation, erosion and difference recognition, which
shows great potential to provide simplified information
processing for subsequent tasks in machine vision.
Index Terms — Human-inspired, logical operation, image
processing, sensing-memory-processing system.
I. I NTRODUCTION
T HE development of artificial intelligence (AI) technology
and Internet-of-Things (IoT) makes it essential to achieve
Manuscript received September 15, 2021; revised October 3, 2021;
accepted October 3, 2021. Date of publication October 5, 2021; date
of current version November 24, 2021. This work was supported in
part by the National Natural Science Foundation of China under Grant
61974014, in part by the Fundamental Research Funds for the Central
Universities under Grant ZYGX2019Z018, in part by the Innovation
Group Project of Sichuan Province under Grant 20CXTD0090, in part by
the University of Electronic Science and Technology of China (UESTC)
Shared Research Facilities of Electromagnetic Wave and Matter Interac-
tion under Grant Y0301901290100201, and in part by the Open Project
of Key Laboratory of Infrared Imaging Materials and Detectors, Chinese
Academy of Sciences. The review of this letter was arranged by Editor
B. Govoreanu. (Corresponding author: Jiang Wu.)
Wen Du, Caihong Li, Yixuan Huang, Lingzhi Luo, and Jihua Zou are
with the Institute of Fundamental and Frontier Sciences, University of
Electronic Science and Technology of China, Chengdu 610054, China.
Ziyang Long is with the Glasgow College, University of Electronic
Science and Technology of China, Chengdu 610054, China.
Jiang Wu is with the Institute of Fundamental and Frontier Sciences,
University of Electronic Science and Technology of China, Chengdu
610054, China, also with the State Key Laboratory of Electronic Thin
Films and Integrated Devices, University of Electronic Science and
Technology of China, Chengdu 610065, China, and also with the Key
Laboratory of Infrared Imaging Materials and Detectors, Shanghai Insti-
tute of Technical Physics, Chinese Academy of Sciences, Shanghai
200083, China (e-mail: jiangwu@uestc.edu.cn).
Color versions of one or more figures in this letter are available at
https://doi.org/10.1109/LED.2021.3118053.
Digital Object Identifier 10.1109/LED.2021.3118053
0741-3106 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Vardhaman College of Engineering. Downloaded on December 09,2021 at 08:51:39 UTC from IEEE Xplore. Restrictions apply.
high computing efficiency and low power consumption [1].
However, the physical separation of memory and process-
ing units, which is imposed by conventional von Neumann
computing architecture, results in the communication bottle-
neck between the logic and memory [2]. Human-inspired
neuromorphic computing can perform complex tasks due
to simulating brain at the physical level [3]. This kind of
in-memory computing breaks the limit of the well-known
“memory wall” or “von Neumann bottleneck” [4]. A variety
of in-memory computing devices have been proposed [5]–[7],
and the perception of sight, olfaction, taste and touch has been
achieved [8]–[10]. Among them, visual perception is particu-
larly important [11], but the processing of non-structural visual
data requires a vast data storage and computation to obtain
meaningful information [12]–[14]. Therefore, multifunctional
devices which integrate sensing, memory and processing func-
tions are desired. It is also highly desired for neuromorphic
computing to perform logical operations with optical signals
because of the low crosstalk and high bandwidth of light [15].
In human retina, ganglion cells carry out specific calcula-
tions on the visual scene to extract visual features of interest,
and then transmit these processed sets to the brain [16].
Inspired by this, an integrated sensing-memory-processing
system based on the image memory logical unit (IMLU) is
presented in this work. Logical operations (AND, OR, and
XOR) on two sequenced optical singles can be performed by
changing the reference voltage of a comparator module. The
output state of the IMLU is stored in the photodetector in
the form of conductance. Furthermore, the image processing
functions of the IMLU array are conceptually demonstrated
including dilation, erosion and difference detection.
II. D EVICE FABRICATION
A. Preparation of the Material
The as-grown Mo1−x Wx S2 (0≤x≤1) alloy was prepared
using CVD method at atmosphere pressure. The Si/SiO2
substrates were cleaned using acetone and isopropanol, respec-
tively. First, a substrate was placed in a quartz boat to hold
the precursor powder. The WO3 powder and NaCl powder
with the mixture ratio of 10:1 was placed on this precursor
substrate, and the MoO3 powder (half weight of WO3 ) and
a small amount of NaOH powder was placed on the other
side. Second, a growth substrate was put over the previous
substrate. Third, the sulfur powder (0.5 g) was placed in
another quartz boat in the upstream of the tube. The tube center
was ramped up to 750 ◦ C in 15 min and kept for 5 min with
DU et al.: IMLU INSPIRED BY HUMAN RETINA
Fig. 1. Schematic diagram of artificial optical synapse and material
measurements. (a) Schematic diagram of human visual perception.
(b) Schematic diagram of device. (c) Schematic illustration of the photo-
carriers trapping and de-trapping process in Mox W1−x S2 alloy.
the Ar of 45 sccm. The sulfur powder was heated to 220 ◦ C
when the tube center was 670 ◦ C. After that, the tube center
reached 875 ◦ C in 5 min and maintained for 5 min with Ar/H2
of 40/5 sccm. Finally, the tube was removed from the furnace
and was cooled down to room temperature naturally.
B. Preparation and Measurement of the Photodetector
The electrodes (Ti/Au) were prepared by photolithography
and electron beam evaporation. The optoelectronic charac-
teristics were measured in a probe station connected to a
semiconductor parameter analyzer (KEITHLEY 4200). The
405 nm laser is used as the light source.
III. R ESULTS AND D ISCUSSION
A. Device Structure and Optoelectronic Characteristics
The retina of human visual system can store and preprocess
visual information while perceiving light (Fig. 1(a)). The
bionic image sensor inspired by this can reduce the redundant
information and improve the computational efficiency.
Transition metal dichalcogenides (TMDs) are effective
candidates in forming the charge-trapping layer of bionic
photoelectric devices [17]–[20]. In addition, defect engineering
has been utilized to create the synaptic behavior [14], [21].
Since alloying is a common method to introduce defects in
TMDs, a large area alloyed Mox W1−x S2 film is prepared as
the photosensitive layer of a photodetector. The structure of
the device is depicted in Fig. 1(b). As shown in Fig. 1(c), the
defects of sulfur vacancies trap some photocarriers, and the
photocarriers are de-trapped gradually after the illumination is
stopped. The slowly de-trapping process enables the persistent
photoconductivity effect of the Mox W1−x S2 alloy.
Fig. 2 shows the optoelectronic characteristics of the pho-
todetector based on Mox W1−x S2 . The curves of the current
over time after illumination with different light intensity are
shown in Fig. 2(a). The current of the device, after being
Authorized licensed use limited to: Vardhaman College of Engineering. Downloaded on December 09,2021 at 08:51:39 UTC from IEEE Xplore. Restrictions apply.
1811
Fig. 2. Optical characteristics of the photodetector. (a) Currents of the
device triggered by an optical pulse with different light intensity. The unit
of the light intensity is mW/cm2 . The pulse duration is 0.2 s. (b) The
dependence of light intensity on the current under different illumination
time. (c) Paired-pulse facilitation (PPF) of the device triggered by two
optical pulses. The unit of the light intensity is mW/cm2 . The pulse
duration is 0.2 s and the interval is 2 s. (d) PPF of the device triggered
by two optical pulses with different interval time.
triggered by a short optical pulse of 0.2 s, decays to initial
state slowly. This is due to the abundant defects (mainly sulfur
vacancies) in the Mox W1−x S2 that prevent the recombination
of photocarriers. A higher current is induced by an increased
light intensity due to the generation of more photocarriers.
The current recover process is divided into rapid process and
slow process. The rapid process occurs because of the rapid
de-trapping process of the trapped carriers by the defects and
the fast relaxation of the untrapped carriers, whereas the slow
process is due to the slow de-trapping process [14], [15], [22].
The fast process is attribute to the de-trapping of carriers near
the interface and in shallow level defects, and the slow process
originate from the de-trapping of carriers far away from
the interface and in deep level defects. Similarly, the spike
response currents are further increased with the raised opti-
cal pulse duration (Fig. 2(b)). In addition, the paired-pulse
facilitation (PPF) of the device is examined by applying two
sequential optical pulses. As shown in Fig. 2(c), the effect of
light intensity on the peak current is similar to that triggered by
single optical spike. When the interval time is smaller than the
photocarriers recombination time, the current after the second
illumination will be higher than the first. As the interval time
increases, the number of recombined photocarriers increases
correspondingly, so that the second photocurrent decreases
(Fig. 2(d)).
B. Sensing-Memory-Processing Procedure of IMLU
The devices with persistent photoconductivity have many
bionic applications because they can sense and store
information in-situ [23], [24]. Here, the proposed IMLU
performs in-situ sensing, memory and preprocess of two
sequenced optical signals. The light-off state is defined as the
1812
Fig. 3. Sensing-memory-logical operations processes of the four pixels.
(a) Current curves of the four pixels under four different light inputs. The
duration is 0.35s and the interval is 2 s. (b) Forgetting curves of the four
pixels. The dash lines are the reference currents. (c).
“0” state whereas the light-on state is defined as the “1” state.
For the four pixels (P1, P2, P3, P4), the input signal states are
(0, 0), (0, 1), (1, 0) and (1, 1), respectively.
Fig. 3 depicts the sensing and memory characteristics of the
alloy-based photodetector. Fig. 3(a) shows the current changes
of the four pixels under two different input states. The light
duration of 0.35 s can be interpreted as the time of information
reading. The forgetting (relaxation) curves of the four pixels
after the second information reading are shown in Fig. 3(b).
The current can be divided into three ranges based on two
reference current curves. The reference current 1 (2) is the
average of the current of P1 (P2) and P3 (P4). It is worth
noting that the indistinguishable P2 and P3 can be resolved
by adding input sequences [25]. However, this situation does
not affect the implementation of subsequent functions. In order
to visually demonstrate the sensing and memory processes of
the four pixels, the evolution of the current map is illustrated
in Fig. 3(c). The four pixels in the initial state read the matrix
of (0, 0, 1, 1), which is the sensing process, and store it. Then,
the second matrix of (0, 1, 0, 1) is inputted after an interval
of 1.8 s.
The currents of the four pixels will be different because
of the influence of the first input, and the logical operations
of the two input states can be accomplished through a sim-
ple circuit design of each pixel (Fig. 4(a)). The currents of
the photodetector are converted to voltages for comparison
with reference voltages. A LED is used to show the digital
output state visually. If the output state is “1”, the LED
is on, otherwise the LED is off. For the “AND” / “OR”
operations, when Vin >Vref , the output state is “1”; otherwise,
Authorized licensed use limited to: Vardhaman College of Engineering. Downloaded on December 09,2021 at 08:51:39 UTC from IEEE Xplore. Restrictions apply.
IEEE ELECTRON DEVICE LETTERS, VOL. 42, NO. 12, DECEMBER 2021
Fig. 4. Simulation results of the circuits and logical operations of the
IMLU. (a) Schematic diagram of single pixel and (left) the block circuit
diagram of the comparator module (right) for “AND”, “OR” and “XOR”
operations. (b)-(c) The input voltages (dash lines) and the output voltages
(pink solid lines) after “AND” / ”OR” operation (b), and “XOR” operation
(c) to the input images in Figure 3. The corresponding inset images are
the output states.
Fig. 5. Logical operations of images of actual scene. The white pixels
represent the “1” state, and the black pixels represent the “0” state.
the output state is “0”. The reference voltage for the “AND”
operation and “OR” operation is converted from Ref-1 and
Ref-2 in Fig. 3(b), respectively. For the “XOR” operation,
when Vref−2 <Vin <Vref−1 , the output state is “1”; otherwise,
the output state is “0”. Fig. 4(b)-(c) describe the corresponding
converted voltages and output voltages of four pixels, and the
high output voltage means the output state is “1”.
In order to demonstrate the application potential of the
IMLU, the functions of large array based on IMLU are simu-
lated to deal with actual scene images (Fig. 5). The simulations
are implemented using Python 3.7 and Open Source Com-
puter Vision Library. As shown in Fig. 5, the inputted binary
images have small difference that human vision can hardly
distinguish. The “AND” (“OR”) operation increases (reduces)
the different pixel information, whereas the “XOR” operation
highlights the changing information, such as moving vehicles
and pedestrians.
IV. C ONCLUSION
In summary, an IMLU based on large area Mox W1−x S2
alloys is proposed. The IMLU, which performs functions of
sensing, memory and logical operations, provides great poten-
tial in the intelligent image sensors by outputting meaningful
information to improve the processing efficiency of subsequent
tasks in machine vision.
DU et al.: IMLU INSPIRED BY HUMAN RETINA
R EFERENCES
[1] X. Feng, S. Li, S. L. Wong, S. Tong, L. Chen, P. Zhang, L. Wang,
X. Fong, D. Chi, and K.-W. Ang, “Self-selective multi-terminal mem-
transistor crossbar array for in-memory computing,” ACS Nano, vol. 15,
no. 1, pp. 1764–1774, Jan. 2021, doi: 10.1021/acsnano.0c09441.
[2] X. Xiong, J. Kang, Q. Hu, C. Gu, T. Gao, X. Li, and Y. Wu,
“Reconfigurable logic-in-memory and multilingual artificial synapses
based on 2D heterostructures,” Adv. Funct. Mater., vol. 30, no. 11,
Mar. 2020, Art. no. 1909645, doi: 10.1002/adfm.201909645.
[3] L. Hu, J. Yang, J. Wang, P. Cheng, L. O. Chua, and F. Zhuge, “All-
optically controlled memristor for optoelectronic neuromorphic comput-
ing,” Adv. Funct. Mater., vol. 31, no. 4, Nov. 2020, Art. no. 2005582,
doi: 10.1002/adfm.202005582.
[4] Z. Sun, J. Li, C. Liu, S. Yang, and F. Yan, “Trap-assisted charge
storage in titania nanocrystals toward optoelectronic nonvolatile mem-
ory,” Nano Lett., vol. 21, no. 1, pp. 723–730, Jan. 2021, doi:
10.1021/acs.nanolett.0c04370.
[5] M. Prezioso, F. Merrikh-Bayat, B. D. Hoskins, G. C. Adam,
K. K. Likharev, and D. B. Strukov, “Training and operation of an inte-
grated neuromorphic network based on metal-oxide memristors,” Nature,
vol. 521, no. 7550, pp. 61–64, May 2015, doi: 10.1038/nature14441.
[6] M.-K. Kim and J.-S. Lee, “Ferroelectric analog synaptic transis-
tors,” Nano Lett., vol. 19, no. 3, pp. 2044–2050, Mar. 2019, doi:
10.1021/acs.nanolett.9b00180.
[7] S. R. Nandakumar, M. Le Gallo, C. Piveteau, V. Joshi,
G. Mariani, I. Boybat, G. Karunaratne, R. Khaddam-Aljameh,
U. Egger, A. Petropoulos, T. Antonakopoulos, B. Rajendran,
A. Sebastian, and E. Eleftheriou, “Mixed-precision deep learning
based on computational memory,” Frontiers Neurosci., vol. 14, p. 406,
May 2020, doi: 10.3389/fnins.2020.00406.
[8] S. Zhang, K. Guo, L. Sun, Y. Ni, L. Liu, W. Xu, L. Yang, and
W. Xu, “Selective release of different neurotransmitters emulated by
a p–i–n junction synaptic transistor for environment-responsive action
control,” Adv. Mater., vol. 33, no. 10, Mar. 2021, Art. no. 2007350, doi:
10.1002/adma.202007350.
[9] J. Yu, G. Gao, J. Huang, X. Yang, J. Han, H. Zhang, Y. Chen, C. Zhao,
Q. Sun, and Z. L. Wang, “Contact-electrification-activated artificial
afferents at femtojoule energy,” Nature Commun., vol. 12, no. 1, p. 1581,
Mar. 2021, doi: 10.1038/s41467-021-21890-1.
[10] T. Wang, H. Huang, X. Wang, and X. Guo, “An artificial olfactory
inference system based on memristive devices,” InfoMat, vol. 3, no. 7,
pp. 804–813, Jul. 2021, doi: 10.1002/inf2.12196.
[11] H. Li, X. Jiang, W. Ye, H. Zhang, L. Zhou, F. Zhang, D. She, Y. Zhou,
and S.-T. Han, “Fully photon modulated heterostructure for neuromor-
phic computing,” Nano Energy, vol. 65, Nov. 2019, Art. no. 104000,
doi: 10.1016/j.nanoen.2019.104000.
[12] S. Wang, C.-Y. Wang, P. Wang, C. Wang, Z.-A. Li, C. Pan,
Y. Dai, A. Gao, C. Liu, J. Liu, H. Yang, X. Liu, B. Cheng, K. Chen,
Z. Wang, K. Watanabe, T. Taniguchi, S.-J. Liang, and F. Miao,
“Networking retinomorphic sensor with memristive crossbar for brain-
inspired visual perception,” Nat. Sci. Rev., vol. 8, no. 2, Feb. 2021,
Art. no. nwaa172, doi: 10.1093/nsr/nwaa172.
[13] F. Zhou, Z. Zhou, J. Chen, T. H. Choy, J. Wang, N. Zhang, Z. Lin,
S. Yu, J. Kang, H.-S.-P. Wong, and Y. Chai, “Optoelectronic resistive
random access memory for neuromorphic vision sensors,” Nature Nan-
otechnol., vol. 14, no. 8, pp. 776–782, Aug. 2019, doi: 10.1038/s41565-
019-0501-3.
Authorized licensed use limited to: Vardhaman College of Engineering. Downloaded on December 09,2021 at 08:51:39 UTC from IEEE Xplore. Restrictions apply.
1813
[14] T. Ahmed, M. Tahir, M. X. Low, Y. Ren, S. A. Tawfik, E. L. H. Mayes,
S. Kuriakose, S. Nawaz, M. J. S. Spencer, H. Chen, M. Bhaskaran,
S. Sriram, and S. Walia, “Fully light-controlled memory and neu-
romorphic computation in layered black phosphorus,” Adv. Mater.,
vol. 33, no. 10, Nov. 2020, Art. no. 2004207, doi: 10.1002/
adma.202004207.
[15] Y.-X. Hou, Y. Li, Z.-C. Zhang, J.-Q. Li, D.-H. Qi, X.-D. Chen,
J.-J. Wang, B.-W. Yao, M.-X. Yu, T.-B. Lu, and J. Zhang, “Large-
scale and flexible optical synapses for neuromorphic computing
and integrated visible information sensing memory processing,” ACS
Nano, vol. 15, no. 1, pp. 1497–1508, Jan. 2021, doi: 10.1021/
acsnano.0c08921.
[16] T. Gollisch and M. Meister, “Eye smarter than scientists
believed: Neural computations in circuits of the retina,” Neuron,
vol. 65, no. 2, pp. 150–164, Jan. 2010, doi: 10.1016/j.neuron.
2009.12.009.
[17] X. Pan, T. Jin, J. Gao, C. Han, Y. Shi, and W. Chen, “Stimuli-
enabled artificial synapses for neuromorphic perception: Progress and
perspectives,” Small, vol. 16, no. 34, Aug. 2020, Art. no. 2001504, doi:
10.1002/smll.202001504.
[18] T. Wang, J. Meng, Z. He, L. Chen, H. Zhu, Q. Sun, S. Ding,
P. Zhou, and D. W. Zhang, “Ultralow power wearable heterosynapse with
photoelectric synergistic modulation,” Adv. Sci., vol. 7, no. 8, Apr. 2020,
Art. no. 1903480, doi: 10.1002/advs.201903480.
[19] Y. Hu, M. Dai, W. Feng, X. Zhang, S. Zhang, B. Tan, H. Shang,
Y. Q. Fu, and P. Hu, “Monolayer hydrophilic MoS2 with strong
charge trapping for atomically thin neuromorphic vision systems,” Mater.
Horizons, vol. 7, no. 12, pp. 3316–3324, Dec. 2020, doi: 10.1039/
D0MH01472A.
[20] B. Yao, J. Li, X. Chen, M. Yu, Z. Zhang, Y. Li, T. Lu, and J. Zhang,
“Non-volatile electrolyte-gated transistors based on graphdiyne/MoS2
with robust stability for low-power neuromorphic computing and
logic-in-memory,” Adv. Funct. Mater., vol. 31, no. 25, Apr. 2021,
Art. no. 2100069, doi: 10.1002/adfm.202100069.
[21] T. Ahmed, S. Kuriakose, S. Abbas, M. J. S. Spencer, M. A. Rahman,
M. Tahir, Y. Lu, P. Sonar, V. Bansal, M. Bhaskaran, S. Sriram, and
S. Walia, “Multifunctional optoelectronics via harnessing defects in lay-
ered black phosphorus,” Adv. Funct. Mater., vol. 29, no. 39, Sep. 2019,
Art. no. 1901991, doi: 10.1002/adfm.201901991.
[22] S. Seo, S.-H. Jo, S. Kim, J. Shim, S. Oh, J.-H. Kim, K. Heo, J.-W. Choi,
C. Choi, S. Oh, D. Kuzum, H.-S.-P. Wong, and J.-H. Park, “Artificial
optic-neural synapse for colored and color-mixed pattern recognition,”
Nature Commun., vol. 9, no. 1, p. 5106, Nov. 2018, doi: 10.1038/s41467-
018-07572-5.
[23] H. Jang, C. Liu, H. Hinton, M. Lee, H. Kim, M. Seol, H. Shin,
S. Park, and D. Ham, “An atomically thin optoelectronic machine vision
processor,” Adv. Mater., vol. 32, no. 36, Jul. 2020, Art. no. 2002431, doi:
10.1002/adma.202002431.
[24] Z.-C. Zhang, Y. Li, J.-J. Wang, D.-H. Qi, B.-W. Yao,
M.-X. Yu, X.-D. Chen, and T.-B. Lu, “Synthesis of wafer-scale
graphdiyne/graphene heterostructure for scalable neuromorphic
computing and artificial visual systems,” Nano Res., early access,
pp. 1–10, Mar. 2021, doi: 10.1007/s12274-021-3381-4.
[25] L. Sun, Z. Wang, J. Jiang, Y. Kim, B. Joo, S. Zheng, S. Lee,
W. J. Yu, B.-S. Kong, and H. Yang, “In-sensor reservoir comput-
ing for language learning via two-dimensional memristors,” Sci. Adv.,
vol. 7, no. 20, May 2021, Art. no. eabg1455, doi: 10.1126/sciadv.
abg1455.