ISSCC 2021 / SESSION 7 / IMAGERS AND RANGE SENSORS / 7.
7.5 A 250fps 124dB Dynamic-Range SPAD Image Sensor Stacked
with Pixel-Parallel Photon Counter Employing Sub-Frame
Extrapolating Architecture for Motion Artifact Suppression
Jun Ogi1, Takafumi Takatsuka1, Kazuki Hizu1, Yutaka Inaoka1, Hongbo Zhu1,
Yasuhisa Tochigi1, Yoshiaki Tashiro1, Fumiaki Sano1, Yusuke Murakawa2,
Makoto Nakamura2, Yusuke Oike1
1
Sony Semiconductor Solutions, Kanagawa, Japan
2021 IEEE International Solid- State Circuits Conference (ISSCC) | 978-1-7281-9549-0/20/$31.00 ©2021 IEEE | DOI: 10.1109/ISSCC42613.2021.9365977
2
Sony Semiconductor Manufacturing, Nagasaki, Japan
Photon-count imaging has been proposed as a promising technology to realize image
capture with noiseless readout and high dynamic range (HDR) [1-7]. In addition, for
industrial and scientific applications, a global shutter exposure with motion artifact
suppression is essential. A single-photon avalanche diode (SPAD) image sensor is well
matched to the photon-counting architecture by shrinking the SPAD pixel size and
stacking a logic chip with pixel-parallel Cu–Cu connections. A pixel-parallel photon
counter, however, requires many counter bits in a pixel for HDR operation. This makes
it difficult to shrink the pixel size and lower the power consumption, owing to the
substantial number of SPAD activations under high light conditions [2]. Inter-frame
mode switching between digital photon count and analog accumulation avoids the power
increase under high light conditions, but it suffers from a dip in the signal-to-noise ratio
(SNR) and/or motion artifact in reproducing an HDR image [3,4]. An approach reducing
SPAD activations under high light conditions can reduce the power consumption [5-7],
but the combination of long- and short-exposure frames for HDR [5,6] still suffers from
the dip in SNR like conventional multi-exposure image sensors [8], even if these
techniques can suppress motion artifacts owing to sub-frame readout.
This paper presents a SPAD-based photon-counting image sensor implemented with a
sub-frame and extrapolating architecture to demonstrate 250fps global-shutter image
capture with 124 dB dynamic range (DR), motion artifact suppression, and no SNR dip.
Figure 7.5.1 shows a simplified block diagram of the prototype chip. It consists of a
SPAD pixel array on a top-tier chip and readout circuit blocks on a bottom-tier chip in
90nm/40nm CMOS technology. The SPAD pixel pitch is 6.12μm, but 2×2 SPADs are
placed on a 12.24μm pitch pixel readout circuit that is constrained by the readout circuit
area with some test equipment, where 3 of 4 SPADs are unconnected to evaluate the
characteristics of the 6.12μm SPAD. On the bottom tier, a 160×264-pixel circuit array
and peripherals are implemented. The pixel circuit consists of a quenching circuit, a
triggering buffer, and a counter (CN) for pixel-parallel photon counting. The counting
results are digitally read out to the sense amplifiers using the vertical scanner. The pixel
outputs are stored in SRAM through a digital adder (ADD) and a decoder generating an
extrapolated code (EC) for the sub-frame and an extrapolating photon count.
Figure 7.5.2 shows a schematic diagram of a pixel unit with SPAD and readout circuits.
The SPAD is connected to VANODE and a quenching circuit on the bottom tier through a
Cu–Cu connection. The quenching circuit controls SPAD activation by an enable signal
(EN) and an internal overflow signal (OF). A buffer circuit outputs a digital pulse DOUT
according to the quenching at VSPAD. A 9b digital ripple counter (CN) counts the incident
photons on the SPAD. A carry from the most significant bit of CN is held in an overflow
latch as an OF signal, which returns to the quenching circuit to control both SPAD
activation and the timing code (TC) latches for HDR. The 14b TC is distributed to all
pixels and latched only when the OF flag turns. The 14 latches making up the upper 7b
of CN are reused to store the TC after the overflow of CN. The overwrite on CN
contributes to the removal of additional latches to shrink a pixel circuit. The 9b count of
photons or the latched 14b TC are selected at a multiplexer (MUX) by an OF signal, and
the selected data and the OF signal are readout by a row select signal (SEL) with up to
15 bit-lines.
Figure 7.5.3 shows a timing chart of photon counting. The exposure period for each
frame is split into 20 sub-frames. Each sub-frame is composed of a global shutter
exposure and a digital readout scan of 264 rows. The sub-frame operation can be
performed within 200μs. The pixel array is scanned 20 times for each frame, and the
digital outputs are summed at the SRAM of the digital unit. In each sub-frame, photon
counting starts after resetting the counter and OF latch by RSTCN and RSTOF, respectively.
The TC is counted until the end of exposure. After the exposure, the readout scan can
be performed in 300ns per row. At first, each pixel begins true photon counting
regardless of light conditions. DOUT is triggered by an incident photon and an avalanche
current at the SPAD, and then the in-pixel counter counts it one by one. In the case of
no overflow of counter in a sub-frame, i.e. low light conditions, the pixel offers the result
of photon count as is. Meanwhile, each pixel autonomously switches its operation when
the 9b counter overflows due to high light conditions, and then the pixel deactivates its
114 • 2021Authorized
IEEE International Solid-State
licensed use limited Circuits
to: Andrew Conference
Ling. Downloaded
on December 05,2022 at 17:11:36 UTC from IEEE Xplore. Restrictions apply. ©2021 IEEE
SPAD and records a 14b TC at the time of the overflow. The light intensity can be
extrapolated from the TC using the equation (214 – 1) × 29 / TC. This technique avoids
consuming a substantial number of SPAD activations in spite of a lot of incident photons
under high light conditions, and reduces the pixel circuit area due to a small number of
counter bits. Because of the 20× sub-frame summation, each pixel can operate true
photon counting for up to 10,220 photons, i.e. 9b × 20 sub-frames. Beyond the
threshold, some sub-frames offer extrapolated counts instead of true photon counts,
where both true photon counts and extrapolated counts can be summed up. Therefore,
this technique offers an image of contrast-changing and fast-moving objects with motion
artifact suppression. The SNR of the extrapolated counts is a maximum of approximately
40dB, determined by 10,220 photons.
Figure 7.5.4 shows the measurement results of the DR and SNR of the prototype chip.
From a count of around 10,220, the output code is switched to extrapolated counts
decoded from the latched TC. The maximum output code reaches 1 Mcounts, which
corresponds to a 124dB DR on the dark count of 0.59 at 1/60 s. At higher frame rate up
to 250fps, DR is expanding because the dark count is reduced by shorter exposure, but
we have confirmed that it stops increasing up to 127dB due to the maximum count rate
determined by the SPAD deadtime. We have also demonstrated HDR characteristics
without any SNR dip at approximately 40dB. This sensor guarantees stable HDR image
quality regardless of the brightness conditions.
Figure 7.5.5 shows HDR images captured by the prototype chip at 250fps. A tone
reproduction for HDR is applied to the captured image to see both dark and bright areas.
This scene contains both a dark area in the lower left and an extremely bright area of
LEDs in the lower right. The true photon counting operation increases the gain of the
dark area with low noise, and the extrapolating architecture helps prevent LEDs from
saturation. Owing to the global shutter operation, we cannot see any motion artifact of
a rotating fan. The captured images of rotating color stripe show a comparison between
a conventional multiple exposure technique and our implemented sub-frame
extrapolating architecture. For the contrast-changing and fast-moving object, the
conventional technique suffers a severe motion artifact, but our sensor can efficiently
suppress it.
Figure 7.5.6 shows the performance summary of the protype chip, and the chip
micrograph is shown in Fig. 7.5.7. This work achieves both 250fps global shutter and
>120 dB DR by pixel-parallel photon counting without any inter-frame mode change,
which also achieves motion artifact suppression and no SNR dip over the 124dB DR.
Compared to the full photon counting consuming a large amount of energy in proportion
to photons under high light conditions [2], the prototype with the extrapolating
architecture can drastically reduce the power by a factor of 100 at 106 incident photons.
References:
[1] J. Ma et al., “Jot Devices and the Quanta Image Sensor,” Proc. IEDM, pp. 247-250,
Dec. 2014.
[2] R. K. Henderson et al., “A 256×256 40nm/90nm CMOS 3D-stacked 120dB Dynamic-
Range Reconfigurable Time-Resolved SPAD Imager,” ISSCC, pp. 106-107, Feb. 2019.
[3] M. Mori et al., “A 1280×720 Single-Photon-Detecting Image Sensor with 100dB
Dynamic Range Using a Sensitivity-Boosting Technique,” ISSCC, pp. 120-121, Feb.
2016.
[4] M. Johnston et al., “Combining Linear and SPAD-Mode Diode Operation In-Pixel for
Wide Dynamic Range CMOS Optical Sensing,” Int. SPAD Sensor Workshop, June 2020.
[5] K. Morimoto et al., “Megapixel Time-Gated SPAD Image Sensor for 2D and 3D
Imaging Applications,” OSA Optica, vol. 7, no. 4, pp. 346-354, Apr. 2020.
[6] N. A. W. Dutton et al., “High Dynamic Range Imaging at the Quantum Limit with
Single Photon Avalanche Diode-Based Image Sensors,” MDPI Sensors, vol. 18, p. 1166,
Apr. 2018.
[7] R. Kazma et al., “High Dynamic Range Readout Architecture for SPAD Array,” DCIS,
pp. 1-4, Nov. 2015.
[8] Y. Sakano et al., “A 132dB Single-Exposure-Dynamic-Range CMOS Image Sensor
with High Temperature Tolerance,” ISSCC, pp. 106-107, Feb. 2020.
978-1-7281-9549-0/21/$31.00
 ISSCC 2021 / February 16, 2021 / 9:02 AM

Figure 7.5.1: Block diagram of the image sensor. Figure 7.5.2: Circuit diagram of a pixel unit.

Figure 7.5.3: Timing diagram of the photon counting implementation. Figure 7.5.4: Measurement results of dynamic range and SNR.

Figure 7.5.5: Captured images from the prototype chip. Figure 7.5.6: Performance summary and comparison table.

DIGEST
Authorized licensed use limited to: Andrew Ling. Downloaded on December 05,2022 at 17:11:36 UTC OF TECHNICAL
from IEEE PAPERS
Xplore. Restrictions apply. • 115
 ISSCC 2021 PAPER CONTINUATIONS

Figure 7.5.7: Chip micrograph.

• 2021 IEEE International Solid-State Circuits Conference 978-1-7281-9549-0/21/$31.00 ©2021 IEEE

Authorized licensed use limited to: Andrew Ling. Downloaded on December 05,2022 at 17:11:36 UTC from IEEE Xplore. Restrictions apply.