applied

sciences
Article
Deep Learning-Based Image Classification through a
Multimode Fiber in the Presence of Wavelength Drift
Eirini Kakkava 1, * , Navid Borhani 1 , Babak Rahmani 2 , Uğur Teğin 1,2 , Christophe Moser 2
and Demetri Psaltis 1
1 Optics Laboratory, École Polytechnique Fédérale de Lausanne, 1015 Lausanne, Switzerland;
navid.borhani@epfl.ch (N.B.); ugur.tegin@epfl.ch (U.T.); demetri.psaltis@epfl.ch (D.P.)
2 Laboratory of Applied Photonic Devices, École Polytechnique Fédérale de Lausanne, 1015 Lausanne,
Switzerland; babak.rahmani@epfl.ch (B.R.); christophe.moser@epfl.ch (C.M.)
* Correspondence: eirini.kakkava@epfl.ch




Received: 29 April 2020; Accepted: 28 May 2020; Published: 30 May 2020


Abstract: Deep neural networks (DNNs) are employed to recover information after its propagation
through a multimode fiber (MMF) in the presence of wavelength drift. The intensity distribution of
the speckle patterns generated at the output of an MMF when an input wavefront propagates along its
length is highly sensitive to wavelength changes. We use a tunable laser to implement a wavelength
drift with a controlled bandwidth, aiming to estimate the DNN’s performance in different cases and
identify the limitations. We find that when the DNNs are trained with a dataset which includes the
noise induced by wavelength changes, successful classification of a speckle pattern can be performed
even for a large wavelength bandwidth drift. A single training step is found to be sufficient for high
classification accuracy, removing the need for time-consuming recalibration at each wavelength.

Keywords: multimode fibers; deep learning; image classification; fiber-based optical computing;
wavelength drift

1. Introduction
Information transmission through multimode fibers (MMFs) has been a relatively dormant field
of research and development for many decades. More recently, MMFs are becoming widely used
in telecommunications because of their high information capacity at a lower manufacturing cost [1].
The number of available channels in MMFs is limited by the number of supported spatial modes.
The spatial modes are the solutions to the wave equation for a MMF and thus propagate unchanged
along the fiber, offering unique mode multiplexing opportunities [2–5]. MMFs have also attracted
significant attention in endoscopy, for which a high information capacity is essential to achieve high
quality images [6–11]. However, the differences in propagation constants between the spatial fiber
modes can hinder the information transmission through a MMF system. When an input image is
coupled in a MMF, it distributes among the supported spatial modes, resulting in a speckle pattern at
the fiber output.
Several methods have been employed for reversing the scrambling caused by MMFs.
Phase conjugation or measurement of the transmission matrix [7,8,11,12] allow to reverse the effects of
propagation and unscramble the information. While phase conjugation and transmission matrix can
control the light propagation through an MMF probe, they are based on interferometric measurements
of the light field using digital holography, and as a result they are susceptible to environmental or
experimental perturbations, requiring dynamic recalibration [8,13–15]. To overcome the sensitivity
of the calibration-based measurements, deep neural networks (DNNs) have been proposed as an
alternative for imaging through MMFs [16–20]. The idea of using artificial neural networks (ANNs)

Appl. Sci. 2020, 10, 3816; doi:10.3390/app10113816 www.mdpi.com/journal/applsci

Appl. Sci. 2020, 10, 3816 2 of 10

to interpret the information after propagation in a MMF was first reported by Aisawa et al. [21,22]
in 1991 using a simple neural network architecture. Recently, the advancements in computational
means allow the realization of more complex neural network architectures that can handle data of
high-dimensionality, bringing DNNs to the forefront of many leading technologies ranging from
research and business to military and entertainment [23–25]. DNNs have shown remarkable capabilities
in recovering information through MMFs for imaging applications, while preserving robustness against
perturbations in the system [16–18,20,26].
The noise induced at the output speckle patterns due to perturbations caused by thermal,
mechanical or source wavelength drift can be catastrophic for a calibration-based technique such as
the transmission matrix but they affect, at a lower level, the performance of DNNs [16]. In addition,
in cases that multiple wavelengths are needed for imaging, the transmission matrix of the fiber needs
to be measured multiple times for different wavelengths [27]. Therefore, it would be time-efficient to
investigate a way to remove recalibration of the system when the wavelength is changed. In the work
presented in this article, we propose an alternative approach for recovering the information through
the MMF system in the presence of a wavelength drift in the light source using DNNs. We investigate
the performance of DNNs to classify the input images in a MMF from their corresponding speckle
patterns in various wavelength drift cases. Remarkable results were obtained for wavelength drift
bandwidth up tp 100 nm, and they are further discussed in the following paragraphs.

2. Materials and Methods

The optical setup built to collect the datasets for training the DNNs is shown in Figure 1. The beam of
a wavelength-tunable laser source (M-squared, SolsTiS 2000 PSX XF with Terascan software), emitting in
the range 700–1000 nm is used to illuminate a graded-index (GRIN) MMF with 62.5 µm core diameter
and numerical aperture (NA) of 0.275 (GIF625, Thorlabs GmbH, Germany). The wavelength of the laser
source can be tuned with 10 fm accuracy. GRIN fibers are often used to avoid temporal distortion in a
signal due to modal dispersion. A beam expansion is achieved by a set of lenses (L1: f = 200 mm, OBJ1:
10×, Newport, Irvine, CA, USA) placed at the laser output to ensure that the beam size is large enough to
illuminate optimally
Appl. Sci. 2020, 10, x FOR a REVIEW spatial light modulator (SLM, 1920 × 1080 pixels, Pluto-Vis, Holoeye,
phase-only
PEER 3 of 10
Berlin, Germany). The SLM is used to display the desired patterns and the SLM plane is imaged onto the
proximal
wavelength facettooffar
the spaced
MMF bywavelength
means of a 4f imaging
values. system (lens
Moreover, the L2: f = stabilization
laser 400 mm, microscope objective
at the specified
OBJ2: 60x, Newport,
wavelength Irvine,The
is much faster. CA,inputs
USA).are
Another 4f system
therefore (theon
projected same as the proximal
the proximal one,inL3-OBJ3)
fiber facet batches
of 100, each
magnifies theone at a specific
speckle patternwavelength.
emerging from For the
the distal
next 100 inputs,
facet on a the
CCD random
camerawavelength array is
(CCD, Chameleon
3, 1024 × 1280sopixels,
recalculated that we always
Mono, get Grey).
Point randomlyAn different wavelength
additional = 200 for
lens (L4: f values mm)theinimages of the full
2f configuration
dataset.monitoring
allows We collected a total generated
the images of 10,000 by
intensity
the SLMimages of speckle
on a second camera patterns. The speckle
at the proximal images
side (CCD2).
captured
A halfwave byplate
CCD1 are cropped
is placed beforeto
the1024
SLM× to
1024 pixels
ensure and
that thethen downsampled
correct polarizationto of32 × 32 pixelsisbefore
illumination used
being
(see imported
Figure 1). Thein fiber
the DNN code.
length used in the experiments is 10 cm.

Figure 1. Optical
Figure 1. Optical setup
setup for
for the
the collection
collection of
of the
the image
image dataset
dataset used
used for
for training the DNN
training the DNN algorithms.
algorithms.
(L = lens, OBJ = microscope objective, HWP = half-wave plate, M = mirror, BS = beam splitter).
(L = lens, OBJ = microscope objective, HWP = half-wave plate, M = mirror, BS = beam splitter).

For the classification experiments, a VGG(Visual Geometry Group)-type DNN is used with an
architecture described in Figure 2 [29]. We use mean squared error (MSE) as a loss function and an
Adam optimizer with a learning rate of 1 × 10−4. The DNN is trained in batches of 100 images for a
maximum of 50 epochs. The collected images of the dataset are split so that 80% are used for training,
wavelength is much faster. The inputs are therefore projected on the proximal fiber facet in batches
of 100, each one at a specific wavelength. For the next 100 inputs, the random wavelength array is
recalculated so that we always get randomly different wavelength values for the images of the full
dataset. We collected a total of 10,000 intensity images of speckle patterns. The speckle images
captured
Appl. by 10,
Sci. 2020, CCD1
3816 are cropped to 1024 × 1024 pixels and then downsampled to 32 × 32 pixels before
3 of 10
being imported in the DNN code.

The dataset used to assess the performance of the DNN is generated by projecting phase images of
handwritten digits at the proximal fiber side [16–18,20]. These images are available online in the MNIST
database, which is widely used to test the capabilities of different neural network architectures [28].
Wavelength drifting is simulated by a Matlab code written to control the laser wavelength, the speckle
acquisition by the camera CCD1 (Figure 1) and the input images on the SLM. We first choose the
wavelength range of the drift in the code and then a random array with 100 wavelength values within
this range is generated. The wavelength values are sorted in an ascending order because this leads to a
more stable operation of the tunable laser instead of tuning the wavelength to far spaced wavelength
values. Moreover, the laser stabilization at the specified wavelength is much faster. The inputs are
therefore projected on the proximal fiber facet in batches of 100, each one at a specific wavelength.
For the next 100 inputs, the random wavelength array is recalculated so that we always get randomly
different wavelength values for the images of the full dataset. We collected a total of 10,000 intensity
imagesFigure 1. Optical
of speckle setup forThe
patterns. thespeckle
collection of the image
images dataset
captured used forare
by CCD1 training the to
cropped DNN algorithms.
1024 × 1024 pixels
(L = lens, OBJ = microscope objective, HWP = half-wave plate, M =
and then downsampled to 32 × 32 pixels before being imported in the DNN code.mirror, BS = beam splitter).
For the classification experiments, a VGG(Visual Geometry Group)-type DNN is used with an
For the classification experiments, a VGG(Visual Geometry Group)-type DNN is used with an
architecture described in Figure 2 [29]. We use mean squared error (MSE) as a loss function and an
architecture described in Figure 2 [29]. We use−4mean squared error (MSE) as a loss function and an
Adam optimizer with a learning rate of 1 × 10 −4 . The DNN is trained in batches of 100 images for a
Adam optimizer with a learning rate of 1 × 10 . The DNN is trained in batches of 100 images for a
maximum of 50 epochs. The collected images of the dataset are split so that 80% are used for training,
maximum of 50 epochs. The collected images of the dataset are split so that 80% are used for training,
10% for validation and 10% for testing. For each one of the collected datasets the DNN training is
10% for validation and 10% for testing. For each one of the collected datasets the DNN training is
repeated five times to obtain the standard deviation values. The VGG classifier is implemented in
repeated five times to obtain the standard deviation values. The VGG classifier is implemented in
Tensorflow 1.5 Python library on a computer equipped with an NVIDIA GeForce GTX 1080Ti graphics
Tensorflow 1.5 Python library on a computer equipped with an NVIDIA GeForce GTX 1080Ti
processing unit.
graphics processing unit.

Figure
Figure 2. Schematic representation
2. Schematic representation of of the
the deep
deep neural
neural network
network (DNN)
(DNN) architecture
architecture used
used for
for the
the
classification of the speckle patterns recorded at the fiber output for different phase inputs. The VGG-type
classification of the speckle patterns recorded at the fiber output for different phase inputs. The VGG-
network is trained
type network to receive
is trained the the
to receive image
imageof aofspeckle
a specklepattern intensity
pattern intensityand
andprovide
provideaalabel
label for
for the
the
input
input digit.
digit.

3. Results
3. Results
The number of fiber modes in MMFs determines the available degrees of freedom (channels) for
The number of fiber modes in MMFs determines the available degrees of freedom (channels) for
transmitting the information. An input image is mapped to the fiber modes basis. Therefore, a MMF
transmitting the information. An input image is mapped to the fiber modes basis. Therefore, a MMF
with only few modes would not be able to transmit a complex signal. Figure 3 shows how the image
with only few modes would not be able to transmit a complex signal. Figure 3 shows how the image
of the digit 5 selected from the MNIST database would be reconstructed through MMFs with the
of the digit 5 selected from the MNIST database would be reconstructed through MMFs with the
same NA that support different total number of spatial modes. It is apparent in Figure 3 that, below a
certain number of modes, the features of the images would not be preserved. Therefore, it is important
to evaluate the classification performance of the DNN at different wavelengths in the range of the
tunable laser to assess its dependence on the number of supported modes in each case. In particular,
in order to test the performance of DNNs in the presence of wavelength drift, it is essential that the
classification accuracy is firstly determined for the different wavelengths within the drift bandwidth in
no-drift conditions. In this way, it can be assumed that any change in the classification performance
 same
same NA NA that
that support
support different
different total
total number
number of of spatial
spatial modes.
modes. It It
is is apparent
apparent in in Figure
Figure 3 that,
3 that, below
below
a acertain
certainnumber
numberof ofmodes,modes,thethefeatures
featuresof ofthetheimages
imageswould wouldnot notbebepreserved.
preserved.Therefore,
Therefore,it itis is
important to evaluate the classification performance of the DNN at different wavelengths in in
important to evaluate the classification performance of the DNN at different wavelengths thethe range
range
of the tunable laser to assess its dependence on the number of supported modes in each case. InIn
of the tunable laser to assess its dependence on the number of supported modes in each case.
particular,
Appl. Sci. 2020,
particular, in
in 10, order
3816to to
order test
test thethe performance
performance of of DNNs
DNNs in in
thethe presence
presence of of wavelength
wavelength drift,
drift, it is
it is essential
4 of 10
essential
that the classification accuracy is firstly determined for the different
that the classification accuracy is firstly determined for the different wavelengths within the drift wavelengths within the drift
bandwidth
bandwidth in in no-drift
no-drift conditions.
conditions. InInthisthis way,
way, it can
it can bebe assumed
assumed thatthat
anyany change
change in in
thethe classification
classification
isperformance
attributed to is theattributed
induced “noise”to the ininduced
the dataset caused
“noise” in by
thethedataset
wavelength
caused perturbation
by the and not
wavelength
performance is attributed to the induced “noise” in the dataset caused by the wavelength
toperturbation
the inherentand incapability
not of the system to support the input images because of the low number of
perturbation and not to to
thethe inherent
inherent incapability
incapability of of
thethe system
system to to support
support thethe input
input images
images because
because
supported modes at of
a certain wavelength.
of of
thethe
low low number
number supported
of supported modes
modes at at a certain
a certain wavelength.
wavelength.

Figure
Figure
Figure 3.3.3. Decomposition
Decomposition
Decomposition ofof
of images
images
images in in cases
cases of of different
different
different numbers
numbers
numbers of supported
ofofsupported
supported fiber
fiber
fiber modes.
modes.
modes. The
The The size
size
size of
of of
the
the the
input input
input images
images
images was
was
was approximately
approximately
approximately 100 100
100 ×
× ×100 100 pixels.
100pixels.
pixels.

Phase-encoded
Phase-encoded
Phase-encoded images
images
images ofof
of handwritten
handwritten
handwritten digits
digits
digits fromfrom
from thethe
the MNIST
MNIST
MNIST database
database
database are
are
are projected
projected
projected byby
by the
the
the SLM
SLM SLM
on
on on
the the
the GRIN
GRIN
GRIN fiber
fiber
fiber facet,
facet,
facet, andand
and thethe
the corresponding
corresponding
corresponding intensity
intensity
intensity images
images
images ofofof the
the
the resulted
resulted
resulted speckle
speckle
speckle patterns
patterns
patterns are
are are
collected.
collected. We
collected. We collect
We collect datasets
collectdatasets
datasetsforfor for different
different
different wavelengths
wavelengths
wavelengths covering covering
covering the range the
the range range
of the of of
tunable the tunable
laser ranging
the tunable laser
laser
ranging
from
ranging 700from
tofrom
1000
700700to to
nm 1000
with
1000 a 50
nm nm nmwith
with a 50
increment.
a 50 nm nm increment.
The
increment. The
classification
The classification
accuracy ofaccuracy
classification accuracy
the system of of
is the
the system
determined
system is is
bydetermined
training theby training
VGG DNN the
forVGG
each DNN
case for each
[30,31]. case
The [30,31].
training The
and training
test
determined by training the VGG DNN for each case [30,31]. The training and test set consist of images set and
consist test
of set consist
images of
captured images at
captured at
a single wavelength,
captured a single wavelength,
without any
at a single wavelength, without
drift during
without any drift during
recording,
any drift during for recording, for
this preliminary
recording, this preliminary assessment.
assessment.assessment.
for this preliminary It has been
It
shown has inbeen
our shown
previous in our
work previous
that with work
increasingthat with
fiber increasing
length,
It has been shown in our previous work that with increasing fiber length, slight environmentalslight fiber length,
environmental slight environmental
perturbations can
perturbations can cause severe speckle drifts [16]. For the 10-cm long GRIN fiber used in thethe
perturbations
cause severe can
speckle cause
drifts severe
[16]. Forspeckle
the drifts
10-cm long [16].
GRIN For the
fiber 10-cm
used in long
the GRIN
experiments,fiber used
no in
further
experiments,
instabilities
experiments, nonofurther
because further instabilities
of mechanical
instabilities or because
thermal
because of of mechanical
changes
mechanical are or orthermal
thermalchanges
observed. Inchanges
Figureare areobserved.
4a, observed.
the average InIn
Figure4a,4a,thethe
classification
Figure average
accuracy
average classification
obtained
classification accuracy
at the accuracy
different obtainedat at
wavelengths
obtained is
thethe different
presented
different wavelengths
together
wavelengths with the ispresented
presented
is normalized
together
confusion
together withwith
matricesthe normalized confusion
for the twoconfusion
the normalized ends of the matrices
wavelength
matrices for
for the the
range, two
two namely ends
ends of 700 of the
the nm wavelength
(Figure 4b)
wavelength range,
and 1000
range, namely
namely nm
700700
(Figure
nmnm (Figure
4c).
(Figure No 4b) 4b) and
significant
and 1000
1000change
nm nm in(Figure 4c).
NoNo
the classification
(Figure 4c). significant
accuracy
significant change in in
is observed
change thethe for classification
the datasets
classification accuracy
across
accuracy is is
the
observed for the datasets across the studied range. Hence, we can assume that the total number of of
observed
studied for
range. the datasets
Hence, we across
can assumethe studied
that the range.
total Hence,
number we
of can
spatial assume
modes that the
supported total number
within this
spatial
wavelength
spatial modes
modes supported
scanning
supported range within
within this
is sufficient
this wavelength
to represent
wavelength scanning
the input
scanning range
range is is
information sufficient
sufficient andto to
it represent
does
represent notthe the
affect input
inputthe
information
performance and
of the it does
VGG not affect
classifier. the performance
information and it does not affect the performance of the VGG classifier. of the VGG classifier.

Figure
Figure
Figure (a)
4.4.4.
(a)(a)Classification
Classification
Classification accuracy
accuracy
accuracy asas
as aafunction
afunction
functionofof
of wavelength
wavelength
wavelength and
and
and number
number
number ofofof
fiber
fiber
fiber spatial modes
spatial
spatial modes
modes
for the
for thewavelength
wavelength range
rangeof the
of thetunable
tunable laser source;
laser source; Normalized
Normalizedconfusion matrix
confusion matrix
for the wavelength range of the tunable laser source; Normalized confusion matrix for the datasets for the
for datasets
the datasets
collected
collected at the (b) 700 nm; and (c) 1000 nm wavelength.
collected at at
thethe
(b)(b)
700700 nm;
nm; andand(c)(c) 1000
1000 nmnm wavelength.
wavelength.

As a anext step, we explore the effect of wavelength drift on the DDN performance for different
AsAs next
a next step,
step, wewe explore
explore thethe effect
effect of of wavelength
wavelength drift
drift onon
thethe DDN
DDN performance
performance forfor different
different
bandwidths
bandwidths of 6, 12, 24, 48, 72 and 96 nm for a central wavelength at 800 nm (Figure 5). We investigate
bandwidths of of
6, 6,
12,12,
24,24,
48,48,
7272 and
and 9696
nm nm
forfor a central
a central wavelength
wavelength at at
800800
nmnm (Figure
(Figure 5).5).
We We investigate
investigate
two different cases of drifting noise [30,31]. The same GRIN fiber of 62.5 µm core and length of 10 cm is
used for all the measurements. In the first case, the DNNs are trained with data acquired with the laser
wavelength fixed at 800 nm, and then tested on speckle patterns recorded at different wavelengths
far from the 800 nm one at which the training set is obtained. In the second case, the wavelengths
within the drifting range are included in the training set by randomly acquiring speckle patterns at
Appl. Sci. 2020, 10, x FOR PEER REVIEW 5 of 10

two different cases of drifting noise [30,31]. The same GRIN fiber of 62.5 μm core and length of 10 cm
is used for all the measurements. In the first case, the DNNs are trained with data acquired with the
laser wavelength
Appl. Sci. 2020, 10, 3816 fixed at 800 nm, and then tested on speckle patterns recorded at different 5 of 10
wavelengths far from the 800 nm one at which the training set is obtained. In the second case, the
wavelengths within the drifting range are included in the training set by randomly acquiring speckle
different at
patterns wavelengths during the measurement,
different wavelengths as describedasindescribed
during the measurement, the Methods section.
in the For comparison,
Methods section. For
comparison, the speckle intensity correlation was calculated for a range of 100 nm aroundwavelength
the speckle intensity correlation was calculated for a range of 100 nm around the central the central
(800 nm) to study
wavelength the DNN
(800 nm) results
to study the as
DNNa function
results of
asthe speckle decorrelation.
a function of the speckle As seen from Figure
decorrelation. As seen5a,
the speckle patterns decorrelate at 50% within a range of approximately 30 nm.
from Figure 5a, the speckle patterns decorrelate at 50% within a range of approximately 30 nm.

(FWHM).

Figure 5.5. (a)

Figure (a)Speckle intensity
Speckle correlation
intensity as a function
correlation of wavelength
as a function drift bandwidth;
of wavelength drift (b) Classification
bandwidth; (b)
Classification accuracy for datasets collected for the two cases discussed in the text: DNN at
accuracy for datasets collected for the two cases discussed in the text: DNN trained 800 nm
trained at
wavelength
800 and tested
nm wavelength andat tested
data collected
at data at differentatwavelengths
collected (red circles)(red
different wavelengths and circles)
DNN trained
and DNNwith
data that
trained include
with data the
thatperturbation (blue circles);(blue
include the perturbation (c) Classification accuracy for aaccuracy
circles); (c) Classification DNN trained in the
for a DNN
presence of 12 nm bandwidth drift and tested at data collected outside this range. All
trained in the presence of 12 nm bandwidth drift and tested at data collected outside this range. All the results are
obtained through the same GRIN fiber (10 cm, GIF625, Thorlabs GmbH, Germany).
the results are obtained through the same GRIN fiber (10 cm, GIF625, Thorlabs GmbH, Germany).

Studying the first case, we train the DNN with data recorded at 800 nm and we test its ability
Studying the first case, we train the DNN with data recorded at 800 nm and we test its ability to
to recover the information from data measured at 803, 806, 812 and 850 nm (Figure 5b, red circles).
recover the information from data measured at 803, 806, 812 and 850 nm (Figure 5b, red circles). The
The classification accuracy drops at 69% for the shortest drift of 803 nm, while the speckle correlation
classification accuracy drops at 69% for the shortest drift of 803 nm, while the speckle correlation has
has only dropped to 90% at this wavelength distance. The results become completely random at
only dropped to 90% at this wavelength distance. The results become completely random at 812 nm,
812 nm, reaching 10% classification. Hence, it is shown that generalization of the VGG network for
reaching 10% classification. Hence, it is shown that generalization of the VGG network for
wavelengths further than the one used in the training is only efficient for a very narrow range of
wavelengths further than the one used in the training is only efficient for a very narrow range of
wavelengths with respect to the one used for the training data measurements.
wavelengths with respect to the one used for the training data measurements.
The classification results also show a decrease in performance with an increase in the wavelength
drifting
Thebandwidth,
classification but at a much
results lowera rate
also show in theinsecond
decrease case forwith
performance which
an we include
increase the wavelength
in the wavelength
drift during
drifting the training.
bandwidth, but atFor 6 nm drift,
a much lowermeaning ±3second
rate in the nm aroundcase the
for central
which wewavelength,
include the thewavelength
VGG DNN
shows
drift minimal
during the change
training. ofFor
classification
6 nm drift,performance.
meaning ±3 Specifically,
nm around we the achieve classificationthe
central wavelength, accuracy
VGG
of 88%shows
DNN in thisminimal
case while the one
change obtained in absence
of classification performance. (∆λ = 0) 90%
of driftSpecifically, we(Figure
achieve 4a). This value
classification
is remarkably
accuracy of 88%better
in thiscompared
case while tothetheone
69%obtained
observed in in the experiments
absence of drift (Δλof the90%
= 0) first(Figure
case discussed
4a). This
beforeisforremarkably
value the same wavelength drift ofto
better compared 3 nm.
the Even for approximately
69% observed 100 nm drift,
in the experiments ofthetheVGGfirstDNN
case
classifies correctly
discussed before for 70%theofsame
the inputs (Figuredrift
wavelength 5b, blue circles).
of 3 nm. EvenThese results indicate100
for approximately highnmrobustness
drift, the
of theDNN
VGG DNNs in the presence
classifies correctly of70% severe
of theperturbations
inputs (Figurein5b, theblue
system as long
circles). These asresults
the training
indicate dataset
high
is collectedofinthe
robustness theDNNs
presence of presence
in the the wavelength
of severedisturbance
perturbations under study.
in the Theas
system ability
long of as the training
DNN to
successfully
dataset classifyinthe
is collected thespeckle
presencepatterns
of thewithin a widedisturbance
wavelength range of wavelengths
under study. can The be ability
beneficial for
of the
multicolor
DNN image transmission
to successfully through
classify the specklea fiber system
patterns [19]. aInwide
within Figure 6, weofshow,
range for comparison,
wavelengths can be
the normalized
beneficial confusionimage
for multicolor matrices for the twothrough
transmission approaches for system
a fiber a drift of 3 nm
[19]. In and
Figure±3 nm,6, we respectively.
show, for
It is evident the
comparison, thatnormalized
many labels are mixed
confusion up in the
matrices for first case;
the two for example,
approaches for the digit
a drift of 39,nmwhich
andis±3quite
nm,
similar to at least
respectively. It is two other
evident digits,
that many such as 7 are
labels andmixed
5 s, is highly misclassified.
up in the first case; for example, the digit 9,
which is quite similar to at least two other digits, such as 7 and 5 s, is highly misclassified.
Appl. Sci. 2020, 10, 3816 6 of 10
Appl. Sci. 2020, 10, x FOR PEER REVIEW 6 of 10

Figure 6. Normalized confusion matrices showing the classification results for the case that the drifting
Figure 6. Normalized confusion matrices showing the classification results for the case that the
wavelength is: (a) not part of the training set and; (b) is included in the drifting wavelengths for which
drifting wavelength is: (a) not part of the training set and; (b) is included in the drifting wavelengths
the VGG classifier is trained.
for which the VGG classifier is trained.
Another aspect evaluated is the effect of including the wavelength drift noise in the training set
Another
when we attempt toaspect
recoverevaluated
informationis the effect the
outside of including the wavelength
training bandwidth. driftthe
We train noise
VGG in the training set
classifier
when we attempt to recover information outside the training bandwidth. We
with data measured under 12 nm drifting situation and we afterwards test the classification accuracy train the VGG classifier
with data measured under 12 nm drifting situation and we afterwards
of the trained model for data outside the 12 nm range, starting from 807 nm to 850 nm. In spitetest the classification accuracy
of the of the efficiency
high trained model for data
reported for outside
recoveringthe 12 nm range, about
information starting from
the 807 within
inputs nm to 850thenm.driftInrange
spite of the
high efficiency reported for recovering information about the inputs within the
(Figure 5b, blue circles), this is not the case for speckle patterns recorded at wavelengths outside this drift range (Figure
range 5b, blue 5c).
(Figure circles),
The this is not theresults
classification case forobtained
speckle forpatterns recorded
the speckles at wavelengths
collected outside this range
at laser wavelengths
(Figure 5c). The classification results obtained for the speckles collected at laser
outside the 12 nm range follow the trend observed in the first case of experiments discussed before, as wavelengths outside
shown in Figure 5c. Based on these results, it is found that training DNNs to infer the inputs fromshown
the 12 nm range follow the trend observed in the first case of experiments discussed before, as
speckleinpatterns
Figure 5c. Based
in the on these
presence results, it isdrift
of wavelength found that assist
cannot training DNNs
their to infer the
performance inputs
outside thisfrom
range.speckle
patterns in the presence of wavelength drift cannot assist their performance
In other words, the trained model does not generalize to new wavelength values which lie outside the outside this range. In
other words, the
bandwidth used for training.trained model does not generalize to new wavelength values which lie outside the
bandwidth
Once crucial used for training.
parameter usually suggested for improving the performance of the DNNs is the
number of available samples in the dataset. At the same time, this parameter is considered to be
Once crucial parameter usually suggested for improving the performance of the DNNs is the
one of the drawbacks of deep learning, because there are cases for which the number of available
number of available samples in the dataset. At the same time, this parameter is considered to be one
data is limited. In order to assess whether the dataset used to train the VGG classifier in the case of
of the drawbacks of deep learning, because there are cases for which the number of available data is
wavelength drifting is sufficiently large, we investigate the classification accuracy for training datasets
limited. In order to assess whether the dataset used to train the VGG classifier in the case of
of different sizes. The dataset of 10,000 speckle images is shuffled in a random way and each time
wavelength drifting is sufficiently large, we investigate the classification accuracy for training
only a part of the total dataset is used for training and testing the DNN. The classification results are
datasets of different sizes. The dataset of 10,000 speckle images is shuffled in a random way and
presented in Figure 7. The classification accuracy is observed to vary with the size of the training
each time only a part of the total dataset is used for training and testing the DNN. The classification
dataset. It is apparent from the values reported in Figure 7 that the more severe the wavelength
results are presented in Figure 7. The classification accuracy is observed to vary with the size of the
drift, the more samples are needed for achieving higher classification accuracies. Nevertheless, for all
training dataset. It is apparent from the values reported in Figure 7 that the more severe the
the tested wavelength bandwidths, a saturation of the classification improvement is observed while
wavelength drift, the more samples are needed for achieving higher classification accuracies.
increasing the size of the dataset; for the largest bandwidth (96 nm), this happens at about 6000 samples,
Nevertheless, for all the tested wavelength bandwidths, a saturation of the classification
as suggested by Figure 7. Interestingly, the accuracies measured for each bandwidth saturate at values
improvement is observed while increasing the size of the dataset; for the largest bandwidth (96
of total available samples that are below 10,000, which implies that an increase in the dataset size
nm), this happens at about 6000 samples, as suggested by Figure 7. Interestingly, the accuracies
would not significantly improve the results.
measured for each bandwidth saturate at values of total available samples that are below 10,000,
Appl.
Appl. Sci.
Sci.2020,
2020,10,
10,x3816
FOR PEER REVIEW 7 7ofof10
10

which implies that an increase in the dataset size would not significantly improve the results.

Figure 7. Classification accuracy of the VGG-type DNN classifier used in the experiment for increasing
Figure 7. Classification accuracy of the VGG-type DNN classifier used in the experiment for
the number of available samples in the training dataset.
increasing the number of available samples in the training dataset.
4. Discussion
4. Discussion
In the analysis presented in the previous paragraphs, we show the effects of wavelength drift
In performance
on the the analysis presented
of a VGGin the previous
classifier paragraphs,
of speckle patternswe at show the effects
the output of wavelength
of a 10-cm long GRIN drift
MMF.on
the performance
Although 10 cm is of considered
a VGG classifiershort forof speckle patterns at the
certain applications, output
such of a 10-cm long GRIN
as telecommunications, it isMMF.
often
Although
sufficient 10forcm is considered
medical imagingshort (e.g.,for certain
brain imagingapplications, such as telecommunications,
and diagnosis). It is observed that DNNs it is often
can
sufficient
recognizefor the medical
digits used imaging
of the (e.g.,
phasebrainimage imaging
inputs on andthediagnosis).
proximal facetIt is observed
of a GRINthat fiberDNNs
from their can
recognize
correspondingthe digits
speckle used of the with
patterns phase image
high inputsThese
accuracy. on theresults
proximalwerefacet of a GRIN
presented in our fiber from their
previous work
corresponding speckle patterns with high accuracy. These results were
and accurately verified here [16]. However, the trained DNN model with a dataset collected at a certain presented in our previous
work and accurately
wavelength verified
can efficiently here [16].
perform However, the
classification of atrained
speckleDNNpatternmodelthatwith a datasetusing
is generated collectedinputsat
arecorded
certain wavelength
at the same can efficiently
wavelength perform
only. For the classification
fiber length of a speckle
used, drifting pattern
of the that is generated
wavelength of theusinglaser
inputs
sourcerecorded at the same
by 3 nm causes wavelength
the speckle patternsonly. For the fiber
recorded length
at this used, driftingvalue
new wavelength of thetowavelength
be incorrectly of
the laser source
classified by 3 nmmodel.
by the trained causesThe theperformance
speckle patterns of therecorded at this and
DNN degrades newthe wavelength
classification value to be
accuracy
incorrectly classified by the trained model. The performance of
abruptly drops, as reported in Figure 5b (red circles). Consequently, it is important that the DNN the DNN degrades and the
classification
model is trained accuracy
to in a abruptly
way which drops,
allows asclassification
reported in of Figure
an input5b (red
even circles). Consequently,
if the corresponding it is
speckle
important that the DNN
pattern is generated model is
at different trained to inWe
wavelengths. a way
foundwhich
thatallows classification
one solution, which of was analso
input even if
proposed
the
for corresponding
the case of bending speckle pattern
by Fan et al.is[20],
generated at different
is to include wavelengths.
the potential We found
perturbations thatmeasurements
in the one solution,
which was also
of the training proposed
dataset so that forthethe
DNN caselearns
of bending by Fancorrectly
to interpolate et al. [20], is to include
and properly classify thethepotential
speckle
perturbations in the measurements of the training dataset so that the
outputs. However, it is observed that the classification accuracy saturates at values dependent on the DNN learns to interpolate
correctly
size of theand properly drift
wavelength classify the speckle
bandwidth, andoutputs.
even when However,
increasingit isthe
observed
size of thethatdataset,
the classification
they never
accuracy
reach thesaturates
maximum at values
of 90%, dependent
which is on thethe size of the
measured wavelength
accuracy in thedrift bandwidth,
absence of drift. and even
One when
possible
increasing the size of the dataset, they never reach the maximum of
reason for this result is the randomization of the input–output mapping; the large wavelength drift, 90%, which is the measured
accuracy
the specklein pattern
the absence of drift.digit
of a specific One captured
possible reason for this
at a certain result is the
wavelength can randomization
be very similar to of the speckle
input–
output
patternmapping;
of another the large
digit wavelength
captured drift, the
at another speckle pattern
wavelength. of a specific
Another interesting digitobservation
captured at based a certainon
wavelength can be very similar to the speckle pattern of another
the measurements reported in the presence of wavelength drift, is that the classification accuracy digit captured at another
wavelength.
does not notably Another interesting
deteriorate afterobservation
increasing based on bandwidth
the drift the measurements
further reported
than 60 nm in the presence of
approximately.
wavelength drift, is that the classification accuracy does not notably
MMFs support modes whose shape is independent of the wavelength within a certain bandwidth. deteriorate after increasing the
drift bandwidth further than 60 nm approximately. MMFs support
These modes are called principal modes [32,33]. However, this effect is usually observed for longer modes whose shape is
independent of the wavelength
fibers of few meters and further within a certain
investigation bandwidth.
is necessary These modes
to evaluate if thereareiscalled principal modes
any contribution in the
[32,33]. However, this
results presented in this work. effect is usually observed for longer fibers of few meters and further
investigation
Until thisispoint,
necessary
we haveto evaluate
discussed if there is anyofcontribution
the aspect training a DNN in the results presented
algorithm to recoverininformation
this work.
through a MMF system despite the wavelength changes by including the noise in the dataset.
Until this point, we have discussed the aspect of training a DNN algorithm to recover
Nevertheless, there are cases where sensitivity in the wavelength change is actually crucial; for example,
information through a MMF system despite the wavelength changes by including the noise in the
MMFs have been proposed as suitable systems for high-resolution spectrometers [34,35]. In those
dataset. Nevertheless, there are cases where sensitivity in the wavelength change is actually crucial;
previous works, the speckle decorrelation with wavelength is used for distinguishing different
for example, MMFs have been proposed as suitable systems for high-resolution spectrometers [34,35].
wavelengths and the resolution is dependent on the speckle decorrelation bandwidth of the specific
In those previous works, the speckle decorrelation with wavelength is used for distinguishing
fiber, which becomes narrower with increases in the fiber length. In the results presented in this work, we
different wavelengths and the resolution is dependent on the speckle decorrelation bandwidth of the
Appl. Sci. 2020, 10, 3816 8 of 10

observed that the classification efficiency of a trained DNN with data recorded at a single wavelength is
sensitive to wavelength changes and deteriorates within a significantly narrower bandwidth (Figure 5b,
red circles) than the decorrelation bandwidth of the system (Figure 5a). Therefore, what is considered
unwanted for information recovery through the system under a wavelength drift situation, it could
potentially be desired for achieving a resolution which is higher than the one given by the decorrelation
bandwidth and has been previously proposed in the literature for spectroscopic applications.

5. Conclusions
We have shown that DNNs can recognize distorted images at the output of a MMF from
their corresponding intensity speckle patterns. In particular, the deep learning approach presents a
remarkable robustness against wavelength drift of the light source within an extended bandwidth
for which total decorrelation of the speckle pattern intensity is observed. Despite the randomization
of the input–output mapping introduced by the various speckle drifting, image classification of
high accuracy of about 70% is reported for almost 100 nm wavelength drift, while more than 80%
classification accuracy is shown for a 50 nm bandwidth. The results presented in this article suggest
that deep learning can overcome distortion in the signal after propagation through MMFs in the
presence of severe wavelength drift. The potential of using a single trained DNN model for retrieving
image information in multiple wavelengths is suggested by using the intensity-only images of the
respective speckle patterns, which could allow multicolor illumination imaging without extensive
system recalibration.

Author Contributions: Conceptualization, E.K., and D.P.; methodology, E.K.; software, E.K. and N.B.; validation,
E.K., N.B. and D.P.; formal analysis, E.K.; investigation, E.K.; resources, E.K.; data curation, E.K.; writing—original
draft preparation, E.K.; writing—review and editing, E.K., N.B., B.R., U.T., D.P. and C.M.; visualization, E.K.;
supervision, E.K. and D.P.; project administration, E.K.; funding acquisition, D.P. and C.M. All authors have read
and agreed to the published version of the manuscript.
Funding: This project was partially conducted with the support of the Swiss program: “CEPF SFA, CERAMIC
X.0: High-precision micro-manufacturing of ceramics”.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Azadeh, M. Fiber Optic Communications: A Review. In Fiber Optics Engineering; Azadeh, M., Ed.; Optical
Networks; Springer US: Boston, MA, USA, 2009; pp. 1–27. ISBN 978-1-4419-0304-4.
2. Aoki, K.; Okamoto, A.; Wakayama, Y.; Tomita, A.; Honma, S. Selective multimode excitation using volume
holographic mode multiplexer. Opt. Lett. 2013, 38, 769–771. [CrossRef] [PubMed]
3. Richardson, D.J.; Fini, J.M.; Nelson, L.E. Space-division multiplexing in optical fibres. Nat. Photon. 2013, 7,
354–362. [CrossRef]
4. Okamoto, K. Fundamentals of Optical Waveguides, 2nd ed.; Academic Press: Cambridge, MA, USA, 2005;
ISBN 978-0-12-525096-2.
5. Agrawal, G. Nonlinear Fiber Optics; Academic Press: Cambridge, MA, USA, 2001; ISBN 978-0-08-047974-3.
6. Papadopoulos, I.N.; Farahi, S.; Moser, C.; Psaltis, D. Increasing the imaging capabilities of multimode fibers
by exploiting the properties of highly scattering media. Opt. Lett. 2013, 38, 2776–2778. [CrossRef] [PubMed]
7. Čižmár, T.; Dholakia, K. Exploiting multimode waveguides for pure fibre-based imaging. Nat. Commun.
2012, 3, 1027. [CrossRef] [PubMed]
8. Kakkava, E.; Romito, M.; Conkey, D.B.; Loterie, D.; Stankovic, K.M.; Moser, C.; Psaltis, D. Selective
femtosecond laser ablation via two-photon fluorescence imaging through a multimode fiber. Biomed. Opt.
Express 2019, 10, 423–433. [CrossRef]
9. Caravaca-Aguirre, A.M.; Niv, E.; Conkey, D.B.; Piestun, R. Real-time resilient focusing through a bending
multimode fiber. Opt. Express 2013, 21, 12881. [CrossRef]
10. Mahalati, R.N.; Askarov, D.; Wilde, J.P.; Kahn, J.M. Adaptive control of input field to achieve desired output
intensity profile in multimode fiber with random mode coupling. Opt. Express 2012, 20, 14321–14337.
[CrossRef]
Appl. Sci. 2020, 10, 3816 9 of 10

11. Loterie, D.; Farahi, S.; Papadopoulos, I.; Goy, A.; Psaltis, D.; Moser, C. Digital confocal microscopy through a
multimode fiber. Opt. Express 2015, 23, 23845–23858. [CrossRef]
12. Popoff, S.M.; Lerosey, G.; Fink, M.; Boccara, A.C.; Gigan, S. Controlling light through optical disordered
media: Transmission matrix approach. New J. Phys. 2011, 13, 123021. [CrossRef]
13. Loterie, D.; Psaltis, D.; Moser, C. Bend translation in multimode fiber imaging. Opt. Express 2017, 25, 6263.
[CrossRef]
14. Farahi, S.; Ziegler, D.; Papadopoulos, I.N.; Psaltis, D.; Moser, C. Dynamic bending compensation while
focusing through a multimode fiber. Opt. Express 2013, 21, 22504. [CrossRef] [PubMed]
15. Conkey, D.B.; Kakkava, E.; Lanvin, T.; Loterie, D.; Stasio, N.; Morales-Delgado, E.; Moser, C.; Psaltis, D.
High power, ultrashort pulse control through a multi-core fiber for ablation. Opt. Express 2017, 25, 11491.
[CrossRef] [PubMed]
16. Borhani, N.; Kakkava, E.; Moser, C.; Psaltis, D. Learning to see through multimode fibers. Option 2018, 5,
960–966. [CrossRef]
17. Kakkava, E.; Rahmani, B.; Borhani, N.; Teğin, U.; Loterie, D.; Konstantinou, G.; Moser, C.; Psaltis, D. Imaging
through multimode fibers using deep learning: The effects of intensity versus holographic recording of the
speckle pattern. Opt. Fiber Technol. 2019, 52, 101985. [CrossRef]
18. Rahmani, B.; Loterie, D.; Konstantinou, G.; Psaltis, D.; Moser, C. Multimode optical fiber transmission with a
deep learning network. Light. Sci. Appl. 2018, 7, 69. [CrossRef]
19. Shabairou, N.; Cohen, E.; Wagner, O.; Malka, D.; Zalevsky, Z. Color image identification and reconstruction
using artificial neural networks on multimode fiber images: Towards an all-optical design. Opt. Lett. 2018,
43, 5603–5606. [CrossRef]
20. Fan, P.; Zhao, T.; Su, L. Deep learning the high variability and randomness inside multimode fibers.
Opt. Express 2019, 27, 20241–20258. [CrossRef]
21. Aisawa, S.; Noguchi, K.; Matsumoto, T. Remote image classification through multimode optical fiber using a
neural network. Opt. Lett. 1991, 16, 645. [CrossRef]
22. Marusarz, R.K.; Sayeh, M.R. Neural network-based multimode fiber-optic information transmission.
Appl. Opt. 2001, 40, 219–227. [CrossRef]
23. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT: Cambridge, MA, USA; London, UK, 2017;
ISBN 0-262-03561-8.
24. Barbastathis, G.; Ozcan, A.; Situ, G. On the use of deep learning for computational imaging. Option 2019, 6,
921–943. [CrossRef]
25. Schmidhuber, J. Deep learning in neural networks: An overview. Neural Networks 2015, 61, 85–117. [CrossRef]
[PubMed]
26. Kürüm, U.; Wiecha, P.R.; French, R.; Muskens, O.L. Deep learning enabled real time speckle recognition
and hyperspectral imaging using a multimode fiber array. Opt. Express 2019, 27, 20965–20979. [CrossRef]
[PubMed]
27. Carpenter, J.; Eggleton, B.J.; Schroder, J. Complete spatiotemporal characterization and optical transfer matrix
inversion of a 420 mode fiber. Opt. Lett. 2016, 41, 5580. [CrossRef] [PubMed]
28. Deng, L. The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the
Web]. IEEE Signal Process. Mag. 2012, 29, 141–142. [CrossRef]
29. Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition.
arXiv 2014, arXiv:1409.1556 [cs].
30. Kakkava, E.; Borhani, N.; Rahmani, B.; Tegin, U.; Moser, C.; Psaltis, D. Efficient Image Classification through
a Multimode Fiber using Deep Neural Networks in presence of Wavelength Drifting. Imaging Appl. Opt.
2019 (COSI, IS, MATH, pcAOP) 2019, CW1A.4. [CrossRef]
31. Kakkava, E.; Borhani, N.; Rahmani, B.; Tegin, U.; Moser, C.; Psaltis, D. Wavelength Independent Image
Classification through a Multimode Fiber using Deep Neural Networks. In Proceedings of the 2019 Conference
on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC),
Munich, Germany, 23–27 June 2019; p. 1.
32. Redding, B.; Alam, M.; Seifert, M.; Cao, H. High-resolution and broadband all-fiber spectrometers. Option
2014, 1, 175. [CrossRef]
33. Redding, B.; Popoff, S.M.; Cao, H. All-fiber spectrometer based on speckle pattern reconstruction. Opt. Express
2013, 21, 6584. [CrossRef]
Appl. Sci. 2020, 10, 3816 10 of 10

34. Fan, S.; Kahn, J.M. Principal modes in multimode waveguides. Opt. Lett. 2005, 30, 135–137. [CrossRef]
35. Xiong, W.; Ambichl, P.; Bromberg, Y.; Redding, B.; Rotter, S.; Cao, H. Principal modes in multimode fibers:
Exploring the crossover from weak to strong mode coupling. Opt. Express 2017, 25, 2709. [CrossRef]

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).