See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/330780298

A Fish Classification on Images using Transfer Learning and Matlab

Conference Paper · September 2018

DOI: 10.1109/INAPR.2018.8627007

CITATIONS READS

27 2,583

7 authors, including:

Suryadiputra Liawatimena Yaya Heryadi

Binus University Binus University
67 PUBLICATIONS 231 CITATIONS 149 PUBLICATIONS 1,191 CITATIONS

SEE PROFILE SEE PROFILE

Lukas Lukas Erland Barlian

Atma Jaya Catholic University of Indonesia Binus University
93 PUBLICATIONS 4,340 CITATIONS 11 PUBLICATIONS 43 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Erland Barlian on 10 February 2019.

The user has requested enhancement of the downloaded file.

 A Fish Classification on Images using Transfer
Learning and Matlab
Suryadiputra Liawatimena1,2
1) Computer Science Department
BINUS Graduate Program - Doctor
Computer Science
2) Computer Engineering Department
Faculty of Engineering
Bina Nusantara University
Jakarta, Indonesia 11480
suryadi@binus.edu/
s.liawatimena.id@ieee.org
Bahtiar Saleh Abbas
Industrial Engineering Department
Faculty of Engineering
Bina Nusantara University
Jakarta, Indonesia 11480
bahtiars@binus.edu
BINUS Business School - Doctor of Research
Yaya Heryadi
Computer Science Department
BINUS Graduate Program - Doctor
Computer Science
Bina Nusantara University
Jakarta, Indonesia 11480
yayaheryadi@binus.edu
Agung Trisetyarso
Computer Science Department
BINUS Graduate Program - Doctor
Computer Science
Bina Nusantara University
Jakarta, Indonesia 11480
atrisetyarso@binus.edu
Erland Barlian
Management Department
in Management
Bina Nusantara University
Jakarta, Indonesia 11480
erland.barlian@binus.ac.id
Lukas
Cognitive Engineering Research Group
(CERG)
Faculty of Engineering
Universitas Katolik Indonesia Atma Jaya
Jakarta, Indonesia, 12930
lukas@atmajaya.ac.id
Antoni Wibowo
Computer Science Department
BINUS Graduate Graduate Program –
Master of Computer Science
Bina Nusantara University
Jakarta, Indonesia 11480
anwibowo@binus.edu

Abstract— The Ministry of Marine Affairs and Fisheries (MMAF) supplier to the United Kingdom, United States of America,
carry out the responsibilities and features associated with the Japan and The European Union (EU). In response to the
policy of marine and fisheries. MMAF carry out the increasing, The pole-and-line fishery has been grown
responsibilities and features associated with marine and fisheries predominantly in the eastern area of Indonesia especially in the
policy. One of their duties and functions are organized marine and
fishery statistics, in accordance to Law of The Republic of
water Sulawesi [3], Maluku and northern off Papua. Fishing
Indonesia Number sixteen of 1997 regarding Statistics. The main operation of pole-and-line is mainly within five out of eleven
problem is the number of enumerators at each Basis Landing Of Indonesian Fisheries Management Areas (FMA) namely FMA
Fish. This paper proposed a fish classification on fish images using 713, 714, 715, 716 and 717. In the 2007-2013 period, the
transfer learning and Matlab as the first stage of tackling the average Katsuwonus Pelamis caught is 83.7%, 80.7% and
problem. FishNet is a modification from AlexNet to classify 77.8% at Sorong-Papua, Bitung-North Sulawesi and Kendari-
Katsuwonus Pelamis (Skipjack tuna or Cakalang), Euthynnus Southeast Sulawesi and respectively (see Fig.1).
Affinis (Tongkol) and Coryphaena Hippurus (Mahi-mahi) that
caught by fishermen. There are 15.120 images of 3 type of fishes,
5.040 for each fish. Data is split into 70:30 for training and
validation set. The training process is done using Matlab 2018a on
Windows 7 operating system in a notebook with single CPU i7 with
8 GB RAM for 124 minutes. The validation accuracy is 99.63%.

Keywords—Image classification, Machine learning, Marine

animals

I. INTRODUCTION
Indonesia is the largest archipelagic country, and it has
more than thirteen thousand islands [1], [2]. Indonesia has
become one of the largest tuna producing countries around the
globe. Indonesia is a significant producer if seafood and a vital

Fig. 1. The Fish Aggregating Devices (FADs) position which indicates as
pole-and-line fishing ground [3]
The Fishermen usually catch Katsuwonus Pelamis
(Skipjack tuna or Cakalang), Euthynnus Affinis (Tongkol) and
Coryphaena Hippurus (Mahi-mahi) [4]. Captain of the boat
must fill in the logbook and get permit issuance data were
obtained from the port authorities of the Oceanic Fishing Port
(OFP). The logbook data includes departure and arrival dates of
vessels into the port, gross tonnage (GT), the overall length of
the vessel (LOA), number of crew, and catch of pole-and-line,
longline, purse-seine and handline vessels for every reported
trip.
Sometimes the captain hesitates to fill the logbook
correctly because he also uses estimation only [5]. At the Basis
Landing Of Fish (Pangkalan Pendaratan Ikan/PPI in Bahasa),
some enumerators who make a record from each fisherman
boats such as the type of fishes and their weight in tonnage. The
record needs to make a national report to FAO [6]. The
enumerator makes the only estimation from by glance at boat
hold. The number of enumerators is minimal compared to the
number of boats. The enumerators cannot catch up the record
when there are many boats came in at the same time. The paper
aims to help the Ministry of Maritime Affairs and Fisheries
(KKP) in improving the sustainability of aquaculture and
capture fisheries implement science and technology innovation
in marine and fisheries area. There are two stages to achieve
automatic fish catch estimation using a web camera, such as
classify fishes from images than from a web camera. This paper
proposed a fish classification on fish images using transfer
learning and Matlab.
II. RELATED WORK
Deep learning is a kind of machine learning that trains a
computer to operate human-like tasks, such as recognizing
speech, figuring out images or making predictions. Instead of
organizing statistics to run via predefined equations, deep
learning sets up fundamental parameters about the records and
trains the computer to analyze on its own through recognizing
patterns the use of many layers of processing. Deep learning
techniques have accelerated the capability to recognize,
classify, discover and describe – in one word, understand. Deep
learning is class methods rest on multilayer neural networks,
draws a lot of researchers attention in machine learning lately.
Deep learning has been recognized as a handy tool in the
application of image processing, natural language processing,
speech recognition, text classification, robotics and control,
computer vision, and so on [7], [8].
A Convolutional Neural network (CNN) is a practical
machine learning approach from the sector of deep learning.
CNNs are trained using massive collections of numerous
images. From those massive collections, CNNs can learn rich
characteristic representations for an extensive range of images.
Those characteristic representations frequently outperform
hand-crafted features which include a Histogram of Oriented
Gradients (HOG), Local binary pattern (LBP), or Speeded Up
Robust Features (SURF) [9]. An easy way to leverage the energy
of CNNs, without investing effort and time into training, is to
use a pretrained CNN as a characteristic extractor.
A comprehensive analysis of essential metrics in practical
applications: accuracy, memory footprint, parameters,
operations count, inference time and power consumption in [10].
As shown in Fig. 2 AlexNet has the smallest Top-1 accuracy and
smallest operations too. In the next stage of this research will put
AlexNet into an embedded system such as Raspberry Pi 3B+
[11] and a web camera as its input to recognize three type of
fishes on fisherman boat.
Fig. 2. Top1 vs. operations, size ∝ parameters. Top-1 one-crop accuracy versus
the number of operations required for a single forward pass [10]
Transfer learning also is known as reuse pretrained network,
refers to the process of using the weights from pre-trained
networks on a large dataset. As the pre-trained networks have
already learned how to identify lower level features such as
edges, lines, curves, and many others. With the convolutional
layers which are frequently the most computationally time-
consuming parts of the process, the use of these weights helps
the network to converge to an excellent score faster than training
from scratch. In this paper, we use AlexNet [12], then modified
its final layer to recognize three classes of fish, namely:
Katsuwonus Pelamis (Cakalang), Euthynnus Affinis (Tongkol)
and Coryphaena Hippurus (Mahi-mahi) (See Fig. 3).
 Fig. 3. Fish Images namely Katsuwonus Pelamis (Cakalang), Euthynnus
Affinis (Tongkol) and Coryphaena Hippurus (Mahi-mahi) Fig. 4. A 5-degree rotation Augmentation Program
(the program is provided at the end of paper for clarity)
To train a CNN model from scratch successfully, it needs a
huge dataset, and machines with higher computational power are
needed, preferably with GPU [13]. AlexNet has been trained on
more than a million images and can classify images into 1000
object categories (such as coffee mug, a keyboard, pencil, and
animals). AlexNet has learned wealthy characteristic
representations for a vast variety of images. AlexNet takes an
image as input and outputs a label for the object in the image Fig. 5. A 25-degree rotation augmentation result
collectively with the chances for every of the object categories. • Step 3: Prepare the data set. In this paper, there are 15.120
Transfer learning is frequently used in deep learning images of fishes, 5.040 for each fish. Data is split into 70:30
applications. A pretrained network can be taken and use it as a for training and validation set. Where a training set is fixed
starting point to learn a new task. Fine-tuning a network with for examples used for learning, this is to match the
transfer learning is usually much faster and more comfortable parameters of the classifier. On the other hand, the validation
than training a network with randomly initialized weights from set is fixed for examples used to tune the parameters of a
scratch [14]. classifier, for instance, to choose the extensive form of
hidden units in a neural network [16].
III. METHOD AND RESULTS • Step 4: Modify the AlexNet into FishNet with Matlab and
There is five-step to teach machine learning to classify use it to predict the class of the fish. FishNet architecture is
fishes, such as: shown in Fig. 6, with the fully connected layer (FC8) only
three classes for each fish.
• Step 1: Collect fish images from the internet by search with
species name or its other popular names, for examples:
Katsuwonus Pelamis is known as skipjack tuna or Cakalang
in Bahasa, Euthynnus Affinis is known as mackerel tuna or
Tongkol in Bahasa, and Coryphaena Hippurus is known as
mahi-mahi or Lemadang in Bahasa.
• Step 2: Image-preprocessing consists centered crop into the
size of the input layer of the CNN and using augmentation
for rotation since fish will come into boat hold in every
possible angle. Augmentation is used to multiple images for
Fig. 6. The FishNet architecture
creating enough images for deep learning training.
Augmentation can be scale, rotation, flip, skew, crop and so • Step 5: Setting hyperparameter of FishNet. In this paper,
on [15]. Since many augmentation programs when it makes Stochastic gradient descent with momentum (SGDM) is
scale and rotation, it leaves the black area of augmented chosen as the learning algorithm.
images. An augmentation program that utilized an alpha
layer, it helps to avoid the black area effect from the
augmentation process is proposed in this paper, and the
rotation result shown in Fig. 5.
 Fig. 8. Confusion matrix of Fishnet

As shown in Fig. 9, there are four images classifying results

with its scores.

Fig. 7. Training process

Mini batch size is set to 512 for faster convergence [17].
Mini batch size is anything from 64 up to 512 are quite typical.
Because of the way computer memory is laid out and accessed,
sometimes code of program runs faster if mini-batch size is a
power of 2. In one of the experiments, mini batch size is set to
1024; however, the notebook hangs due to the fact it does not
have sufficient memory to fit all weight calculations.
As shown in Fig. 7, the output of training as expected when
a transfer learning is used, and higher accuracy achieved faster.
The training process is done using Matlab 2018a on Windows 7 Fig. 9. The sample of fishes classifying
operating system in a notebook with single CPU i7 with 8 GB
RAM for 124 minutes. The FishNet can recognize an image outside data set
Fig. 8 shows the confusion matrix of FishNet. There are (training and validation set) correctly, see Fig. 10.
errors in classifying 2 images of Euthynnus Affinis as
Coryphaena Hippurus and 15 images of Euthynnus Affinis as
Katsuwonus Pelamis. The validation accuracy is 99.63%.

Fig. 10. The sample of fishes classifying
CONCLUSION
An augmentation program that utilized an alpha layer, it
helps to avoid the black area effect from the augmentation
process. FishNet can achieve higher validation accuracy faster
because it utilizes transfer learning from AlexNet. Further work,
the research will put AlexNet into an embedded system such as
Raspberry Pi 3B+ and a web camera as its input to recognize
three type of fishes on fisherman boat.
REFERENCES
[1] Y. Yang, “The prior notification issue of military
activities in EEZ,” pp. 158–170, Mar. 2018.
[2] “The World Factbook — Central Intelligence
Agency.” [Online]. Available:
https://www.cia.gov/library/publications/the-world-
factbook/geos/id.html. [Accessed: 22-Aug-2018].
[3] Wayongkere, “Melaut 40 Mil Dapat Tuna, Bitung
Surga Nelayan Filipina - Tribun Manado.” [Online].
Available:
http://manado.tribunnews.com/2014/08/02/melaut-40-
mil-dapat-tuna-bitung-surga-nelayan-filipina.
[Accessed: 15-Aug-2018].
[4] A. A. Widodo, W. Wudianto, and F. Satria, “Current
Status Of The Pole-And-Line Fishery In Eastern Part
Of Indonesia,” Indones. Fish. Res. J., vol. 22, no. 1, p.
43, Dec. 2016.
[5] S. Yuniarta, P. A. M. van Zwieten, R. A. Groeneveld,
S. H. Wisudo, and E. C. van Ierland, “Uncertainty in
catch and effort data of small- and medium-scale tuna
fisheries in Indonesia: Sources, operational causes and
magnitude,” Fish. Res., vol. 193, no. April, pp. 173–
# Import libraries
import os
from os import walk
from os.path import basename
from PIL import Image
import numpy as np
183, 2017.
[6] R. A. P. Publication, the Fishing Fleet in Aceh
Province, Indonesia, no. January. 2009.
[7] R. Hof, “Deep Learning - MIT Technology Review,”
2018. [Online]. Available:
https://www.technologyreview.com/s/513696/deep-
learning/. [Accessed: 22-Aug-2018].
[8] J. Han, D. Zhang, G. Cheng, N. Liu, and D. Xu,
“Advanced Deep-Learning Techniques for Salient and
Category-Specific Object Detection: A Survey,” IEEE
Signal Process. Mag., vol. 35, no. 1, pp. 84–100, Jan.
2018.
[9] A. Lee, “Comparing Deep Neural Networks and
Traditional Vision Algorithms in Mobile Robotics,”
2016.
[10] A. Canziani, A. Paszke, and E. Culurciello, “An
Analysis of Deep Neural Network Models for
Practical Applications,” May 2016.
[11] “Machine Learning on Arm | Running AlexNet on
Raspberry Pi with Compute Lib – Arm Developer.”
[Online]. Available:
https://developer.arm.com/technologies/machine-
learning-on-arm/developer-material/how-to-
guides/running-alexnet-on-raspberry-pi-with-compute-
library. [Accessed: 25-Aug-2018].
[12] A. Krizhevsky, I. Sutskever, and G. E. Hinton,
“ImageNet Classification with Deep Convolutional
Neural Networks,” Adv. Neural Inf. Process. Syst., pp.
1–9, 2012.
[13] T. Dettmers, “A Full Hardware Guide to Deep
Learning - Tim Dettmers.” [Online]. Available:
http://timdettmers.com/2015/03/09/deep-learning-
hardware-guide/. [Accessed: 22-Aug-2018].
[14] J. Brownlee, “A Gentle Introduction to Transfer
Learning for Deep Learning.” [Online]. Available:
https://machinelearningmastery.com/transfer-learning-
for-deep-learning/. [Accessed: 21-Aug-2018].
[15] A. Jung, “imgaug — imgaug 0.2.6 documentation,”
2017. [Online]. Available:
https://imgaug.readthedocs.io/en/latest/. [Accessed:
21-Aug-2018].
[16] Z. Q. John Lu, “The Elements of Statistical Learning:
Data Mining, Inference, and Prediction,” J. R. Stat.
Soc. Ser. A (Statistics Soc., vol. 173, no. 3, pp. 693–
694, Jan. 2010.
[17] D. Yin, A. Pananjady, M. Lam, D. Papailiopoulos, K.
Ramchandran, and P. Bartlett, “Gradient Diversity: a
Key Ingredient for Scalable Distributed Learning,”
2018.
import cv2
import imutils
import matplotlib.pyplot as plt

# Path of image source and target output directory

source = ["C:\\Users\\user\\0rotator\\katsuwonuspelamis", "C:\\Users\\user
\\0rotator\\euthynnusaffinis", "C:\\Users\\user\\0rotator\\coryphaenahippurus"]
target= "C:\\Users\\user\\0rotator\\output\\katsuwonuspelamis", "C:\\Users\\user
\\0rotator\\output\\euthynnusaffinis", "C:\\0rotator\\output\\coryphaenahippurus"]
# Loop for each fish directory
for i in range (0, 3):
mypath = source[i]
targetpath=target[i]
os.chdir(mypath)
# For each file in the directory target
for root, directory, files in os.walk(mypath):
for eachfile in files:
if ".jpg" in eachfile:
file_name = os.path.splitext(eachfile)[0]
# Read one image
imgsrc = Image.open(eachfile)
# Rotate it by 5 degree
for degree in range(0, 360, 5):
imgrotated = imgsrc.convert('RGBA').rotate(degree)
#create a composite image the use of the alpha layer of imgrotated as a mask
fff = Image.new('RGBA', imgrotated.size, (255,)*4)
out = Image.composite(imgrotated, fff, imgrotated)
imgconvert = out.convert(imgsrc.mode)
outputfilename = os.path.join(targetpath, file_name) + '-' + \
str("%03d" % degree) + ".jpg"
imgconvert.save (outputfilename)

View publication stats