Professional Documents
Culture Documents
Automated Molting Detection System For Commercial Soft-Shell Crab (Portunus Pelagicus) Production
Automated Molting Detection System For Commercial Soft-Shell Crab (Portunus Pelagicus) Production
doi: 10.2306/scienceasia1513-1874.2022.090
ABSTRACT: Cost, availability and reliability of labor are major problems in any aquaculture operation, and particularly
in the soft-shell crab industry, since its success depends on the precise timing and accuracy of the workers to observe
and harvest the newly molted crabs before their shells harden. To improve efficiency and reduce the dependence on
human intervention, we have developed an automated molting detection system. The detection system utilizes the fact
that the crab’s carapace reflects infrared light (appearing as white pixels) much more strongly than the surrounding
areas. By using Internet Protocol (IP) cameras, network video recorder (NVR), personal computer and newly designed
image analysis software, molting can be detected by continuously measuring relative changes in white pixel area. Two-
dimensional Gaussian function and Otsu’s thresholding method are incorporated into the detection software. Snapshot
images, date, and time of molting are recorded automatically and displayed via user interface. Test results indicated
that the highest (100%) hit rate and lowest precision (13.92%) were obtained when detection threshold was 20%.
Lower hit rates and higher precision were observed at higher threshold levels. The optimum threshold for detecting
molting in commercial operations is discussed.
www.scienceasia.org
2 ScienceAsia 48 (2022)
www.scienceasia.org
ScienceAsia 48 (2022) 3
Fig. 3 System overview of the molting detection system. IP cameras acquire images of crabs from the raft and transmit them
to NVR for recording. Then, image analysis software in personal computer analyzes crab images from each area of interest
and displays the result via user interface.
the user to maximize the molting detection accuracy. gential distortion. By using these equations, both types
Capture Time Interval is the duration between each of distortion were corrected.
picture in frames per second. Maximum Capture Time Radial distortion correction:
is the maximum recording duration. Minimum crab
size is the least possible size (area in pixels) of each x corrected = x 1 + k1 r 2 + k2 r 4 + k3 r 6
crab in the area of interest, while Beware and Molt ycorrected = y 1 + k1 r 2 + k2 r 4 + k3 r 6
threshold are the size values that will trigger the pro-
gram to give warning and molting messages for each Tangential distortion correction:
crab. The size of the area of interest (detection area)
x corrected = x + 2p1 x y + p2 (r 2 + 2x 2 )
for each crab can also be adjusted.
Detection area identification: This component man- ycorrected = y + p1 (r 2 + 2 y 2 ) + 2p2 x y
ages the areas of interest and their locations by la-
belling a snapshot from the IP camera with green Combined radial and tangential distortion correction:
squares. These green squares can be added, moved or
x corrected = x + 2p1 x y + p2 (r 2 + 2x 2 )
removed at any point. The program only monitors the
ratio of white: total pixels within each green square. + x(1 + k1 r 2 + k2 r 4 + k3 r 6 )
Control panel for molting detection: This compo-
ycorrected = y + p1 (r 2 + 2 y 2 ) + 2p2 x y
nent can be used to activate, pause, reset and stop the
program during molt monitoring. + y(1 + k1 r 2 + k2 r 4 + k3 r 6 )
www.scienceasia.org
4 ScienceAsia 48 (2022)
www.scienceasia.org
ScienceAsia 48 (2022) 5
Fig. 4 Uncalibrated (left) and calibrated (right) images. Camera calibration transforms pixel point at (x, y) from distorted
image to corrected pixel point at (x corrected , ycorrected ); thus, the distorted area on each side of the raft is corrected to a
rectangular shape.
www.scienceasia.org
6 ScienceAsia 48 (2022)
Fig. 7 User interface with molting information. When molting occurs, the green square changes to red to be easily noticed by
user.
www.scienceasia.org
ScienceAsia 48 (2022) 7
uations, such as in the presence of a predator [21]. To images were taken every 10 s to reduce computational
reduce the risk of being preyed upon, some crab species workload. Since actual molting events usually took
exhibit synchronized molting behavior, in which the under 10 min to complete, a 5-min calculation window
majority of the animals molt during the same period was deemed sufficient to detect any molting. The
and in the same location. In the case of the Tanner crab program’s user interface was also designed to present
(Chionoecetes bairdi) in Alaska, an estimated 11 500 all necessary molting information. It also provides
crab exuviae were reported in an area of approxi- actual molting details such as date and time, which
mately 0.034 km2 [22]. From a farm management may be valuable for future research. To the best of our
perspective, this molting plasticity has the potential knowledge, this is the first attempt to detect molting
to be useful for production planning, both in labor by using infrared light reflection and image processing
management and in the harvesting schedule. For in this manner.
example, changing the lighting scheme, temperature The testing showed that low threshold settings
or even the direction of water flow to avoid conspecific (20% and 30%) had the highest hit rates (100%) but
cues could stimulate or prolong molting, according to also the lowest precision (13.92% and 22.45%). This
the farm’s requirements. is because the low-threshold levels were extremely
Wild-caught crabs had lower molting rate and sensitive to any changes in crab area, resulting in
higher mortality rate than cultured crabs. This was 100% detection of actual molting cases. However,
not unexpected, since gill nets are the primary fish- the majority of the detections (77.55%–86.08%) were
ing gear for blue swimming crab in Thailand [23]. false positives, making precision very low. At 40
Trauma from gear entanglement, air/temperature ex- and 50% thresholds, the hit rate dropped to 90.91%
posure and mishandling during transportation [24] and 72.73%, respectively, while precision increased to
contribute to high mortality rates. However, to achieve 33.33% and 53.33%. At thresholds of 80–90%, it was
commercial-scale blue swimming crab culture that is rare for the program to detect any molting. The hit rate
profitable, better survival rates, especially at early and was very low at 9.09% while the precision was 100%,
late larval stages [25] and higher stocking density must without a single case of false positive detection.
be realized. It is tempting to believe that a good detection
The proposed molting detection method consists system must have both high hit rate and high precision
of three main stages: (1) preprocessing, (2) crab detec- at the same time, but this is not always the case.
tion, and (3) molting classification. The preprocessing In some situations such as in facial recognition, it is
stage prepares the images to eliminate noise. In more important to have high precision than high hit
our trial, noise was caused by low-light photography rate to avoid the nuisance of too many false positive
(higher ISO light sensitivity) as well as by the netting detections [28]. However, in the case of molting, each
covering each box, which interfered with the crab molt is very important because it is a source of income
detection system. Most noise interference was char- for the farm, while false positive detection only causes
acterized as high frequency objects. Thus, a Gaussian minor nuisance, and can be further improved upon
low-pass filter [26] was used to reduce the effect of in the program. In the case of a 40% threshold, the
that particular noise, while maintaining relatively good farm would lose almost 10% of the total production but
crab area images. The standard deviation variable have little improvement in precision. This loss would
in the Gaussian filter was related to the size and be even greater in the case of a higher threshold. For
frequency of the netting. these reasons, it is recommended that the threshold for
The crab detection stage is designed to detect the current version of the detection program be set at
the crab area in each green square. As the crab’s 30% to provide the highest hit rate of molting and to
carapace strongly reflects infrared light in contrast to maximize the soft-shell crab production.
the background, Otsu’s method [27] was chosen as the Although a perfect system with 100% hit rate and
most appropriate tool for this stage. Since the refection precision is nearly impossible to achieve, one can still
of infrared light across the raft was non-uniform due improve the system so that both values are relatively
to the location of light sources, the appropriate value high across different thresholds. One possibility to im-
of Otsu’s threshold of each green square had to be prove both hit rate and precision is to improve lighting
adaptive. To accommodate this, Otsu’s method was conditions. Most of the IP cameras have infrared array
applied for each green square separately. LED lights near the lens that cause water reflection
The molting classification equation was designed in the center of images and uneven distribution of
to calculate the rate of change in white (crab shell) area light across the raft. This reflection can interfere with
over a fixed time, in this case 5 min. Since each crab the detection program because the bright white light
has a different size and posture, it is far more effective results in a higher number of false positive/negative
to define the molting classification conditions based detections. Meanwhile, around the edge of the raft,
on percentage change of each individual crab area reflection is very dim. To avoid this, the source of
rather than by a universal fixed constant. Snapshot infrared light should be relocated from the top-center
www.scienceasia.org
8 ScienceAsia 48 (2022)
to the top-left and top-right of the raft to avoid direct workload, an autonomous system to detect molting
reflection and provide more even light distribution. It has been developed. We have demonstrated that the
has also been demonstrated that the accuracy of finger reflection of infrared light from the crab’s carapace
vein authentication technology could be improved by in combination with image-analysis software can be
adjusting the intensity of infrared light [29]. coupled to detect molting.
Movements of rafts due to maintenance activities The authors consider this detection program use-
such as feeding or harvesting also interfere with the ful not only for the soft-shell crab industry, but also as a
detection system, causing more false positive/negative research platform for any future molting experiments.
detection. Installing augmented reality markers (AR Molting frequency, date, and time, along with crab
markers) to track orientation and position of the raft behavior could be tracked and analyzed in a real-
might resolve this issue. A similar method has been time fashion. It is also possible to collect data from
demonstrated [30] to help visually impaired persons various off-site locations simultaneously via internet
by recognizing dynamic content and fixed positions. network, thus enabling even greater molting detection
Crab posture also needs to be considered in the use and prediction.
of this system, and as a matter for further development.
Some crabs align themselves with the wall in a vertical Acknowledgements: The authors would like to express
position, which makes the detectable area much less their sincere gratitude to Thailand Science Research and In-
than for a horizontal position. When a crab changes novation (TSRI) for sponsoring this research (RDG6220021).
posture from vertical to horizontal, the detectable We are also thankful to Dr. Pongthep Wiliapun and Dr.
area increases, causing false positive detection. As a Shettapong Meksumpun for their most valuable support and
result, “Minimum Crab Size%” was introduced to the constant encouragement.
detection configuration so the program would ignore
any area less than a predetermined level. REFERENCES
The conventional method of soft-shell crab (mud 1. Dana SS, Ghosh A, Bandyopadhyay UK (2015) Socio-
crab) production in Thailand is by floating crab boxes economic profile and problems of mud-crab farmers of
in earthen ponds, exposing crabs to both sunlight and South 24-Parganas, West Bengal: an explorative study. J
rain. Workers physically inspect each box using a Crop Weed 11, 120–123.
flashlight and personal expertise. The detection system 2. Hungria DB, Tavares CPDS, Pereira LA, Silva UDATD,
introduced in this study was designed not only for Ostren-sky A (2017) Global status of production and
commercialization of soft-shell crabs. Aquac Int 25,
blue swimming crab (which cannot be cultured by
2213–2226.
conventional methods for biological and physiological 3. Oesterling MJ (1988) Manual for Handling and Shedding
reasons) but also could be adapted to improve the Blue Crabs (Callinectes sapidus), Virginia Institute of
conventional method for mud crab. The detection Marine Science, Gloucester Point, Virginia, USA.
software also lays the groundwork for a novel soft- 4. Gaude AR, Anderson JA (2011) Soft Shell Crab Shedding
shell crab production method by incorporating image Systems, Southern Regional Aquaculture Centre, USDA,
processing technologies with aquaculture. For the National Institute of Food and Agriculture, USA.
detection system to achieve highest effectiveness, the 5. Rahman MM, Haque SM, Wahab A, Egna H, Brown C
entire production facility should be redesigned from (2018) Soft-shell crab production in coastal Bangladesh:
the ground up. Environmental control (to prevent prospects, challenges and sustainability. World Aquac 49,
43–47.
light interference) and a monitoring system need to
6. Sanguansuttigul P, Saleewong T, Khamwan K,
be factored in from the beginning. Monitoring via Bongsebandhu-phubhakdi S (2021) A diffusion
cameras would enable the farmer to use less space and model of 18F-FDOPA in the diagnosis of Parkinson’s
resources by stacking crab boxes vertically, reducing disease and its numerical simulation. ScienceAsia 47,
both the workload and skill required to detect and 241–250.
collect molted crabs. This, in turn, would enable the 7. Mehtre BM (1993) Fingerprint image analysis for auto-
farmer to increase not only the production output but matic identification. Mach Visual Appl 6, 124–139.
also the quality of the soft-shell crabs. The size of the 8. Senthilkumar K, Vijayan E, Arun AR (2017) Traffic anal-
production facility is not limited by the system per se ysis and control using image processing. IOP Conf Ser:
but by the availability of crab seeds and investment Mater Sci Eng 263, 042047.
9. Miranda JL, Gerardo BD, Iii BT (2014) Pest detection
level. It would also require farmers with a technology-
and extraction using image processing techniques. Int J
and data-driven mindset, and initial financial invest- Comput Commun Eng 3, 189–192.
ment for the system. 10. Thenmohiz K, Srinivasaulu UR (2017) Image processing
techniques for insect shape detection in field crops. In:
CONCLUSION 2017 International Conference on Inventive Computing
and Informatics (ICICI), pp 699–704.
Labor shortage and unskilled human error contribute 11. Dutta MK, Issac A, Minhas N, Sarkar B (2016) Image
to the significant loss of soft-shell crab production processing based method to assess fish quality and fresh-
annually. To improve efficiency and reduce human ness. J Food Eng 177, 50–58.
www.scienceasia.org
ScienceAsia 48 (2022) 9
www.scienceasia.org