full report_group-5

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 36

A PROJECT PHASE-I REPORT ON

MUSICAL INSTRUMENT SOUND CLASSIFICATION USING


DEEP CONVOLUTIONAL NEURAL NETWORK

SUBMITTED TO THE SAVITRIBAI PHULE PUNE UNIVERSITY, PUNE


IN THE PARTIAL FULFILLMENT FOR THE AWARD OF THE DEGREE

OF

BACHELOR OF ENGINEERING (SEM-1)


IN
ELECTRONICS AND TELECOMMUNICATION
BY

1. AMAN SRIVASTAVA 71711908E


2. ANKUR AADITYA 71711917D
3. C RANGRAJAN KUMAR 71712026M

UNDER THE GUIDANCE OF

Prof. R. S. Kothe

DEPARTMENTOF ELECTRONICS AND TELECOMMUNICATION ENGG.


STES’S
SMT. KASHIBAI NAVALE COLLEGE OF ENGINEERING
VADGAON BK., OFF. SINHGAD ROAD,
PUNE 411041

2019-2020
CERTIFICATE

This is to certify that the project phase-1 report entitled

MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP


CONVOLUTIONAL NEURAL NETWORK

Submitted by

1. AMAN SRIVASTAVA 71711908E


2. ANKUR AADITYA 71711917D
3. C RANGRAJAN KUMAR 71712026M

is a bonafide work carried out by them under the supervision of Prof. R. S. Kothe and it is
approved for the partial fulfillment of the requirement of Savitribai Phule Pune University for
the award of the Degree of Bachelor of Engineering (Electronics and Telecommunication
Engineering)

This project phase-1 report has not been earlier submitted to any other Institute or University
for the award of any degree or diploma.

Prof. S. M. Ingawale/ Prof. P. G.


Prof. R. S. Kothe
Chilveri
Guide Coordinator
Department of E&TC Department of E&TC

Prof. Dr. S. K. Jagtap Dr. A. V. Deshpande


Head Principal
S.K.N.C.O.E,
Department of E&TC
Pune-41

Place: Pune

Date

I
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL
NETWORK

ACKNOWLEDGEMENT
It gives us great pleasure in presenting the preliminary project report on ‘Musical
Instrument Sound Classification using Deep Convolutional Neural Network’.

I would like to take this opportunity to thank my internal guide Prof. R. S. Kothe,
Department of Electronics and Telecommunication Engineering, SMT. KASHIBAI
NAVALE COLLEGE OF ENGINEERING, for her unconditional guidance. I am really
grateful for her kind support. An erudite teacher, a magnificent person and a strict
disciplinarian, we consider ourselves fortunate to have worked under her supervision.

We are also thankful to Honorable Principal, SMT. KASHIBAI NAVALE COLLEGE OF


ENGINEERING, Pune Dr. A. V. Deshpande for the support.

We are highly grateful to Dr. S. K. Jagtap, Head of Department, Electronics and


Telecommunication Engineering, SMT. KASHIBAI NAVALE COLLEGE OF
ENGINEERING, for providing necessary facilities during the course of the work.

We admit thanks to project coordinator, Department of Electronics and Telecommunication


Engineering, for giving us such an opportunity to carry on such mind stimulating and
innovative Project.

AMAN SRIVASTAVA 71711908E

ANKUR AADITYA 71711917D

C RANGRAJAN KUMAR 71712026M

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


II
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL
NETWORK

LIST OF FIGURES

3.1 Block Diagram 14


5.1 Sample Input of Signal 21
5.2 Features of Tabla 22

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


III
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL
NETWORK

CONTENTS
CERTIFICATE I
ACKNOWLEDGEMENT II
LIST OF FIGURES III

CHAPTER TITLE PAGE NO.

1. INTRODUCTION 1
1.1 BACKGROUND 2
1.2 PROJECT UNDERTAKEN 3
1.3 OBJECTIVES 3
1.4 ORGANIZATION OF PROJECT REPORT 4
2. LITERATURE SURVEY 5
2.1 LITERATURE SURVEY 6
3. DESIGN AND DRAWING 13
3.1 SYSTEM ARCHITECTURE 14
3.2 ALGORITHMS 16
4. SYSTEM REQUIREMENT 17
4.1 HARDWARE REQUIREMENT 18
4.2 SOFTWARE REQUIREMENT 18
5. EXPERIMENTATION AND RESULT 20
6. OTHER SPECIFICATIONS 23
6.1 ADVANTAGES 24
6.2 LIMITATIONS 24
7. CONCLUSIONS 25
8. REFERENCES 27

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL
NETWORK

ABSTRACT

Acoustic is the branch of physics that deals with the study of mechanical waves such as
vibration, sound etc. It is important to find a way to classify musical instruments for obtaining
high level information about the musical signal. For this purpose, the MUMS musical
instrument sound database is used for feeding the data to machine. Sound recognition problem
consists of three different stages as pre-processing of signals, extraction of specific features
and their classification. Signal pre-processing divides the input signal to different segments
which used for extracting related features. Feature extraction reduces the size of data and
represents the complex data as feature vectors. The trimmed sound signals along with
characteristics timbre of the sound signals are fed to Convolutional Neural Network. Deep
learning methods used to perform feature extraction and classification from these signals.
CNN detects the features and builds a network to identify different instruments. All the work
including the processing of database and creation of neural network will be done using suitable
software.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL
NETWORK

CHAPTER 1

INTRODUCTION

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

1.1 BACKGROUND

Music is one of greatest creation of human mankind in course of history. Music


speaks what cannot be expressed, makes mind peaceful. Music is used not only for
entertainment and pleasure but also for psychological effects and treatments. Music
makes mind peaceful, happy and relaxed. Musical instrument classification deals
with recognition and classification of musical instruments. The demand for online
access of music data on internet is increasing day by day. Therefore, the need for
development of computational tools for the analysis, summarization, classification,
and indexing of music data has increased.

Over a decade, Music Information Retrieval (MIR) has been making efforts to
provide a reliable solution for several music related tasks. Musical instruments sound
classification is a sub-task of MIR and deals with identification of individual musi-
cal instruments and its family. Musical instrument classification problem consists of
three steps: pre-processing, feature extraction and classification. Majority of the
research on musical instrument classification is focused on feature extraction and
feature analysis, of which feature extraction is vital. Finding compact, efficient and
robust feature set is a major challenge in instrument classification. In addition, se-
lection of a suitable classifier plays an important role in improving classification
accuracy.

Automated instrument recognition has been at the core of sound research for a long
time. Taking care of the issue would permit gains in related region, for example,
automated translation, source partition and some more. In spite of the fact that it
appears to be a clear issue, computer battle in the assignment set for basic sound
recordings that comprise just of a only one instrument.

Musical Instrument Classification (MIC) is one of the major critical tasks for
acquiring the high-level information about the musical signal. The musical notes are
classified into monophonic and polyphonic. In monophonic, a single instrument is
played and in polyphonic two are more instruments are played concurrently.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


2
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

The instrument classification in monophonic notes is practically successful but polyphonic is


harder to recognize the annotation of musical signal. The challenges in MIC are the variance
in timbre and performance style of musical signal is combined with the perceptual similarity
of some other musical instrument and superposition of multiple instruments in frequency and
time. Another major issue in classifying the musical signal is choosing the best feature set
for given classification algorithm. Hence, we are choosing CNN.

1.2 PROJECT UNDERTAKEN

1.2.1 Motivation

Audio classification of the dataset, is what inspired me to try and apply CNN to
instrument recognition .In many ways, instrument classification is very similar to
object recognition in images: both contain one or more objects/instruments, both
could come in different qualities and in both the object / instrument can appear in
different positions of the image/song. As mentioned, Deep neural networks, and
CNN in particular, have shown incredible performance when faced with complex
problems. Despite that, there has been little research seeking to apply these
techniques to Automatic Musical Instrument Recognition. This project investigates
whether CNN can improve the current state-of-the-art in this field.

1.3 Objectives

• Study and analysis of different features of musical instrument sounds for ac-
curate identification of musical instruments.

• Find novel feature scheme to maximize the accuracy of musical instrument


classification.

• Classification of musical instruments using reduced feature set.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


3
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

• Selecting proper classifier model for the classification of musical instruments


to get maximum classification accuracy.

1.4 ORGANIZATION OF PROJECT REPORT

• Chapter 2 deals with the Project Related Work i.e. Literature Survey.

• Chapter 3 giving an overall view of the techniques used in the system i.e.
Design and Drawing.

• Chapter 4 deals with System Requirement.

• Chapter 5 with Experimentation and Result

• Chapter 6 with Other Specification of Project

• Chapter 7 contains conclusion and References

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


4
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

CHAPTER 2

LITERATURE SURVEY

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


5
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

2.1 LITERATURE SURVEY

1. G. Jawaherlalnehru and S. Jothilakshmi, ” Music Instrument Recogni-


tion from Spectrogram Images Using Convolution Neural Network”, In-
ternational Journal of Innovative Technology and Exploring Engineering
(IJITEE), Volume-8 Issue-9, July 2019.

Proposed system, the model tries to solve the trouble of MIR, the use of feature
gaining knowledge of system based on the network of deep convolution.
Music signal is represented by spectrograms which performance on the input
to deep CNNs. The CNN model consisting of 3 convolution and 3 absolutely
layers extract features of spectrograms and predictions of ten instrumental
classes. In this work, CNN model used to be skilled created on the
spectrograms produced for the IRMAS dataset. The results had been
performed the use of the model for most of the instruments.

2. S. Prabavathy, V. Rathikarani and P. Dhanalakshmi, “An Enhanced Mu-


sical Instrument Classification using Deep Convolutional Neural Network”,
International Journal of Recent Technology and Engineering (IJRTE),
Volume-8 Issue-4, November 2019.

Presents an enhanced deep convolutional neural network for recognizing mu-


sical instrument from large musical database. MFCC feature extraction algo-
rithm is used to extract the features from the sample musical instruments. The
extracted features are trained for predicting the instrument exactly. The pro-
posed algorithm classifies the instrument according to the extracted features
and classified correctly. The result shows that the proposed algorithm classi-
fies the musical instrument with the accuracy of 97.5%.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


6
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

3. Arun Solanki and Sachin Pandey “Music instrument recognition using


deep convolutional neural networks”, School of Information and
Communication Technology, Gautam Buddha University, Greater Noida,
Uttar Pradesh 201308, India , 2019

Proposed work, music instrument recognition using deep convolution neural


network is accomplished. The system receives input in the form sampled audio
signal which further converted Mel spectrogram form. The network receives
input in the matrix representation form of Mel spectrogram. This work is
accomplished using eight layers deep convolution neural network. The eight-
layer provide the fitting of large datasets. It was found that ReLu activation
function performs better in proposed work. Max Pooling function is used for
dimension reduction. The dropout is used for preventing the result from
overfitting. In the final layer, softmax function is used which calculate the
probability of each musical instrument used in the audio signal. On compar-
ing by with different function change it is found that the combination of ReLu
activation function and Max Pooling is giving the best result in this work.

4. Juan S. Gomez, Jakob Abeber and Estefania Cano, “Jazz Solo Instru-
ment Classification with Convolutional Neural Networks, Source Separa-
tion, and Transfer Learning”, Proceedings of the 19th ISMIR Conference,
Paris, France, September 23-27, 2018

Investigated two methods to improve upon a system for AIR on multitimbral


ensemble recordings. They first evaluated two state-of-the-art source
separation methods and showed that on multitimbral audio data, analyzing the
harmonic and solo streams can be beneficial compared to the mixed audio data.
For the specific use-case of jazz solo instrument classification, which involves
classifying six instruments with high timbral similarity, combining
solo/accompaniment source separation and transfer learning methods seems to
lead to AIR models with better generalization to unseen data.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


7
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

5. Taejin Park and Taejin Lee, “Musical Instrument Sound Classification


with Deep Convolutional Neural Network using Feature Fusion Approach”,
Electronics and Telecommunications Research Institute (ETRI), Republic
of Korea,2017

Musical instrument classification method using convolutional neural networks


(CNNs) was presented, Unlike the traditional methods, they investigated a
scheme for classifying musical instruments using the learned features from
CNNs. To create the learned features from CNNs, they not only used a
conventional spectrogram image, but also proposed multiresolution recurrence
plots (MRPs) that contain the phase information of a raw input signal.
Consequently, they fed the characteristic timbre of the particular instrument
into a neural network, which cannot be extracted using a phase-blinded
representation such as a spectrogram. By combining our proposed MRPs and
spectrogram images with a multi-column network, the performance of our
proposed classifier system improves over a system that uses only a
spectrogram.

6. Siddharth Bhardwaj,”Audio Data Augmentation with respect to Musical


Instrument Recognition”,Universitat Pompeu Fabra,2017

Identifying musical instruments in a polyphonic music recording is a difficult


yet crucial problem in music information retrieval. It helps in auto- tagging of
a musical piece by instrument, consequently enabling searching music
databases by instrument. Other useful applications of instrument recognition
are source separation, genre recognition, music transcription, and instrument
specific equalizations. We review the state of the art methods for the task,
including the recent Convolutional Neural Networks based approaches. These
deep learning models require large quantities of annotated data, a problem
which can be partly solved by synthetic data augmentation.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


8
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

7. Nevo Segal, “Automatic Musical Instrument Recognition Using Convolu-


tional Neural Networks”, MSc in Digital Signal Processing.2016

In the proposed system instrument identification with convolutional neural net-


works three different experiments were designed. For all approaches, a multi-
track classical music dataset incorporating 13 different instruments was
employed. After applying the herein developed crosstalk reduction method on
the spot microphone tracks, the individual instrument activities were first
determined to then binarize, Features were extracted in form of mel
spectrograms from the according stereo mixtures. To examine the effects of
frame size to classification performance, labels as well as features were
generated with different frame sizes. After the training phase, the learned model
is able to predict the instrument composition for the given frame size. While
the former can improve classification for small frame sizes by utilizing
classifiers which were trained on larger frames but only classify center frames,
the latter does not show any enhancements.

8. Bhalke, C. Rao, and D. Bormane, (2013) “Musical Instrument Recogni-


tion Using Higher Order Moments,” Digit. Signal Process.vol. 5, no. 4, pp.
133–138.

This paper presents a study on performance of western musical instrument


recognition using higher order moments, such as skewness and kurtosis. Higher-
order moments contain supplementary statistical information of the signal over
the conventional first or second order moments. Higher-order moments pro-
vide segment level features. Experimental result shows that low level features
integrated using higher order moments improve recognition accuracy.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


9
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

9. E. Hall, H. Ezzaidi, and M. Bahoura, (2012) “Study of Feature Categories


for Musical Instrument Recognition State of Art,” Springer-VerlagBerlin
Heidelb., pp. 152–161.

The extraction of the timbre for musical identification from the audio signal
has been subject of several researches, where various spectro-temporal param-
eters have been proposed and compared. Classification strategies are generally
based on two discrimination approaches: direct classification and hierarchical
classification. In both strategies, the feature vector is static and is used during
the whole treatment process. In this paper, author proposed a hierarchical clas-
sification where the feature vector is dynamic and changes depending on each
level and each node of the hierarchical tree. The feature vector was optimized
and was determined with the sequential backward selection (SBS) algorithm.
Using a large database (RWC), the results show, a score gain in musical in-
strument recognition performances with the proposed approach.

10. Chandwadkar, D M, Sutaone, M, (2012) “Role of Features and Classifiers


on Accuracy of Identification of Musical Instruments,” CISP2012—Proceedings,
pp. 66–70.

In this paper, the role of various features with different classifiers on automatic
identification of musical instruments is discussed. Piano, flute, trumpet, gui-
tar, xylophone and violin are identified using various features and classifiers.
Spectral features like spectral centroid, spectral slope, spectral spread, spectral
kurtosis, spectral skewness and spectral roll-off are used along with zero cross-
ing rate, autocorrelation and Mel Frequency Cepstral Coefficients (MFCC) for
this purpose. The dependence of instrument identification accuracy on these
features is studied for different classifiers. The analysis also confirms the se-
lection of features and classifiers as the results are better.

11. M. Akamineand J. Ajmera, (2012) “Decision tree-based acoustic models


for speech recognition,” EURASIP J. Audio, Speech, Music Process., vol.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


10
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

1, pp. 2-10.

In this paper, the author presented a new innovative method for quantitative
estimation of the musical instrument categories which compose a music piece.
The method uses a wavelet-based music source (i.e., musical instrument)
separation algorithm and consists of two steps. In the first step, a source
separation technique based on wavelet packets is applied to separate the
musical instruments which compose a music piece. In the second step, a
classification algorithm based on support vector machines was applied to
estimate the musical category of each of the musical instruments identified in
the first step. The method is evaluated on the publicly available Iowa Musical
Instrument Database and found to perform quite successfully.

12. E. Hall, H. Ezzaidi, and M. Bahoura, (2012) “Hierarchical Parametri-


sation and Classification for Musical Instrument Recognition,” 11th Int.
Conf. Inf. Sci. Signal Process. Their Appl. Main Tracks.

This paper presents a new acoustic model using decision trees (DTs) as re-
placements for Gaussian mixture models (GMM) to compute the observation
likelihoods for a given hidden Markov model state in a speech recognition
system. DTs have a number of advantageous properties, such as that they do
not impose restrictions on the number or types of features, and that they
automatically perform feature selection. Context-dependent DT-based models
are highly compact compared to conventional GMM-based acoustic models.

13. S. Lampropoulou, A. S. Lampropoulos, and G. A. Tsihrintzis, (2008)


“Musical Instrument Category Discrimination Using Wavelet-Based
Source Separation,” pp. 127–136.

The main research content in this paper was to examine some topic often
ignored in lastly works. Several features inspired by both the psychoacoustic
and perceptive knowledges were reexamined to explore the 16 efficiency of
the timbre dimension contribution for each feature and each category (similar

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


11
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

processing and representation form), on a large database. Also, the impact of


the normalization, time duration decomposition (short term vs. long term) and
three feature selection algorithms were examined. Using the Real-World
Computing (RWC) music database, results show that using 7 envelope
features, the score is better than using 14 LPC coefficients and lower than
using 13 MFCC coefficients. After applying Sequential Forward Selection
(SFS) re- duction technique to the observation vectors composed of all the 38
proposed features, 5 envelope features were retained when only 7 and 11
features were kept for MFCC and LPC, respectively.

14. D. Deng, C. Simmermacher, and S. Cranefield, (2008) “A Study on Fea-


ture Analysis for Musical Instrument Classification,” IEEE Trans. Syst.
Man. Cybern., vol. 38, no. 2, pp. 429–438.

This paper presents an empirical study on classical musical instrument clas-


sification. A methodology with feature extraction and evaluation is proposed
and assessed with a number of experiments. The final aim of this work was to
detect instruments in solo passages. In feature selection it is found that similar
but different rankings for individual tone classification and solo passage instru-
ment recognition are reported. Total 20 Musical instruments from 4 families
have been used for this research work.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


12
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

CHAPTER 3

DESIGN AND DRAWING

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


13
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

3.1 SYSTEM ARCHITECTURE

Figure 3.1: Block Diagram

Various attempts have been made to construct automatic musical instrument classi-
fication system. Researchers have used different approaches and scopes, achieving
different performances. But, the literature survey study states that, finding reduced
and efficient feature scheme is a major challenge for musical instrument classifica-
tion. In order to achieve this objective, we have proposed automatic musical instru-
ment classification using MFCC features using CNN for isolated musical instrument
sounds. This work can be extended for polyphonic musical instrument sound clas-
sification. In polyphonic musical instrument classification, several instruments are
played simultaneously. The timbral and similar features can be extracted and used in
same way for classification of polyphonic musical sound signals. The proposed
automatic musical instrument classification system is depicted. The input to the
system is musical instrument sound in the form of wave file and output is name of
instrument and its family. The proposed automatic musical instrument classification-
system comprises of three main stages such as pre-processing, feature extraction and
classification which has been described in the following section.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20

14
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

1. Pre-processing

In this stage, the silence part of the music sound signal is removed, which helps
to improve the time complexity of the system. The energy threshold value is
used to remove the silence part of the signal.

2. Feature extraction

Feature extraction is a vital part of the system. In this stage, acoustic features
were extracted to obtain the effective classification using CNN. Acoustic fea-
tures such as temporal, spectral, the MFCC features were constructed and the
significant statistical values of the features were used as feature components.
The detail of the feature extraction is explained below.

• Mel frequency cepstral coefficient (MFCC)

MFCC are the short time power spectral representation of a signal and
represents psychoacoustic property of the human auditory system.MFCC
has been used extensively in speech analysis over past few decades and
recently received more attention in music analysis. Here, attempts have
been made to identify and classify musical instruments using MFCC fea-
tures. Computation of MFCC features for a segment of music signal
consists of pre-emphasis, framing, windowing, FFT, triangular mel scale
band pass filtering and DCT which is described below.

Pre-emphasis The aim of pre-emphasis is to compensate the high fre-


quency formants which were suppressed during the sound production
mechanism by instruments.

Frame blocking The input musical instrument sound is segmented into


frames of 20 ms and overlapped with 10 ms.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20

15
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

3.2 ALGORITHMS:-

Convolutional Neural Networks

• Step 1: Convolution Operation


The first building block in our plan of attack is convolution operation. In this
step, we will touch on feature detectors, which basically serve as the neural
network’s filters. We will also discuss feature maps, learning the parameters
of such maps, how patterns are detected, the layers of detection, and how the
findings are mapped out.
Step 1(b): ReLU Layer
The second part of this step will involve the Rectified Linear Unit or ReLU.
We will cover ReLU layers and explore how linearity functions in the context
of Convolutional Neural Networks. Not necessary for understanding CNN’s,
but there’s no harm in a quick lesson to improve your skills.

• Step 2: Pooling
In this part, we’ll cover pooling and will get to understand exactly how it
generally works. Our nexus here, however, will be a specific type of pooling;
max pooling. We’ll cover various approaches, though, including mean (or
sum) pooling. This part will end with a demonstration made using a visual
interactive tool that will definitely sort the whole concept out for you.

• Step 3: Flattening
This will be a brief breakdown of the flattening process and how we move from
pooled to flattened layers when working with Convolutional Neural Networks.

• Step 4: Full Connection


In this part, everything that we covered throughout the section will be merged
together. By learning this, you’ll get to envision a fuller picture of how Con-
volutional Neural Networks operate and how the ”neurons” that are finally
produced learn the classification of images.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20

16
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

CHAPTER 4

SYSTEM REQUIREMENT

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


17
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

4.1 HARDWARE REQUIREMENT

• System : Windows 7/8/10

• Hard Disk : 500 GB.

• Monitor : 15”LED

• Input Devices : Keyboard, Mouse

• Ram :4GB

4.2 SOFTWARE REQUIREMENT

Matlab Version 2019: - 9.6v

MATLAB is a high-performance language for technical computing. It inte-


grates computation, visualization, and programming in an easy-to-use environment
where problems and solutions are expressed in familiar mathematical notation. Typ-
ical uses include:

• Math and computation

• Algorithm development

• Modeling, simulation, and prototyping

• Data analysis, exploration, and visualization

• Scientific and engineering graphics

• Application development, including Graphical User Interface building

MATLAB is an interactive system whose basic data element is an array that does not
require dimensioning. This allows you to solve many technical computing prob-
lems, especially those with matrix and vector formulations, in a fraction of the time
it would take to write a program in a scalar noninteractive language such as C or
Fortran.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


18
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

The name MATLAB stands for matrix laboratory. MATLAB was originally
written to provide easy access to matrix software developed by the LINPACK and
EISPACK projects, which together represent the state-of-the-art in software for
matrix computation.
MATLAB has evolved over a period of years with input from many users. In
university environments, it is the standard instructional tool for introductory and
advanced courses in mathematics, engineering, and science. In industry, MATLAB
is the tool of choice for high-productivity research, development, and analysis.
MATLAB features a family of application-specific solutions called toolboxes.
Very important to most users of MATLAB, toolboxes allow you to learn and apply
specialized technology. Toolboxes are comprehensive collections of MATLAB
functions (M-files) that extend the MATLAB environment to solve particular classes
of problems. Areas in which toolboxes are available include signal processing,
control systems, neural networks, fuzzy logic, wavelets, simulation, and many
others.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


19
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

CHAPTER 5

EXPERIMENTATION AND RESULT

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


20
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

Figure 5.1: Sample Input of Signal

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20

21
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

Figure 5.2: Features of Tabla

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20

22
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

CHAPTER 6

OTHER SPECIFICATIONS

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


23
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

6.1 ADVANTAGES

• Quick and easy searching.

• Less problems with text labeling

6.2 LIMITATIONS

1. Difficulties in sound recognition.

2. The information is not enough to represent the content of the music.

3.There are enormous amounts of music without any textual information

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


24
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

CHAPTER 7

CONCLUSION

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


25
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

Conclusion
An epic component extraction plot for programmed order of melodic instruments
sound utilizing MFCC highlights have been proposed. The Proposed highlights
expand the between-class variety and limit the inside class variety and increment the
separating capacity of the highlights with same computational multifaceted nature as
MFCC. The proposed highlights catch the dynamic time fluctuating nature of sound
sign due to trill like portion premise capacity and help to catch acoustic qualities of
music sound flag superior to MFCC. CNN has been used for prediction and
classification.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20

26
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

CHAPTER 8

REFERENCES

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


27
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL NETWORK

1. G. Jawaherlalnehru and S. Jothilakshmi, ” Music Instrument Recognition from


Spectrogram Images Using Convolution Neural Network”, International Jour- nal
of Innovative Technology and Exploring Engineering (IJITEE), Volume-8 Issue-
9, July 2019

2. S. Prabavathy, V. Rathikarani and P. Dhanalakshmi, “An Enhanced Musical


Instrument Classification using Deep Convolutional Neural Network”, Inter-
national Journal of Recent Technology and Engineering (IJRTE), Volume-8
Issue-4, November 2019.

3. Arun Solanki and Sachin Pandey”Music instrument recognition using deep


convolutional neural networks”,School of Information and Communication
Technology,Gautam Buddha University, Greater Noida, Uttar Pradesh 201308,
India , 2019

4. Juan S. Gomez, Jakob Abeber and Estefania Cano, “Jazz Solo Instrument
Classification with Convolutional Neural Networks, Source Separation, and
Transfer Learning”, Proceedings of the 19th ISMIR Conference, Paris, France,
September 23-27, 2018

5. Taejin Park and Taejin Lee, “Musical Instrument Sound Classification with Deep
Convolutional Neural Network using Feature Fusion Approach”, Electronics
and Telecommunications Research Institute (ETRI), Republic of Korea,2017

6. Siddharth Bhardwaj, “Audio Data Augmentation with respect toMusical


Instrument Recognition”,Universitat Pompeu Fabra,2017

7. Nevo Segal, “Automatic Musical Instrument Recognition Using Convolutional


Neural Networks”, MSc in Digital Signal Processing.2016

8. Bhalke, C. Rao, and D. Bormane, (2013) “Musical Instrument Recognition


Using Higher Order Moments,” Digit. Signal Process. vol. 5, no. 4, pp. 133–
138.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20

28
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL
NETWORK

9. E. Hall, H. Ezzaidi, and M. Bahoura, (2012) “Study of Feature Categories for


Musical Instrument Recognition State of Art,” Springer-Verlag Berlin Heidelb.,
pp. 152–161.

10. Chandwadkar, D M, Sutaone, M, (2012) “Role of Features and Classifiers


on Accuracy of Identification of Musical Instruments,” CISP2012—
Proceedings, pp. 66–70.

11. M. Akamineand J. Ajmera, (2012) “Decision tree-based acoustic models


for speech recognition,” EURASIP J. Audio, Speech, Music Process., vol.
1, pp. 2-10.

12. E. Hall, H. Ezzaidi, and M. Bahoura, (2012) “Hierarchical Parametrization


and Classification for Musical Instrument Recognition,” 11th Int. Conf. Inf.
Sci. Signal Process. Their Appl. Main Tracks.

13. S. Lampropoulou, A. S. Lampropoulos, and G. A. Tsihrintzis, (2008)


“Musical Instrument Category Discrimination Using Wavelet-Based Source
Separation,” pp. 127–136.

14. D. Deng, C. Simmermacher, and S. Cranefield, (2008) “A Study on Feature


Analysis for Musical Instrument Classification,” IEEE Trans. Syst. Man.
Cybern., vol. 38, no. 2, pp. 429–438.

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20


29
MUSICAL INSTRUMENT SOUND CLASSIFICATION USING DEEP CONVOLUTIONAL NEURAL
NETWORK

SKNCOE, DEPT. OF ELECTRONICS & TELECOMMUNICATION ENGINEERING – 2019-20

You might also like