Prediction using Machine Learning techniques
(LSTM MODEL)
Rohith Anil Kumar Sai Anirudh Guduru
Computer Science and Engineering Computer Science and Engineering
specializing in Information Security Vellore Institute of Technology,
Vellore Institute of Technology,
Vellore
Abstract—The prediction of time series data is very
difficult. Due to the various environmental factors such as
global warming the temperature, pressure and various
components of the planet is found to increase every year.
Based on certain features, we can predict the temperature of
a time in future. Therefore, we can make the necessary
changes and take enough precautions to reduce the
temperature and other components of the Earth. To ensure
this, we can create some sort of a time series forecasting
machine learning model, that will allow us to predict the
temperature and other components of the earth in future.
The traditional recurrent neural network RNN wants to
remember all the knowledge whether it is useful or not, so
there may be problems such as gradient disappearance and
gradient explosion.
Keywords—recurrent neural networks, long short-term
memory model (LSTM model), root mean square error,
preprocessing, dataset, training
I. INTRODUCTION
A time series is a set of repeated measurements of the
same phenomenon taken sequentially over time. Time is
usually the independent variable in a time series. The
temperature data represents temperature anomalies. We
will not be working with absolute temperature data as in
climate change studies, anomalies are more important than
absolute temperature. From the given dataset, various
features that have good correlation with the feature that is
predicted would be used by a machine learning prediction
model. In this way, the temperature or pressure features
can be predicted before-hand based on a test data set and
necessary measures can be taken by organisations like the
UN to reduce the variation. Some of the common
prediction models are clustering models, classification
model, neural network (convolutional and recurrent), time
series model. Here, a variation of the generic recurrent
neural network model is used as the prediction model and
it is the LSTM RNN model. A common LSTM unit is
composed of a cell, an input gate, an output gate and a
forget gate. The cell remembers values over arbitrary time
intervals and the three gates regulate the flow of
information into and out of the cell.
II. PROBLEM STATEMENT
The main idea is to predict the values of any important
feature like temperature for a forecast dataset. For this a
good machine learning model must be chosen with a low
RMSE value. The existing common prediction models
such as recurrent neural networks and convolutional
neural networks have certain shortcomings. LSTM
networks are an extension of recurrent neural networks
Thodunupuru Sahithi
Computer Science and Engineering
Vellore Institute of Technology,
Vellore Vellore
mainly introduced to handle situations where RNN’s fail.
RNN’s fail to store information for a longer period of
time. At times, a reference to certain information stored
quite a long time ago is required to predict the current
output. But RNN’s are absolutely incapable of handling
such “long-term dependencies” Another problem of RNN
is that they do not have control over which part of the
context needs to be carried forward and how much of the
past needs to be forgotten. One of the most prominent
issues with RNNs are exploding and vanishing gradients
which occur during the training process of a network
through backtracking. The problem with CNN is that for
smaller datasets the accuracy of prediction is found to be
low. While model fitting the CNN model was required to
need more epochs, and hence its performance reduced for
smaller datasets.
III. PROPOSED METHODOLOGY
LSTM is designed in such a manner that the vanishing
gradient problem is almost completely removed, while the
training model is left unaltered. Long time lags in certain
problems are bridged using LSTMs where they also
handle noise, distributed representation, and continuous
values. LSTM take care of RNN’s long term dependency
problem. LSTMs have feedback connections which make
them different to more traditional feedforward neural
networks. This property enables LSTMs to process entire
sequences of time series data without treating each point
in the sequence independently but rather retaining useful
information about previous data in the sequence to help
with the processing of new data points. Also, the
performance and accuracy for both larger datasets and
smaller datasets is good for LSTMs and this overcomes
the drawback of using the CNN model for smaller
datasets.
The LSTM model is chosen to predict the temperature.
The behaviour of the preceding X years helps us forecast
what the current temperature/future temperature would be.
LSTMs are better at identifying complex pattern logics
from data by remembering what’s useful and what’s not.
Programming language that is going to be used is Python.
Some of the libraries that have to be used are:
Pandas: for data manipulation and analysis
Numpy: for working with arrays
Seaborn: data visualization library based on matplotlib
Matplotlib: plotting library for python programming
language
Plotly: Scientific graphing library
Tensorflow: used to create machine learning models
Keras: Keras is used for creating deep models which can
be productized on smartphones. It is also used for
distributed training of deep learning models. Keras is the
high-level API of Tensorflow 2.
DATASET USED:
https://storage.googleapis.com/tensorflow/tf-keras-
datasets/jena_climate_2009_2016.csv.zip
Number of records: 420551
Number of features: 15
First of all, the necessary libraries are imported. The
dataset is then loaded. It is a csv file and is read using
pandas. The number of rows in the chosen dataset is
420551 and the number of columns is 15. Pre-processing
of this dataset then takes places. Starting from the fifth
row every sixth row is taken and another smaller dataset is
created. This is then stored as another data frame. The first
column which is the index column is changed to the Date
time column. From plotting the temperature on a graph,
we understand that the variations in temperature follow a
particular pattern. This point is kept in mind while training
the machine learning model.
For deep learning models, data should always be in an
input matrix. Each row is taken as the input and has its
corresponding label. The data is then reshaped to the form
which is required for training the machine learning model.
For training the first 60000 rows would be used. The rest
of the rows would be used for testing. We plot the graph
to see the training data and testing data. The model is then
created by adding the necessary layers. Initially we create
a model with the LSTM layer present. The LSTM layers
contain a Dropout to prevent overfitting in the model. The
output layer consists of a dense layer with 1 neuron with
activation as ReLU. We compile this model with loss
calculated as mean squared error and the optimizer used is
Adam optimizer. After compiling, model fitting is done.
The feature that is used to for training the machine
learning model is temperature and the feature that is
predicted is also the temperature feature. Then the test
data set is given to the machine learning model to predict
the values and result is represented in a tabular format.
The predicted values are then compared with the actual
values and are plotted graphically for better
understanding. The RMSE value for the values predicted
by the machine learning model is calculated and displayed
along with the graph. Here, the LSTM model is the
univariate LSTM model.
The same dataset is pre-preprocessed, compiled with CNN
layer. The output layer consists of a dense layer with 1
neuron with activation as ReLU. We compile this model
with loss calculated as mean squared error and the
optimizer used is Adam optimizer. After model fitting is
done, test data is given to the machine learning model to
predict the values and the result is represented in a tabular
format. The predicted values are then compared with the
actual values and are plotted graphically for better
understanding. The RMSE value for the values predicted
by the machine learning model is calculated and displayed
along with the graph.
The same dataset is pre-processed, compiled with LSTM
layer. The LSTM layers contain a Dropout to prevent
overfitting in the model. The output layer consists of a
dense layer with 1 neuron with activation as ReLU. We
compile this model with loss calculated as mean square
error and the optimizer used is Adam optimizer. After
compiling, model fitting is done. The features that are
used for training the machine learning model is
temperature and timestamp which is processed and made
into four other features. These features are the date time
converted to Day sin, Day cos, Year sin, Year cos. The
periodic change in temperature can be noticed from the
graph which has plotted date-time against temperature.
This is why temperature along with date-time is
considered for training. The test data values are fed into
the machine learning model to predict the values and the
result is represented in a tabular format. The predicted
values are then compared with the actual values and are
plotted graphically for better understanding. The RMSE
value for the values predicted by the machine learning
model is calculated and displayed along with the graph.
The same process is again repeated for another LSTM
machine learning model where the features used for
training the model are temperature and pressure. The
RMSE value for the values predicted by the machine
learning model is calculated and displayed along with the
graph.
To test the performance, the machine learning models
(univariate LSTM and univariate CNN) mentioned have
also been tested with smaller datasets. The layers added to
 the model here are only the LSTM layer with the ReLU temperature values. The RMSE value is
activation function and a Dense layer. We compile this calculated based on the 5086 rows and 2
model with loss calculated as mean squared error and the columns. The graph plot shows the actual and
optimizer used is Adam optimizer. After model fitting is predicted values of the temperature for the first
done, test data is given to the machine learning model to 100 instances.
predict the values and the result is represented in a tabular
format. The predicted values are then compared with the
actual values and are plotted graphically for better
understanding. The RMSE value for the values predicted
by the machine learning model is calculated and displayed
along with the graph.

IV. RESULTS
• When the univariate LSTM model is used to
predict the temperature by only taking the
temperature as a feature, the RMSE value is
found to be 0.625. The orange color plotted in
the graph shows the actual value and the blue
color plotted in the graph shows the predicted
value of the model. The X axis shows the index
number of the temperatures and the Y axis shows
the temperature values. The RMSE value is
calculated based on the 5086 rows and 2
columns. The graph plot shows the actual and
predicted values of the temperature for the first
100 instances.

• When the multivariate LSTM model is used to

predict the temperature by taking the temperature
as one feature and date-time divided into four
other features, the RMSE value is found to be
0.504. The orange color plotted in the graph
shows the actual value and the blue color plotted
in the graph shows the predicted value of the
model. The X axis shows the index number of
the temperatures and the Y axis shows the
temperature values. The RMSE value is
calculated based on the 5085 rows and 2
columns. The graph plot shows the actual and
predicted values of the temperature for the first
100 instances.

• When the CNN model is used to predict the

temperature by only taking the temperature as a
feature, the RMSE value is found to be 0.6384.
The orange color plotted in the graph shows the
actual value and the blue color plotted in the
graph shows the predicted value of the model.
The X axis shows the index number of the
temperatures and the Y axis shows the

• When the multivariate LSTM model is used to
predict the temperature by taking the temperature
as one feature and pressure as the other feature,
the RMSE value is found to be 0.584. The orange
color plotted in the graph shows the actual value
and the blue color plotted in the graph shows the
predicted value of the model. The X axis shows
the index number of the temperatures and the Y
axis shows the temperature values. The RMSE
value is calculated based on the 5084 rows and 2
columns. The graph plot shows the actual and
predicted values of the temperature for the first
100 instances.
• When very small datasets are used to train the
LSTM recurrent neural network model, the
accuracy of the model when the model fitting is
done for 200 epochs is very good. The same
accuracy for the same dataset can be achieved by
the CNN machine learning model when the
model fitting is done for 1000 epochs. But even
in this case the accuracy of the LSTM model is
found to be more as the predicted value is closer
to the actual value.
LSTM MODEL
CNN MODEL
V. CONCLUSION
From the RMSE values of univariate models of CNN and
LSTM, LSTM is found to have lower RMSE value and
hence, it is said to have better accuracy. The accuracy of
LSTM models is good for both large datasets and smaller
datasets while the accuracy of CNN models are low for
smaller datasets and for CNN the model requires more
training to achieve the same accuracy as that of a LSTM
model for lesser training. The LSTM is said to solve
vanishing gradient and explosive gradient problem of the
recurrent neural networks, hence achieving a good
accuracy for a prediction model.
For the RMSE values of the multivariate models of
LSTM, the model in which the features used for training
are temperature and four other features derived from date-
time feature are said to have lower RMSE value as
compared to the LSTM model in which the features used
for training are temperature and pressure. Hence, the
model with temperature and four other features derived
from date-time as said to have better correlation to the
label which is temperature.
Also, from the RMSE values of LSTM (univariate and
multivariate), we can understand that multivariate models
give lesser RMSE value compared to univariate LSTM
models. Hence, the predicted values provided by them is
more accurate.
From the above results, it can be concurred that the LSTM
model is a better predictive model than the CNN model
and generic RNN model (as it takes care of the
dependency problem in RNN while doing predictive
analysis and also as it takes care of the
vanishing/explosive gradient problem).
VI. REFERENCES
1. Li, Xiangang; Wu, Xihong (2014-10-15).
"Constructing Long Short-Term Memory based
Deep Recurrent Neural Networks for Large
Vocabulary Speech Recognition".
2. Graves, A.; Liwicki, M.; Fernández, S.;
Bertolami, R.; Bunke, H.; Schmidhuber, J. (May
2009). "A Novel Connectionist System for
Unconstrained Handwriting Recognition". IEEE
Transactions on Pattern Analysis and Machine
Intelligence
3. Tax, N.; Verenich, I.; La Rosa, M.; Dumas, M.
(2017). Predictive Business Process Monitoring
with LSTM neural networks. Proceedings of the
International Conference on Advanced
Information Systems Engineering (CAiSE).
Lecture Notes in Computer Science. Vol.
4. Thireou, T.; Reczko, M. (2007). "Bidirectional
Long Short-Term Memory Networks for
predicting the subcellular localization of
eukaryotic proteins". IEEE/ACM Transactions
on Computational Biology and Bioinformatics.
5. Zhao, Z.; Chen, W.; Wu, X.; Chen, P.C.Y.; Liu,
J. (2017). "LSTM network: A deep learning
approach for Short-term traffic forecast". IET
Intelligent Transport Systems.
6. Klaus Greff; Rupesh Kumar Srivastava; Jan
Koutník; Bas R. Steunebrink; Jürgen
Schmidhuber (2015). "LSTM: A Search Space
Odyssey". IEEE Transactions on Neural
Networks and Learning Systems.
7. A. Vohra, N. Pandey and S. K. Khatri, "Decision
Making Support System for Prediction of Prices
in Agricultural Commodity," 2019 Amity
International Conference on Artificial
Intelligence (AICAI), 2019, pp. 345-348,
doi:10.1109/AICAI.2019.8701273.
8. T. M. N. P. Karunarathna et al., "Consumer and
Farmer Centric Subscription Based Organic
Vegetable/Fruit Delivery System," 2021 3rd
InternationalConference on Advancements in
Computing (ICAC), 2021, p p. 109-115,
doi:10.1109/ICAC54203.2021.9671159.
9. D. K, R. M, S. V, P. N and I. A. Jayaraj, "Meta-
Learning Based Adaptive Crop Price
prediction for Agriculture Application," 2021 5th
International Conference on Electronics,
Communication and Aerospace Technology
(ICECA2021, pp. 396-402, doi :
10.1109/ICECA52323.2021.9675891.
10. G. S. Sajja, S. S. Jha, H. Mhamdi, M. Naved, S.
Ray and K. Phasinam, "An Investigation on Crop
Yield Prediction Using Machine Learning," 2021
Third International Conference on Inventive
Research in Computing Applications(ICIRCA),
2021, pp. 916-921, do :
10.1109/ICIRCA51532.2021.9544815.
11. D. Elavarasan and P. M. D. Vincent, "Crop Yield
Prediction Using Deep Reinforcement Learning
Model for Sustainable Agrarian Applications," in
IEEE Access, vol. 8, pp. 86886-86901, 2020,
doi: 10.1109/ACCESS.2020.2992480.