Mobile Networks and Applications (2021) 26:256–265

https://doi.org/10.1007/s11036-020-01647-8

Financial Times Series Forecasting of Clustered Stocks

Felipe Affonso 1 & Thiago Magela Rodrigues Dias 1 & Adilson Luiz Pinto 2

Published online: 17 September 2020

# Springer Science+Business Media, LLC, part of Springer Nature 2020

Abstract
Predicting the stock market is a widely studied field, either due to the curiosity in finding an explanation for the behavior of
financial assets or for financial purposes. Among these studies the best techniques use neural networks as a prediction technique.
More specifically, the best networks for this purpose are called recurrent neural networks (RNN) and provide an extra option
when dealing with a sequence of values. However, a great part of the studies is intended to predict the result of few stocks,
therefore, this work aims to predict the behavior of a large number of stocks. For this, similar stocks were grouped based on their
correlation and later the algorithm K-means was applied so that similar groups were clustered. After this process, the Long Short-
Term Memory (LSTM) - a type of RNN - was used in order to predict the price of a certain group of assets. Later, predicted prices
are compared to the correct prices in order to analyze prices tendency. Results showed that clustering stocks did not influence the
effectiveness of the network, once tendency was predicted correct for an average of 48% of time. Investors and portfolio
managers can use proposed techniques to simply their daily tasks.

Keywords Financial forecasting . Clustering stocks . Artificial neural networks

1 Introduction The latest discoveries in machine learning (ML) area shows

Forecasting can be defined as the prediction of some future
event or events by analyzing the historical data [1]. Times
series forecasting plays an imperative role in several real-
world problems, such as the financial markets, network traffic,
weather forecasting, and petroleum (or oil) industry, among
others [2]. As it represents an attempt to estimate a future
event based on its past history, the financial market attracts
the attention of several researchers interested in this area of
knowledge. Some studies argue that it is possible to predict
stock market movement [3, 4] once, it is believed that today’s
stock price will be dependent on yesterday’s price.
* Felipe Affonso
felipe-affonso@hotmail.com
Thiago Magela Rodrigues Dias
thiagomagela@gmail.com
Adilson Luiz Pinto
adilson.pinto@ufsc.br
1
Centro Federal de Educação Tecnológica de Minas Gerais
(CEFET-MG), Belo Horizonte, Minas Gerais, Brazil
2
Universidade Federal de Santa Catarina (UFSC),
Florianópolis, Santa Catarina, Brazil
that algorithms can learn by themselves, recognize patterns,
find the best features for a model and many others [5]. By
using some deep learning (a ML subarea) method it is possible
to better understand financial market than just analyzing
stocks prices. Deep Neural Networks have been used several
times in order to predict the financial market movement.
Studies show that this method is obtaining 48%–54% right
predictions [6]. Therefore, it is believed that there is a wide
field of research open to improve such results. Overall, deep
learning provides some key advantages, as the ability of
performing feature abstraction and detecting complex interac-
tions between these features – resulting in state-of-the-art per-
formance across many applications [7]. In this context, recur-
rent neural network (RNN), a type of neural network, offers
the best outcomes when dealing with sequential data, for ex-
ample, stock returns.
Despite having several benefits, the application of neural
networks is a complex task, as it is necessary to have a large
amount of data and organize it in order to make it possible to
use it as an input through some deep learning algorithm. Also,
as it has several layers of neurons, the computational cost is
high, and sometimes specific equipment is often necessary for
the development of a given application. Therefore, this study
has as its main objective to estimate the movement of group of
similar stocks at some specific stock exchange. If so, investors
and fund managers will be able to carry out analyzes different
Mobile Netw Appl (2021) 26:256–265
from the traditional ones, thus facilitating their work and en-
abling new opportunities.
In order to predict the movement of a group of stocks, it
must be previously clustered together. There are several recent
studies proposing different ways of clustering stocks [8–11].
Most of them use more than one technique and focus on port-
folio management. In this paper, K-Means algorithm will be
used once it outperforms other methods in terms of clusters
compactness [8, 11]. Later recurrent neural networks are used
in order to predict clustered stocks future prices.
The main contributions of this work are: (i) propose a tech-
nique for grouping stocks based on their historical prices, (ii)
use a machine learning technique in order to predict stock
prices, as well as (iii) predict financial market trend in a future
moment of time.
The remainder of this paper is organized as follows.
Section 2 presents some necessary background in order un-
derstand some key concepts. Section 3 briefly covers some
previous studies in this area. Later, Section 4 provides an in-
depth discussion of the methodology, explaining which tech-
niques were used in each part of the process. In Section 5 the
results are presented. Finally, Section 6 presents some discus-
sions about most relevant findings and some future works.
2 Background
As explained by Krauss et al. [7], one of the motivators of
projects related to neural networks applied to finance is the
small number of articles published in this area. When
conducting a search for the term “neural networks” in The
Journal of Finance, one of the main ones in the area, only 17
results were found, while approximately 2000 articles have
been published in the last 30 years. According to the paper,
this is a unique study because they use three different state-of-
the-art machine learning techniques and compare than with a
deep learning algorithm. Also, they follow the financial liter-
ature and provide a holistic performance evaluation, making it
easy for investors to understand the results achieved. And
lastly, the authors focus on a daily investment horizon instead
of months and weeks, as normally on the literature. By using
daily values, the model can be trained with more data and
consequently perform better. The main objective of the paper
is to strengthen the links between the academic area and pro-
fessional investors, providing a new perspective for both
fields equally.
Längkvist et al. [12] explains the importance of time-series
in data mining and how it can be used as a feature to perform
better tasks in ML. By adopting unsupervised training
methods, it is possible to use time-series as an important fea-
ture of the model and consequently achieve better results. The
authors also explain that dealing with times series is an ardu-
ous task, as the data can be noisy, and usually have several
257
dimensions, so it is possible to use techniques to reduce the
complexity of the datasets, but important information can also
be lost during this process. Different algorithms are shown
and explained, each one of them present their own character-
istics and purposes. Focusing on the stock market, deep learn-
ing strategies can solve most of the problems on dealing with
such type of data and provide new approaches that have not
yet been tested.
Bini and Mathew [8] present some data mining techniques
applied in clustering and information prediction, where the
goal is finding the best companies in the market using clus-
tering techniques and prediction of future stock prices by
using regression algorithms. Among the methods used for
clustering, K-Means and Expectation Maximization algo-
rithms have shown better results when compared to hierarchi-
cal and density-based techniques. For the prediction, multiple
linear regression algorithm performed better than simple linear
regression.
Kumar and Ravi [13] describes past studies and surveys on
text mining field including several applications of these tech-
niques. They also present the advantages and disadvantages of
methods used on these researches. The paper consists of a
literature survey on different databases about the text mining
subject applied to financial domain. In order to transform a
normal data set into a structured format several techniques can
be applied. The first step is called pre-process, it plays an
important role in text mining tasks. Usually, when the pre-
processing of information is done with excellence, results tend
to be better. In a second step, feature selection must be carried
out. In this step, information considered irrelevant to the prob-
lem in question is removed, causing the data set to contain
only the most relevant fields according to the proposed
objective.
Cavalcante et al. [14] proposes a literature review on recent
approaches designed to solve financial market problems,
where the main purpose is to present machine learning
methods applied in this context published from 2009 to
2016. Besides summarizing the main studies, the authors have
also identified the key challenges in this research field. The
articles are categorized by their pre-processing, forecasting
and text mining methods. Some concepts are detailed trough
the article, as the difference between technical and fundamen-
tal analysis, and some artificial intelligence concepts, like
clustering, artificial neural networks, support vector machines,
and others. The authors also explain the advantages of using
these techniques in financial data and how it can be applied in
order to achieve better results.
3 Related works
Chong et al. [15] aims to use deep learning techniques for
financial market prediction for 38 stocks from the Korean
258
stock market. The authors compare three deep neural net-
works methods against a standard regressive model and an
artificial neural network. The authors utilize the Normalized
Mean Squared Error (NMSE), Root Mean Squared Error
(RMSE), Mean Absolute Error (MAE) and Mutual
Information (MI) as measures to evaluate the prediction per-
formance. As a result, the deep neural network performs better
on the training set, but on the test set the autoregressive model
performs better. Both methods can be joined in order to com-
bine the advantages of each one and eliminate the
disadvantages.
Fischer and Krauss [16] propose a new method for finan-
cial market predictions using deep learning with long short-
term memory (LSTM) networks. This work outperforms
Krauss et al. [7] and achieves returns of 0.46% per day. To
this end, the authors explain some concepts related to the use
of artificial neural networks, as well as some challenges in
using them, and finally proposes the use of LSTM in the
financial market. In this case, the application of this technique
meant that long-term dependencies were understood and the
results, when compared with a previous study, were
improved.
Nelson et al. [17] studies the applicability of recurrent neu-
ral networks (RNN) in order to predict stock prices movement
for four Brazilian companies. It was possible to observe that
by the application of LSTMs, the predictions outperform the
baselines with few exceptions. It proves that this technique
can offer good predictions when compared to other ap-
proaches, mainly in terms of accuracy and gains obtained.
Even when compared to other finance strategies the developed
model performs better than traditional methods and offers less
risks when observing the maximum losses.
Similar to the work presented here Shao et al. [18] seek to
use the K Means clustering algorithm to group, based on dif-
ferent time windows, a single stock exchange action. In this
way, the neural network will have several observations to
learn, but all of the same asset. From the methods presented,
the authors then compare the presented methodology and con-
clude that the use of several layers for training the network
favored and improved the performance of the model
presented.
Zhang et al. [4] created a model called Xuanwu, which
utilizes unsupervised pattern recognition methods in order to
remove the human factor in some steps of the neural network
model creation. Experimental results show that the proposed
framework outperforms the best methods in stock movement
prediction. On a more detailed explanation, the first fact is that
it is impossible for human to compete with big data algo-
rithms, so, the separation between train set and test set should
be made by the algorithm. The training sample differs from
other literature models because it works utilizing the probabil-
ity of a known shape be formed in a fixed duration of time
when only the first days of analyze were conducted. The
Mobile Netw Appl (2021) 26:256–265
authors claim that this framework can be used by small
startups that does not trade in the stock market so frequently,
the analysts could use the method to narrow down their search
for good stocks to invest in.
Comparing with related works, the present paper shows
itself using two different ML techniques, where stock ex-
change actions are made or grouped using the KMeans algo-
rithm, with this number or number of activities used by a
portfolio manager is considerably less and later, groups of
actions are used as input to a neural network. Shao et al.
[18] presents the most correlated work, however, the authors
use only one company, while 2.029 are used in the work
presented here.
4 Materials and methods
The data set used in this paper was downloaded from Kaggle.1
It contains the stock code, date, and values of close, high, low
and volume for a given date. The first step is to define the
period in which the data must be extracted. Later, it must be
analyzed in order to obtain the period that contains the biggest
number of stocks traded. Data preparation and handling was
entirely conducted in Python 3.5 [19]. The deep learning
LSTM network was developed using Keras [20].
In order for the proposed objectives to be achieved, a series
of steps is necessary: (i) initially the data set must be prepared
and organized, this process consists of downloading the infor-
mation, and filtering the period, stock exchange and stocks to
be used. (ii) The second step is to group the stocks according
to their historical price. In this way, it will be possible to use a
smaller set of observations as input to the neural network.
Therefore, the K-Means algorithm will be applied to a corre-
lation matrix previously generated from the use of Pearson’s
correlation coefficient. Once the data is grouped by its simi-
larities, neural networks algorithms will be implemented to
(iii) predict stock market prices. Finally, the results generated
in last step will be evaluated.
4.1 Clustering
Mirkin [21] defines clustering as a mathematical technique
designed for revealing classification structures in the data col-
lected in some real-world phenomena. It can also be explained
as the process in which clusters of similar objects are created
[8]. K-means clustering aims to partition n observations into k
clusters in which each observation belongs to the cluster with
the nearest mean [11]. However, for any clustering algorithm
to be applied, a data set must be used as an input. In this
1
https://www.kaggle.com/borismarjanovic/price-volume-data-for-all-us-
stocks-etfs
Mobile Netw Appl (2021) 26:256–265
specific case, the information used refers to the closing prices
of the shares in a given period.
The stock return was calculated using Eq. 1, where Rt rep-
resents the return for a specific period t. Pt is defined as the
price of the stock at time t, and Pt-1 represents the price of that
same stock at the instant just before Pt analysis [22]. In this
study, 1-day interval was used to perform the calculations.


pt
Rt ¼ ln ð1Þ
pt−1
Pearson’s correlation coefficient is used to measure how
two linear variables are correlated. The result goes from −1
to 1, the signal indicates the direction of the relationship and
the value suggests how strong it is. By other side, if the value
is zero, there is no linear correlation between the variables [3].
   
R i R j −h R i i R j
C i; j ¼ rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
D ED ffi ð2Þ
2  2 E
R i −hR i i
2
Rj− Rj
2
The formula for calculating Pearson’s correlation coeffi-
cient is presented in Eq. 2, where “Ri” represents the average
of the closing prices of a given stock, thus “RiRj” can be
explained as the average of the sum of the mean log return
values of two corresponding stocks. In other words, it is the
mean of the mean. The correlation coefficient is measured for
all pairs of stocks in both exchanges.
After performing the calculation of the chosen similarity
metrics, we must separate the observations analyzed in clus-
ters, however, it is necessary to define how many clusters will
be formed. As we are dealing with an unsupervised method of
machine learning, the definition of this number is a major
challenge found in the literature [23, 24]. Several techniques
can be applied to overcome this challenge, in this article the
“elbow method” was chosen. It consists in plotting a graph
where the y-axis represents some error metric and the x-axis
represents the number of clusters. A marked flattening of the
graph suggests that the clusters being combined are very dis-
similar, thus the appropriate number of clusters is found at the
“elbow” of the graph [25].
After obtaining the correct number of clusters, K-Means
algorithm was applied on all the correlation matrices and the
output was a file containing the stock and its correspondent
group.
4.2 Neural networks
At this moment, still addressing the artificial intelligence tech-
niques used in this work, neural networks are largely respon-
sible for the results shown in the literature. It consists of at
least three layers namely: 1) an input layer, 2) hidden layers,
and 3) an output layer [26]. Each one of these layers contains
259
some simple modules, all (or most) of which are subject to
learning, and many of which compute non-linear input–output
mappings [5]. These modules are the neurons in a network,
and in other for the information to flow through the whole
structure, they must be connected to each other. After that,
deep learning methods are applied, causing different layers
of these groups of neurons to be developed [27]. These
methods have been used to achieve state-of-the-art results on
several benchmark data sets and for solving difficult AI tasks
[12].
A recurrent neural network (RNN) is a special case of
neural network where the objective is to predict the next
step in the sequence of observations with respect to the
previous steps observed in the sequence [26], which is ex-
actly the case for stock prices. It processes an input se-
quence one element at a time, maintaining in their hidden
layers a ‘state vector’ that implicitly contains information
about the history of all the past elements of the sequence
[5]. This is the key difference in RNN, by using the state
vector, it is possible to store important information during
the learning process.
One of the biggest challenges when using this type of tech-
nique is that when it is performed several times, over several
sequential data, important information can be lost throughout
the learning process. It uses a back-propagation algorithm in
order to replicate its results through layers. Although their
main purpose is to learn long-term dependencies, theoretical
and empirical evidence shows that it is difficult to learn to
store information for very long [28]. For example, it would
be harder for an RNN to predict the next correct word for a
2000 words text than for a 10 words sentence. This situation is
called vanishing gradient problem.
LSTM is a special kind of RNNs with additional fea-
tures to memorize the sequence of data [26]. This type of
architecture is specifically designed to learn long-term de-
pendencies and overcome the previously inherent prob-
lems of RNNs, i.e., vanishing and exploding gradients
[16, 29]. It does so by keeping the error flow constant
through special units called “gates” which allows for
weights adjustments as well as truncation of the gradient
when its information is not necessary [17]. Hence, the
gates, which are based on sigmoidal neural network layer,
enable the cells to optionally let data pass through or
disposed [26], they are:
& Forget Gate: Decides the fraction of the information to be
allowed [1]. This value ranges from 0 to 1, where the
bigger it is, more information will be remembered.
& Output Gate: It combines all the information processed
inside of the cell and decides what will be transferred to
the next one.
& Memory Gate: Through some steps, it chooses which new
information is going to be maintained in the cell.
260
Table 1 Representation of the dataset before and after extraction
Exchange Number of Companies Companies After Extraction
NYSE 3.795 1.225
NASDAQ 3.066 804
5 Results
The results are presented in three stages. First, information
about the data is presented. The second step consists in using
the data set in order to cluster stocks by its similarity. Lastly,
by using the groups created in the last step, the LSTM model
is trained, and the predictions are made. The results are
discussed in Section 5.
After analyzing the dataset, it was possible to observe that
the period between 01/01/2008 and 29/03/2018 contained the
biggest portion of the data. The result of the extraction is
shown in Table 1, where both exchanges had their data re-
duced drastically.
5.1 Clustering
In order to illustrate the results obtained during this part of the
process, a correlation matrix was plotted, and it is presented in
Fig. 1. For each stock, in each stock exchange, the correlation
between all of them was measured. It is possible to confirm
that the main diagonal equals to one and the rest of the matrix
is between −1 and 1. Therefore, it is possible to use this result
to group similar stocks.
The elbow method was used to obtain the optimal number
of clusters. It consists in plotting a graph, where it is possible
to check the number of clusters versus the generated error, in
this case Within-Cluster Sum of Squares was used. It is im-
portant to emphasize that we cannot just choose the biggest
Fig. 1 Correlation Matrix. a) NASDAQ (Left); b) NYSE (Right)
Mobile Netw Appl (2021) 26:256–265
number of clusters, with the minimum error value, because it
would cause an over-fit.
Using the information generated by K-Means algorithm,
the data was grouped using simple average, which means that
for each day the values of all group components where
summed and then divided by the number of components, it
was done for every day since 01/01/2008 in both exchanges.
The Table 2 shows the number of files in the beginning of the
process, and the number of groups for each exchange. By
doing that, instead of conducting 2.029 predictions, it will be
necessary to run the model through 40 clusters.
5.2 Predicting values
Prior to executing the predictions, the two sets of data were
sliced, where a part was separated to execute the training of
the algorithms, and another part for testing. In this way it is
possible to validate if there really was a learning experience,
therefore the division was done by selecting 80% of the data
for training and 20% for testing. After going through scaling
process, each group of stocks was trained and tested separate-
ly. An LSTM network was created with the aim of predicting
future stock prices considering the previous values. It was
developed using the mean squared error (MSE) as the loss
function, “adam” optimizer and 10 epochs, which means that
each group of stocks was used ten times, in different patterns,
for training.
After running the prediction algorithms on the selected data
set, the results were generated and are shown in Table 3 sorted
by their respective MSE for both exchanges. It is possible to
observe that the error is small, and that the groups had a sim-
ilar performance. The results are also shown in Fig. 2, for
groups formed by NASDAQ, and in Fig. 3, composed of
groups formed by NYSE. In both cases the blue line repre-
sents the real value, and the orange line represents the predic-
tions made.
Mobile Netw Appl (2021) 26:256–265 261

Table 2 Number of Groups

Exchange Companies Companies After Extraction Number of Groups

NYSE 3.795 1.225 20

NASDAQ 3.066 804 20

After running the predictions in the neural network, it was
observed that, many times, the algorithm misses the exact
value of the prediction, but the trend seems to go in the right
direction. Thus, the separate vectors for testing were used to
assess whether the results indicated a bullish or bearish market
for the following period. Therefore, two vectors were gener-
ated using this data, one with the real values, and the other
with the values predicted by the network and they are com-
pared against each other. The results of this analysis are also
shown in Table 3, where the tendency indicates whether the
trend of the predictions was right or not. It is important to note
that no specialized classification algorithms were used for this
purpose. In other words, the data presented is based only on
the values predicted by the neural networks in the previous
step, where it was analyzed the number of times that predicted
value corresponded to the real one.
When comparing the results of the two exchanges, the re-
sults for Nasdaq showed an average error of 0.011127, and the
trend was correct 47.80% of the time. On the NYSE, the
average error was 0.003407, and the trend was correct
49.26% of the time, demonstrating that this exchange
achieved the best results, especially considering the MSE.
However, the trend did not show such a significant difference.
Considering the trend of predictions, the groups that ob-
tained the best and worst results from each exchange will now
be analyzed. In this case, the best group for Nasdaq was found
in group 1, and the worst in group 12. It is interesting to note
that, even having the highest success rate related to trend,
group 1 showed the worst result when using the MSE.
When analyzing Fig. 2, it is possible to make some observa-
tions about what happened. The prediction and the correct
value distanced themselves at the beginning, thus increasing
the difference between the predicted and the real. However,
the algorithm was able to follow the movements performed by
the actions. Already considering the worst result related to the
trend, group 12 shows the eighth smallest error, however, this
group of shares showed no market direction, after a huge drop
in the beginning of the process, it presented an stable move-
ment, making it difficult to predict the trend.
This time analyzing the best and the worst result taking into
account the proportion of errors related to the trend, the NYSE
group 5 hit the market direction 56.14% of the time, with an

Table 3 Results for both

Exchanges Exchange Group MSE Tendency Exchange Group MSE Tendency

Nasdaq Group 10 0.000003 50.29% NYSE Group 2 0.000150 50.29%

Nasdaq Group 16 0.000004 50.29% NYSE Group 12 0.000338 50.10%
Nasdaq Group 13 0.000005 47.95% NYSE Group 7 0.000385 43.27%
Nasdaq Group 7 0.000784 47.75% NYSE Group 10 0.000434 52.63%
Nasdaq Group 4 0.000862 45.80% NYSE Group 5 0.000495 56.14%
Nasdaq Group 19 0.001014 47.36% NYSE Group 1 0.000744 46.78%
Nasdaq Group 15 0.001016 48.34% NYSE Group 18 0.001026 54.58%
Nasdaq Group 12 0.001507 43.46% NYSE Group 9 0.001077 48.73%
Nasdaq Group 8 0.001739 50.48% NYSE Group 16 0.001123 45.61%
Nasdaq Group 18 0.001979 45.22% NYSE Group 17 0.001223 50.68%
Nasdaq Group 2 0.002017 47.17% NYSE Group 11 0.001275 47.95%
Nasdaq Group 3 0.002117 44.44% NYSE Group 14 0.001479 49.12%
Nasdaq Group 11 0.002658 48.92% NYSE Group 8 0.002089 49.71%
Nasdaq Group 14 0.003343 46.39% NYSE Group 6 0.002255 47.76%
Nasdaq Group 9 0.003704 48.34% NYSE Group 4 0.003121 45.22%
Nasdaq Group 0 0.004064 48.92% NYSE Group 19 0.004496 46.78%
Nasdaq Group 6 0.004315 44.44% NYSE Group 15 0.005739 43.27%
Nasdaq Group 17 0.005058 48.34% NYSE Group 0 0.005979 50.29%
Nasdaq Group 5 0.005611 49.90% NYSE Group 13 0.006704 52.24%
Nasdaq Group 1 0.180757 52.24% NYSE Group 3 0.028008 54.19%
262 Mobile Netw Appl (2021) 26:256–265
Group_10 Group_16 Group_13 Group_13
$0

$0 $2
$0 $0
$1
Scaled Price

Scaled Price
$0

Scaled Price
Scaled Price
$0 $1
$0 $0

$0 $1

$0 $1 $0 $0

$0 $1
$0 $0
$0 $1

$0
0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500

Period Period Period Period

Group_4 Group_19 Group_15 Group_12

$4 $0

$1 $0 $4 $0

$1 $3 $0

Scaled Price
Scaled Price

Scaled Price

Scaled Price
$0
$1 $3 $0

$1 $3 $0
$0
$1 $2 $0

$1 $0 $2 $0

$1 $2 $0
$0

0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500

Period Period

Group_8 Group_18 Group_2 Group_2

$1 $1
$2 $2

$1 $1

$1 $1 $2 $2
Scaled Price

Scaled Price
Scaled Price

Scaled Price
$1 $0
$2 $1
$0 $0

$0 $0
$1 $1
$0 $0

$0 $0 $1 $1

$0 $0
0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500

Period Period Period Period

Group_11 Group_14 Group_9 Group_0

$2
$1 $1 $2
$2
$2
$1 $1
Scaled Price

$2
Scaled Price

Scaled Price
Scaled Price

$1 $1
$1 $1
$1
$1
$1 $1 $1

$1 $0

$0 $0
$0
0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500

Period Period Period Period

Group_6 Group_17 Group_5 Group_1

$0
$1
$2
$1
$1 $0
Scaled Price
Scaled Price

Scaled Price

Scaled Price

$2

$1 $1
$0 $1

$1
$1 $1
$0
$1
$1
$0
$1 $0
$1

0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500

Period Period Period Period

Predicted Actual

Fig. 2 Predictions for Nasdaq

MSE of 0.000434, this being the best prediction of the study. two analyzes can be made. At first, group 7 presents the third
Accomplished. When analyzing Fig. 3, it is possible to notice smallest error of this exchange, however, as can be seen in
that this group of stocks showed an upward behavior, where at Fig. 3, there are significant drops in value at times, but most of
different times the stock prices rose for several consecutive the time prices rise and fall for small periods of time., that is,
periods, thus demonstrating a strong bull market. Still analyz- they do not present a definite trend. Group 15, on the other
ing the NYSE, groups 7 and 15 presented the worst results hand, has a similar behavior, but with significant increases in
related to the trend, both with 43.27% of correct answers, so value and later a lateral behavior.
Mobile Netw Appl (2021) 26:256–265 263
Group_12 Group_7 Group_10
Group_2
$1 $1
$2
$1 $1
$0 $2
$1 $1

Scaled Price

Scaled Price
Scaled Price

Scaled Price
$0 $1
$1 $1
$1
$0 $0 $1
$1
$0 $0 $1
$1
$0 $1
$0
$1
$0 $1
$0 $1
0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500
0 100 200 300 400 500

Period Period Period Period

Group_5 Group_1 Group_18 Group_9

$1 $2
$0
$0 $2
$1
Scaled Price

Scaled Price

Scaled Price
Scaled Price

$0
$1
$1
$0
$1 $0
$1

$1 $0
$0 $1
$1
$1 $0
$0 $1
$1 $0

0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500
E
Period Period Period Period

Group_16 Group_17 Group_11 Group_14

$0
$1 $2 $2
$0
$1 $2
$2

Scaled Price
Scaled Price

Scaled Price
Scaled Price

$0
$1
$2 $2
$0
$1
$1 $1
$0
$1

$1 $1
$0
$0

$0 $1 $1

0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500

Period Period Period Period

Group_8 Group_6 Group_4 Group_19

$6 $0 $5

$1
$0
$5 $4
Scaled Price
Scaled Price

Scaled Price

Scaled Price
$1 $0
$4 $4

$1 $0
$4 $4

$0
$1
$4 $3
$0

0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500

Period Period Period Period

Group_15 Group_0 Group_13 Group_3

$0
$1 $1 $2
$0
$1
$0 $1
Scaled Price
Scaled Price

Scaled Price
Scaled Price

$1
$1
$0
$0 $1
$1
$0
$0
$0 $1
$1 $0
$0
$0 $1
$0 $0
$0
$1
$0
0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500

Period Period Period Period

Predicted Actual

Fig. 3 Predictions for NYSE

6 Discussion Means algorithm. Finally, an LSTM network was applied in

the data set with the purpose of predicting the value of a group
We have shown how two different machine learning tech- of similar stocks. The process was done successfully, and it
niques can be used in pursuit of a single goal. The raw data proves that stocks can be clustered using K-Means algorithm
was organized, and important information was extracted. The in a way that the network prediction capacity did not suffer
rest of the stocks were analyzed in order to find the most any impact, making it possible to manage a large group of
similar ones, and the clusters were generated using the K- stocks using less computational power.
264
Taking into account the trend of the predictions made, an
average of 48.53% of correct values was reached. Where, in
one of the groups, the right trends were predicted 56.14% of
the time, and the worst group having a hit rate of 43.27%.
Such values are within the literature and demonstrate that
there is still much to be studied when predicting stock prices
on the stock exchange.
These results can have a huge impact for investors and
portfolio managers. By using the proposed techniques, it is
possible to buy stocks that have the same behavior. Thus, less
time and less computational power will be necessary in order
to evaluate a whole group of companies. However, even using
LSTMs, specialized in forecasting times series, the results still
need to be improved before being used by finance profes-
sionals. Predicting stock prices is a major challenge for any
technique, tool or professional, since most of the time, it de-
pends on the psychological factor of buyers and sellers in a
given period. While neural networks, in this case, are analyz-
ing only the price of the previous day, along with some “hid-
den” factors, investors analyze news, balance sheets released
by companies and several other scenarios. However, the
values found here are within those presented in the literature,
demonstrating that the methodology was successfully imple-
mented and can be replicated.
Potential directions of future work include analyzing the
impact of different similarity measures and how it can influ-
ence the process of clustering stocks. Also, when dealing with
clustering, it will be necessary to study further options and
algorithms that better represent the similarity between stocks.
To increase the results of the predictions, a new dimension can
be added to the data, adding information related to the news,
for example, making the prediction not only depend on the
stock price, but that there is some type of monitoring of the
psychological factors that influence share purchase and sale
decisions.
Availability of data and material The dataset used is available
at: https://www.kaggle.com/borismarjanovic/price-volume-
data-for-all-us-stocks-etfs
Compliance with ethical standards
Conflicts of interest/competing interest Not Applicable.
Code availability The code developed is available at: https://github.
com/felipe-affonso/Financial-Times-Series-Forecasting-MONET
References
1. Selvin S, Vinayakumar R, Gopalakrishnan EA, Menon VK, Soman
KP (2017) Stock price prediction using LSTM, RNN and CNN-
sliding window model. In 2017 international conference on
Mobile Netw Appl (2021) 26:256–265
advances in computing, communications and informatics (icacci)
(pp 1643-1647). IEEE
2. Sagheer A, Kotb M (2019) Time series forecasting of petroleum
production using deep LSTM recurrent networks. Neurocomputing
323:203–213
3. Filho DBF, Júnior JAdS (2009) Desvendando os mistérios do
coeficiente de correlacão de pearson (r). Universidade Federal de
Pernambuco
4. Zhang J, Cui S, Xu Y, Li Q, Li T (2018) A novel data-driven stock
price trend prediction system. Expert Syst Appl 97:60–69
5. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature
521(7553):436
6. Gerlein EA, McGinnity M, Belatreche A, Coleman S (2016)
Evaluating machine learning classification for financial trading:
an empirical approach. Expert Syst Appl 54:193–207
7. Krauss C, Do XA, Huck N (2017) Deep neural networks, gradient-
boosted trees, random forests: statistical arbitrage on the S&P 500.
Eur J Oper Res 259(2):689–702
8. Bini BS, Mathew T (2016) Clustering and regression techniques for
stock prediction. Procedia Technol 24:1248–1255
9. Jung SS, Chang W (2016) Clustering stocks using partial correla-
tion coefficients. Phys A: Statistical Mech Appl 462:410–420
10. Momeni M, Mohseni M, Soofi M (2015) Clustering stock market
companies via K-means algorithm. Kuwait Chapter Arabian J
Business Manag Rev 4(5):1–10
11. Nanda SR, Mahanty B, Tiwari MK (2010) Clustering Indian stock
market data for portfolio management. Expert Syst Appl 37(12):
8793–8798
12. Längkvist M, Karlsson L, Loutfi A (2014) A review of unsuper-
vised feature learning and deep learning for time-series modeling.
Pattern Recogn Lett 42:11–24
13. Kumar BS, Ravi V (2016) A survey of the applications of text
mining in financial domain. Knowl-Based Syst 114:128–147
14. Cavalcante RC, Brasileiro RC, Souza VL, Nobrega JP, Oliveira AL
(2016) Computational intelligence and financial markets: A survey
and future directions. Expert Syst Appl 55:194–211
15. Chong E, Han C, Park FC (2017) Deep learning networks for stock
market analysis and prediction: methodology, data representations,
and case studies. Expert Syst Appl 83:187–205
16. Fischer T, Krauss C (2018) Deep learning with long short-term
memory networks for financial market predictions. Eur J Oper
Res 270(2):654–669
17. Nelson DM, Pereira AC, de Oliveira RA (2017) Stock market's
price movement prediction with LSTM neural networks. In neural
networks (IJCNN), 2017 international joint conference on (pp
1419-1426). IEEE
18. Shao X, Ma D, Liu Y, Yin Q (2017) Short-term forecast of stock
price of multi-branch LSTM based on K-means. In 2017 4th
International Conference on Systems and Informatics (ICSAI) (pp
1546-1551). IEEE
19. Python Software Foundation (2018) Python 3.5.5 documentation.
Available at https://docs.python.org/3.5/
20. Chollet, F. (2015). Keras. Available at https://keras.io
21. Mirkin BG (1996) Mathematical classification and clustering.
Kluwer Academic Publishing, Dordrecht
22. Affonso F, de Oliveira F, Dias TMR (2017) Uma Análise dos
Fatores que Influenciam o Movimento Acionário das Empresas
Petrolíferas. In Ibero-Latin Am Congress Computat Methods Eng
(CILAMCE)
23. Gan G, Ma C, Wu J (2007) Data clustering: theory, algorithms, and
applications (Vol. 20). Siam
24. Liu Y, Li Z, Xiong H, Gao X, Wu J (2010) Understanding of
internal clustering validation measures. In ICDM:911–916
25. Ketchen DJ, Shook CL (1996) The application of cluster analysis in
strategic management research: an analysis and critique. Strateg
Manag J 17(6):441–458
Mobile Netw Appl (2021) 26:256–265
26. Siami-Namini S, Tavakoli N, Namin AS (2018) A comparison of
ARIMA and LSTM in forecasting time series. In 2018 17th IEEE
International Conference on Machine Learning and Applications
(ICMLA) (pp. 1394-1401). IEEE
27. Li Y, Jiang W, Yang L, Wu T (2018) On neural networks and
learning systems for business computing. Neurocomputing 275:
1150–1159
265
28. Bengio Y, Simard P, Frasconi P (1994) Learning long-term depen-
dencies with gradient descent is difficult. IEEE Trans Neural Netw
5(2):157–166
29. Hochreiter S, Schmidhuber J (1997) Long short-term memory.
Neural Comput 9(8):1735–1780
Publisher’s Note Springer Nature remains neutral with regard to jurisdic-
tional claims in published maps and institutional affiliations.