Deep learning-based feature engineering for stock

price movement prediction

Tuesday 12th January, 2021 (19:43)

1st Abhishek Tripathi 2nd Amitoj Singh Kalra 3rd Aman

ECE 00113202817 ECE 00713202817 ECE 00513202817
Guru Tegh Bahadur Institute Guru Tegh Bahadur Institute Guru Tegh Bahadur Institute
of Technology of Technology of Technology
Delhi, India Delhi, India Delhi, India

4th Wen Long 5th Zhichen Lu 6th Lingxiao Cui

School of Economics Management, Research Center on Fictitious Economy Key Laboratory of Big Data Mining
University of Chinese Academy of Sciences and Data Science, and Knowledge Management,
Beijing, China Chinese Academy of Sciences Chinese Academy of Sciences
Beijing, China Beijing, China

Abstract—Stock price prediction and its modeling have been
always a matter of challenge for the speculators, researchers, and
experts. It is due to the characteristics of the samples. With the
purposely designed network and broadening of deep learning, the
task of feature learning can be performed with more efficiency.
This paper proposed an end to end model for price movement
prediction of task and sampling of series on financial time.
The model is named MFNN i.e Multi Filters Neural Network.
The paper featured the engineering of financial time series and
deep learning for feature learning. Backed up with mathematical
theories, data sets, and statistical tables, the model is applied
to the China stock market index for the prediction of extreme
market and signal-based trading simulation.
Index Terms—Stock price prediction, Feature engineering,
Deep learning
I. I NTRODUCTION
Stock Price forecasting plays a vital role in defining the
trading strategies in order to get the opportunity to trade i.e
buy and sell a stock. Financial forecasting has been a difficult
problem for speculators and researchers [1]. Due to inter-
pretability and comprehensibility, statistical methods such as
linear and autoregression, GACH [2], moving average(AMRA
[3]) was much favorable for financial especially stock price
forecasting[4]–[6]. Apart from the above-mentioned traditional
statistical methods, machine learning algorithms such as ran-
dom forest(RF), logistic regression(LF), k-nearest neighbors,
naive Bayes(NB) is also used in financial forecasting or
the relationship between the future prices and the features
from the technical analysis[7], [8]. This method of machine
learning was comprised many algorithms out of which, support
vector machine(SVM) and artificial neural networks(ANN)
were the leading and promising algorithms owing to the
promising capability of the non-linear mapping and fitting[6],
[9], [10]. However, there were certain flaws of the nonlinear
mapping and fitting because of their black box features and
overfitting presents one of the biggest obstacles in practical
application. Moreover, some universal used methods included
the principal component analysis(PCA [11]), locally linear
embedding(LLE [12]), independent component analysis(ICA
[13]). They focused on the distributing samples and the kernel-
based extraction operation was performed on data sets [14].
Likewise, Among deep learning methodologies, the extraction
process is mainly handled by the neural networks. The neural
network plays a key role in the feature of the extraction
process and by the series of available neural networks feature
engineering can also be performed by purposely designing
an integrated network structure(INS) [1]. These neural link
layers are further connected by non-linear mapping and to
make the deeper feature maps suitable for the prime tasks;
features are transformed by a high layer nonlinear mapping.
The listing status of Chinese-listed companies [15] is now
being determined by the recently proposed model of decision
tree C4.5 and C5.0 combined with an improved filter feature
selection method. The characteristics of the network led to
the designing of the structures of the network. These designed
networks, GoogleNET [16], and AlexNET [17] was designed
to enhance the learning process and extracting information
from the pixel data. CNN-blSTM [18] was proposed for voice
recognition [19] and R-CNN [20] was designed for scalable
design object detection. Attention-over-attention neural net-
work [21] was intended to extract data from the text for the
reading comprehension task. In a nutshell, the study proposed
a novel end to end model in order to perform feature trans-
formation on sequential market data. The model is MFNN,
a multi-feature neural network designed especially for the
feature extraction on the financial time series samples. In the
proposed network structure, samples are mapped into feature
space where different patterns are likely distinguishable[1]. They also enhance the efficiency of the training. Also, some
of these deep learning methodologies have been used in stock
price prediction. In essence, the restricted Boltzmann machine
M ETHODOLOGIES has been used to update the classical deep brief network to
Stock price movements and predictions are quite com- model continuous data [23]. DNN with the (2D) 2PCA feature
plicated, unlike text, pictures, and images whose raw input extractor [11] was proposed to yield better profitability and
contains information needed for final objectives[23]. It is accuracy of the hit rate than recurrent neural networks and
because stock price movements are the result of multiple fac- radial basis function neural networks. Experimental settings
tors such as investor sentiments, macroeconomy, the financial and training methodologies are used as a baseline while
situation of the company, etc. Machine learning methods and evaluating the performance of the model MFNN on 30 data
traditional statistics are existing models for the prediction of sets, especially building the single type filter networks, long
financial time series and stock prediction. These models are short term memory unit as a comparison to determine the
developed and built on some specific methodologies. Technical effectiveness of MFNN in table1[1].
analysis and statistical methodologies are two commonly used
feature engineering methodologies. Technical analysis focuses Table 1:training times of MFNN on 30 data sets
on historical patterns as it presumes that future scopes are
correlated with it. Technical indicators that describe such (θ) t=5 t=10 t=15 t=20 t=25 t=30
correlation and patterns are Bollinger band [24], momentum
[25], moving average(MA), etc. Statistical methods focus on 0.1 72.40 64.20 120.00 84.20 106.40 114.40
the feature information extracted by information compression 0.15 149.45 135.80 28.00 210.00 158.20 210.00
and dimension reduction used in machine learning methods. 0.2 182.92 165.42 250.00 233.33 220.83 170.00
The universally accepted methods include locally linear em- 0.25 184.97 268.67 310.00 276.93 310.00 310.00
bedding(LLE), independent component analysis(ICA), princi- 0.3 247.00 390.00 85.80 390.00 334.75 390.00
pal component analysis(PCA [11]), etc. They focus on the
extraction operation and distribution of samples. Commonly
used statistical methods include GARCH [2], VAR, Learning R ESULTS
the potential relationship between patterns in labels, and fea- The result of the accuracy of each model is represented by
tures are considerably provided by machine learning methods. the figure and it can be concluded that the accuracy of the
Moreover, the quality of features determines the performance test is inversely proportional to the (θ) . Implying that when
of either machine learning methods or statistical models. The (θ) increases the accuracy of the test decreases as we can
listing status of Chinese-listed companies [15] is now being see in fig.1[1], fig.1[2],fig.1[3]. These results are based on the
determined by the recently proposed model of decision tree 30 data sets on which MFNN has been applied. To evaluate
C4.5 and C5.0 combined with an improved filter feature se- performance of the MFNN in the market simulation, the total
lection method. For further integrating the feature engineering return (R) [1] is measured by
process into models in order to learn it with consideration of
objectives. It leads to the further methodology of deep learning portf olioT
R=( − 1) × 100%
methodologies on which network structure can be designed for portf oliot0
specific data information. Deep learning methodologies have
achieved prime results in recent years in speech recognition,
computer vision, and some path recognition such as hand-
written character recognition. The prime difference between
the deep learning methodologies and traditional methods is
that the network structure can be used to determine specific
data information and objective task. The characteristics of
the network led to the designing of the structures of the
network. These designed networks, GoogleNET [16], and
AlexNET [17] was designed to enhance the learning process
and extracting information from the pixel data. CNN-blSTM Fig. 1. Performance of MFNN and RNN on 30 data sets.[1]
[18] was proposed for voice recognition and R-CNN [20]
was designed for scalable design object detection [20], [22].
Attention-over-attention neural network [21] was intended to C ONCLUSION
extract data from the text for the reading comprehension task. This paper proposed the end-to-end model for price move-
However, learning ability can be enhanced with the depth of ment prediction of task and sampling of series on financial
network growth. However, problems can be arisen such as time. The model is named MFNN i.e multi filters neural
vanishing gradient and exploding which can be handled by network. The paper featured the engineering of financial time
trucks such as dropout, residual, batch normalization [4]–[6]. series and deep learning for feature learning. The best results

Fig. 2. Performance of CNN and LSTM on 30 data sets.[1]
Fig. 3. Performance of SVM and Random Forest on 30 data sets.[1]
from the machine learning method and statistical method
have been outperformed by the prediction result of MFNN
by 11.47% and 19.75% respectively. As compared to RNN
and CNN, the purposely designed network and the predic-
tion filters enhanced the network accuracy for 6.28% and
7.91% respectively. In the market simulation, other baselines
failed to present better profitability than the deep learning
methodologies. The proposed model MFNN was 15.41% more
accurate and better than the best result presented by the
traditional methods and 22.41% better than statistical models
at returns. In future works, we hope to explore and analyze
financial stock pricing and predictions based on how the
quality of trading is affected by the way of integration in
terms of profitability, risk, and stability. Considering the deep
learning, neural networks, and different methodologies we can
further design the networks to extract data and other important
informational aspects from the multi informational and raw
sources.
R EFERENCES
[1] W. Long, Z. Lu, and L. Cui, “Deep learning-based
feature engineering for stock price movement prediction,”
Fig. 4. Performance of Lofistic Regression and Linear Regression on 30 data
sets.[1]
Knowl.-Based Syst., vol. 164, pp. 163–173, Jan. 2019, doi:
10.1016/j.knosys.2018.10.034.
[2] Z. Lin, “Modelling and forecasting the stock market
volatility of SSE Composite Index using GARCH models,”
Future Gener. Comput. Syst., vol. 79, pp. 960–972, Feb.
2018, doi: 10.1016/j.future.2017.08.033.
[3] C. Kocak, “ARMA(p,q) type high order fuzzy time
series forecast method based on fuzzy logic relations,”
Appl. Soft Comput., vol. 58, pp. 92–103, Sep. 2017, doi:
10.1016/j.asoc.2017.04.021.
[4] L. G. Valiant, “A theory of the learnable,” Commun.
ACM, vol. 27, no. 11, pp. 1134–1142, Nov. 1984, doi:
10.1145/1968.1972.
[5] T. Zhou, S. Gao, J. Wang, C. Chu, Y. Todo, and Z. Tang,
“Financial time series prediction using a dendritic neuron
model,” Knowl.-Based Syst., vol. 105, pp. 214–224, Aug.
2016, doi: 10.1016/j.knosys.2016.05.031.
[6] T. Xiong, C. Li, Y. Bao, Z. Hu, and L. Zhang, “A
combination method for interval forecasting of agricultural
commodity futures prices,” Knowl.-Based Syst., vol. 77, pp.
92–102, Mar. 2015, doi: 10.1016/j.knosys.2015.01.002.
[7] J. Patel, S. Shah, P. Thakkar, and K. Kotecha, “Predicting
stock and stock price index movement using Trend
Deterministic Data Preparation and machine learning
techniques,” Expert Syst. Appl., vol. 42, no. 1, pp. 259–268,
Jan. 2015, doi: 10.1016/j.eswa.2014.07.040.
[8] J. Sen and T. Chaudhuri, “A Robust Predictive Model
for Stock Price Forecasting,” Social Science Research
Network, Rochester, NY, SSRN Scholarly Paper ID 3068204,
Dec. 2017. Accessed: Sep. 27, 2020. [Online]. Available:
https://papers.ssrn.com/abstract=3068204.
[9] G. S. Atsalakis and K. P. Valavanis, “Surveying stock
market forecasting techniques – Part II: Soft computing
methods,” Expert Syst. Appl., vol. 36, no. 3, Part 2, pp.
5932–5941, Apr. 2009, doi: 10.1016/j.eswa.2008.07.006.
[10] Y. Pan, Z. Xiao, X. Wang, and D. Yang, “A
multiple support vector machine approach to stock index
forecasting with mixed frequency sampling,” Knowl.-
Based Syst., vol. 122, pp. 90–102, Apr. 2017, doi:
10.1016/j.knosys.2017.01.033.
[11] R. Singh and S. Srivastava, “Stock prediction using
deep learning,” Multimed. Tools Appl., vol. 76, no. 18, pp.
18569–18584, Sep. 2017, doi: 10.1007/s11042-016-4159-7.
[12] S. T. Roweis and L. K. Saul, “Nonlinear Dimensionality
Reduction by Locally Linear Embedding,” Science,
vol. 290, no. 5500, pp. 2323–2326, Dec. 2000, doi:
10.1126/science.290.5500.2323.
[13] A. D. Back and A. S. Weigend, “A First Application
of Independent Component Analysis to Extracting Structure
from Stock Returns,” Int. J. Neural Syst., vol. 08, no. 04, pp.
473–484, Aug. 1997, doi: 10.1142/S0129065797000458.
[14] Y. Bengio, P. Simard, and P. Frasconi, “Learning
long-term dependencies with gradient descent is difficult,”
IEEE Trans. Neural Netw., vol. 5, no. 2, pp. 157–166, Mar.
1994, doi: 10.1109/72.279181.
[15] L. Zhou, Y.-W. Si, and H. Fujita, “Predicting the listing
statuses of Chinese-listed companies using decision trees
combined with an improved filter feature selection method,”
Knowl.-Based Syst., vol. 128, pp. 93–101, Jul. 2017, doi:
10.1016/j.knosys.2017.05.003.
[16] C. Szegedy et al., “Going deeper with convolutions,”
in 2015 IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), Jun. 2015, pp. 1–9, doi:
10.1109/CVPR.2015.7298594.
[17] “(5) ImageNet Classification with Deep Convolutional
Neural Networks,” ResearchGate. (accessed Sep. 27, 2020).
[18] K. J. Han, S. Hahm, B.-H. Kim, J. Kim, and I. Lane,
“Deep Learning-Based Telephony Speech Recognition in the
Wild,” in Interspeech 2017, Aug. 2017, pp. 1323–1327, doi:
10.21437/Interspeech.2017-1695.
[19] A. Graves, A. Mohamed, and G. Hinton, “Speech
recognition with deep recurrent neural networks,” in 2013
IEEE International Conference on Acoustics, Speech
and Signal Processing, May 2013, pp. 6645–6649, doi:
10.1109/ICASSP.2013.6638947.
[20] R. Girshick, J. Donahue, T. Darrell, and J. Malik,
“Rich Feature Hierarchies for Accurate Object Detection
and Semantic Segmentation,” in 2014 IEEE Conference on
Computer Vision and Pattern Recognition, Jun. 2014, pp.
580–587, doi: 10.1109/CVPR.2014.81.
[21] Y. Cui, Z. Chen, S. Wei, S. Wang, T. Liu, and G.
Hu, “Attention-over-Attention Neural Networks for Reading
Comprehension,” Proc. 55th Annu. Meet. Assoc. Comput.
Linguist. Vol. 1 Long Pap., pp. 593–602, 2017, doi:
10.18653/v1/P17-1055.
[22] “Papers with Code - Object Detection.”
https://paperswithcode.com/task/object-detection (accessed
Sep. 27, 2020).
[23] F. Shen, J. Chao, and J. Zhao, “Forecasting exchange rate
using deep belief networks and conjugate gradient method,”
Neurocomputing, vol. 167, pp. 243–253, Nov. 2015, doi:
10.1016/j.neucom.2015.04.071.
[24] C. Lento, N. Gradojevic, and C. S. Wright, “Investment
information content in Bollinger Bands?,” Applied Financial
Economics Letters, vol. 3, no. 4, pp. 263–267, Jul. 2007, doi:
10.1080/17446540701206576.
[25] C. M. C. Lee and B. Swaminathan, “Price Momentum
and Trading Volume,” The Journal of Finance, vol. 55, no. 5,
pp. 2017–2069, 2000, doi: 10.1111/0022-1082.00280.