2017 2nd IEEE International Conference on Computational Intelligence and Applications

Regression Model for Appraisal of Real Estate Using Recurrent Neural Network
and Boosting Tree

Junchi Bin∗, Shiyuan Tang†, Yihao Liu∗, Gang Wang‡, Bryan Gardiner§, Zheng Liu∗, Eric Li¶
∗School of Engineering, University of British Columbia, Kelowna, BC, Canada
e-mail: junchibin@alumni.ubc.ca, yihaoliu007@gmail.com, zheng.liu@ubc.ca
†School of Foreign Languages and Literature, Tianjin University, Tianjin, China, e-mail: lglsbd1992@163.com
‡College of Architecture and Civil Engineering, Guilin University of Technology, Guilin, Guangxi, China
e-mail: 1059020529@qq.com
§Data Nerds, Kelowna, BC, Canada, e-mail: bryan@datanerds.com
¶Faculty of Management, University of British Columbia, Kelowna, BC, Canada
e-mail: eric.li@ubc.ca

Abstract—Automated valuation model (AVM) is a mathemat-
ical program to estimate the market value of real estates based
on the analysis of locations, neighborhood characteristics, and
relevant property characteristics. The most common AVMs
em-ployed by the appraisal industry are based on multiple
regression analysis. Other analytic tools such as statistical
learning and fuzzy algorithms have become more popular
because of the increasing capability of collecting a high volume
of data and the advancement of machine learning. The new
analytic model thus becomes possible to build a more
sophisticated model to exploit the information embedded in the
collected data. In this work, we proposed a boosting tree model
facilitated with a Recurrent Neural Network (RNN) to forecast
the average price of an area. The experimental results indicate
that our model outperforms the existing models adopted in the
appraisal industry.
Keywords-appraisal of real estate; regression model;
boosting tree; recurrent neural network
I. INTRODUCTION
For most of the consumers in North America, housing
has been one of the largest expenses. Purchasing a house is
a high involvement decision. Consumers’ judgment on the
value of the property and their estimation on the future
value of the property would influence their purchasing
decision and their budget allocation. Moreover, the price of
real estate is one of the important factors to reflect the
economic activities. An accurate prediction of the price of
land, therefore, can help governments or companies make a
crucial decision for manipulating the financial condition for
future fiscal years. From this perspective, the process of
estimating the price of real estate is closely related to both
people lives and national economy.
Automated valuation model (AVM) is a mathematical
pro-gram to assess the market value of real estate based on
the analysis of locations, neighborhood conditions and the
characteristics of properties. Regression analyses and
machine learning algorithms are widely used in the current
appraisal industry. Some businesses in estate industry
provide easy- access web applications of AVM to estimate
the market value of properties. For example, our partner
organization, Data
Nerds, provides an extensive amount of data to evaluate
the price of real estate based on property data, transaction
history, and area information(https://factory.datanerds.com/).
Clients could browse these data and decide their next move.
The house price index (HPI) is a time-series parameter
H
to measure the movement of single-family houses published
by the federal government. It serves as an accurate indicator
of the overall trends of house price in geographic levels
such as CBSA and zip code. For example, there is an HPI
H1 and a variable P1 which indicates house price in the
same area in a certain year. Given H2 in another year, we
can calculate the house price in another year with an
equation P2 = P1H2 . Therefore, AVM firstly predicts the
market value of a house based on attributes of properties,
and forecast HPIs next few years or few months. Then, it
calculates the future house price with above equation.
There are multiple machine learning methods introduced
to the field in the past decade. LASSO regression (LASSO)
and support vector regression (SVR) are modern machine
learning approaches for both prediction and forecasting
HPIs in AVM. Boosting tree model is a promising machine
learning method in data analysis competition. The basic
idea of the boosting tree model is to combine many low-
accuracy regression trees models into a model with high
accuracy [1], [2]. The model will continue to iterate in the
direction of gradient descent. In AVM, boosting tree model
is to predict the house price based on attributes of
properties.
The Artificial Neural Network (ANN) is a powerful tool
for nonlinear regression inspired by the mechanism of how
the brain works. In recent years, ANN has played a critical
role in many industrial applications due to its capability of
learning feature from data, which produces a highly accu-
rate performance on pattern recognition tasks such as image
classification and natural language process. However,
previous studies reported that the predictions with ANN for
house properties were not stable [3], [4]. One research
suggested that linear regression may be even better than

978-1-5386-2030-4/17/$31.00 ©2017 IEEE 209

ANN with selected small number of the sample [5]. The
recurrent neural network is a state-of-art model for h(t) = f (Ux(t) + Wh(t−1)) (1)
processing sequential data such as time series data or
natural languages. Long short-term memory (LSTM) is one
where f is generally a non-linear activation function such as
of the most successful RNNs architectures. Some designs of
LSTM were used to forecast stock market and solar power tanh or ReLU with shared parameters U, W [7]. The O(t) is
output which demonstrate its power in forecasting [1], [6]. the output of step t which depends on activation function in
In this study, we propose an ensemble learning method, current neuron as in Eq.(2).
which combines the LSTM for forecasting HPIs with a
boost-ing tree model to predict house price in Chicago. O(t) = 6(Vh(t)) (2)
Conventional machine learning methods such as LASSO
and SVR in AVM are the baseline algorithms for the
proposed model. where 6 represents the activation function for the output
In this study, we propose an ensemble learning method, layer.
which combines the LSTM for forecasting HPIs with a Theoretically, RNNs can handle context from the
boost-ing tree model to predict house price in Chicago. beginning of the sentence which will allow more accurate
Conventional machine learning methods such as LASSO predictions of a word at the end of a sentence. However, the
and SVR in AVM are the baseline algorithms for the longer length of sequence requires more hidden layers
proposed model. which cause vanishing gradient problem for preventing
from optimization of RNN.
II. M ODELING WITH LSTM AND BOOSTING TREE LST is architecture to solved the problem. Each LSTM
splits the whole neural networks into multiple cells
In this section, we briefly introduce the major
components of the proposed model. First, we introduce the C(1),...,C(T )}. Each cell contains an input gate, forget gate
basic archi-tecture of RNN and long-short term memory and output gate as illustrated in Fig. 2, which is capable
(LSTM). Then, we introduced the boosting tree model and of memorizing the error in forward propagation stage. The
baseline machine learning methods respectively. forget gate drops the error from cell to solve vanishing
gradient.
A. Long Short-Term Memory
In natural language processing (NLP), the whole
sentence is defined as a sequential data and each word based
on an understanding of previous words. When ANNs
perform natural language processing, it requires a structure
to reason the next word depending on the context of the
sentence, which combines the previous outputs as inputs for
inference. Recurrent Neural Networks (RNNs) are a family
of neural networks for processing sequential data [7].

Figure 2. The architecture of Long Short-Term Memory (LSTM).

Wf , Wc, and Wo are corresponding parameters for input,

Figure 1. An illustration of RNN.
Fig. 1 illustrates the structure of a simple RNN. {O(1),...,
O(T )} are the hidden layers of the neural network given
the input sequence {x(1), ..., x(T )} and hidden units
h(1), ..., h(T )}. There is a one-way flow of information
from the input units to reach the hidden units, while another
one-way flow of information from the hidden units to reach
the output units. h(t) is calculated based on the output of the
current input layer and the state of the previous hidden layer
h(t−1), which is usually estimated as in Eq.(1) [7].
forget and output gate, respectively. Input gate combines the
current input with previous output with activation function
σ and bias b f in the neuron. Then, the tanh creates new
candidates for cell value and compares to the previous value
for decision of update with bias bi and bc respectively. The
following equations give the details of the calculation [7].
ft = σ (Wf [h(t−1), x(t)] + b f ) (3)
(4)
ot = σ (Wo[h(t−1), x(t)] + bo) ∗ tanh(ct + ft ) (5)

210
B. Boosting Tree
A boosting tree can be represented by Eq.(6):
m
F(x; P) = ∑ βmh(x; αm)
m=1
where P denotes parameters or a set of multiple
parameters, P = {p0, p1, p2,...}. F(x; P) represents a
function with x as set of variables given parameter P. The
boosting ensembles multiple learning models {h(x; α1),
h(x; α2),..., h(x; αM )} with different weights βm and
parameters αm [2].
The next step is to compute the residual, which is the
difference between the observe value and the current
predicted value yi for each iteration. Then, We further
optimize the parameters according to the previous model to
the end of the iteration along the direction of gradient
descent. The following equations illustrate how the process
evolves with the learning rate ρ [2], [8], [9].
N
ρm = argmin ∑ L(yi, Fm−1(xi)+ ρh(xi; αm, βm))
i=1
Fm(x) = Fm−1(x)+ ρmh(x; αm)
C. Baseline Models
1) Support Vector Regression: The support vector
Figure 3. The results after feature selection, and the accepted features are indicated by green.
regres- sion (SVR) is given by
(6) (10)
where g j (x), j = 1, 2...m, denotes a set of non-linear
transfor- mations, y j stands for the predicted value and b is
the bias term. SVR always use ε -intensive loss as loss
function for training [4].
2) LASSO Regression: LASSO is a linear regression
model with l1[7]
n
βˆlasso = arg min ∑(yi − (β0 + β Txi))2 + λ "β "1 (11)
III. EXPERIMENTAL RESULTS
In the experiments, all the data are obtained from our
partner organization, Data Nerds. The collected data are
from Chicago of Illinois, one of the biggest city in the USA.
This section presents how to pre-process the data and
(7) evaluate the proposed model in comparison with the SVR
and LASSO regression.
A. Real Estate Data Pre-processing
(8)
The HPI dataset is offered by the federal government.
The whole dataset contains all HPIs across the United States
(9) for all geographical levels from 1975 to 2015. In this
experiment, we extracted 60 series of HPIs in Chicago over
zip codes level. Moreover, we used the HPIs before 2013 to
forecast the HPIs on 2014 and 2015.
211
 The original dataset contains many variables such as the
qualities of houses, geographic information of properties. It
also contains transactional records of house prices across
time. Here, we only selected the most recent records (within
2013) and convert the prices of 2014 and 2015 by HPIs as
ground truths of this experiment. We filtered out the house
with either extremely high or low price across the city.
Table I shows the summary of filtered data.
TABLE I. THE AVERAGE PRICE AND STANDARD DEVIATION OF THE
PRICE IN CHICAGO
# of Houses Avg. Std.
Chicago 5179 208.2 118.6
Feature selection is a major step in the application of
machine learning methods. The data sets come with too
many variables for modeling. There are two reasons to
select the features. One is that too large feature sets will
slow down the algorithm and the other reason is that it will
cause inaccuracy of machine learning when the number of
variables is signif-icantly higher than optimal. Therefore, it
is critical to select the optimal features according to
contribution and correlation of the ground truth. Boruta is a
method of feature selection based on random forest and
employed in our experiment to conduct the feature
extraction [10]-[12]. After feature selection, only 19
features were used to build the model. The result of feature
selection is shown in Fig. 3. The green indicators refer to
the accepted features.
To train and validate our models and prevent overfitting,
we apply the 5-folds cross-validation technique. The
algorithm randomly split the complete data into five
subsets. A unique subset is regarded as the validation data
for testing, and the remaining four subsets are used for
training in each validation procedure. After 5-fold cross
validation, we can obtain a predicting price of each house.
Figure 4. Training process.
212
B. Training Methods
Fig. 4 shows the training process for the entire model.
First, the filtered data from pre-processing contains
attributes of houses and HPIs from 1975 to 2013 as
described previously. Second, multiple LSTMs were
employed to forecast the HPIs in each zip code level
respectively. It is a single hidden layer LSTM with 4
neurons of activation of ReLu. The window size is three
which means the forecasting HPIs are predicted by three
previous HPIs. Simultaneously, the boosting tree model is
obligated to predict the house price on 2013 based on
attributes of properties. Finally, we use the forecasting HPIs
and house price on 2013 to assess the house price on 2014
and 2015.
C. Evaluating Process
We randomly split the entire data into training (80%)
and testing set (20%). The evaluation metrics employed are
mean absolute error (MAE) and mean absolute percentage
error (MAPE). Eq.(10) and (11) give the definitions of the
two metrics, where ti is the ground truth and pi is the
predicted value.
(12)
(13)
D. Results
We use the same training and testing set to evaluate all
the models. Table II gives the regression results for all the
different models. The results indicate that the proposed
model has better performance than the other two models.
TABLE II. RESULT COMPARISON
MAE MAPE
LASSO 58.661 31.24%
SVR 57.225 29.83%
Proposed Model 46.926 24.03%
IV. CONCLUSIONS
In this paper, we proposed an ensemble learning
regression model for real estate appraisal. The proposed
model is capable of taking qualities of houses, location, and
trend of market price into account. The experimental results
show the effec-tiveness of the design. The work also offered
a new approach to ensemble deep learning methods with the
statistical learning algorithm. From this project, we discover
that LSTM has great potential in AVM or other regression
applications. However, our model didn’t consider spatial
and temporal factors in estimating housing value. In future
studies, we are intending to use LSTM or other deep
learning techniques to build a spatial-temporal model. We
hope our work can give a new insight to real estate
appraisals.

REFERENCES
[1] Kai Chen, Yi Zhou, and Fangyan Dai. A LSTM-Based method for
stock returns prediction: A case study of China Stock. In 2015 IEEE
International Conference on Big Data, 2015.
[2] Villius Kontrimas and Antans Verikas. The mass appraisal of the
real estate by computational intelligence. Applied Soft Computing,
11(1): 443–448, 2011.
[3] Nils Raabe. Deep Learning for Solar Power Forecasting An
Approach Using Autoencoder and LSTM Neural Networks. pages
2858–2865, 2016.
[4] Vladimir N. Vapnik. Statistical Learning Theory. JOHN WILEY &
SONS INC, 1998.
[5] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep
learning. MIT Press, Cambridge, 2016.
[6] K. C. Y. Lam, C. Y. Yu, and K. C. Y. Lam. An Artificial Neural
Network and Entropy Model for Residential Property Price
Forecasting in Hong Kong. Journal of Property Research, 25(4):321–
342, 2008.
[7] Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The
Elements of Statistical Learning. Springer-Verlag New York Inc.,
2009.
[8] Tianqi Chen and Carlos Guestrin. XGBoost: Reliable Large-scale
Tree Boosting System. arXiv, pages 1–6, 2016.
[9] Jerome H. Friedman. Greedy Function Approximation: A Gradient
Boosting Machine. The Annals of Statistics, 29(5):1189–1232, 2011.
[10] Max Kuhn and Kjell Johnson. Applied Predictive Modeling. 2013.
[11] Miron B. Kursa, Aleksander Jankowski, and Witold R. Rudnicki.
Boruta-A system for feature selection. Fundamenta Informaticae,
101(4):271– 285, 2010.
[12] Miron B Kursa and Witold R Rudnicki. Feature Selection with the
Boruta Package. Journal Of Statistical Software, 36(11):1–13, 2010.

213