CN115705322 Database management system, data processing method and equipment

Abstract

The application discloses a database management system, a data processing method and equipment,
wherein the system comprises: the system comprises an optimizer comprising n models, a training
data collector, a model manager and a model evaluator. The optimizer is used for obtaining a target
physical plan through the n models according to the SQL statement; the training data collector is
used for constructing m training sets according to the running data of the database process; the
model manager is used for carrying out fine tuning (finetune) on a first target model (which belongs
to n models and needs to meet preset requirements, such as performance degradation) by adopting
a target training set (which belongs to m training sets) to obtain a second target model; and the
model evaluator is used for evaluating the performance of the second target model and updating the
first target model into the second target model when the performance meets preset requirements
(such as performance improvement). The present application combines with machine learning to
achieve the functionality of automatically performing database tuning, updates, and other database
management tasks traditionally performed by DBAs without manual intervention.

Description

Database management system, data processing method and equipment

Technical Field

The present application relates to the field of database management, and in particular, to a database
management system, a data processing method, and a device.

Background

A database (database) is a "warehouse that organizes, stores, and manages data according to a data
structure," which is an organized, sharable, and uniformly managed collection of large amounts of
data that is stored in computers for a long period of time. The database is crucial to the efficient
operation of modern enterprises.

In practical applications, the management mode of the database mainly depends on a database
administrator (DBA), and the DBA usually needs to spend a lot of time and effort to manually manage
and maintain the database, which is very easy to make mistakes, and brings catastrophic influences
to the normal operation time, performance and safety of the database. For example, untimely and
correct patching and installation of security updates may result in database vulnerabilities,
weakening or even complete failure of database safeguards, and thus expose enterprises to serious
risk of data leakage, serious financial impact and loss of reputation.

However, the traditional database management method is a traditional optimization technology

based on expert experience, such as cost estimation, connection order selection, parameter tuning
and the like, and cannot meet the requirements of multi-scenario services, massive applications and
extreme performance.

Disclosure of Invention

The embodiment of the application provides a database management system, a data processing
method and equipment, which are combined with a machine learning technology to realize the
functions of automatically executing database tuning, protection, updating and other conventional
database management tasks traditionally executed by a DBA without manual intervention.

1|Page
Based on this, the embodiment of the present application provides the following technical solutions:

in a first aspect, an embodiment of the present application first provides a database management
system, which may specifically include: the optimization method comprises a self-learning optimizer
(note that part of kernel components included in the optimizer provided by the application can be
native kernel components, but n models included in the optimizer are newly added in the
application, n is greater than or equal to 1), a training data collector, a model manager and a model
evaluator, wherein the optimizer is used for obtaining a physical plan to be executed finally (which
can be called a target physical plan) through the n models according to SQL statements input into a
database, wherein the target physical plan refers to a physical plan of which the execution overhead
meets a certain preset requirement (which can be called a first preset requirement). The training
data collector is used for obtaining training data according to the operation data (such as database
operation indexes, query logs, system logs and the like) of the processes in the database, and
constructing m training sets based on the obtained training data, wherein m is larger than or equal to
1. And the model manager is used for performing fine tuning (finetune) on the first target model by
adopting a target training set corresponding to the first target model under the condition that the
first target model meets a certain preset requirement (which can be called as a second preset
requirement), so as to obtain a second target model (the second target model is the first target
model with updated model parameters). Wherein the first target model is one of the n models, and
the target training set is one of the m training sets. And then, the model parameters of the second
target model after finetune can be transmitted to the model evaluator again for model performance
evaluation. It should be noted that, the values of m and n may be the same or different, if m = n, it is
described that each model in n models corresponds to one training set used by itself, and if m ≠ n, it
is described that there may be a case where a plurality of models share one training set in n models
(that is, the case where m < n), or there may be a case where one model may have a plurality of
training sets for training (that is, the case where m > n), and this is not limited in the present
application. And the model evaluator is used for evaluating the performance of the obtained second
target model and updating the first target model in the optimizer to the second target model when
the performance of the second target model meets a certain preset requirement (which can be
called as a third preset requirement). As an example, the updating process may specifically be: the
model evaluator sends the model parameters of the second objective model to the optimizer, which
assigns the received updated model parameters to the first objective model, thereby obtaining the
second objective model. It is noted that in some embodiments of the present application, the model
evaluator may be a graph-embedding based performance prediction model.

In the above embodiments of the present application, the database management system includes an
optimizer with n models, a training data collector, a model manager, and a model evaluator, where
the optimizer with n models replaces a traditional heuristic optimizer, and by combining with a
machine learning technique, a logical query can be converted into a physical execution plan with
higher execution efficiency; the training data collector can generate training data of a model related
to the database based on the running data of the process in the database, can continuously optimize
the database and provides credible autonomous operation and maintenance service; the model
manager can call the corresponding target training set in the training data collector to finely adjust
the first target model so as to dynamically update and replace the corresponding model used in the
database according to the real-time running state of the database; and the model evaluator is used
for providing secondary verification capability and ensuring that an optimal and effective model is
provided. The database management system constructed in the embodiment of the application can
realize the functions of automatically executing database tuning, protection, updating and other

2|Page
conventional database management tasks traditionally executed by the DBA by combining with a
machine learning technology without manual intervention.

In a possible implementation manner of the first aspect, the database management system may
further include a recommender, which may include p models, where p ≧ 1; the advisor is used for
discovering abnormal conditions (namely discovering abnormal data) existing in the running data
(such as CPU utilization rate, user response time and the like) of the processes in the database,
diagnosing abnormal reasons based on the obtained abnormal data, and then optimizing an
optimization module (the optimization module is also positioned in the advisor and used for
performing parameter optimization on the database) corresponding to the abnormal reasons based
on the obtained abnormal reasons so as to reduce the probability of the abnormal running data of
the processes in the database.

In the above embodiment of the present application, the constructed database management system
may be additionally provided with a recommender based on a machine learning method, so as to
implement the functions of self-monitoring, self-diagnosis and self-optimization, so as to
automatically and intelligently optimize the database. Therefore, the problems that database
monitoring, configuration, diagnosis and optimization methods (such as parameter tuning, slow SQL
diagnosis and index/view consultants) in the prior art depend on DBA, are high in cost and cannot
adapt to large-scale instances (such as cloud databases) are solved.

In a possible implementation manner of the first aspect, the recommender may specifically include
three models, which may be respectively referred to as a codec, a first model, and a second model,
for performing three steps of self-monitoring, self-diagnosis, and self-optimization of the database.
Specifically, the recommender encodes and decodes the running data of the process in the database
through the codec to obtain encoded data, and compares the encoded data with the running data
input to the codec to obtain abnormal data. It should be noted here that the principle of obtaining
the abnormal data by using the codec is as follows: the coder and the decoder can restore normal
original data but can not restore abnormal original data, so that the input original data is encoded
and then decoded to obtain encoded data, and the encoded data is compared with the original data
to know whether abnormal data exists or not. After obtaining the abnormal data, if the operating
data belongs to system index data (e.g., a page fault), the recommender may further diagnose the
cause of the abnormality according to the abnormal data through a first model, where the first
model is a model constructed based on a deep learning algorithm; if the operation data belongs to
query index data (such as average delay), the recommender can further diagnose the cause of the
abnormality according to the abnormal data through a second model, wherein the second model is a
model constructed based on a deep learning algorithm. It should be noted here that in other
embodiments of the present application, the recommender may further include more or fewer
models for implementing the self-monitoring and self-diagnosis process of the database, and in the
embodiment of the present application, the recommender includes three models for illustration
only.

In the above embodiments of the present application, how the advisor implements the self-
monitoring and self-diagnosing functions of the database through the model included therein is
specifically set forth, and the purpose of automatically and intelligently managing the operation data
of the database by the database management system is achieved.

In a possible implementation manner of the first aspect, the first model may include an LSRM model
and a classifier, and specifically, the recommender encodes the found abnormal data into a
compressed vector (i.e., a reduced/increased vector) by calling the LSRM model, and then uses a

3|Page
learning-type classifier (e.g., a binary classifier, a multi-classifier, etc.) to infer a corresponding root
(e.g., a database backup operation); the second model may include a Tree-LSTM model and a
softmax function, and in particular, the recommender encodes the slow query (i.e., the query is long
in execution time) by calling the Tree-LSTM model, locates the physical operator (i.e., the execution
operator) operations that caused the anomaly, and identifies the root cause that caused the anomaly
by using the softmax function.

In the above embodiments of the present application, a typical implementation of the first model
and the second model is specifically described, and the realizability is provided.

In a possible implementation manner of the first aspect, if the database management system
constructed in the embodiment of the present application includes the recommender, the first target
model described above may be any one of n models in the optimizer, and may also be any one of p
models in the recommender.

In the above embodiment of the present application, it is clear that the first objective model may be
any one of the optimizer and the recommender, and the selection range of the first objective model
is expanded, so that the subsequent model evaluator may evaluate not only the performance of the
model in the optimizer, but also the performance of the model in the recommender, and the
realizability of the solution is achieved.

In a possible implementation manner of the first aspect, the optimizer may specifically include three
models, which may be respectively referred to as a third model, a fourth model, and a fifth model,
for performing three steps of logical query rewrite, cost estimation, and physical plan generation.
Specifically, the optimizer performs logical query rewrite on an SQL statement (also referred to as
SQL query) input into the database through a third model, so as to obtain a rewritten logical plan,
where the third model is a model constructed based on a tree search algorithm, for example, a
monte carlo tree search algorithm; then, the optimizer generates q physical plans according to the
logic plan by a fourth model, wherein the fourth model is a model constructed based on a deep
learning algorithm, for example, the fourth model can be a model based on Tree-LSTM, and q is more
than or equal to 1; finally, the optimizer calculates Q execution costs corresponding to the Q physical
plans (one physical plan corresponds to one execution cost) by using a fifth model, and determines a
target physical plan to be executed finally according to the Q execution costs, wherein the fifth model
is a model constructed based on a reinforcement learning algorithm, for example, a model based on
deep reinforcement learning (DQN). It should be noted here that in other embodiments of the
present application, the optimizer may further include more or fewer models for implementing the
processes of logical query rewrite, cost estimation, and physical plan generation, and in the
embodiment of the present application, the optimizer includes three models for illustration only.

In the above embodiments of the present application, how the optimizer implements the functions
of logical query rewrite, cost estimation, and physical plan generation of the database through the
included model is specifically described, so that a traditional heuristic optimizer is replaced, the
logical query is converted into a physical execution plan with higher execution efficiency by
combining with a machine learning technology, and the problems of inaccurate cost evaluation and
poor physical plan generation of complex SQL statements caused by the current database
architecture problem can be effectively solved.

In a possible implementation manner of the first aspect, if the performance of the second target
model does not meet the third preset requirement, the model evaluator is further configured to
trigger the database to generate a target physical plan to be finally executed by using the native

4|Page
kernel component in the database, for example, the index selection module enables a traditional hill-
climbing algorithm to create an index, so as to execute the logic of the SQL statement. That is, if the
performance of the second target model after finetune still does not meet the requirement, the
target physical plan is generated by using the database conventional kernel component algorithm.

In the above embodiment of the present application, because the native kernel component of the
database is not deleted, but is stored in the database software together with the newly added
optimizer, in the running process of the database, if the performance of the second target model
does not meet the third preset requirement, the model evaluator will also trigger the database to
generate the final target physical plan to be executed by using the native kernel component in the
database (because in this case, the execution effect of the native kernel component is better), so that
the method of dynamically adjusting the generated target physical plan in real time based on the
evaluation of the current performance of the second target model can be performed, thereby
improving the performance of the database as a whole.

In a possible implementation manner of the first aspect, if the performance of the second target
model does not meet the third preset requirement, the model evaluator may be configured to
trigger the database to generate the final target physical plan to be executed by using the native
kernel component in the database, and also may feed back information of a failure of model update
(that is, the performance of the second target model does not meet the third preset requirement) to
the model manager, so that the model manager adjusts the fine-tuning policy for the first target
model based on the information.

In the above embodiment of the present application, the model evaluator may further feed back
information of the model update failure to the model manager, so that the model manager provides
reference for a subsequent model training strategy, thereby improving the model training capability
and the training efficiency.

In a possible implementation manner of the first aspect, the satisfying of the first preset requirement
by the execution overhead includes, but is not limited to:

1) The execution overhead of the target physical plan is lowest in q execution overheads, the q
execution overheads are the execution overheads corresponding to q physical plans generated based
on SQL statements of the input database, one physical plan corresponds to one execution overhead,
and q is greater than or equal to 1;

2) The execution overhead of the target physical plan is below a certain preset value (which may be
referred to as a first preset threshold). For convenience of illustration, in the following embodiments
of the present application, the situation that the execution overhead of the target physical plan is the
lowest among the q execution overheads and is taken as the execution overhead to meet the first
preset requirement is all taken, and details are not described in the following.

In the above embodiments of the present application, several specific situations that the execution
overhead meets the first preset requirement are specifically described, and the method has wide
applicability and flexibility.

In a possible implementation manner of the first aspect, the first target model satisfying the second
preset requirement includes, but is not limited to:

1) The performance of the first target model begins to be degraded;

5|Page
2) The performance of the first target model is not only degraded, but the degradation degree
reaches a certain preset value (which can be called as a second preset threshold value);

3) Evaluating the real-time performance of the first target model and predicting the next
performance, for example, predicting the performance of the first target model by using a model
evaluator, assuming that the predicted probability of performance degradation of the first target
model reaches a certain preset value (which may be referred to as a third preset threshold), for
example, the predicted probability of performance degradation reaches 80%;

4) The first target model continues to operate for a predetermined period of time, for example, the
first target model continues to operate for 30 minutes.

In the above embodiments of the present application, several specific situations that the first target
model meets the second preset requirement are specifically described, and the method has wide
applicability and flexibility.

In a possible implementation manner of the first aspect, the second target model satisfying the third
preset requirement may include, but is not limited to:

1) The performance of the second target model is improved by a certain preset value (which may be
referred to as a fourth preset threshold) compared with the performance of the first target model,
and as an example, the fourth preset threshold may be zero, which means that the second target
model is considered to meet the third preset requirement as long as the performance of the second
target model reaches the level of the performance of the original first target model; as another
example, the fourth preset threshold may also be a certain value or a certain proportion greater than
zero, which indicates that the second target model is considered to meet the third preset
requirement only if the performance of the second target model is improved to a certain degree
compared with the performance of the original first target model;

2) The performance of the second target model is increased by a fifth preset threshold compared to
the performance of the native kernel component within the database. And if the performance
improvement reaches a certain threshold value, the model used by the corresponding module of the
data is actually replaced, otherwise, the traditional database algorithm is adopted to execute the
target physical plan. The value of the fifth preset threshold may be zero, or may be a certain value or
a certain ratio greater than zero, which is specifically referred to the above first manner, and is not
described herein again.

In the above embodiments of the present application, several specific situations that the second
target model meets the third preset requirement are specifically described, and the method has
wide applicability and flexibility.

A second aspect of the embodiments of the present application further provides a data management
method, where the method includes: firstly, local computer equipment receives SQL statements sent
by client equipment to a database deployed in the computer equipment, wherein the database
comprises an optimizer and a database native kernel component, the optimizer comprises n models,
and n is greater than or equal to 1. After receiving an SQL statement sent by a client device, a
computer device may first determine whether n models in an optimizer have a model that does not
satisfy a predetermined requirement (which may be referred to as a second predetermined
requirement), and if a first target model (i.e., one of the n models) does not satisfy the second
predetermined requirement, the computer device may obtain a target physical plan through the n
models included in the optimizer according to the SQL statement, where the target physical plan is a

6|Page
physical plan whose execution overhead satisfies the first predetermined requirement. After
obtaining the final target physical plan, the computer device executes the target physical plan, and
the essence of the execution process is to execute the actual logic of the input SQL statement by
using the generated target physical plan.

In the above embodiments of the present application, how a computer device obtains a target
physical plan based on an optimizer included in a database and finally executes the target physical
plan is described, where the database deployed in the computer device includes an optimizer having
n models, so that a traditional heuristic optimizer is replaced, and by combining with a machine
learning technique, a logical query is converted into a physical execution plan with higher execution
efficiency.

In a possible implementation manner of the second aspect, the optimizer may specifically include
three models, which may be respectively referred to as a third model, a fourth model and a fifth
model, for performing the steps of logical query rewrite, cost estimation and physical plan
generation. In this case, the manner in which the computer device obtains the target physical plan
through the n models included in the optimizer according to the SQL statement may specifically be:
firstly, the computer device performs logic query rewriting on an SQL statement (also called SQL
query) input into the database through a third model to obtain a rewritten logic plan, wherein the
model a is a model constructed based on a tree search algorithm, for example, a monte carlo tree
search algorithm; then, generating q physical plans according to the logic plan by using a fourth
model, wherein the fourth model is a model constructed based on a deep learning algorithm, for
example, the fourth model can be a model based on Tree-LSTM, and q is more than or equal to 1;
and finally, calculating q execution overheads corresponding to the q physical plans through a fifth
model (one physical plan corresponds to one execution overhead), and determining a target physical
plan to be executed finally according to the q execution overheads, wherein the fifth model is a
model constructed based on a reinforcement learning algorithm, and may be a model based on DQN,
for example. It should be noted here that in other embodiments of the present application, the
optimizer may further include more or fewer models for implementing the processes of logical query
rewrite, cost estimation, and physical plan generation, and in the embodiment of the present
application, the optimizer includes three models for illustration only.

In the above embodiments of the present application, how the optimizer implements the functions
of logical query rewrite, cost estimation, and physical plan generation of the database through the
included model is specifically described, so that a traditional heuristic optimizer is replaced, the
logical query is converted into a physical execution plan with higher execution efficiency by
combining with a machine learning technology, and the problems of inaccurate cost evaluation and
poor physical plan generation of complex SQL statements caused by the current database
architecture problem can be effectively solved.

In a possible implementation manner of the second aspect, the computer device may further send
the running data of the process in the database deployed therein to a recommender, and the
recommender may be deployed in the computer device or may not be deployed in a remote device,
which is not limited herein. And the recommender receives the operation data, can send abnormal
data based on the operation data, diagnoses the abnormal reason based on the obtained abnormal
data, and optimizes a self-optimization module corresponding to the abnormal reason based on the
abnormal reason so as to reduce the abnormal probability of the operation data of the process in the
subsequent database, wherein the recommender comprises p models, and p is more than or equal
to 1.

7|Page
In the above embodiment of the present application, it is specifically stated that the computer device
may further feed back the operation data of the process in the database to the advisor, and the
advisor may give an all-directional optimization advice of the database based on the operation data,
so that unattended database performance monitoring and root cause identification may be achieved,
operation and maintenance manpower may be greatly liberated, and the database system may be
helped to quickly recover from an abnormality or improve performance.

In a possible implementation manner of the second aspect, the computer device may further send,
to the advisor, the operation data of the process in the database deployed therein, and may also
send, to a training data collector, where the training data collector may be deployed in the computer
device, and may also be deployed in a remote device, which is not limited herein. After the training
data collector receives the operation data, training data can be obtained according to the operation
data, and m training sets are constructed based on the training data, wherein m is larger than or
equal to 1.

In the above embodiments of the present application, it is specifically stated that the computer
device may also feed back the running data of the process in the database to the training data
collector, and the training data collector may generate the training data of the model related to the
database based on the running data of the process in the database, so as to achieve continuous
optimization of the database system, reduce the misjudgment probability of the database system,
and provide a trusted autonomous operation and maintenance service.

In a possible implementation manner of the second aspect, if the first target model meets the
second preset requirement, the computer device may further send a first instruction to a model
manager (the model manager may be deployed in the computer device, or may not be deployed in
the computer device, which is not limited herein), the first instruction being used to instruct the
model manager to perform fine tuning on the first target model, and, in a case that performance of
the second target model meets a certain preset requirement (which may be referred to as a third
preset requirement), the computer device receives the model parameters of the second target
model sent by the model manager, where the second target model is a model obtained by the model
manager performing fine tuning on the first target model by using a target training set corresponding
to the first target model, the target training set being one of m training sets, and finally, the
computer device updates the first target model to the second target model, and obtains the updated
n models (at this time, the updated n models do not include the first target model, and the second
target plan is a physical target plan).

In the foregoing embodiment of the present application, it is specifically stated that, when a first
target model does not meet a second preset requirement, a model manager is further required to
perform fine tuning on the first target model by using a target training set corresponding to the first
target model, and when performance of a second target model obtained after the fine tuning meets
a third preset requirement, the first target model is updated, so as to dynamically update and replace
a corresponding model used in a database according to a real-time operating state of the database.

In a possible implementation manner of the second aspect, if the performance of the second target
model does not meet the third preset requirement, the computer device further receives a second
instruction sent by a model evaluator, where the second instruction is used to instruct the database
to generate a final target physical plan by using a native kernel component in the database, and the
model evaluator may be deployed in the computer device or may not be deployed in a remote
device, which is not limited herein. The model evaluator is used to evaluate the performance of the
second object model.

8|Page
Database management system, data processing method and equipment

In the foregoing embodiments of the present application, it is specifically stated that when the
trimmed second target model still does not satisfy the third preset requirement, the computer
device receives a second instruction of the model evaluator, so as to instruct the database to
generate the target physical plan by using a conventional database algorithm (i.e., a native kernel
component). The embodiment of the application provides multiple choices for generation of the
target physical plan, and has flexibility.

In one possible implementation manner of the second aspect, the first target model satisfying the
second preset requirement includes but is not limited to:

1) The performance of the first target model begins to be degraded;

2) The performance of the first target model is not only degraded, but the degradation degree
reaches a certain preset value (which can be called as a second preset threshold value);

3) Evaluating the real-time performance of the first target model and predicting the next
performance, for example, predicting the performance of the first target model by using a model
evaluator, assuming that the predicted probability of performance degradation of the first target
model reaches a certain preset value (which may be referred to as a third preset threshold), for
example, the predicted probability of performance degradation reaches 80%;

4) The first target model continues to operate for a predetermined period of time, for example, the
first target model continues to operate for 30 minutes.

In the above embodiments of the present application, several specific situations that the first target
model meets the second preset requirement are specifically described, and the method has wide
applicability and flexibility.

In a possible implementation manner of the second aspect, the second target model satisfying the
third preset requirement may include, but is not limited to:

1) The performance of the second target model is improved by a certain preset value (which may be
referred to as a fourth preset threshold) compared with the performance of the first target model,
and as an example, the fourth preset threshold may be zero, which means that the second target
model is considered to meet the third preset requirement as long as the performance of the second
target model reaches the level of the performance of the original first target model; as another
example, the fourth preset threshold may also be a certain value or a certain proportion greater than
zero, which indicates that the second target model is considered to meet the third preset
requirement only if the performance of the second target model is improved to a certain degree
compared with the performance of the original first target model;

2) The performance of the second target model is increased by a fifth preset threshold compared to
the performance of the native kernel component within the database. And if the performance
improvement reaches a certain threshold value, the model used by the corresponding module of the
data is actually replaced, otherwise, the traditional database algorithm is adopted to execute the
target physical plan. The value of the fifth preset threshold may be zero, or may be a certain value or
a certain ratio greater than zero, which is specifically referred to the above first manner, and is not
described herein again.

9|Page
In the above embodiments of the present application, several specific situations that the second
target model meets the third preset requirement are specifically described, and the method has
wide applicability and flexibility.

In one possible implementation manner of the second aspect, the satisfying of the first preset
requirement by the execution overhead includes but is not limited to:

1) The execution overhead of the target physical plan is lowest in q execution overheads, the q
execution overheads are the execution overheads corresponding to the q physical plans generated
based on SQL statements of the input database, and one physical plan corresponds to one execution
overhead, wherein q is more than or equal to 1;

2) The execution cost of the target physical plan is below a certain preset value (which may be
referred to as a first preset threshold). For convenience of illustration, in the following embodiments
of the present application, the situation that the execution overhead of the target physical plan is the
lowest among the q execution overheads and is taken as the execution overhead to meet the first
preset requirement is all taken, and details are not described in the following.

In the above embodiments of the present application, several specific situations that the execution
overhead meets the first preset requirement are specifically described, and the method has wide
applicability and flexibility.

A third aspect of embodiments of the present application provides a computer device having a
function of implementing a method according to any one of the second aspect and the second
possible implementation manner. The function can be realized by hardware, and can also be realized
by executing corresponding software by hardware. The hardware or software includes one or more
modules corresponding to the functions described above.

A fourth aspect of the embodiments of the present application provides a computer device, which
may include a memory, a processor, and a bus system, where the memory is used to store a program,
and the processor is used to call the program stored in the memory to execute the method of any
one of the second aspect or the possible implementation manner of the second aspect of the
embodiments of the present application.

A fifth aspect of the embodiments of the present application provides a computer-readable storage
medium, which stores instructions that, when executed on a computer, enable the computer to
perform the method of any one of the second aspect and the possible implementation manner of
the second aspect.

A sixth aspect of embodiments of the present application provides a computer program which, when
run on a computer, causes the computer to perform the method of any one of the second or third
possible implementation manners.

A seventh aspect of the embodiments of the present application provides a chip, where the chip
includes at least one processor and at least one interface circuit, the interface circuit is coupled to
the processor, the at least one interface circuit is configured to perform a transceiving function and
send an instruction to the at least one processor, and the at least one processor is configured to
execute a computer program or an instruction, and has a function of implementing the method
according to any one of the above-mentioned second aspect or any one of the above-mentioned
second aspect, where the function may be implemented by hardware, software, or a combination of
hardware and software, and the hardware or software includes one or more modules corresponding

10 | P a g e
to the above-mentioned function. In addition, the interface circuit is used for communicating with
other modules outside the chip.

Drawings

FIG. 1 is a schematic diagram of a system architecture for constructing a database management

system according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a logical architecture of a database management system according to

an embodiment of the present application;

FIG. 3 is a schematic diagram of an optimizer provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of a model evaluator provided in an embodiment of the present

application;

FIG. 5 is a schematic diagram of a recommender provided in accordance with an embodiment of the

present application;

FIG. 6 is a schematic diagram of an optimizer comparing three re-strategies according to an

embodiment of the present application;

FIG. 7 is a schematic diagram of a model evaluator provided in an embodiment of the present

application in comparison to two known performance evaluation methods;

FIG. 8 is a schematic flow chart illustrating a data processing method according to an embodiment of
the present application;

FIG. 9 is a schematic structural diagram of a computer device according to an embodiment of the

present application;

fig. 10 is another schematic structural diagram of a computer device according to an embodiment of

the present application.

Detailed Description

The embodiment of the application provides a database management system, a data processing
method and equipment, which are combined with a machine learning technology to realize the
functions of automatically executing database tuning, protection, updating and other conventional
database management tasks traditionally executed by a DBA without manual intervention.

The terms "first," "second," and the like in the description and claims of this application and in the
foregoing drawings are used for distinguishing between similar elements and not necessarily for
describing a particular sequential or chronological order. It is to be understood that the terms so
used are interchangeable under appropriate circumstances and are merely descriptive of the various
embodiments of the application and how objects of the same nature can be distinguished.
Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are
intended to cover a non-exclusive inclusion, such that a process, method, system, article, or
apparatus that comprises a list of elements is not necessarily limited to those elements, but may
include other elements not expressly listed or inherent to such process, method, article, or
apparatus.

The embodiments of the present application relate to a lot of related knowledge about databases,
models, etc., and in order to better understand the solution of the embodiments of the present

11 | P a g e
application, the following first introduces related terms and concepts that may be related to the
embodiments of the present application. It should be understood that the related conceptual
explanations may be limited by the specific details of the embodiments of the present application,
but do not mean that the present application is limited to the specific details, and that the specific
details of the embodiments may vary from one embodiment to another, and are not limited herein.

(1) Database with a plurality of databases

A database is a computer software system that stores and manages data in a data structure. The
concept of a database actually includes two layers of meaning: a. the database is an entity, which is a
"warehouse" capable of reasonably keeping data, in which a user stores transaction data to be
managed, and the two concepts of "data" and "library" are combined into the database. b.
Databases are new methods and techniques for data management that enable more appropriate
organization of data, more convenient maintenance of data, tighter control of data, and more
efficient use of data.

(2) Database software

The database software is deployed on a local device, such as a local server, a local terminal device
(e.g., a mobile phone, a smart watch, a personal computer, etc.), and usually exists in the form of a
single or multiple processes, so the database software may also be referred to as a database process.

(3) Neural network

The neural network can be composed of neural units, and can be specifically understood as a neural
network with an input layer, a hidden layer and an output layer, wherein generally the first layer is
the input layer, the last layer is the output layer, and the middle layers are hidden layers. Among
them, a neural network with many hidden layers is called a Deep Neural Network (DNN). The
operation of each layer in the neural network can be expressed mathematically

Figure BDA0003211156120000091

Describing, from the physical level, the work of each layer in the neural network can be understood
as performing the transformation of the input space to the output space (i.e. the row space to the
column space of the matrix) through five operations on the input space (the set of input vectors),
which include: 1. ascending/descending dimensions; 2. zooming in/out; 3. rotating; 4. translating; 5.
"bending". Wherein 1, 2, 3 are operated by

Figure BDA0003211156120000092

The operation of 4 is completed by "+ b", and the operation of 5 is realized by "a ()". The expression
"space" is used here because the object to be classified is not a single thing, but a class of things,
space refers to the set of all the individuals of such things, where W is the weight matrix of the
neural network layers, the weight matrix in the matrixEach value represents a weight value for one
neuron of the layer. The matrix W determines the spatial transformation of the input space to the
output space described above, i.e. W at each layer of the neural network controls how the space is
transformed. The purpose of training the neural network is to finally obtain the weight matrix of all
layers of the trained neural network. Therefore, the training process of the neural network is
essentially a way of learning the control space transformation, and more specifically, the weight
matrix.

(4) Loss function (loss function)

12 | P a g e
In the process of training the neural network, because the output of the neural network is expected
to be as close as possible to the value really expected to be predicted, the weight matrix of each
layer of the neural network can be updated according to the difference between the predicted value
of the current network and the really expected target value (of course, an initialization process is
usually carried out before the first updating, namely, parameters are configured in advance for each
layer in the neural network), for example, if the predicted value of the network is high, the weight
matrix is adjusted to be lower in prediction, and the adjustment is carried out continuously until the
neural network can predict the really expected target value. Therefore, it is necessary to define in
advance how to compare the difference between the predicted value and the target value, which are
loss functions or objective functions (objective functions), which are important equations for
measuring the difference between the predicted value and the target value. Taking the loss function
as an example, if the higher the output value (loss) of the loss function indicates the larger the
difference, the training of the neural network becomes a process of reducing the loss as much as
possible.

(5) Back propagation algorithm

In the training process of the neural network, a Back Propagation (BP) algorithm can be adopted to
correct the size of parameters in the initial neural network model, so that the reconstruction error
loss of the neural network model is smaller and smaller. Specifically, the error loss is generated by
passing the input signal forward until the output, and the parameters in the initial neural network
model are updated by back-propagating the error loss information, so that the error loss is
converged. The back propagation algorithm is a back propagation motion with error loss as a
dominant factor, aiming at obtaining the optimal parameters of the neural network model, such as a
weight matrix.

(6) Machine learning

Machine learning is a multi-field interdiscipline, relates to multiple disciplines such as probability

theory, statistics, convex analysis, algorithm complexity and the like, and is used for specially
researching how a computer simulates or realizes the learning behavior of human beings so as to
obtain knowledge or skills and reorganizing the existing knowledge structure to continuously
improve the functions of the computer. Several machine learning models used in the embodiments
of the present application are described below:

a. monte Carlo tree search

Monte Carlo Tree Search (MCTS) is a method for policy optimization on artificial intelligence
problems, and generally combines the generality of stochastic simulations with the accuracy of tree
searches for those parts that require movement planning in a composite game. Due to its
achievements on computer go and the potential for solving certain problems, the application field of
the monte carlo tree search algorithm can already cover any field (e.g. the problem of selection of a
rewrite sequence in query rewrite) that can be described in (state, action) form to predict results
through simulation, in addition to gaming.

b. Recurrent Neural Networks (RNN)

The RNN is a type of neural network, and the purpose of the RNN is to process sequence data. In the
traditional neural network model, from an input layer to a hidden layer to an output layer, all layers
are connected, and nodes between each layer are connectionless. But such a general neural network
is not capable of failing to address many problems. For example, to predict what the next word of a

13 | P a g e
sentence is, the previous word is typically used because the previous and next words in a sentence
are not independent. The RNN is called a recurrent neural network, i.e., the current output of a
sequence is also related to the previous output. The concrete expression is that the network
memorizes the previous information and applies the previous information to the calculation of the
current output, namely, the nodes between the hidden layers are not connected any more but
connected, and the input of the hidden layer not only comprises the output of the input layer but
also comprises the output of the hidden layer at the last moment. In theory, RNNs can process
sequence data of any length.

Because a simple RNN cannot deal with the problems of exponential weight explosion or gradient
disappearance along with recursion, long-term time association is difficult to capture; and combining
different LSTM may solve this problem well. Time-cycled neural networks can describe dynamic time
behavior because, unlike feed-forward neural networks (feed-forward neural networks) which accept
inputs of more specific structures, RNNs cycle states through their own network and thus can accept
more extensive time-series structural inputs.

c. Long-short term memory artificial neural network (LSTM)

The LSTM, which may also be referred to as a long-short term memory network, is a time-cycling
neural network designed specifically to solve the long-term dependence problem of the general
RNNs, all of which have a chain form of repeating neural network modules. In the standard RNN, this
repeated structure block has only a very simple structure, e.g. one tanh layer.

d. Tree-short term memory artificial neural network (Tree-LSTM)

The traditional LSTM can keep the sequence information and has better performance on the task of
the sequence model. However, many problems in the database are modeled as a tree structure, such
as a logical plan and a physical plan of a query statement, and LSTM can only input a linear sequence,
so that the input of the tree cannot be well processed. The Tree-LSTM mainly expands the LSTM to a
Tree-shaped input structure, and is superior to a traditional LSTM model in semantic relevance and
semantic classification tasks on a prediction Tree.

e. Convolutional Neural Networks (CNN)

The CNN is a deep neural network with a convolution structure, and the convolution neural network
comprises a characteristic extractor consisting of convolution layers and sub-sampling layers. The
feature extractor may be viewed as a filter and the convolution process may be viewed as convolving
an input image or convolved feature plane (feature map) with a trainable filter. The convolutional
layer is a neuron layer for performing convolutional processing on an input signal in a convolutional
neural network. In convolutional layers of convolutional neural networks, one neuron may be
connected to only a portion of the neighbor neurons. In a convolutional layer, there are usually
several characteristic planes, and each characteristic plane may be composed of several neural units
arranged in a rectangular shape. The neural units of the same feature plane share weights, where the
shared weights are convolution kernels. Sharing weights may be understood as the way image
information is extracted is location independent. The underlying principle is: the statistics of a certain
part of the image are the same as the other parts. Meaning that image information learned in one
part can also be used in another part. We can use the same learned image information for all
locations on the image. In the same convolution layer, a plurality of convolution kernels can be used
to extract different image information, and generally, the greater the number of convolution kernels,
the more abundant the image information reflected by the convolution operation.

14 | P a g e
The convolution kernel can be initialized in the form of a matrix of random size, and can be learned
to obtain reasonable weights in the training process of the convolutional neural network. In addition,
sharing weights brings the direct benefit of reducing connections between layers of the
convolutional neural network, while reducing the risk of overfitting.

f. Graph convolutional neural network (GCN)

Motivated by the enormous success of CNN in the field of computer vision, a number of approaches
have recently emerged to redefine the concept of convolution for image data. These methods fall
into the category of GCN. Since spectral methods typically process the entire graph simultaneously
and are difficult to parallelize or extend onto large graphs, space-based GCNs perform convolution
directly on the graph structure by aggregating the information of neighboring nodes. In combination
with a sampling strategy, calculation can be performed in a batch of nodes instead of the whole
graph, so that the processing efficiency of problems (such as concurrent query modeling) related to
the graph model in the database can be effectively improved.

g. Reinforcement Learning (RL)

RL is one area in machine learning that emphasizes how to act on an environmental basis to achieve
maximum expected benefit. RL is the third basic machine learning method in addition to supervised
learning and unsupervised learning. Unlike supervised learning, the RL does not require tagged
input/output pairs, nor does it require precise correction of non-optimal solutions. The focus is to
find the balance of exploration (for unknown domains) and exploitation (for known domains), the
"exploration-exploitation" exchange in reinforcement learning, and the most studied in the dobby
slot machine problem and the limited Markov Decision Process (MDP).

For example, in the embodiment of the present application, for the query statement of the multi-join
operation, the RL controls the execution sequence of each join operation, and guarantees that the
total query execution overhead is minimum. The RL also has a training process, and needs to
continuously execute actions, observe the effect after the actions are executed, and accumulate
experiences to form a model. Unlike supervised learning, where each action generally has no directly
calibrated tag value as a supervisory signal, the system gives only one feedback to the action
performed by the algorithm.

(7) Remote procedure call protocol (RPC)

RPC is a protocol that requests services from a remote computer program over a network without
knowledge of the underlying network technology.

In the embodiment of the application, the RPC is used for realizing the rapid interaction between the
database kernel component and the external model manager, such as the proposal of a model
updating request, the creation of a new model and the like.

In addition, before the embodiments of the present application are introduced, a few management
architectures of the current database management system are briefly introduced, so that the
embodiments of the present application are convenient to understand in the following.

Mode one, database management system SageDB

The core idea of the SageDB is to construct a plurality of Cumulative Distribution Functions (CDFs)
related to data distribution, and generate a learning index, a replacement cost estimation model, an
acceleration physical operator, and the like by using the CDFs.

15 | P a g e
As a conceptual system, sageDB first assumes that it can learn a "perfect" CDF model, i.e. the
probability distribution of the model exactly matches the data distribution of the corresponding data
table, and then inserts the CDF model in different modules of the database, respectively, to provide
reasoning capability based on machine learning: 1) For the optimizer, the SageDB directly replaces
the cost estimation model with the CDF model obtained by learning on the single table, and
estimates the cost (cost) and cardinality (cardinality) of different queries; 2) For a data structure, the
SageDB directly replaces blocks (blocks) on a traditional multipath search Tree (B-Tree) with learned
CDF models, summarizes abnormal condition lists and is used for correcting the problem of CDF
positioning errors; 3) For physical operator acceleration, taking a sorting operation as an example,
the SageDB performs rough sorting (inputting a data value and outputting a relative position serial
number) on the underlying data according to the learned CDF model, and then obtains a final sorting
result by using a traditional sorting algorithm (e.g., fast sorting).

SageDB is still in research gas, and a simple CDF experimental model is adopted, so that the problems
of large-scale data set adaptation and the like are difficult to realize; moreover, all learning functions
of the SageDB are based on a CDF model cluster obtained by learning on a single table, so that the
SageDB is suitable for a simple single-table query scene, but cannot effectively solve the problem of
multi-table connection; furthermore, CDFs are only used to learn data or load distribution, and
cannot provide intelligent decision-making functions such as query rewrite, query plan generation,
anomaly diagnosis, etc., and SageDB does not provide model management and model update
mechanisms for multiple CDFs.

Second mode, relational database management system Oracle

Oracle databases have long invested in automated operation and maintenance of databases, and
Oracle 10g introduces various self-management functions to simplify management, improve
efficiency, and reduce the overall costs associated with system management. These management
functions include: 1) Statistical analysis related to SQL query optimization; 2) An automated storage
manager: simplifying the storage modes of data files, control files and log files; 3) Automated
workload storage library: storing and managing information for self-adjustment; 4) Automated
database diagnostic monitor: analyzing the stored statistical data, identifying possible performance
bottlenecks and providing suggestions for solving the discovered problems; 5) Automatic query
optimization: determining an efficient way to execute a Structured Query Language (SQL) query by
using query rewrite rules and a cost model; 6) Tuning suggestions are automatically generated for
SQL statements or workloads. These recommendations are provided to the user, who subjectively
accepts or rejects them; 7) The SQL tune suggestions make decisions based on information provided
by the query optimizer, including an automated database diagnostic monitor and an automated
workload repository; 8) Indexes (including bitmap indexes, function-based indexes and B-tree
indexes), materialized views and table partitions and indexes are recommended according to the
current load amount. Acquiring contents from the SQL cache, and selecting proper indexes and
materialized views after analysis; 9) An optimizer statistics collector to collect statistics on the
optimization; 10 ) coordinate all autonomous management within the server by managing database
snapshots and storing information; 11 Server generates alerts, configuring the system to
automatically generate alerts upon triggering events; 12 Automated pre-installation and post-
installation tasks: the system is automatically checked before installation to ensure success of the
installation process and suggest changes; 13 Automatically managing shared memory used by the
Oracle database instance and enabling an administrator to not need to manually configure shared
memory components; 14 Database resource manager allows the DBA to logically divide the workload
into different units and allocate Central Processing Unit (CPU) resources to these units without

16 | P a g e
additional overhead. During peak hours, the on-line transaction processing (OLTP) should take
precedence over the on-line analytical processing (OLAP), and vice versa. Its scheduling mechanism is
used for fixed time intervals and controls the number of active sessions performed at one time.
When the available slots of an active session are filled by a new session, the remaining sessions will
be queued until a slot is available. The first autonomous database release version is 19c, the service
is provided for the outside in a public cloud mode, and the functions of automatic indexing,
distribution columns, materialized view recommendation and the like are included.

Database management system, data processing method and equipment

However, the optimization capability of the Oracle database is mainly presented in the form of
single-point and independent functions, the functions are not unified to form a closed loop, and a
user needs to call the functions as required according to the requirement of the user; in addition, the
autonomous function of the Oracle database is mainly embodied in the stages of data statistics,
analysis and management, and the optimization capability of the database exception is limited based
on a limited rule or traditional statistical learning; moreover, the Oracle database does not provide a
unified model and a training data management and updating mechanism, does not provide a
verification function of component performance, and belongs to passive tuning.

In summary, to solve the above problems, embodiments of the present application provide a new
database management system, which implements self-learning kernel and model optimization based
on machine learning algorithm and expert experience, and constructs an all-directional autonomous
function of a database.

Embodiments of the present application are described below with reference to the accompanying
drawings. As can be known to those skilled in the art, with the development of technology and the
emergence of new scenarios, the technical solution provided in the embodiments of the present
application is also applicable to similar technical problems.

First, a system architecture of a database management system constructed in an embodiment of the

present application is described, specifically referring to fig. 1, fig. 1 is a schematic diagram of a
system architecture of a database management system constructed in an embodiment of the present
application, where the system architecture includes database software 101, a machine learning
platform component 102, and a self-learning recommender 103 (may be simply referred to as a
recommender 103), and module functions of each structure are introduced below:

(1) Module function of database software 101

The database software 101 (like server software) is deployed on a local device, for example, a local
server, a local terminal device (such as a mobile phone, a personal computer, etc.), and usually exists
in the form of a single or multiple processes. The system architecture of the embodiment of the
application comprises a self-learning kernel component of the database, replaces or replaces the
traditional algorithm or implementation of the native kernel component of the database (note that
the native kernel component is not deleted or is in the database process), and realizes the
improvement of the overall reliability or performance of the database. The self-learning kernel
component means that a single point replaces an algorithm through a machine learning method, the
outstanding capability is to automatically adapt to a scene based on system load change or service
state change, automatically update a training model based on the algorithm, access a feedback
mechanism and a verification mechanism, automatically drift the model and continuously maintain
the usability of the model.

17 | P a g e
It should be noted that, in the embodiment of the present application, the self-learning kernel
component may be identified as a self-learning optimizer 1011 (may be simply referred to as the
optimizer 1011), a self-learning index, a self-learning storage, a self-learning executor, and the like,
and different self-learning kernel components may correspondingly implement functions of
corresponding modules in the native kernel component.

(2) Modular functionality of machine learning platform component 102

In the embodiment of the present application, regardless of which machine learning algorithm is
used, the data source is a database system, and includes, but is not limited to, database internal
indicators (e.g., number of Transactions Per Second (TPS), cache hits, active transactions, resource
usage, etc.), operating system information, log information, and the like. The information is used for
model training, and specifically, the information may be written into a training data collector 1021
(which may also be referred to as a training data collection platform) through an information
collector, and a model manager 1022 (which may also be referred to as a model management
platform) completes model training based on an algorithm to be trained in combination with the
data information. The trained models will be pushed to the model evaluator 1023, and only those
models that are evaluated to meet the business expectations will be identified as the model to be
applied, otherwise readjustment and training will be required.

It should be noted here that the information collector may be deployed in the database software
101, or may be a process separately deployed outside the database software 101, where the purpose
of the separate deployment is to decouple from the database and is used to implement a specific
function (i.e., a function of collecting operation data of a process in the database).

It should be further noted that in other embodiments of the present application, since the function
of the information collector is to collect data, for convenience of illustration, the function of the
information collector may be integrated in the optimizer 1011 or the training data collector 1021,
which is not limited in this application.

It should be noted that, in this embodiment of the present application, the model included in the
machine learning platform component 102 may be pre-trained in advance, and the models deployed
in the system architecture may refer to pre-trained models, in this case, the training data collected by
the training data collector 1021 in this embodiment of the present application may be used to
perform fine tuning on the pre-trained models (for example, when the performance of the models
decreases after a period of time elapses, the fine tuning may be performed); the model included in
the machine learning platform component 102 may also be a model that is not pre-trained in
advance, but an initialized model is directly deployed in the system architecture, and then training
and subsequent fine tuning are performed on each model by using training data collected by the
training data collector 1021, specifically, the present application does not limit the state (i.e.,
whether pre-trained) of the model included in the machine learning platform component 102 when
deployed.

It is further noted that, in some embodiments of the present application, the machine learning
platform component 102 can be deployed on a remote device (e.g., a remote server, which is
illustrated in fig. 1 as being deployed on a remote device), can be deployed on a local device (e.g., a
local server), or even be implemented with a database co-process (i.e., implemented in a database
kernel). The individual component program is a minimally invasive mode for the existing capacity of
the database, and can be used as an attachment for iteratively and gradually replacing the capacity
of the core module of the database. The method is also deployed in local equipment, so that the

18 | P a g e
method is less invasive to the database, but the same equipment resource is forced by the database,
and a new scheduling component is generally required to balance and manage the resource. A
machine learning component platform is integrated in a database kernel, namely the database
provides machine learning (including deep learning, reinforcement learning and the like), the
intrusiveness to the database is high, the data privacy protection is good, the communication
overhead is reduced, the interface is indirectly and conveniently realized, and the model tuning or
fine tuning is easier.

(3) Module function of the advisor 103

The advisor 103 is used for finding out possible problems in the database operation process,
diagnosing and optimizing the problems, and is used for intelligent operation and maintenance
management of the database. The advisor 103 also needs a machine learning platform component
for algorithm model management used in the intelligent operation and maintenance process. The
machine learning platform used may be homologous to the machine learning platform component
used by the database system, that is, both may use the same machine learning platform, or may be
managed separately, that is, the model in the recommender 103 may deploy one machine learning
platform component separately, which is not limited in this application, but the function or
mechanism of the machine learning platform component does not change, and the model is
automatically updated to provide a learning and feedback mechanism, so as to ensure the usability
of the model.

It should be noted here that the recommender 103 is not embodied in the database kernel
capability, but is used for database management, and may adjust or enhance the capability provided
by the database kernel module. Through interaction with the database, the recommender 103
obtains more information and recommendations, and the model is more optimized, thereby
facilitating the intelligent implementation of the system.

Usually, the recommender 103 needs to report the diagnosis information and the health index, and
also needs to receive an instruction from the user, and at this time, a web front end is needed to
implement the function, and the implementation manner is a conventional operation, which is not
described herein again.

It should be further noted that, in the above embodiments of the present application, the system
architecture of the data management system includes the advisor 103, and as can be seen from the
above description, the module function of the advisor 103 is to find out that there may be a problem
in the database operation process, and perform diagnosis and tuning. Thus, in other embodiments of
the present application, the advisor 103 may not be needed.

Based on the above description of the system architecture of the data management system, specific
functions and calling logic of each function module under the database management system
provided in the embodiment of the present application are specifically described below, where each
function module is respectively an optimizer 1011, a training data collector 1021 (a training data
collector may also be referred to as a training data collection platform), a model manager 1022 (a
model manager may also be referred to as a model management platform), a model evaluator 1023,
and a recommender 103 (in some embodiments, the recommender 103 may not be needed).
Referring to fig. 2 in detail, fig. 2 is a schematic diagram of a logical architecture of a database
management system according to an embodiment of the present application, where an SQL query
parser and a storage engine in the database management system are existing modules, and others
are newly added modules of the system, and the newly added modules include a self-learning

19 | P a g e
optimizer 201 (note that, some of the kernel components included in the optimizer 201 provided by
the present application may be native kernel components, but n models included in the optimizer
201 are newly added by the present application), a training data collector 202, a model manager 203,
a model evaluator 204, and a self-learning recommender 205 (in some embodiments, the
recommender 205 may not be needed), and each functional module is described below from specific
functions and call logics:

1. optimizer 201

Most of the problems in the optimizer (e.g., query rewrite, cost estimation, and physical plan
generation) are NP-hard problems (it is difficult to solve the non-deterministic problem with an
efficient algorithm), and existing optimization techniques employ heuristic methods that may fall into
local optimality. Therefore, to address these issues, the learning-type optimizer provided by
embodiments of the present application uses machine learning techniques to improve performance.

Specifically, in the embodiment of the present application, n models are included in the optimizer
201, where n ≧ 1. The optimizer 201 is configured to obtain a final physical plan to be executed
(which may be referred to as a target physical plan) through the n models according to the SQL
statements input into the database, where the target physical plan refers to a physical plan whose
execution overhead meets a certain preset requirement (which may be referred to as a first preset
requirement).

It should be noted that, in some embodiments of the present application, the execution overhead
satisfying the first preset requirement includes, but is not limited to: 1) The execution overhead of
the target physical plan is lowest in q execution overheads, the q execution overheads are the
execution overheads corresponding to the q physical plans generated based on SQL statements of
the input database, and one physical plan corresponds to one execution overhead, wherein q is more
than or equal to 1; 2) The execution overhead of the target physical plan is below a certain preset
value (which may be referred to as a first preset threshold). For convenience of illustration, in the
following embodiments of the present application, the situation that the execution overhead of the
target physical plan is the lowest among the q execution overheads and is taken as the execution
overhead to meet the first preset requirement is all taken, and details are not described in the
following.

It should be noted that in some embodiments of the present application, the optimizer 201 may
specifically include three models, which may be respectively referred to as model a, model B, and
model C, for performing three steps of logical query rewrite, cost estimation, and physical plan
generation. It should be noted here that in other embodiments of the present application, the
optimizer 201 may further include more or fewer models for implementing the processes of logical
query rewriting, cost estimation, and physical plan generation, in this embodiment, the three models
included in the optimizer 201 are only exemplary, and are not described in detail later.

Specifically, the optimizer 201 performs logical query rewrite on an SQL statement (also referred to
as SQL query) input to the database through a model a, so as to obtain a rewritten logical plan,
where the model a is a model constructed based on a tree search algorithm, for example, a monte
carlo tree search algorithm; then, the optimizer 201 generates q physical plans according to the logic
plan by using a model B, where the model B is a model constructed based on a deep learning
algorithm, for example, a model based on Tree-LSTM, and q is greater than or equal to 1; finally, the
optimizer 201 calculates q execution costs corresponding to the q physical plans (one physical plan
corresponds to one execution cost) through a model C, and determines a target physical plan to be

20 | P a g e
executed finally according to the q execution costs, where the model C is a model constructed based
on a reinforcement learning algorithm, and may be a model based on DQN, for example.

It should be noted that, in some embodiments of the present application, the three steps performed
by the optimizer 201 may be generated by three sub-modules, namely, a learning rewriter, a learning
cost estimator, and a learning plan generator, and the process may specifically be: firstly, the rewriter
provided by the embodiment of the application converts an initial SQL statement input into a
database system into a semantically equivalent logic plan A which can be recognized by the database
system by using a model based on a tree search algorithm; and obtaining a rewritten logic plan B
with higher execution efficiency based on the cost estimator. The plan generator generates x physical
plans (for example, x = 5) based on the logic plan B, obtains respective execution overheads of the x
physical plans based on the cost estimator, selects a physical plan (i.e., a target physical plan) whose
execution overheads satisfy a first preset requirement, and finally executes actual logic of the initial
SQL statement using the generated target physical plan.

For ease of understanding, the optimization 201 includes three sub-modules, namely, a learning
rewriter, a learning cost estimator, and a learning plan generator, and the processes of logical query
rewrite, cost estimation, and physical plan generation performed by the optimizer 201 are described
in detail:

(1) Logical query rewrite

The learning rewriter rewrites an input SQL statement into an equivalent but less costly query using
MCTS-based methods. Firstly, a policy tree is constructed, wherein a root node is an original query,
and a tree node is a query obtained by rewriting from a parent node thereof by applying a rewrite
rule (the rewrite rule is a known technology, such as sub-query promotion, redundancy filtering
deletion and the like, which is not described herein). The rewriter utilizes an MCTS method, firstly, a
strategy tree which takes an input logic plan as a root node is constructed, wherein each child node
represents a semantic equivalent logic plan obtained by a father node through a rewriting operation;
then, iteratively selecting an equivalent logic plan with the minimum overhead or the minimum
selected probability on the strategy tree, and expanding the strategy tree (namely, adding new child
nodes under the corresponding tree nodes of the selected plan according to all the rewriting
strategies of the selected plan); and finally, selecting the logic plan with the minimum execution
overhead on the strategy tree as the output of the rewriter.

(2) Cost estimation

The learning-type cost estimator uses a deep learning-based approach to estimate the cost and
cardinality of the query, which can capture the correlation between different columns. The method
designs a tree structure model of a physical plan matching a query statement, wherein the analog
physical plan is composed of a plurality of sub-plans, each tree structure model can be composed of
a plurality of sub-models, and the tree structure model is used for estimating the cost or the
cardinality of the plan.

(3) Physical plan generation

The learning-based plan generator uses a reinforcement learning based approach to generate an
optimized physical plan (i.e., a target physical plan), where the logic is: the generated equivalent logic
plan corresponds to a plurality of execution plan trees, each execution plan tree comprises one or
more execution operators (also called physical operators), a plurality of execution paths under each
execution plan tree are possible, the execution paths relate to different execution operators, one

21 | P a g e
execution plan tree corresponds to one total cost, and the aim is to find the physical plan with the
minimum total cost. The method comprises the steps of selecting a connection sequence by using
reinforcement learning of LSTM with a Tree structure, specifically, encoding a current physical plan
into a compressed vector by using Tree-LSTM as a deep reinforcement learning state, iterating for
multiple times, selecting connection operation with the highest long-term benefit each time, and
finally outputting a physical plan with the lowest execution cost as actual logic for executing SQL
sentences. Embodiments of the present application may employ GCN to capture the structure of a
connection tree that supports database schema updates and table name multi-aliases. The model
may automatically select the appropriate physical operators.

To sum up, the calling logic of the optimizer 201 is: in the learning-based optimizer 201 provided in
the embodiment of the present application, for a logic plan input through the SQL query parser, the
rewriter first constructs a policy tree with the input logic plan as a root node, where each child node
represents an equivalent logic plan obtained by a rewrite operation on a parent node. The rewriter
searches the policy tree for the least expensive equivalent logic plan based on MCTS, and then inputs
the plan generator. The plan generator iteratively adjusts the order of the join operations to obtain a
plurality of different physical plans. For each physical plan, a cost estimator is used to estimate the
execution overhead and then the physical plan output with the least execution overhead is selected.

For understanding, the following describes the execution process of the optimizer 201 in detail based
on the principle of the optimizer 201, and referring to fig. 3 in particular, fig. 3 is a schematic diagram
of the optimizer provided in the embodiment of the present application, which rewrites SQL
statements, estimates costs, and selects a physical execution plan in the database execution process
through a machine learning model. The model in the process can be updated through feedback and
incremental training, so that the change of the load is dynamically adapted. The core steps are as
follows:

1) Before an SQL statement of an input database is executed, a logic query rewrite function (namely
a morphological study type rewriter) of an optimizer is called to rewrite the statement level of the
SQL statement, so as to prevent the performance problem caused by poor SQL writing habit. The
specific process is as follows:

a. first, a policy tree is constructed, where the root node is the input SQL query and the non-root
nodes are rewritten query statements. Discovering the rewriting order for obtaining the maximum
profit through an MCTS search algorithm, namely iteratively selecting an equivalent logic plan with
the minimum overhead or the least frequently selected overhead on the strategy tree, and
expanding the strategy tree (namely adding new child nodes under the corresponding tree nodes of
the selected plan according to all the rewriting strategies of the selected plan); and finally, selecting
the logic plan with the minimum execution overhead on the strategy tree as the output of the
rewriter.

b. Second, the potential benefit of each tree node is confirmed, a neural network-based benefit
estimation model is designed (e.g., the similarity between the attention level calculation rules and
the rules on the rewrite operator) according to the given query statement (the query in the original
or rewrite), the available rewrite rules and the data column information, and the execution cost
which can be reduced subsequently by the query statement is predicted.

c. Thirdly, in order to improve the searching efficiency, particularly when the number of query logic
operators is large, the optimal first N nodes without ancestor-descendant relations are selected from
each node and the subtree thereof by utilizing dynamic programming and calculation from bottom to

22 | P a g e
top, and the maximum total benefit value is ensured; and then outputting a node selection scheme
corresponding to the root node, wherein the node selection scheme indicates that the optimal
rewrite query is acquired from the highest probability of the corresponding N node expansion
strategy trees.

d. And fourthly, outputting the rewriting query statement with the minimum execution cost when
the maximum iteration times are reached or no new leaf node exists.

2) And calling a learning type cost estimator, calculating cost estimation by using a machine learning
model, and estimating the cardinality and cost of any execution plan by using a Tree-LSTM model for
the path selection of the optimizer. The specific process is as follows:

a. for off-line training, training data is input into a model to be trained through feature extraction
based on collected historical query sentences. During training, the weight of the model is updated in
a back propagation mode based on the current training loss.

b. When online use cost estimation is carried out, if the sub-plan of the current node root is already
estimated, acquiring sub-plan estimation information from a cache pool; if the current sub-plan is
not evaluated, the root is encoded, the encoded plan vector is input into the Tree-LSTM model, then
the model returns the evaluated cost and plan to the optimizer, and the new evaluation information
is put into a cache pool for use by subsequent query statements.

3) When the complex SQL sentences are detected, the model based on deep reinforcement learning
is called to perform execution path enumeration, and compared with heuristic algorithms based on
genetic algorithm and the like in the traditional database, a feasible SQL execution plan can be
obtained more quickly and efficiently. The specific process is as follows:

a. the best plan is found by combining the Deep Q Network (DQN) with the Tree-LSTM model.

b. Firstly, initializing an empty state, only containing basic information of a query, and then further
designing a plurality of intermediate states, wherein each intermediate state contains a part of
selectable plans of a plan tree to form a plan forest.

c. The training process is divided into cost training and delay adjustment. The cost training
continuously selects segments in the execution plan by a reinforcement learning method, and judges
whether the operation of the newly selected plan conforms to the optimal plan or not; in the
process, the Q value retrieval method in the Tree-LSTM model is used for preliminarily determining
the quality of the plan. In delay adjustment, only a few of the planned delays are used as training
data for model fine-tuning.

d. DQN is estimated using the Q network and finds that the execution tree is better. In the plan tree,
there are three types of leaf nodes, including columns, tables, and operations. And traversing the
plan Tree by using depth-first search, and judging the performance cost of each leaf node by using
the Tree-LSTM network layer.

4) When the simple SQL sentence is detected, searching an optional execution plan list in a plan
buffer area by using a machine learning algorithm based on sentence characteristics and data
distribution, and matching a similar plan to be used as a final execution plan. The specific process is
as follows:

a. and constructing a buffer area, and adding the plan of the simple SQL sentence into the buffer
area when the simple SQL sentence is executed for the first time.

23 | P a g e
b. In the second and subsequent executions of the simple SQL statement, based on the 90. And if the
regenerated plan does not exist in the buffer, adding the regenerated plan into the buffer as a
candidate plan, wherein at most y candidate plans exist in each statement, wherein y can be
customized, and for example, y =5.

c. And matching the characteristics of the new execution statement with the information in the
buffer plan through a K-nearest neighbor (KNN) classification algorithm, and selecting the execution
plan which is successfully matched. That is, the matching plan is a true executable plan for the
statement.

5) And the database execution engine executes the optimized SQL statement execution plan, namely,
executes a final target physical plan.

2. Training data collector 202

In the embodiment of the present application, the training data collector 202 is configured to obtain
training data according to the running data of the processes in the database, and construct m
training sets based on the obtained training data, where m is greater than or equal to 1.

Specifically, in some embodiments of the present application, the training data collector 202 may
automatically collect statistical information of a database, including database operation indexes,
query logs, system logs, and the like, generate training data of all learning models involved in the
database management system (i.e., models included in the recommender 205 (if any), the optimizer
201, and the model evaluator 204) by using the statistical information, and may generate
corresponding training sets (i.e., construct m training sets) for different models respectively.

As an example, assuming that the optimizer 201, the recommender 205, and the model evaluation
204 include a total of 6 models, a corresponding 6 different training sets, i.e., m =6, may be
generated based on respective model characteristics, in which case one model corresponds to one
training set. In other embodiments of the present application, the number of training sets
constructed may also be less than 6, that is, m < 6, in which case some models may share one
training set, and the present application does not limit the number of the constructed training sets to
the corresponding relationship between their own training sets and models.

It should be noted that the model according to the embodiment of the present application may be
pre-trained in advance, that is, the models deployed in the database management system are pre-
trained models, and in this case, the training data collected by the training data collector 202
according to the embodiment of the present application may be used to perform fine tuning on the
pre-trained models (for example, fine tuning may be performed when the performance of the
models is degraded after the models are applied for a period of time); the model related to the
embodiment of the present application may also be a model that is not pre-trained in advance, but
an initialized model is directly deployed in the database management system, and then training and
subsequent fine tuning are performed on each model by using the training data collected by the
training data collector 202.

It should also be noted that in the embodiment of the present application, the training data collector
202 may collect the operation data of the processes in the database from a plurality of aspects,
including but not limited to: 1) Database indexes are as follows: the running state of the database,
such as Query Per Second (QPS), CPU utilization, cache hit, etc. These are typically represented by
time series data; 2) SQL query: it collects SQL queries and their statistical information such as
physical plans, response times and durations, etc. 3) Database logging: the method collects the

24 | P a g e
running logs, and because different models in the database management system need different
training data, the embodiment of the application can intelligently organize the training data for
different learning modules, including organizing related columns into the same table to reduce
connection overhead, selecting the training data for the models and the like.

Database management system, data processing method and equipment

In summary, the call logic of the training data collector 202 is: the training data collector 202 receives
the operating data (e.g., information collected by an Agent program of a database) from the process
in the database, and performs data cleaning and data processing operations on the received
operating data (e.g., through data cleaning, data merging, multi-index direct correlation analysis,
etc., so that the data is more suitable for subsequent model training or fine tuning), to obtain
training data, which is constructed into m training sets for training or fine tuning each model in the
database management system. If the model is not pre-trained, training the model based on a
specified algorithm and training data; if the model is a model which is already pre-trained, evaluating
the relationship between newly obtained training data and the pre-trained model, continuously
monitoring and determining whether the model needs to be updated; the monitoring time is based
on a data source of a training model, and whether the data belongs to data which has high frequency
and is easy to change or not is judged.

3. Model manager 203

In this embodiment, the model manager 203 is configured to, when a first target model meets a
certain preset requirement (which may be referred to as a second preset requirement), perform fine
tuning (finetune) on the first target model by using a target training set corresponding to the first
target model, so as to obtain a second target model (which is substantially the first target model with
updated model parameters). Wherein the first target model is one of the n models, and the target
training set is one of the m training sets. Then, the model parameters of the second target model
after finetune may be further transmitted to the model evaluator 204 for model performance
evaluation.

It should be noted here that values of m and n may be the same or different, where m = n indicates
that each model in the n models corresponds to a training set used by itself, and where m ≠ n
indicates that there may be a case where a plurality of models share one training set (i.e., a case
where m < n) or a case where one model may have a plurality of training sets for training (i.e., a case
where m > n), and the present application is not limited to this.

It is further noted that, in some embodiments of the present application, the first target model
satisfies the second preset requirement including, but not limited to: 1) The performance of the first
target model begins to be degraded; 2) The performance of the first target model is not only
degraded, but the degradation degree reaches a certain preset value (which can be called as a
second preset threshold value); 3) Evaluating the real-time performance of the first target model and
predicting the next performance, for example, predicting the performance of the first target model
by the model evaluator 204, assuming that the predicted probability of performance degradation of
the first target model reaches a certain preset value (which may be referred to as a third preset
threshold), for example, the predicted probability of performance degradation reaches 80%; 4) The
first target model continues to operate for a predetermined period of time, for example, the first
target model continues to operate for 30 minutes.

In the embodiment of the present application, the model manager 203 integrates a common
machine learning capability, and is configured to provide a uniform application access interface and

25 | P a g e
support management and scheduling of a learning model. Specifically, the model manager 203
generates a more optimal model to fit the current system operating state based on the training data
updated by the training data collector 202.

In summary, the calling logic of the model manager 203 is: after receiving the training data, the
model manager 203 determines whether the model needs to be updated, and if the model needs to
be updated, the model manager transmits the model parameters of the model to the model verifier
204 after completing updating the model.

4. Model evaluator 204

In this embodiment, the model evaluator 204 is configured to evaluate performance of the obtained
second target model, and update the first target model in the optimizer 201 to the second target
model when the performance of the second target model meets a certain preset requirement (which
may be referred to as a third preset requirement). As an example, the updating process may
specifically be: the model evaluator 204 sends the model parameters of the second object model to
the optimizer 201, and the optimizer 201 assigns the received updated model parameters to the first
object model, thereby obtaining the second object model. It is noted that in some embodiments of
the present application, the model evaluator 204 may be a graph-embedding based performance
prediction model.

It should be noted that, in some embodiments of the present application, the second target model
may meet the third preset requirement, which includes but is not limited to: 1) The performance of
the second target model is improved by a certain preset value (which may be referred to as a fourth
preset threshold) compared with the performance of the first target model, and as an example, the
fourth preset threshold may be zero, which means that the second target model is considered to
meet the third preset requirement as long as the performance of the second target model reaches
the level of the performance of the original first target model; as another example, the fourth preset
threshold may also be a certain value or a certain proportion greater than zero, which indicates that
the second target model is considered to meet the third preset requirement only if the performance
of the second target model is improved to a certain degree compared with the performance of the
original first target model; 2) The performance of the second target model is increased by a fifth
preset threshold compared to the performance of the native kernel component within the database.
And if the performance improvement reaches a certain threshold value, the model used by the
corresponding module of the data is actually replaced, otherwise, the traditional database algorithm
is adopted to execute the target physical plan. The value of the fifth preset threshold may be zero, or
may be a certain value or a certain ratio greater than zero, which is specifically referred to the above
first manner, and is not described herein again.

It should be further noted that, in some embodiments of the present application, if the performance
of the second target model does not meet the third preset requirement, the model evaluator 204
will further be configured to trigger the database to generate the target physical plan to be finally
executed by using the native kernel component in the database, for example, the index selection
module enables a conventional hill-climbing algorithm to create a new index, so as to execute the
logic of the SQL statement. That is, if the performance of the second target model after finetune still
does not meet the requirement, the target physical plan is generated by using the database
conventional kernel component algorithm. Because the native kernel component of the database is
not deleted, but is stored in the database software together with the newly added optimizer, the
database performance can be improved in any mode in the database operation process, and the

26 | P a g e
dynamic adjustment is carried out in real time, so that the database performance is integrally
improved.

It should be further noted that, in other embodiments of the present application, if the performance
of the second target model does not meet the third preset requirement, the model evaluator 204
may be configured to trigger the database to generate the target physical plan to be executed finally
by using the native kernel component in the database, and also may feed back information of the
failure of model update (that is, the performance of the second target model does not meet the third
preset requirement) to the model manager 203, so that the model manager 203 adjusts the fine-
tuning policy for the first target model based on the information, and provides a reference for the
subsequent model training policy.

Based on the above, the model evaluator is to verify whether the model is valid for the workload,
and if a learning model is used for the database, the model evaluator 204 can be used to predict the
model performance. For the models deployed in the database management system provided in the
embodiments of the present application, the present application can perform performance
prediction through the model evaluator 204. If the performance of the model is good (for example,
the performance improvement reaches a certain threshold), identifying the obtained new model
(namely, the original model updates the model parameters) as the optimal model, and actually
deploying the optimal model to the database management system; otherwise, the model is marked
to be required to be updated, and the deployment is abandoned.

In summary, the calling logic of the model evaluator 204 is: the model evaluator 204 obtains the
latest model generated by the model manager 203 and verifies whether the model is stable and
reliable, and at the same time, the system performance can be improved. The verification results are
fed back to the model manager 203 identifying it as the best model or requiring a model update.

For convenience of understanding, the following describes the execution process of the model
evaluator 204 in detail based on the principle of the model evaluator 204, and referring to fig. 4 in
particular, fig. 4 is a schematic diagram of a principle of the model evaluator provided in the
embodiment of the present application, the model evaluator performs performance evaluation on a
model (e.g., a model included in an optimizer or a recommender) deployed in the database
management system constructed in the embodiment of the present application, checks whether the
model can obtain a performance improvement, and abandons the deployment of the model if the
new model does not improve the performance. In particular, in some embodiments of the present
application, the model evaluator may be GNN-based, with the structure shown in fig. 4. The method
comprises the following core steps:

1) Firstly, vectorization representation is carried out on loads to be executed by a user, then a

performance prediction model based on GNN is called for evaluation, and estimated execution
effects can be given by combining load characteristics of a new model and an old model, so that
whether the new model is effective or not is compared.

2) Secondly, the workload map is input into the prediction model, and if the map is too large, the
prediction efficiency may be affected. Thus, embodiments of the present application propose a graph
compression algorithm that removes redundant vertices and merges similar vertices. The specific
implementation process is as follows:

a. first, in load graph construction, a graph model is used to capture workload characteristics,
wherein vertices represent operator characteristics extracted from a query plan, and edges between
two operators represent query correlation and resource competition between them.

27 | P a g e
b. Secondly, the characteristics of the performance prediction model are input into the prediction
model, in the model, a graph embedding algorithm is provided, graph characteristics (such as
operator characteristics and K-hop neighbors) are embedded at an operator level, and a deep
learning model is constructed to predict query performance.

c. In addition, if the graph is too large, the size of the load graph is reduced by combining the nodes
with time overlap through a graph compression algorithm in the load graph optimization program,
and the used method is that the nodes with time overlap are clustered according to the execution
time range of each node, and then the nodes without edge relation are combined in each class
through minimum complete connection subgraph (clique) division.

d. And finally, predicting the query performance and verifying whether the input model brings
benefits.

5. Advisor 205

Existing database monitoring, configuration, diagnostics, optimization methods (e.g., parameter

tuning, slow SQL diagnostics, index/view advisors) rely on DBA, are costly, and cannot accommodate
large-scale instances (e.g., cloud databases). Therefore, in order to solve the problem, the present
application may implement functions of self-monitoring, self-diagnosis, and self-optimization based
on a machine learning method so as to automatically and intelligently optimize the database.

Therefore, in the embodiment of the present application, the database management system may
further include a recommender 205, where the recommender 205 includes p models, where p ≧ 1.
The recommender 205 is configured to discover abnormal conditions (i.e., abnormal data is
discovered) existing in the running data (e.g., CPU utilization, user response time, etc.) of the
processes in the database, diagnose an abnormal cause based on the obtained abnormal data, and
then optimize an optimization module corresponding to the abnormal cause based on the obtained
abnormal cause (the optimization module is also located in the recommender 205, and the
optimization module is used to perform parameter optimization on the database), so as to reduce
the probability of occurrence of abnormal running data of the processes in the database. It should be
noted here that, if the database management system constructed in the embodiment of the present
application includes the recommender 205, the first target model described above may be any one
of the n models in the optimizer 201, and may also be any one of the p models in the recommender
205.

It should be noted that, in some embodiments of the present application, the recommender 205
may specifically include three models, which may be respectively referred to as a codec, a model D,
and a model E, for performing three steps of self-monitoring, self-diagnosis, and self-optimization of
the database. It should be noted here that in other embodiments of the present application, the
recommender 205 may further include more or fewer models for implementing the processes of self-
monitoring, self-diagnosis, and self-optimization of the database, and in the embodiments of the
present application, the recommender 205 includes three models which are only illustrated and will
not be described in detail later.

Specifically, the recommender 205 encodes and decodes the running data of the process in the
database through a codec to obtain encoded data, and compares the encoded data with the running
data input to the codec to obtain abnormal data. It should be noted here that the principle of
obtaining the abnormal data by using the codec is as follows: the coder and the decoder can restore
normal original data but can not restore abnormal original data, so that the input original data is
encoded and then decoded to obtain encoded data, and the encoded data is compared with the

28 | P a g e
original data to know whether abnormal data exists or not. After obtaining the abnormal data, if the
operating data belongs to system index data (e.g., a page fault), the advisor 205 may further
diagnose the abnormal cause according to the abnormal data through a model D, where the model D
is a model constructed based on a deep learning algorithm, for example, the model D may include an
LSRM model and a classifier, specifically, the advisor 205 encodes the found abnormal data into a
compressed vector (i.e., a reduced/increased vector) by calling the LSRM model, and then uses a
learning-type classifier (e.g., a binary classifier, a multi-classifier, etc.) to infer a corresponding root
cause (e.g., a database backup operation); if the operation data belongs to query indicator data (e.g.,
average delay), the recommender 205 may further diagnose the cause of the abnormality according
to the abnormality data by using a model E, where the model E is a model constructed based on a
deep learning algorithm. For example, the model E may include a Tree-LSTM model and a softmax
function, and in particular, the recommender 205 encodes a slow query (i.e., the execution duration
of the query) by invoking the Tree-LSTM model, locates the physical operator (i.e., the execution
operator) operation that caused the exception, and identifies the root cause that caused the
exception by using the softmax function.

Finally, after finding the root cause of the data anomaly based on self-monitoring and self-diagnosis,
the recommender 205 selects a corresponding optimization module for optimization according to
the root cause of the performance degradation of the database system, for example, if the
performance degradation is caused by the lack of the index, the recommender 205 may invoke the
index selection module to establish a new index based on deep reinforcement learning, so that the
performance of the query load is improved (the root cause is that the index has no suggestion); if the
performance is degraded due to parameters, the recommender 205 may perform parameter tuning
based on empirical rules or reinforcement learning from query level, connection level, and system
level, respectively, and the purpose of the optimization is to minimize abnormal operation data of
the system.

It should be noted that there is a corresponding relationship (but not necessarily a one-to-one
relationship) between the optimization modules included in the recommender 205 and the root
cause, for example, one optimization module may correspond to a series (i.e. multiple) of root
causes, or one optimization module may correspond to one root cause, which is not limited in this
application. As an example, table 1 is a schematic table of the correspondence between the
optimization modules and the partial root factors in the recommender 205.

TABLE 1 correspondence between optimization modules and partial root causes

Figure BDA0003211156120000231

Based on the above, the recommender 205 is mainly used to implement the following three
functions:

(1) Self-monitoring

The database state is self-monitored, and operational data (e.g., CPU usage, response time,
operational logs) during database runtime is provided. For anomaly detection, the present
application utilizes a codec to automatically detect anomalies based on data distribution and index
correlation. Specifically, the run data is converted into a low-dimensional representation by an
encoder and the low-dimensional representation is restored using a decoder. Data that the codec
cannot reconstruct well is considered anomalous data.

(2) Self-diagnosis

29 | P a g e
The self-diagnosis is intended to automatically diagnose an abnormality for finding the root cause of
generation of abnormal data. If the abnormal data is system index data (such as lock conflict), the
found abnormal data is coded into a compressed vector (namely the vector after dimension
reduction/dimension lifting) by calling an LSRM model, and then corresponding root factors (such as
database backup operation) are inferred by utilizing a learning type classifier (such as a classifier II, a
classifier III and the like); if the abnormal data is query index data (such as slow query), encoding the
slow query by calling a Tree-LSTM model, positioning a physical operator (namely an execution
operator) operation causing the abnormality, and identifying a root cause causing the abnormality by
using a softmax function.

(3) Self-optimization

Self-optimization automatically optimizes the database for query workload, e.g., index/view
recommendations. Index is automatically recommended using deep reinforcement learning, and
learned view recommendation utilizes a coder-decoder model to automatically recommend views.
Self-optimization is carried out aiming at a database system, and the learning parameter adjusting
module adopts a deep reinforcement learning technology to adjust parameter values. According to
the method and the device, an Actor-Critic model can be used for automatically selecting
appropriate parameter values, and SQL level, session level and system level parameter tuning can be
supported.

In summary, the calling logic of the advisor 205 is: the database and the status index of the query
execution are dynamically collected first, and then abnormal data is discovered by using a self-
monitoring module (namely a codec). For abnormal data, the self-diagnosis module finds out root
causes causing the performance reduction of the database by using a system diagnosis function
(namely, the model D) and an inquiry diagnosis function (namely, the model E) respectively, and then
appoints the self-optimization module to execute corresponding optimization functions. For example,
if the root is that the accessed column is not indexed, the self-diagnostic module may invoke the
index selection module of the self-optimization module to optimize.

It should be noted that, in some embodiments of the present application, the module function of the
advisor 205 is to find out that there may be a problem in the database operation process, and to
perform diagnosis and tuning. Therefore, in some embodiments of the present application, the
recommender 103 may not be required, and is not limited in this application.

For the sake of understanding, the following describes the implementation process of the advisor
205 in detail based on the principle of the advisor 205, and referring to fig. 5 in particular, fig. 5 is a
schematic diagram of the advisor provided in the embodiment of the present application, and the
advisor includes three parts of self-monitoring, self-diagnosis, and self-optimization. The self-
monitoring can judge whether the database has problems in history, current and future according to
the performance indexes of the running data of the processes in the database, and judge the
abnormal or possible abnormal state of the database. The actual problems of the database are
solved by discovering the abnormal state of the database to realize the self-diagnosis and self-
optimization functions of the database. For convenience of explanation, it is assumed that the
advisor includes a self-monitoring module, a self-diagnosis module, and a self-optimization module,
respectively, for implementing self-monitoring, self-diagnosis, and self-optimization, respectively, and
its core steps are as follows:

1) The self-monitoring module continuously collects database performance indexes, and when
abnormality occurs inside or outside the database, the database performance indexes can be

30 | P a g e
reflected through corresponding indexes and system logs. Therefore, openGauss monitors and
discovers the real-time abnormality by analyzing the database and the operating system index. The
specific process is as follows:

a. first, the training data collector continuously collects metrics and logs, such as QPS, runtime logs,
etc., from the database and operating system, and then puts these data together to form time series
data.

b. Secondly, an algorithm based on reconstruction is adopted to find the abnormity, namely, the
normal time sequence data always has a regular change pattern, and the change pattern of the
abnormity is very likely to be system abnormity. The present embodiment employs an LSTM-based
automatic codec with an attention layer. The original time series data is encoded into a low
dimensional representation, which a decoder parses and attempts to recover the original data. The
training loss is the reconstruction quality. The model learns the distribution of the multi-dimensional
data to obtain the reconstruction capability. Data that cannot be reconstructed (errors exceed a
threshold) is reported as anomalous. The embodiment of the application adopts a statistical method
of extreme value theory to determine the dynamic threshold value. Thus, the user needs to set the
system sensitivity to 1% or 5%, which will calculate the corresponding threshold from the historical
data: the method comprises the steps of firstly standardizing training data, then inputting the
processed data into an automatic time sequence encoder to update model parameters, and after the
model has the capability of reconstructing normal database indexes, collecting reconstruction errors
by openGauss and calculating a threshold value.

2) If no abnormity is found, after waiting for a period of time (namely preset duration), repeatedly
executing the step 1); if the abnormality is found, step 3) is executed.

3) When a problem or potential problem exists in the database history, the current or future, the
self-diagnosis module is called to carry out root cause analysis.

4) And the self-diagnosis module judges the found faults, and if the faults do exist, a system-level or
SQL statement-level problem root is given. The specific process is as follows: the database self-
diagnostic function may identify system-level, SQL statement-level failures or abnormal root causes.
The system-level fault analysis method is realized by adopting an LSTM + KNN algorithm; the SQL
statement level fault root is realized through a Tree-LSTM algorithm. And for the root cause with the
fault, calling a self-optimization function, and giving a corresponding optimization suggestion so as to
solve the problem.

5) And (4) suggesting the database system through the root factor given by self-diagnosis, and calling
a self-optimization module to optimize. The specific process is as follows:

a. the self-optimization module includes optimizing parameter configurations for database system
characteristics. Parameter recommendation is realized through deep reinforcement learning: firstly,
modeling is carried out through database parameter configuration and corresponding performances
which are learned through history, namely, a parameter combination with the optimal corresponding
performance is searched in a search space formed by selected parameters; then, the deep
reinforcement learning model takes the database state and the load characteristics as input states,
and selects proper parameter configuration as output behaviors according to parameter adjusting
experience learned from historical data, so that an optimal database parameter optimization scheme
is given.

31 | P a g e
b. The self-optimization module also includes tuning for database SQL statements, e.g., materialized
view recommendations and index recommendations. The materialized view recommendation is
realized through RNN and reinforcement learning, the materialized view which can be created is
recommended by analyzing the load of a user and adopting an enumeration and evaluation means,
and the user can accelerate the load by creating the materialized view. The index recommendation is
directed at the index recommendation of the load level, and an optimal index configuration scheme
adapted to the index recommendation is given according to different increasing, deleting and
modifying ratios of the users.

In order to more intuitively recognize the beneficial effects brought by the embodiments of the
present application, the following technical effects brought by the embodiments of the present
application are further compared, and the comparison results are respectively as follows:

(1) Comparison of the optimizer 201 with three rewrite strategies (random rewrite, top-down
rewrite, heuristic rewrite).

Referring specifically to fig. 6, fig. 6 is a schematic diagram illustrating comparison between an

optimizer and three re-policies provided in this embodiment of the present application, taking query
rewrite as an example, the embodiment of the present application compares query rewrite in
openGauss with three rewrite policies (random rewrite, top-down rewrite, and heuristic rewrite). For
random rewrite and top-down rewrite, the embodiment of the present application extracts 82
rewrite rules in the query optimization engine call and rewrites the query with the corresponding
policy. In addition, the application embodiment uses the tool SQL-smith to generate 15,750 and
10,673 slow queries (> 1 s) for TPC-H and JOB, respectively. As shown in fig. 6, the present rewrite
strategy is superior to other methods in all cases, i.e., the execution time of TPC-H is reduced by
more than 49.7%, and the execution time of JOB is reduced by more than 36.8%. The reason is
mainly divided into two aspects: first, openGauss explores the rewrite order with lower execution
cost than the default top-down order in PostgreSQL. For example, using outer-joins, postgreSQL
cannot push predicates down to the input table, while openGauss solves the problem by first
converting the outer-joins to inner-joins and then pushing the predicates down. Second, the
estimation model in openGauss predicts the potential cost reduction, from which openGauss
chooses a lower execution overhead rewrite order. Furthermore, openGauss works better on TPC-H
than JOB because TPC-H queries contain many sub-queries that can be optimized by query rewrite,
while multiple joins in a JOB query will be further optimized by the plan enumerator.

To sum up, the optimizer 201 included in the database management system provided in the
embodiment of the present application can perform fine-grained optimization in the SQL statement
execution process according to the optimization method provided by the AI model, thereby
improving the execution efficiency of the SQL statement and improving the performance of the
database.

(2) The result of the comparison of the recommender 205 (if any) with the two indexing strategies
(default index, artificially designed index).

Specifically, referring to tables 2 and 3, tables 2 and 3 are schematic diagrams for comparing the
recommender provided in the embodiments of the present application with two indexing strategies,
and taking index selection as an example, the present invention performs experiments on TPC-H and
TPC-C, and compares the recommender provided in the embodiments of the present application
with the default index and the manually designed index. The results are shown in tables 2 and 3. The
index selection algorithm of the present application performs better on both workloads than the

32 | P a g e
default index and the manual index. Because the present index selection algorithm encodes system
statistics into state representations and can optimize index selection strategies based on historical
data to dynamically update index configurations.

Database management system, data processing method and equipment

TABLE 2 indexing program

TPC-H(s) TPC-C(tpmC)

OpenGauss database management system 122.9 10202

Database administrator DBA 130.1 10001

Default setting 140.8 9700

TABLE 3 anomaly detection (TPC-C)

Rate of accuracy Recall rate Fraction of F1

OpenGauss database management system 0.795 0.776 0.785

Variational self-encoder VAE 0.302 0.821 0.441

Generative countermeasure network GAN 0.554 0.745 0.635

In summary, the optimizer 205 (if any) included in the database management system provided in the
embodiment of the present application may find in time whether there is a failure or an abnormality
in the database history, current, and future, and provide a corresponding failure root factor
according to the service type and the characteristics, and provide an optimal optimization and
configuration scheme.

(3) The results of the comparison of the model evaluator 204 with two known performance
evaluation methods.

Referring specifically to fig. 7, fig. 7 is a schematic diagram comparing a model evaluator provided in
an embodiment of the present application with two known performance evaluation methods, BAL
and DL, wherein BAL estimates an average buffer access delay and predicts a query delay of a
concurrent query using linear regression; the DL employs a neural network designed from the query
plan structure to predict the performance of individual queries. The prediction accuracy and the
prediction time are compared on JOB in the embodiment of the present application, and the result is
shown in fig. 7, and it can be known from the comparison result in fig. 7 that the error rate of the
model evaluator provided in the embodiment of the present application is the lowest, which is about
29.9 times lower than BAL and 22.5 times lower than DL. The reason is twofold: first, the workload
map in the model evaluator provided in the embodiment of the present application encodes
concurrent factors such as resource contention, which increases the query delay of JOB by more than
20% compared with serial execution. In contrast, BAL collects buffer access latency, while DL relies on
a single query function. Secondly, openGauss utilizes the embedded network of picture to map the
structural information directly to the performance factor, can improve the commonality when the
work load changes. In contrast, BAL uses a linear regression method that requires many statistical
samples to be provided for a single workload. Furthermore, as can be seen from fig. 7, the prediction
delay of the model evaluator provided by the embodiment of the present application is less than that
of both BAL and DL, and is relatively stable when the concurrency level increases. For openGauss, the
model evaluator provided in the embodiment of the present application predicts the execution time

33 | P a g e
of all vertices simultaneously. It embeds the localized graph of all vertices in the workload graph so
that the total predicted time of the workload is close to predicting the vertex with the largest
localized graph. While for BAL, it requires the longest prediction time because it predicts
performance when executing the workload. For DL, it propagates intermediate data features in the
query plan tree in a bottom-up manner, which takes relatively longer than openGauss.

In summary, the model evaluator 204 included in the database management system provided in the
embodiment of the present application can effectively and timely verify whether the new model is
valid, and if so, the new model is deployed, otherwise, the model is abandoned.

(4) The training data collector 202 and the model manager 203 are not present in the known
database management system, and the database management system constructed by the present
application includes these two modules to ensure the reliability of data processing.

Based on the description of the database management system constructed in the embodiment of the
present application in fig. 1 to fig. 7, the embodiment of the present application provides an
autonomous database framework, which implements a self-learning kernel and an advisor based on
a machine learning algorithm and expert experience to construct an omnidirectional autonomous
function of a database.

Specifically, the learning-type optimizer with a built-in database kernel in the database management
system constructed in the embodiment of the application can specifically comprise an MCTS-based
rewriter, a Tree-LSTM-based cost estimator and an RL-based plan generator, so that the optimizer can
efficiently query and optimize, and the multi-scenario service appeal can be met; the database
management system provided by the embodiment of the invention can also comprise a learning-type
recommender of the database, and the recommender is based on a machine learning technology, so
that the functions of automatic abnormity monitoring, automatic system diagnosis, automatic slow
query diagnosis and automatic performance optimization (such as parameter tuning, index
recommendation and view recommendation) are realized, one-click operation and maintenance
management operation of a client is met, and the operation and maintenance efficiency and the
database execution efficiency are improved; the database management system provided by the
embodiment of the invention can also comprise an efficient model evaluator, which can pre-estimate
the performance of a deployed model in the database management system based on a machine
learning technology, judge the income brought by applying a corresponding model and ensure that
the database management system is always in high-performance and high-reliability operation; the
database management system provided by the embodiment of the invention also can comprise a
training data collector and a model manager based on a uniform interface, wherein the training data
collector automatically collects the running data of the process in the database, including database
running indexes, query logs, system logs and the like, and generates the training data of the deployed
model in the database management system by using the information; the model manager provides
uniform interface management, controls model versions, and dynamically updates and replaces the
models used by the modules.

Based on the database management system, the following technical problems in the existing
traditional database technology can be specifically solved:

1) Traditional database technologies have different modules and functions, and it is difficult to select
a suitable automation algorithm to obtain maximum performance benefits. If the database module,
which may have performance bottlenecks, needs to be replaced with a learning type model, a
suitable machine learning model or algorithm needs to be selected first. For example, a deep

34 | P a g e
learning model can be applied to the cost estimation problem because deep learning can fit data
features and access correlations of high-dimensional data columns; the deep reinforcement learning
can be applied to the parameter tuning problem, because the deep reinforcement learning does not
require training data to be provided in advance, and continuous high-dimensional parameter space
can be efficiently explored under the condition of a small number of samples. According to the
embodiment of the application, through performance evaluation of the model evaluator, a suitable
machine learning model or algorithm is provided for complex and changeable service scenes, large
concurrency and high-performance services.

2) And the evaluation of the model effectiveness is difficult. Before a new learning model is designed
and deployed in a database, whether the model has performance improvement compared with a
traditional algorithm needs to be confirmed in advance. The traditional method usually depends on
expert experience or a large number of running tests, the cost is high, and the evaluation efficiency is
low. The embodiment of the application designs a model evaluator for prejudging the performance
of different learning models in a typical application scene.

3) The various modules of a conventional database are based on classical heuristics or rule-defined
algorithms. However, after each module is replaced by a machine learning model, each machine
learning model needs to collect data and train the model, and update the model when the scene
changes, and if the above operations are performed separately, the training and management costs
of the model are very high. The embodiment of the application provides a training data collector and
a model manager with unified interfaces, and the model is automatically updated by evaluating the
usability of the model according to the change of the collected information.

On the basis of the embodiments corresponding to fig. 1 to fig. 5, a method for applying the
database management system is described below, specifically referring to fig. 8, where fig. 8 is a
schematic flow chart of a data processing method provided in the embodiment of the present
application, and the method specifically includes the following steps:

801. the method comprises the steps that computer equipment receives SQL sentences sent by a
client to a database, the database is deployed in the computer equipment, the database comprises
an optimizer and a native kernel component, the optimizer comprises n models, and n is larger than
or equal to 1.

First, a local computer device (i.e., the local device described in fig. 1) receives an SQL statement sent
by a client device to a database deployed in the computer device, where the database includes an
optimizer including n models and a database native kernel component, and n ≧ 1.

802. And under the condition that the first target model does not meet the second preset
requirement, the computer equipment obtains a target physical plan through the n models included
in the optimizer according to the SQL statement, wherein the target physical plan is a physical plan
with execution cost meeting the first preset requirement, and the first target model is one of the n
models.

After receiving an SQL statement sent by a client device, a computer device may first determine
whether n models in an optimizer have a model that does not satisfy a predetermined requirement
(which may be referred to as a second predetermined requirement), and if a first target model (i.e.,
one of the n models) does not satisfy the second predetermined requirement, the computer device
may obtain a target physical plan through the n models included in the optimizer according to the
SQL statement, where the target physical plan is a physical plan whose execution overhead satisfies
the first predetermined requirement.

35 | P a g e
It should be noted that, in some embodiments of the present application, the execution overhead
satisfying the first preset requirement includes, but is not limited to: 1) The execution overhead of
the target physical plan is lowest in q execution overheads, the q execution overheads are the
execution overheads corresponding to the q physical plans generated based on SQL statements of
the input database, and one physical plan corresponds to one execution overhead, wherein q is more
than or equal to 1; 2) The execution overhead of the target physical plan is below a certain preset
value (which may be referred to as a first preset threshold). For convenience of illustration, in the
following embodiments of the present application, the situation that the execution overhead of the
target physical plan is the lowest among the q execution overheads and is taken as the execution
overhead to meet the first preset requirement is all taken, and details are not described in the
following.

It is further noted that, in some embodiments of the present application, the first target model not
satisfying the second preset requirement may include, but is not limited to: 1) The performance of
the first target model is not degraded; 2) Although the performance of the first target model is
reduced, the degree of the reduction does not reach a certain preset value (which may be called a
second preset threshold); 3) Evaluating the real-time performance of the first target model and
predicting the next performance, assuming that the predicted probability of performance reduction
of the first target model does not reach a certain preset value (which may be called a third preset
threshold), for example, the predicted probability of performance reduction does not reach 80%; 4)
The duration of the continuous operation of the first target model does not reach a preset duration,
for example, the duration of the continuous operation of the first target model does not reach 30
minutes.

It should also be noted that, in some embodiments of the present application, as an example, the
optimizer may specifically include three models, which may be respectively referred to as model a,
model B, and model C, for performing the steps of logical query rewrite, cost estimation, and physical
plan generation. In this case, the manner in which the computer device obtains the target physical
plan through the n models included in the optimizer according to the SQL statement may specifically
be: firstly, the computer device performs logic query rewriting on an SQL statement (also called SQL
query) input into a database through a model a, so as to obtain a rewritten logic plan, wherein the
model a is a model constructed based on a tree search algorithm, and may be a monte carlo tree
search algorithm, for example; then, generating q physical plans according to the logic plan by using
a model B, wherein the model B is a model constructed based on a deep learning algorithm, for
example, the model can be a model based on Tree-LSTM, and q is more than or equal to 1; and
finally, calculating Q execution overheads corresponding to the Q physical plans by using a model C
(one physical plan corresponds to one execution overhead), and determining a target physical plan to
be executed finally according to the Q execution overheads, wherein the model C is a model
constructed based on a reinforcement learning algorithm, and for example, may be a model based
on deep reinforcement learning (DQN).

It should be noted here that in other embodiments of the present application, the optimizer may
further include more or fewer models for implementing the processes of logical query rewrite, cost
estimation, and physical plan generation, and in this embodiment of the present application, the
three models included in the optimizer are merely illustrative and are not described in detail later.

It should be further noted that, in the embodiment of the present application, specific functions and
call logic of the optimizer related to the computer device may refer to the portion of the optimizer
201 described in the embodiment corresponding to fig. 2, which is not described herein in detail.

36 | P a g e
803. The computer device executes the target physical plan.

The computer device executes the target physical plan after obtaining the final target physical plan.
The essence of this execution is the actual logic that executes the input SQL statement using the
generated target physical plan.

In the above embodiment of the present application, how a computer device obtains a target
physical plan based on an optimizer included in a database and finally executes the target physical
plan is described, where the database deployed in the computer device includes an optimizer with n
models, so as to replace a traditional heuristic optimizer, and by combining with a machine learning
technology, a logical query is converted into a physical execution plan with higher execution
efficiency, and the problems of inaccurate cost evaluation and poor physical plan generation by a
complex SQL statement caused by the current database architecture problem can be effectively
solved.

It should be noted that, in some embodiments of the present application, the computer device may
further send the running data of the process in the database deployed therein to the advisor, and the
advisor may be deployed in the computer device or may not be deployed in a remote device, which
is not limited herein. And the recommender receives the operation data, sends abnormal data based
on the operation data, diagnoses an abnormal reason based on the obtained abnormal data, and
optimizes a self-optimization module corresponding to the abnormal reason based on the abnormal
reason so as to reduce the abnormal probability of the operation data of the process in the
subsequent database, wherein the recommender comprises p models, and p is more than or equal
to 1.

It should be noted that, in the embodiment of the present application, specific functions and call
logic of the recommender related to the embodiment of the present application may refer to the
portion of the recommender 205 described in the embodiment corresponding to fig. 2, which is not
described herein again in detail.

In the above embodiment of the present application, it is specifically stated that the computer device
may further feed back the operation data of the process in the database to the advisor, and the
advisor may give an all-directional optimization advice of the database based on the operation data,
so that unattended database performance monitoring and root cause identification may be achieved,
operation and maintenance manpower may be greatly liberated, and the database system may be
helped to quickly recover from an abnormality or improve performance.

It should be further noted that, in other embodiments of the present application, the computer
device may further send, to the advisor, the operation data of the process in the database deployed
therein, and also send, to the training data collector, where the training data collector may be
deployed in the computer device, and may also be deployed in a remote device, which is not limited
herein. After the training data collector receives the operation data, training data can be obtained
according to the operation data, and m training sets are constructed based on the training data,
wherein m is larger than or equal to 1.

It should be noted that, in the embodiment of the present application, specific functions and call
logic of the training data collector related to the embodiment of the present application may refer to
the part of the training data collector 202 described in the embodiment corresponding to fig. 2,
which is not described herein in detail.

37 | P a g e
In the foregoing embodiment of the present application, it is specifically stated that the computer
device may further feed back the operation data of the process in the database to the training data
collector, and the training data collector may generate the training data of the model related to the
database based on the operation data of the process in the database, so that the continuous
optimization of the database system may be implemented, the misjudgment probability of the
database system may be reduced, and a trusted autonomous operation and maintenance service
may be provided.

It should be noted that, in the above embodiments of the present application, a case where the first
target model does not satisfy the second preset requirement, including but not limited to, is
described, whereas a case where the first target model satisfies the second preset requirement,
including but not limited to, is: 1) The performance of the first target model begins to be degraded;
2) The performance of the first target model is not only degraded, but the degradation degree
reaches a certain preset value (which can be called as a second preset threshold value); 3) Evaluating
the real-time performance of the first target model and predicting the next performance, for
example, predicting the performance of the first target model by the model evaluator 204, assuming
that the predicted probability of performance degradation of the first target model reaches a certain
preset value (which may be referred to as a third preset threshold), for example, the predicted
probability of performance degradation reaches 80%; 4) The first target model continues to operate
for a predetermined period of time, for example, the first target model continues to operate for 30
minutes.

Therefore, in other embodiments of the present application, if the first target model meets the
second preset requirement, the computer device may further send a first instruction to a model
manager (the model manager may or may not be deployed in the computer device, and is not
limited herein), the first instruction being configured to instruct the model manager to perform fine
tuning (finetune) on the first target model, and, in a case that performance of the second target
model meets a certain preset requirement (which may be referred to as a third preset requirement),
the computer device receives the model parameters of the second target model sent by the model
manager, where the second target model is a model obtained by the model manager performing fine
tuning on the first target model by using a target training set corresponding to the first target model,
the target training set is one of m training sets, and finally, the computer device updates the first
target model to the second target model, and obtains the updated n target models (at this time, the
updated n target models include the second target model).

It should be noted that, in some embodiments of the present application, the second target model
may meet the third preset requirement, which includes but is not limited to: 1) The performance of
the second target model is improved by a certain preset value (which may be referred to as a fourth
preset threshold) compared with the performance of the first target model, and as an example, the
fourth preset threshold may be zero, which means that the second target model is considered to
meet the third preset requirement as long as the performance of the second target model reaches
the level of the performance of the original first target model; as another example, the fourth preset
threshold may also be a certain value or a certain proportion greater than zero, which indicates that
the second target model is considered to meet the third preset requirement only if the performance
of the second target model is improved to a certain degree compared with the performance of the
original first target model; 2) The performance of the second target model is increased by a fifth
preset threshold compared to the performance of the native kernel component within the database.
And if the performance improvement reaches a certain threshold value, the model used by the
corresponding module of the data is actually replaced, otherwise, the traditional database algorithm

38 | P a g e
is adopted to execute the target physical plan. The value of the fifth preset threshold may be zero, or
may be a certain value or a certain ratio greater than zero, which is specifically referred to the above
first manner, and is not described herein again.

In the foregoing embodiment of the present application, it is specifically stated that, when a first
target model meets a second preset requirement, a model manager calls a corresponding target
training set in a training data collector to perform fine tuning on the first target model, so that a
corresponding model used in a database can be dynamically updated and replaced according to a
real-time running state of the database.

It should be noted that, in the embodiment of the present application, specific functions and call
logic of the model manager according to the embodiment of the present application may refer to the
part of the model manager 203 described in the embodiment corresponding to fig. 2, which is not
described herein in detail.

It should be further noted that, if the performance of the second target model does not meet the
third preset requirement, the computer device further receives a second instruction sent by a model
evaluator, where the second instruction is used to instruct the database to generate a final target
physical plan by using a native kernel component in the database, and the model evaluator may be
deployed in the computer device or may not be deployed in a remote device, which is not limited
herein. The model evaluator is used to evaluate the performance of the second object model.

In the foregoing embodiments of the present application, it is specifically stated that when the
trimmed second target model still does not satisfy the third preset requirement, the computer
device receives a second instruction of the model evaluator, so as to instruct the database to
generate the target physical plan by using a conventional database algorithm (i.e., a native kernel
component). The embodiment of the application provides multiple choices for generation of the
target physical plan, and has flexibility.

It should be noted that, in the embodiment of the present application, specific functions and call
logic of the model evaluator related to the embodiment of the present application may refer to the
model evaluator 204 portion described in the embodiment corresponding to fig. 2, which is not
described herein in detail.

On the basis of the embodiment corresponding to fig. 8, in order to better implement the above-
described scheme of the embodiment of the present application, a computer device for
implementing the above-described scheme is also provided below. Referring to fig. 9 specifically, fig.
9 is a schematic diagram of a computer device provided in the embodiment of the present
application, where the computer device 900 specifically includes: a receiving module 901, a
determining module 902 and an executing module 903, where the receiving module 901 is
configured to receive an SQL statement sent by a client to the database; a determining module 902,
configured to, when the first target model does not meet the second preset requirement, obtain,
according to the SQL statement, a target physical plan through the n models, where the target
physical plan is a physical plan whose execution cost meets the first preset requirement, and the first
target model is one of the n models; an executing module 903, configured to execute the target
physical plan.

In one possible design, the n models include a third model, a fourth model, and a fifth model, in
which case, the determining module 902 is specifically configured to: through the third model, logic
query rewriting is carried out on the SQL statement to obtain a rewritten logic plan, and the third
model is a model constructed based on a tree search algorithm; generating q physical plans

39 | P a g e
according to the logic plan through the fourth model, wherein the fourth model is a model
constructed based on a deep learning algorithm, and q is more than or equal to 1; and calculating q
execution overheads corresponding to the q physical plans through the fifth model, and determining
the target physical plan according to the q execution overheads, wherein one physical plan
corresponds to one execution overhead, and the fifth model is a model constructed based on a
reinforcement learning algorithm.

In one possible design, computer device 900 further includes a sending module 904, the sending
module 904 to: and sending the running data of the process in the database to a recommender, so
that the recommender finds abnormal data based on the running data, diagnoses an abnormal
reason based on the abnormal data, and optimizes a self-optimization module corresponding to the
abnormal reason based on the abnormal reason to reduce the probability of the abnormal running
data, wherein the recommender comprises p models, and p is more than or equal to 1.

In one possible design, the sending module 904 may be further configured to: and sending the
running data of the process in the database to a training data collector so that the training data
collector obtains training data according to the running data, and constructing m training sets based
on the training data, wherein m is more than or equal to 1.

In one possible design, the sending module 904 may be further configured to: sending a first
instruction to a model manager under the condition that the first target model meets the second
preset requirement, wherein the first instruction is used for instructing the model manager to finely
adjust the first target model, and the first target model is one of the n models; the receiving module
901 may further be configured to: under the condition that the performance of a second target
model meets a third preset requirement, receiving model parameters of the second target model
sent by the model manager, wherein the second target model is a model obtained by fine tuning the
first target model by the model manager by using a target training set corresponding to the first
target model, and the target training set is one of the m training sets; the determining module 902
may further be configured to: and updating the first target model into the second target model, and
obtaining the target physical plan through the updated n models.

In one possible design, the receiving module 901 may further be configured to: and receiving a
second instruction sent by a model evaluator under the condition that the performance of the
second target model does not meet the third preset requirement, wherein the second instruction is
used for instructing the database to generate the target physical plan by adopting a native kernel
component in the database, and the model evaluator is used for evaluating the performance of the
second target model.

In one possible design, the first target model satisfying the second preset requirement includes at
least any one of: a degradation of the performance of the first target model; or the degree of
performance degradation of the first target model reaches a second preset threshold, or the
predicted probability of performance degradation of the first target model reaches a third preset
threshold; or the continuous running time of the first target model meets the preset time.

In one possible design, the performance of the second target model meeting the third preset
requirement at least includes any one of the following items: the performance of the second target
model is improved by a fourth preset threshold compared with the performance of the first target
model; or, the performance of the second target model is increased by a fifth preset threshold
compared to the performance of the native kernel component within the database.

40 | P a g e
Database management system, data processing method and equipment

In one possible design, the implementation cost satisfying the first preset requirement at least
includes any one of the following: the execution overhead of the target physical plan is lowest in q
execution overheads, the q execution overheads are the execution overheads corresponding to the q
physical plans generated based on the SQL statement, one physical plan corresponds to one
execution overhead, and q is greater than or equal to 1; or the execution cost of the target physical
plan is lower than a first preset threshold value.

It should be noted that, the contents of information interaction, execution process, and the like
between the modules/units in the computer device 900 are based on the same concept as the
method embodiment corresponding to fig. 8 in the present application, and specific contents may
refer to the description in the foregoing method embodiment in the present application, and are not
described herein again.

Referring to fig. 10, fig. 10 is a schematic structural diagram of a computer device provided in an
embodiment of the present application, where the computer device 1000 may be deployed with the
computer device 900 described in the embodiment corresponding to fig. 9, and is used to implement
the functions of the computer device 900 in the embodiment corresponding to fig. 9, specifically, the
computer device 1000 is implemented by one or more servers, and the computer device 1000 may
generate relatively large differences due to different configurations or performances, and may
include one or more Central Processing Units (CPUs) 1022 and a memory 1032, and one or more
storage media 1030 (e.g., one or more mass storage devices) storing an application program 1042 or
data 1044. Memory 1032 and storage medium 1030 may be transitory or persistent storage, among
other things. The program stored on the storage medium 1030 may include one or more modules
(not shown), each of which may include a sequence of instructions for operating on the computer
device 1000. Still further, the central processor 1022 may be disposed in communication with the
storage medium 1030, and execute a series of instruction operations in the storage medium 1030 on
the computer device 1000.

The computer device 1000 may also include one or more power supplies 1026, one or more wired or
wireless network interfaces 1050, one or more input/output interfaces 1058, and/or one or more
operating systems 1041, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, and so
forth.

In this embodiment of the application, the central processing unit 1022 is configured to execute the
steps executed by the computer device in the embodiment corresponding to fig. 8. For example,
central processor 1022 may be used to: firstly, receiving SQL statements sent by client equipment to
a database deployed in the computer equipment, wherein the database comprises an optimizer and
a database native kernel component, the optimizer comprises n models, and n is greater than or
equal to 1. After receiving an SQL statement sent by a client device, it may be determined first
whether n models in an optimizer have a requirement that a model does not satisfy a predetermined
requirement (which may be referred to as a second predetermined requirement), and if a first target
model (i.e., one of the n models) does not satisfy the second predetermined requirement, a target
physical plan is obtained through the n models included in the optimizer according to the SQL
statement, where the target physical plan is a physical plan whose execution cost satisfies the first
predetermined requirement. After the final target physical plan is obtained, the target physical plan
is executed, and the essence of the execution process is to execute the actual logic of the input SQL
statement by using the generated target physical plan.

41 | P a g e
It should be noted that, the specific manner in which the central processing unit 1022 executes the
above steps is based on the same concept as that of the method embodiment corresponding to fig. 8
in the present application, and the technical effect brought by the method embodiment is also the
same as that of the above embodiment in the present application, and specific contents may refer to
the description in the foregoing method embodiment in the present application, and are not
described herein again.

It should be noted that the above-described embodiments of the apparatus are merely schematic,
where the units described as separate parts may or may not be physically separate, and the parts
displayed as units may or may not be physical units, may be located in one place, or may be
distributed on multiple network units. Some or all of the modules may be selected according to
actual needs to achieve the purpose of the solution of the present embodiment. In addition, in the
drawings of the embodiments of the apparatus provided in the present application, the connection
relationship between the modules indicates that there is a communication connection
therebetween, and may be implemented as one or more communication buses or signal lines.

Through the above description of the embodiments, those skilled in the art will clearly understand
that the present application can be implemented by software plus necessary general-purpose
hardware, and certainly can also be implemented by special-purpose hardware including special-
purpose integrated circuits, special-purpose CPUs, special-purpose memories, special-purpose
components and the like. Generally, functions performed by computer programs can be easily
implemented by corresponding hardware, and specific hardware structures for implementing the
same functions may be various, such as analog circuits, digital circuits, or dedicated circuits.
However, for the present application, the implementation of a software program is more preferable.
Based on such understanding, the technical solutions of the present application may be substantially
embodied in or contributed by a software product, which is stored in a readable storage medium,
such as a floppy disk, a usb disk, a removable hard disk, a read-only memory (ROM), a random-access
memory (RAM), a magnetic disk or an optical disk of a computer, and includes instructions for
causing a computer device (which may be a personal computer, an exercise device, or a network
device) to execute the method according to the embodiments of the present application.

In the above embodiments, the implementation may be wholly or partially realized by software,
hardware, firmware, or any combination thereof. When implemented in software, it may be
implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When loaded and
executed on a computer, cause the processes or functions described in accordance with the
embodiments of the application to occur, in whole or in part. The computer may be a general
purpose computer, a special purpose computer, a network of computers, or other programmable
device. The computer instructions may be stored in a computer readable storage medium or
transmitted from one computer readable storage medium to another, for example, from one website
site, computer, training device, or data center to another website site, computer, training device, or
data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g.,
infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available
medium that a computer can store or a data storage device, such as a training device, a data center,
etc., that incorporates one or more available media. The usable medium may be a magnetic medium
(e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor
medium (e.g., solid State Disk (SSD)), among others.

CLAIMS

42 | P a g e
1. A database management system, comprising:

the system comprises an optimizer, a training data collector, a model manager and a model evaluator,
wherein the optimizer comprises n models, and n is more than or equal to 1;

the optimizer is used for obtaining a target physical plan through the n models according to the SQL
sentences input into the database, wherein the target physical plan is a physical plan with execution
cost meeting first preset requirements;

the training data collector is used for obtaining training data according to the running data of the
processes in the database and constructing m training sets based on the training data, wherein m is
more than or equal to 1;

the model manager is configured to, when a first target model meets a second preset requirement,
perform fine tuning on the first target model by using a target training set corresponding to the first
target model to obtain a second target model, where the first target model is one of the n models,
and the target training set is one of the m training sets;

and the model evaluator is used for evaluating the performance of the second target model and
updating the first target model into the second target model under the condition that the
performance of the second target model meets a third preset requirement.

2. The system of claim 1, further comprising:

the recommender comprises p models, wherein p is more than or equal to 1;

the advisor is used for finding abnormal data in the operating data and diagnosing an abnormal
reason based on the abnormal data;

the recommender is further configured to optimize an optimization module corresponding to the

abnormal reason based on the abnormal reason to reduce a probability of occurrence of an
abnormality in the operation data.

3. The system according to claim 2, wherein the p models comprise a codec, a first model and a
second model, and wherein the recommender is configured to:

the running data is coded and then decoded through the coder-decoder to obtain coded data, and
the coded data is compared with the running data to obtain the abnormal data;

through the first model, under the condition that the operation data belong to system index data,
diagnosing according to the abnormal data to obtain the abnormal reason, wherein the first model is
a model constructed based on a deep learning algorithm;

and diagnosing and obtaining the abnormal reason according to the abnormal data under the
condition that the operating data belong to query index data through the second model, wherein the
second model is a model constructed based on a deep learning algorithm.

4. The system of claim 3,

the first model comprises a long-time memory model (LSTM) and a classifier;

the second model comprises a Tree long-time memory model Tree-LSTM and a softmax function.

5. The system of any of claims 2-4, wherein the first object model further comprises:

any one of the p models.

43 | P a g e
6. The system according to any of claims 1-5, wherein the n models comprise a third model, a fourth
model, and a fifth model, and the optimizer is specifically configured to:

performing logical query rewriting on the SQL sentences input into the database through the third
model to obtain a rewritten logical plan, wherein the third model is a model constructed based on a
tree search algorithm;

generating q physical plans according to the logic plan through the fourth model, wherein the fourth
model is a model constructed based on a deep learning algorithm, and q is more than or equal to 1;

and calculating q execution overheads corresponding to the q physical plans through the fifth model,
and determining the target physical plan according to the q execution overheads, wherein one
physical plan corresponds to one execution overhead, and the fifth model is a model constructed
based on a reinforcement learning algorithm.

7. The system of any of claims 1-6, wherein the model evaluator is further configured to:

and under the condition that the performance of the second target model does not meet the third
preset requirement, triggering the database to generate a target physical plan by adopting a native
kernel component in the database.

8. The system of claim 7, wherein the model evaluator is further configured to:

and feeding back information of model updating failure to the model manager so that the model
manager adjusts the strategy for fine tuning the first target model based on the information, wherein
the model updating failure is that the performance of the second target model does not meet the
third preset requirement.

9. The system according to any one of claims 1-8, wherein the execution overhead satisfying the first
predetermined requirement comprises at least any one of:

the execution overhead of the target physical plan is lowest in q execution overheads, the q
execution overheads are the execution overheads corresponding to the q physical plans generated
based on the SQL statement, one physical plan corresponds to one execution overhead, and q is
greater than or equal to 1;

or the like, or a combination thereof,

the execution cost of the target physical plan is lower than a first preset threshold.

10. The system according to any one of claims 1-9, wherein the first target model satisfying a second
predetermined requirement comprises at least any one of:

a decrease in performance of the first target model occurs;

or the like, or, alternatively,

the performance reduction degree of the first target model reaches a second preset threshold value;

or the like, or, alternatively,

the predicted probability of performance degradation of the first target model reaches a third preset
threshold;

or the like, or, alternatively,

44 | P a g e
the continuous operation time of the first target model reaches a preset time.

11. The system according to any one of claims 1-10, wherein the performance of the second target
model meeting the third predetermined requirement comprises at least any one of:

the performance of the second target model is improved by a fourth preset threshold compared with
the performance of the first target model;

or the like, or, alternatively,

the performance of the second target model is increased by a fifth preset threshold compared to the
performance of a native kernel component within the database.

12. A data processing method is applied to computer equipment, the computer equipment is
provided with a database, the database comprises an optimizer and a native kernel component, the
optimizer comprises n models, n is more than or equal to 1, and the method comprises the following
steps:

the computer equipment receives an SQL statement sent to the database by a client;

under the condition that a first target model does not meet a second preset requirement, the
computer equipment obtains a target physical plan through the n models according to the SQL
statement, wherein the target physical plan is a physical plan with execution cost meeting a first
preset requirement, and the first target model is one of the n models;

the computer device executes the target physical plan.

13. The method of claim 12, wherein the n models include a third model, a fourth model, and a fifth
model, and wherein the obtaining, by the computer device, the target physical plan from the n
models according to the SQL statement comprises:

the computer equipment performs logical query rewriting on the SQL statement through the third
model to obtain a rewritten logical plan, wherein the third model is a model constructed based on a
tree search algorithm;

the computer equipment generates q physical plans according to the logic plan through the fourth
model, the fourth model is a model constructed based on a deep learning algorithm, and q is larger
than or equal to 1;

the computer equipment calculates q execution overheads corresponding to the q physical plans
through the fifth model, determines the target physical plan according to the q execution overheads,
wherein one physical plan corresponds to one execution overhead, and the fifth model is a model
constructed based on a reinforcement learning algorithm.

14. The method according to any one of claims 12-13, further comprising:

the computer equipment sends the running data of the processes in the database to a
recommender, so that the recommender finds abnormal data based on the running data, diagnoses
abnormal reasons based on the abnormal data, and optimizes a self-optimization module
corresponding to the abnormal reasons based on the abnormal reasons to reduce the probability of
the abnormal running data, wherein the recommender comprises p models, and p is larger than or
equal to 1.

15. The method according to any one of claims 12-14, further comprising:

45 | P a g e
and the computer equipment sends the running data of the processes in the database to a training
data collector so that the training data collector obtains training data according to the running data,
and m training sets are constructed based on the training data, wherein m is more than or equal to 1.

16. The method of claim 15, further comprising:

under the condition that the first target model meets the second preset requirement, the computer
equipment sends a first instruction to a model manager, wherein the first instruction is used for
instructing the model manager to finely adjust the first target model, and the first target model is
one of the n models;

under the condition that the performance of a second target model meets a third preset
requirement, the computer equipment receives model parameters of the second target model sent
by the model manager, the second target model is a model obtained by fine tuning the first target
model by the model manager by using a target training set corresponding to the first target model,
and the target training set is one of the m training sets;

and the computer equipment updates the first target model into the second target model and
obtains the target physical plan through the updated n models.

17. The method of claim 16, further comprising:

and under the condition that the performance of the second target model does not meet the third
preset requirement, the computer equipment receives a second instruction sent by a model
evaluator, wherein the second instruction is used for instructing the database to generate the target
physical plan by adopting a native kernel component in the database, and the model evaluator is
used for evaluating the performance of the second target model.

18. The method according to any one of claims 16-17, wherein the first target model satisfying a
second predetermined requirement comprises at least any one of:

a decrease in performance of the first target model occurs;

or the like, or, alternatively,

the performance reduction degree of the first target model reaches a second preset threshold value;

or the like, or, alternatively,

the predicted probability of performance degradation of the first target model reaches a third preset
threshold;

or the like, or, alternatively,

the duration of the continuous operation of the first target model meets the preset duration.

19. The method according to any of claims 16-18, wherein the performance of the second target
model meeting the third predetermined requirement comprises at least any one of:

the performance of the second target model is improved by a fourth preset threshold compared with
the performance of the first target model;

or the like, or, alternatively,

the performance of the second target model is increased by a fifth preset threshold compared to the
performance of a native kernel component within the database.

46 | P a g e
20. The method according to any of claims 12-19, wherein the execution overhead meeting a first
predetermined requirement comprises at least any one of:

the execution overhead of the target physical plan is lowest in q execution overheads, the q
execution overheads are the execution overheads corresponding to the q physical plans generated
based on the SQL statement, one physical plan corresponds to one execution overhead, and q is
greater than or equal to 1;

or the like, or a combination thereof,

the execution cost of the target physical plan is lower than a first preset threshold.

21. A computer device having functionality for implementing the method of any one of claims 12-20,
the functionality being implemented by hardware or by hardware executing corresponding software,
the hardware or the software comprising one or more modules corresponding to the functionality.

22. A computer device comprising a processor and a memory, the processor coupled with the
memory,

the memory is used for storing programs;

the processor to execute the program in the memory to cause the computer device to perform the
method of any of claims 12-20.

23. A computer-readable storage medium comprising a program which, when run on a computer,
causes the computer to perform the method of any one of claims 12-20.

24. A computer program product comprising instructions which, when run on a computer, cause the
computer to perform the method of any one of claims 12-20.

25. A chip comprising a processor and a data interface, the processor reading instructions stored on a
memory through the data interface to perform the method of any one of claims 12-20.

47 | P a g e