2012 Fourth International Conference on Multimedia Information Networking and Security
Research on system logs collection and analysis model of the network and
information security system by using multi-agent technology
Shi Shengyan
North China Electric Power University
School of Control and Computer Engineering
Beijing, China
shishengyan_2009@163.com
Zhao jianbao
Henan Provincial Electric Power Company
Department of science and technology
Henan, China
jbzhao120@sohu.com
Abstract—In order to realize the full rang of information
security, a variety of network equipment, safe equipment have
been applied to deal with all aspects of information security
and protection by many enterprise. These devices, systems
produce a lot of security event log in the network security
protection, and these event log data format are different, and
different safety equipment may generate the same alerts logs,
not only resulting in generating redundant events, but not
conducive to the next work of network security situational
awareness. Therefore, this paper proposed a method by using
the multi-agent technology to collect and analysis the log data
generated by network devices and security devices, and then
generating a fixed-format data structure and building the log
collection and analysis systems to facilitate the later
maintenance and use of data.
Keywords-component; Multi-agent technology; Log collection
agent ; log collection and analysis system
I. INTRODUCTION
Information security management is a complex and
important work. Information security behavior monitoring is
the basis of the information security management work.
Network information security behavior monitoring refer to
collect a variety of network security events and network
security log data from the safety equipment and information
security systems and proceed data mining and conform the
security situation and provide services for the later research
of the information security and network trend.
At present, in order to realize the full rang of information
security, a variety of network equipment, safe equipment
have been applied to deal with all aspects of information
security and protection by many enterprise, but these systems
are variety, monitoring an aspect of information security
each other, the security event and log data format produced
by these security equipment are different, and different
safety equipment may produce more than one alarm for the
978-0-7695-4852-4/12 $26.00 © 2012 IEEE
DOI 10.1109/MINES.2012.181
Authorized licensed use limited to: Consortium - Algeria (CERIST). Downloaded on November 29,2023 at 09:13:37 UTC from IEEE Xplore. Restrictions apply.
Shen Xiaoliu
North China Electric Power University
School of Control and Computer Engineering
Beijing, China
shenxiaoliu@126.com
Ma Xinke
North China Electric Power University
School of Control and Computer Engineering
Beijing, China
maxinke2005@163.com
same time intrusion event, thus resulting in the generation of
redundant events, and not conducive to promote of the later
network security situational awareness work. Therefore, we
need to centralized collect the security event and log data
produced by security equipment, and then generate a fixed
format data structure for the later maintenance and use of the
data.
This paper proposed a method by using the multi-agent
technology to collect and analysis the log data generated by
network devices and security devices by analyzing the status
of the network and information security work, and created
the centralized data source to providing services for the later
research of information security and network and network
security situation assessment model by building the central
data source, and do some collection, collation analysis and
removing the redundant information from the central data
source.
II. OVERVIEW OF MULTI-AGENT TECHNOLOGY
Agent is an important concept in the field of computer
science, has been widely used in AI(Artificial Intelligence),
distributed computing, CSCW(computer supported
cooperative work), man-machine interface and other areas of
computer science.
IA(Intelligent Agent) is a software entities on behalf of
users or other programs to finish a set of autonomy
operations, and can acquire the knowledge and express of the
user’s goals or aspiration.
IA has autonomy, perception and response capability,
initiative, communication ability, persistence, reasoning and
planning capabilities.
IA is a dynamic and distributed directory service, that
can provide function both client and service procedures used.
In case of the user does not has explicitly require, IA can
replace user to do a variety of complex work, such as
information filtering, query, management and other functions
according to the needs of user.
23
 The multi-agent is combine with many agent, each agent
collaborate, interact with others and finish some complex
tasks or goals together.
III. LOG COLLECTION AGENT
Based on the needs of multi-agent technology and
information security, the log collection agent which this
paper related is combined with seven modules, there are: the
control module, the user interface module, the data
collection module, the communication module, the rule
module, the memory module and the message passing
module , the log collection agent model is shown in figure 1.
Figure 1. The model of log collection agent.
From the above figure, we can know that:
The control module is the core of the IA, is responsible
for the allocation of tasks, coordinating and insuring other
modules are in normal work.
The user interface module is the channel of exchanging
information between agent user and agent itself and the
user’s intent through the user interface module to pass to the
control module for processing and the processing result is
feedback to the users also through the user interface.
The data collection module is mainly to collect the log
information of each devices and filter the log information,
and produce the log data which interest to users by using the
related rules.
The communication module is responsible for
communication between the agent with others.
The rule module is used to store some format standard of
system log and some aspects knowledge of control,
monitoring and communication.
The memory module with a powerful memory provide
services for the log collection agent, is mainly responsible
for storing the log information. The memory module
uploaded the data to the database of log management by
using the message passing module.
24
Authorized licensed use limited to: Consortium - Algeria (CERIST). Downloaded on November 29,2023 at 09:13:37 UTC from IEEE Xplore. Restrictions apply.
The message passing module is responsible for the
message transferring between the database of log
management and the memory module.
At first, the log collection agent collects the data, and
then filters it according to the rules defined in the rule
module. Users can flexibly adjust the parameters of filter and
filter out the data which the user be in interested from the
many log information according to the actual needs.
This article include: network devices, security devices,
applications, host system, the different devices produce
different format security event, such as security log, security
message and security incident. The different safety
equipment may generate many security alerts on the same
intrusion incident, and result in the generation of redundant
events. Therefore, we need a unified data format to
standardized deal with those security event. Therefore, we
define the standard log format as follows:
Log format standard: [log time, log source, protocol,
source address, source port, destination address, destination
port, device type, device name, rules, source MAC].
The log time refers to the time of log generated.
The log source includes: system configuration,
connections, access control, user authentication, and so on.
The protocol includes : TCP, UDP, ICMP, and others.
The source address is the source IP address.
The source port is the port number of source address.
The destination address is the destination IP address.
The destination port is the port number of the destination.
The device type is type of message produced device.
The rule include permit, deny and other.
The source MAC is the source Physical address.
Based on the above function of log collection agent,
build a log collection and analysis system model based on
the multi-agent technology, the model is mainly to collect
and analysis the system log information generated by a
variety of network security equipment and systems during
operation, realizing the goal of data centralized collection,
analysis and management by preprocessing operation these
log information.
IV. LOG COLLECTION AND ANALYSIS SYSTEM
Using multi-agent data integration technology can realize
the functions which can centralized collect, preprocess and
analysis the system log of network equipment, safety
equipment and related security system. The logs that this
article related which log collection agent need to collect are
network equipment logs, safety equipment logs, applications
logs and system logs. Because the log collection agent that
this article related is mainly used to receive the system log
and SNMP log, therefore these devices and systems need to
meet some characteristics:
(1) Network devices
The log collection and analysis system obtain the
information from the monitored network devices via SNMP
protocol. The information includes system information,
interface status ˈ interface port mapping, IP address
forwarding, routing table, the MAC address forwarding,
CUP load, dynamic interface bandwidth and interface
historical traffic data and so on and it can open and close the
network devices interface via SNMP. Therefore the network
device must start the SNMP service. Routing device must
support the MIB II(RFC 1213)(RFC, Request For Comments,
is a set of scheduled numbered files, the basic internet
communication protocols are detailed in RFC
documents),the switching equipment must support the
MIB(management information base, specified some
variables maintained by network elements, are the
management process query and set information, MIB
provides a collection of all possible managed object data
structure ) II(RFC 1213)and Bridge MIB(RFC 1493).
(2) Safety devices
aˊFirewall system
The firewall system of the log collection and analysis
system must support the network device SNMP MIB, MIB
II(RFC 1213), and start the SNMP service.
bˊIntrusion detection system
The intrusion detection system of the log collection and
analysis system must support the network device SNMP
MIB, MIB(RFC 1213), and start the SNMP service.
(3) Application systems
The application systems must provide a log interface to
Figure 2. The log collection and analysis system
get the log information, running information and business
information.
The log collection and analysis system consists of some
(4)The host systems
log collection agent and log storage system. The log
The log collection and analysis system obtain the
collection agent consist of network device log collection
information from the monitored host via the SNMP service,
agent, safety device log collection agent, application system
include system information, network connections, TCP
log collection agent and the host system log collection agent.
connection, the running program, software installation, CUP
The log storage system consists of central storage and
load, storage devices, system configuration, the Windows
original log database. In order to provide some convenient
network services, the Windows user account information etc.
for the later work of data use and analysis, the log collection
The host SNMP service provide all of the above information
and analysis system also provide some data interface to
must support the MIB II(RFC 1213)and the Host Resources
communicate with other system, such as safety management
MIB(RFC 1514).
system, network security situation awareness system and so
The log collection and analysis system mainly consists of
on.
log collection agents and log storage system. The log storage
The log central storage and original log database are the
system consists of log central storage and the original log
core of the log data collection and analysis behavior and are
database. At first, the log collection agent mainly collect the
the way of log collection of the log collection and analysis
log information of all the network devices, security devices,
system. This log collection method can ensure the affairs of
applications systems and host system, and then format these
data collection finished independently. The log collection
log data information with log format which pre-defined and
and analysis system can collect the raw log and not affected
store to the log central storage. We can directly extract the
by the behind program of analysis, database reading and
data from the log storage system without from the network
writing, and at the same time the behind analysis program
device, safety device, application system and host system for
are not affected by the log collection and analysis system and
the later network security situation assessment work. This
not appear the scene which slow down the speed. On the one
centralized storage logs way can improve the security and
hand can ensure all the log data from the safety device are
facilitate the management of the log. In order to maximize
not lost and the data security can be completely guaranteed;
reduction the original information and provide guarantee for
On the other hand can shorten the time of data collection
accurately extraction of the log information, the original
from the safety management, and also can get the real-time
format log stored in the original log database.
log information.
The log collection and analysis system model is shown in
figure 2: V. CONCLUSION
The main content of this paper is to build the log
collection and analysis system. At first, using the intelligent
agent technology to realize log collection agent, and then
using the log collection agent and some log storage structure
to realize the structure of the log collection and analysis
system.

25

Authorized licensed use limited to: Consortium - Algeria (CERIST). Downloaded on November 29,2023 at 09:13:37 UTC from IEEE Xplore. Restrictions apply.
 Through the establishment of the log collection and [2] Shizhongzhi.Intelligent agent and its application[M].Beijing:Science
analysis system, to collect, analysis, integrate the log data press,2000
information from the variety network device, safety device, [3] Shaofengjing,Yuzhongqing.Data mining principles and
algorithms.China WaterPower Press,2003
application systems and host systems and then form the data
[4] Zhangjianpin,Chentianyun,Wanglixing.Research on the IA model for
structure with unified data format, and provide data support network collaborative learning[J]. e-EDUCATION
for the later part of calculate the network security situation RESEARCH,2004 ,(5)
assessment, network security situation assessment and the [5] Tipton HF, Krause M.Information security management
network security situation prediction. handbook[M].Wangweiwei,Yangbo,translation.
[6] Wujunhua,Zhengyu.A mail server log data analysis
REFERENCES methods[J].Journal of Nanjing University of technology, 2003,Vo25
[7] Lvjia.Web log mining application[J].Journal of Chongqiong Normal
[1] Lixiao,Wuzhenghong.The design of blended learning model based on University,2006,Vo23
the agent[J].Guangdong Radio and Television University.2005,Vo14.

26

Authorized licensed use limited to: Consortium - Algeria (CERIST). Downloaded on November 29,2023 at 09:13:37 UTC from IEEE Xplore. Restrictions apply.