Accepted Manuscript

A survey on forensic investigation of operating system logs

Hudan Studiawan, Ferdous Sohel, Christian Payne

PII: S1742-2876(18)30398-0
DOI: https://doi.org/10.1016/j.diin.2019.02.005
Reference: DIIN 835

To appear in: Digital Investigation

Received Date: 30 October 2018

Revised Date: 31 January 2019
Accepted Date: 26 February 2019

Please cite this article as: Studiawan H, Sohel F, Payne C, A survey on forensic investigation of
operating system logs, Digital Investigation (2019), doi: https://doi.org/10.1016/j.diin.2019.02.005.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to
our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and all
legal disclaimers that apply to the journal pertain.
 ACCEPTED MANUSCRIPT

A survey on forensic investigation of

operating system logs

Hudan Studiawan∗, Ferdous Sohel, Christian Payne

Discipline of Information Technology, Mathematics, and Statistics,

PT
Murdoch University, Australia

RI
Abstract

SC
Event logs are one of the most important sources of digital evidence for forensic investigation because they
record essential activities on the system. In this paper, we present a comprehensive literature survey of the
forensic analysis on operating system logs. We present a taxonomy of various techniques used in this area.
Additionally, we discuss the tools that support the examination of the event logs. This survey also gives a

U
review of the publicly available datasets that are used in operating system log forensics research. Finally,
we suggest potential future directions on the topic of operating system log forensics.
AN
Keywords: operating system logs, event logs, log forensics, log tamper detection, event correlation, event
reconstruction, event anomaly
M

1. Introduction 2011; Studiawan et al., 2017). Accordingly, meth-

ods to maintain the integrity of logs and to de-
There are various forms of digital evidence. tect any modification have previously been studied
For example, the browsing history, chat logs, au- 25 (Boeck et al., 2010; Sato and Yamauchi, 2012). In
D

thentication log files, and deleted files or im- general, log forensics have been investigated exten-
5 ages. Event logs are files that record the im- sively and a rich volume of literature is available on
TE

portant activities performed by the user, appli-
cation software, or operating systems. There-
fore, these files are considered one of the main
pieces of evidence for digital forensic analy-
EP
this topic. In this paper, we provide a comprehen-
sive survey of this existing literature.
30 This paper reviews various aspects of the forensic
analysis of event logs focusing on operating system

10 sis. For example in Windows operating system,
the log files are usually found in the directory
C:\Windows\System32\Winevt\Logs\, while in the
Linux environment they are located in /var/log/.
C
(OS) logs. We construct a taxonomy on the ba-
sis of a generic investigation pipeline such as event
logs recovery, event correlation, event reconstruc-
35 tion, and visualization. Based on a generic forensic

Log files serve various purposes. An event log framework (Yusoff et al., 2011), we categorize ex-
15 can be used as evidence in court (Ibrahim et al., isting publications within this taxonomy. We then
AC

2012). These files may assist in the reconstruction
of an attack (Liao and Langweg, 2014). They can
also support identification of relationships between
separate events (Herrerı́as and Gómez, 2010; Amato
20 et al., 2017). Log files can be used to detect anoma-
lous user behavior or system activity (Corney et al.,
∗ Corresponding
author
Email addresses: hudan.studiawan@murdoch.edu.au
(Hudan Studiawan), f.sohel@murdoch.edu.au (Ferdous
Sohel), c.payne@murdoch.edu.au (Christian Payne)
Preprint submitted to Digital Investigation
thoroughly discuss the advantages and disadvan-
tages of the techniques in each category.
40 The paper is structured as follows. Section 2 pro-
vides a brief description of other relevant surveys
and then outlines the contributions made by this
work. Section 3 defines the terminology used in
this paper. We present the survey methodology for
45 event log forensics in Section 4. A mind map of the
event log forensics taxonomy is also depicted in this
section to describe the high level structure of the as-
sorted methods used in this area. The subsequent
February 27, 2019
 ACCEPTED MANUSCRIPT
sections (Section 5 to Section 9) will describe a re-
50 view for each step in the generic framework. We
also describe the forensic tools in Section 10 and
the publicly available datasets to support OS log
forensics experiments in Section 11. The discussion
about datasets will assist future researchers to se-
55 lect the most appropriate data and case study. We
outline the current challenges and potential future
directions of research on this topic in Section 12.
Finally, Section 13 concludes this study.
2. Relevant surveys and our contributions
60 There are several other surveys connected with
event log forensics. A recent survey by Khan et al.
(2016) focuses exclusively on event log forensics in
the cloud environment. Khan et al. (2016) analyze
the accessibility of cloud logs, logging as a service,
U
65 security requirements, and possible security chal-
lenges faced in the cloud. There are several other
AN
surveys on the cloud log forensics (Mishra et al.,
2012; Almulla et al., 2014; Farina et al., 2015).
However, they are not as comprehensive as Khan
70 et al. (2016).
Mishra et al. (2012) focus on reviewing existing
M
cloud forensics frameworks. Almulla et al. (2014)
give other insights by classifying the cloud log foren-
sics not only by the method, but also by trending
D
75 technology and the forensic framework. A different
view of cloud log forensics is presented by Farina
TE
et al. (2015) by providing a review of remote foren-
sics techniques, live forensics, and cloud-facilitated
forensics analysis.
80 Other related survey paper (Chabot et al., 2015)
discusses event reconstruction based on log files for
EP
forensic purposes. However, the study by Chabot
et al. (2015) does not provide any clear classifica-
tion of existing techniques and describes only a few
studies on event reconstruction. A review of web
C
85
log forensics was presented in Lazzez and Slimani
(2015). In addition to discussing the methodology,
AC
Lazzez and Slimani (2015) provide a comparison of
several investigation tools for conducting the web
90 application forensics.
Compared with existing related survey articles,
the contributions of this study are listed below.
(a) This study provides a broad range of topics
across a large number of papers, including
95 event log security and recovery, event recon-
struction and correlation, event anomalies, and
visualization.
(b) We present a list of existing methods for each
topic, provide a critical summary, and analyze
100 the respective advantages and disadvantages.
(c) This paper reviews methods for the forensic
analysis of OS logs as this evidence is com-
monly found when extracted from a forensic
PT
disk image.
105 (d) We present a mind map taxonomy to assist
with the classification of research across a
range of areas related to forensic investigation
RI
of OS logs.
(e) We discuss forensic tools as well as public
SC
110 datasets focusing on OS logs.
3. Terminology
For clarity, this section outlines the terminology
used in this survey. An event is an identifiable ac-
tion that happens on a device and is recorded in a
115 log entry (European Commission, 2010). An event
log is a record of events, usually implemented in a
log file or a table in a database. A log file is a file
that records activities from applications or operat-
ing system. This artifact is the primary focus in
120 event log forensics and contains log entries. A log
entry is defined as a single record in a log file. An
event message refers to the main message in a log
entry excluding timestamp and any other fields such
as hostname and application process name. Oper-
125 ating system logs are log files in a particular OS
such as Windows and Linux.
Additionally, digital evidence is defined as any
data stored or transmitted using a computer that
supports a theory of how an offense occurred or how
130 the data addresses critical elements of the offense
(Casey, 2011). Although the digital data is not di-
rectly related to an attack, it will be considered as
a digital evidence as long as it is discovered in a
crime scene. The forensic investigator is a person
135 who analyzes the digital evidence, specifically event
logs in this case. An artifact is a digital object that
will be investigated such as log file, disk, memory,
or an image file. However in this survey, an artifact
will always refer to an event log file from an OS
140 unless otherwise indicated.
4. Survey methodology
In this section, we describe the methodology for
constructing the taxonomy of event log forensics.
2
 ACCEPTED MANUSCRIPT
The detailed aspects of the methodology are ex-
145 plained in each subsection below.
4.1. Forensic framework for classifying studies
We refer to Generic Computer Forensic Investiga-
tion Model (GCFIM) (Yusoff et al., 2011) and map
existing publications into this framework. We chose
150 this framework because it provides completeness
and can accommodate various methods for event
log forensics. Another general advantage of GC-
FIM is that it is created based on the identification
of the common processes in forensic investigation.
155 GCFIM is built based on a detailed review of 15
digital forensic investigation models.
The following are the typical steps in event log
forensics based on GCFIM framework.
(a) Pre-processing step as forensic readiness of OS
160 logs
U
This step concerns secure handling of logs and
provides an explanation of event logs as a dig-
AN
ital evidence. If the event logs are secured by
design, then they will be forensically ready to
165 be examined when a cyber incident has oc-
curred.
M
(b) Acquisition of OS logs
Acquisition refers to the recovery of event logs.
This artifact may be already available or the
D
170 investigator may need to recover event logs
from a device because they have been deleted.
TE
(c) Main analysis of OS log investigation
This is the main phase of the framework. The
investigator will check if there are any modifi-
175 cations in the logs. They can be accessed using
EP
a special query or retrieval technique. In this
phase, the most common analyses conducted
are event correlation, event reconstruction, or
anomaly detection.
C
180 (d) Visualization of OS logs from investigation re-
sults
AC
After the analysis is complete, the result needs
to be visualized in order to provide the forensic
investigator with relevant insights. Visualiza-
185 tion can also produce both general and detailed
events for further investigation.
(e) Post-process of OS log investigation
The last phase of the framework is a review
of the overall investigation process. This step
190 includes evaluation of digital forensic investi-
gation frameworks. The investigator needs to
evaluate which model is more appropriate for
a particular case.
The description of these aspects in the event log
195 forensics framework is summarized in the Fig. 1.
Sections 5 - 9 give a detailed description of each
topic. A mind map of the taxonomy of OS log foren-
PT
sics is provided in Fig. 2. Besides the framework,
the mind map includes the tools and open datasets
200 that are publicly available to promote reproducible
research.
RI
4.2. Inclusion and exclusion criteria for literature
SC
We define the inclusion and exclusion criteria for
papers discussed in this survey as there are many
205 papers that relevant to OS log forensics topics. The
inclusion criteria are explained as follows.
(a) The papers consider an issue in one of steps in
forensic investigation. The steps include pre-
processing for forensic readiness, acquisition of
210 OS logs, and main forensic examination such
as tamper detection and event reconstruction.
The content of the paper should discuss one of
these forensic aspects explicitly.
(b) The papers discuss forensic tools such as tools
for extracting Windows event logs.
215
(c) The papers discuss case study and forensic
dataset which is considering OS logs.
(d) The publication is written in English and stan-
dardized paper style.
220 Moreover, we specify the exclusion criteria for
some papers that are not included in this survey.
The criteria are described below.
(a) Papers discussed forensic analysis of network
logs or network traffic saved in a packet capture
225 (pcap) file. For further information about this
subject, the readers are referred to a survey of
network forensics (Pilli et al., 2010).
(b) Event logs in forensic topics are not to be con-
fused with logs from business process as this
230 topic are intensively discussed in the process
mining research area. For more detail about
business event logs, the readers can check a
survey paper on process mining (Van der Aalst,
2013).
3
 ACCEPTED MANUSCRIPT

Pre­processing step as ­ Guarantee the security of OS logs
forensic readiness of OS logs ­ OS logs as a digital evidence

Acquisition of OS logs ­ Recovery of the OS logs from the device

PT
­ Retrieval of OS logs

RI
Main analysis of  ­ Detection of tamper operation
OS log investigation ­ Correlation and reconstruction of events
­ Detection of anomaly 

SC
Visualization of OS logs ­ Visualization to present the analysis results

U
AN
Post­process of  ­ Review the investigation process
OS log investigation
M

Figure 1: Proposed framework of OS log forensics based on Generic Computer Forensic Investigation Model (Yusoff et al.,
2011)
D

235 (c) The method and tools which do not explicitly application actions. Therefore, provenance is
write about forensics. For example, there are useful for event log forensics to provide digi-
papers discussing anomaly detection in event tal evidence for post-investigation. For a com-
TE

log data, but they do not explicitly discuss the
forensic investigation. Therefore, we do not
240 include these kinds of papers in this survey.
EP
260 plete review of secure data provenance for var-
ious purposes and applications, we recommend
the reader to refer to a survey by Zafar et al.
(2017).

(d) Cloud log forensics

Cloud computing has been an emerging area 4.3. Paper collection
in information technology and therefore there 265 There are three phases for paper collection in a
are many studies that examine event logs in survey paper. The first phase is to search the liter-
C

245 a cloud environment. As discussed previously, ature. We explore the papers on event log forensics
we have excluded cloud log forensics from the in the following online libraries from several lead-
AC

scope of this paper. Interested readers are re-
ferred to Khan et al.’s survey of cloud-based
log forensics (Khan et al., 2016).
250 (e) Secure data provenance
Provenance is information containing where
and how a data, such as log files, was written,
who was created the data, and modifications
involved (Zafar et al., 2017). Provenance can
255 determine suspicious activities in a system be-
cause of its tracking ability to both kernel and
ing publishers: ACM Digital Library, ScienceDi-
270 rect, IEEE Xplore, and Springer Link. We also
search through Google Scholar as this search engine
is specially designed for searching scientific litera-
ture. The term used for searching through these
digital libraries is “event log forensics”. We con-
275 sider all the papers related to the area from 1997-
2018.
The second phase is filtering the search results.
We filter the results whether or not relevant to OS
log forensics. We first read title, abstract, and then
4
 ACCEPTED MANUSCRIPT

Cryptographic approaches
Log centralization

OS log security Cryptographic log centralization

Pre-processing
Virtual machines
Secure data structure

OS logs as digital evidence

PT
Acquisition of OS logs

RI
XML-based

OS log retrieval Database

Live capture

SC
Rule-based

Tamper detection Cryptographic hashes

Hardware-based
Rule-based

U
Database
Semantic model
AN
Main analysis
Tree or graph-based
Event correlation and reconstruction
Timestamp-based
Finite state machines
M

OS log forensics Virtual machines

Live event reconstruction
User profiling and machine learning
D

Anomaly detection Timestamp-based

Log clustering
TE

Event log abstraction

Forensic timelines
EP

Visualization of OS logs Tree-based

Graph-based

Post-process of OS log investigation

C

General tools
Tools for OS log forensics
AC

Libraries
Digital Corpora
DFRWS Challenge
Public datasets for OS log forensics CFReDS Project
e Honeynet Project
SecRepo

Figure 2: A taxonomy of OS log forensics literature

5
 ACCEPTED MANUSCRIPT
280 the contents of a paper. The filtering is based on
inclusion and exclusion criteria described in Section
4.2.
The third phase is to conduct a recursive search
for references. In each paper found from the second
285 phase, we check the references and open the po-
tential article. We apply the second phase to each
paper. We want to trace the reference to its first
publication discussed about a certain topic. This
process is assisted by Google Scholar feature “cited
290 by” represented by the quotation mark button from
a particular paper title.
Furthermore, we identify some information about
the papers when collecting specifically:
(a) full citation reference, such as author names,
295 title, journal or conference name, and year,
(b) classification of the paper to GCFIM as the
U
formal forensic framework used in this survey,
(c) sub-classification of the paper inside the main
AN
framework of GCFIM,
300 (d) summary of the proposed method, its advan-
tages, as well as its disadvantages.
M
5. Pre-processing step as forensic readiness
of OS logs
D
This section discusses the pre-processing step in
305 OS log forensics. According Tan (2001), digital
TE
forensic readiness (DFR) has two aims. First, DFR
is designed to maximize an organization’s ability
to acquire credible digital evidence. Second, DFR
intends to minimize the cost of investigation when
EP
310 a security incident has happened. In the context
of OS log forensics, DFR is related to build secure
log infrastructure, so it will be forensically ready to
be analyzed. In addition, Elyas et al. (2015) sug-
C
gested that DFR not only related to infrastructure
315 but also be legal-ready. Therefore, we also discuss
AC
OS logs as digital evidence that can be presented
in courts.
5.1. OS log security
This phase deals with securing OS logs from mod-
320 ification so that they can be forensically examined.
This process is also referred as ante-mortem foren-
sics where the system is ready to be investigated in
future when an attack or a security incident hap-
pens. We classify security of OS logs into five cate-
325 gories based on the methods used: 1) cryptographic
approaches; 2) log centralization; 3) cryptographic
log centralization; 4) virtual machines; and 5) se-
cure data structures.
5.1.1. Cryptographic approaches
330 The standard technique to identify whether or
not data is modified is using a hash function. It
PT
produces a fix-length string from arbitrary data in-
put called hash value. A cryptographic hash pro-
vides a benefit that the hash value is infeasible
RI
335 to be converted back to the original input data.
Schneier and Kelsey (1999) propose a basic method
for securing event logs for forensic purposes based
on cryptographic hashes. It provides security and
SC
integrity for event logs even on an untrusted de-
340 vice. This work became the foundation for sub-
sequent research on log security. In general, the
method creates an authentication key which is as-
sumed securely generated and saved on the ma-
chine. This key is used to cryptographically hash
345 each log entry. Every single log entry becomes part
of a hash chain to authenticate all previous log en-
tries (Schneier and Kelsey, 1999). However, the
Schneier and Kelsey’s method has a limitation. If
an attacker is able to gain access to an insecure
350 machine, where the event logs are saved, then the
attacker can continue to write log entries as if it is
authorized (Schneier and Kelsey, 1999).
To make the OS logs more secure, there is a
method to prevent any edit or delete operation in
355 the log files (Etoh et al., 2010). They apply a de-
centralized management of log files in several log
servers based on a cryptographic hash function. It
means the log files are split into chunks and each
chunk possibly located in different servers. Each
360 chunk is copied not only to one server to provide
redundancy for backup purposes. However, if re-
dundancy in these dispersion files is removed by
the attacker, the detection of an attack is not al-
ways possible (Etoh et al., 2010).
365 5.1.2. Log centralization
Traditional logging systems such as syslogd
(Wettstein et al., 2018), and its extension
syslog-ng (One Identity, 2018), can provide
backup and security to event logs on a Unix-based
370 OS. These systems can be configured on either lo-
cal or remote machine. Research has found that
the centralization of event logs can increase secu-
rity and also standardizes the logging mechanism
for forensic purposes (Sahoo et al., 2012). Sahoo
6
 ACCEPTED MANUSCRIPT
375 et al. (2012) present a system for securing Win-
dows event logs against software or hardware fail-
ures. The Windows event logs are converted to
syslog format because syslog is the common log-
ging standard for various devices. The event logs
380 are collected proactively at local machine and these
logs are then sent to remote logging server.
The advantage of the centralization method is
that it provides an automatic approach to record
Windows event logs on a dedicated logging server
385 and also gives a monitoring interface. The proposed
method employs a service attached to a native Win-
dows process. This approach leaves the original log
files on the local machine and sends the duplicates
to the server. Therefore, it is less vulnerable when
390 the machine is compromised as there are copies of
event logs. The centralization method is more fo-
cused on maintaining the confidentiality of event
U
logs. Although this architecture can be extended
to other platforms, the method currently only sup-
ports Windows event logs. In addition, this ap-
AN
395
proach does not encrypt the log entries when they
are sending to server. The log duplicates in the
server are also not encrypted. Therefore, we need
cryptographic log centralization to provide more se-
M
400 cure event logs as discussed in the next section.
5.1.3. Cryptographic log centralization
D
While centralization techniques and crypto-
graphic log entries are separate methods, they are
provide greater security if combined into a hybrid
TE
405 architecture. A system can defend event logs from
malicious attacks by building a network processor
and secure operating system as proposed by Ra
and Park (2009). A network processor is a hard-
EP
ware similar to central processing unit (CPU) and is
410 programmable for networking-related applications.
For the secure OS, the authors use a hardened ver-
sion of Linux utilizing a security module based on
C
US Department of Defense requirements. This se-
curity module is well-known as Security-Enhanced
AC
415 Linux (SELinux). The evaluation of this method
shows it can successfully detect unauthorized ac-
cess attempts on a machine. However, there is a
disadvantage in Ra and Park’s approach. The pro-
posed method imposes a time overhead for crypto-
420 graphic process which needs to be reduced to make
the performance acceptably faster (Ra and Park,
2009). In addition, this architecture only supports
Linux-based environments in the given implemen-
tation.
425 Another work proposes a mechanism to validate
and authenticate syslog (Monteiro and Erbacher,
2008). This study adds various fields such as user-
name, application, and system to each syslog entry.
It also adds an authentication mechanism before
430 sending the logs to the server. However, a proof of
concept with experiments is not available in that
work (Monteiro and Erbacher, 2008).
PT
To deal with event logs from multiple sources in-
cluding OS logs, Lin et al. (2009) provide an auto-
435 matic forensic analysis by implementing a collection
agent. This procedure first aggregates the event
RI
logs, then normalizes and analyzes them in a cen-
tral location. In addition, a combination of hash
function, digital signature, and timestamp are used
SC
440 to preserve the integrity and authenticity of event
logs (Lin et al., 2009). However, there is a prob-
lem with duplication of logs from the proposed log
collection agent and some attacks are unable to be
reconstructed (Lin et al., 2009).
445 In addition, one can improve the authenticity
of event logs using a distributed log architecture,
called BBox, along with event log encryption and
tamper detection (Accorsi, 2011). One drawback
of the BBox architecture is adding entries to the
log file is rather expensive and becomes a bottle-
450
neck. Therefore, the method needs more efficient
data structures (Accorsi, 2011).
As centralization and cryptography provides
many advantages, we outline a typical model for
455 this approach in Fig. 3. There are two main parts
of the architecture, namely monitored client and
forensic server. The first part contains a log col-
lection agent, which processes OS logs and creates
a cryptographic hash. The log entries are sent to
the forensic server with encryption enabled. On
460
the server side, the log entries are decrypted and
the hash is verified. The server also normalizes OS
logs for further analysis such as insertion into se-
cure database or tamper detection operation. The
465 investigator can view and monitor all log entries in
this server side.
5.1.4. OS log security using virtual machines
The use of virtual machines can preserve the in-
tegrity of OS logs as demonstrated by Chou et al.
470 (2008). The authors also propose using a kernel
module to write OS logs so it can be securely loaded
together with other kernel modules. They test two
approaches and chose the virtualization technique
because it offers a more secure environment as it is
475 isolated from the host operating system. However,
there is no explanation about how can the host and
7
 ACCEPTED MANUSCRIPT
Monitored client
OS logs
Cryptographic hash
Encryption
Log collection agent
Figure 3: A typical model for OS log security using centralization and cryptographic approach
guest operating systems can communicate securely
(Chou et al., 2008).
In 2012, a study addressed secure methods for
U
480 logging by employing a separate virtual machine to
store the OS logs (Sato and Yamauchi, 2012). The
proposed mechanism compares the logs in a mon-
AN
itored host operating system and the virtual ma-
chine. Although this method is able to detect log
485 loss and data tampering, it imposes a large over-
head in sending the logs to the virtual machine.
M
5.1.5. Secure data structures
D
In order to identify the modifications of an event
log, a secure data structure can be constructed as
an indexing engine. The tree-based data structure
TE
490
can provide incremental and membership proof of
each log entry (Crosby and Wallach, 2009). This
mechanism will provide evidence that the OS logs
are authentic and detect any tampering attempt.
EP
495 The merit of this approach is it produces only a
very small sized hash chain for very large log en-
tries. Although this method was applied to syslog,
it can be extended to the various types of event
C
logs thanks to the generic nature of the tree data
500 structure.
AC
Another approach starts with building a model
for the secure data structure based on combina-
torial group testing (Goodrich et al., 2005). The
data structures included are arrays, linked lists,
505 binary search trees, skip lists, and hash tables.
However, this secure data structure does not
consider existing log entries that get rewritten
as usually the system will overwrite the old log
entries. This generic protocol can be applied to
510 secure OS logs.
Forensic server
Decryption and 
hash verification
Secure 
Log normalizer
PT
database
Log viewer Tamper detection
RI
SC
Besides the research in event log security to sup-
port the forensic investigation, there are articles
that provide a framework or a review in this area.
Ahmad and Ruighaver (2003) define the require-
515
ments of an audit management infrastructure to
improve log security. The framework is based on a
top-down approach to fill the gap between organi-
zational security policy and event log configuration.
520 Ayrapetov et al. (2002) present an improvement of
the secure OS logging mechanism. It presents a
taxonomy of existing secure logging systems, ana-
lyzes the unaddressed issues, and provides possible
solutions for future research.
525 Two similar papers by Accorsi (Accorsi, 2009a,b)
review existing secure OS log protocols. This work
presents the comparison of the security require-
ments fulfilled by the existing secure protocols for
OS logs as digital evidence. Accorsi emphasizes
530 that the event log security mechanism needs formal
verification to guarantee the correctness of the pro-
posed algorithm. Furthermore, there is a need for a
standard format for event logs as a digital evidence
that will be presented in court (Accorsi, 2009a,b).
535 Finally, we summarize the various methods in event
log security in Table 1.
5.2. OS logs as digital evidence
Event logs have been evaluated for their qual-
ity as digital evidence in courts, with one of the
540 earliest works that provides a framework of tests
to evaluate digital evidence being found in Som-
mer (1997). A log file can be used as an evidence
when it has passed several tests such as acquisi-
tion process test, chain of custody test, and quality
545 of forensic presentation test. Acquisition process
test has two main aspects specifically accurate and
8
 ACCEPTED MANUSCRIPT

Table 1: A summary of key publications in OS log security

Method Publication Advantage Disadvantage
Cryptographic Schneier and Kelsey Able to run in unsecure machine Invalid log entries are still processed
approaches (1999)
Etoh et al. (2010) Disperse the log files Attack detection depends on
redundancy
Log centralization Sahoo et al. (2012) Simplify logging mechanism Only support Windows
Cryptographic log Ra and Park (2009) Double security mechanism Produce cryptographic time

PT
centralization overhead
Lin et al. (2009) Deal with multi source event logs A few reduplicate logs
Accorsi (2011) Complete package of log security A little bottleneck when adding log
entry
Virtual machines Chou et al. (2008) Demonstrate that virtualization No explanation how host and guest

is better than kernel module
Sato and Yamauchi (2012) Prevent both event log loss and
tampering
Secure data structures Goodrich et al. (2005) Provide security in regular data
RI
OS communicate securely
Produce a big overhead when
sending logs
Do not consider data that changes

SC
structure over time
Crosby and Wallach Produce a small hash chain for This method can be extended to
(2009) large logs various type of logs

U
complete. First, accurate means free from any con-
tamination when an acquisition is performed. The
AN
procedure must be conducted by certified forensic
550 investigator. Second, the acquired evidence can tell
a complete chronology of particular set of event se-
quences. In a chain of custody, the log evidence has
been acquired on a device, viewed, and investigated.
M
580 ciency of Windows event logs as a digital evidence.
Assessment of event logs based on admissibility and
weight regarding legal evidence is presented. The
general standard for admissibility of evidence is to
prove that the evidence is relevant, authentic, and
585 reliable. It is also required that the evidence satis-
fies the legal rules and does not contain hearsay ma-

This evidence is possibly copied several times by terial. In assessing the weight of evidence, a number
555 the police or forensic experts. In this case, the log of features are put into consideration based on re-
evidence must be securely preserved and tamper- quirements from Sommer (1997). The features are
D

proof. To test the quality of forensic presentation, 590 authenticity, accuracy, completeness, clear chain of
the evidence in electronic form needs to be printed custody, and transparency of forensic procedure.
and presented in a court. The authorities should However, this method was only tested on Windows
TE

560 offer two presentation forms, specifically raw and 2003 whereas the paper was written in 2012.
detailed version and an edited one. The later ver-
sion provides narrative to explain what was done
and why to be clearly understood by court. 6. Acquisition of OS logs
EP

Another research article assesses the evidential 595 In the acquisition phase, event logs are acquired
565 weight of OS logs (Ahmad and Ruighaver, 2004). from a device, specifically a hard disk. The problem
This work defines three specific criteria namely ac- arises when the log files are removed, so we need to
C

curacy, completeness, and utility to evaluate event recover them. This section discusses acquisition OS
logs as digital evidence. Accuracy and completeness logs. In some cases, OS logs are deleted by the at-
AC

are based on criteria from Sommer (1997). More-
570 over, Ahmad and Ruighaver (2004) add utility as
one additional criteria. The utility is a desirable
quality of event logs that makes the evidence more
exposes some factors, specifically: 1) a proof of the
correct working activities on the investigated sys-
575 tems; 2) an identification of the host system and
incidents in detail; and 3) an identification of any
information contained in event logs.
For Windows as the most common operating sys-
tem, Ibrahim et al. (2012) have analyzed the suffi-
600 tacker to wipe the digital evidence. Additionally,
the deletion of a log file is a typical malware behav-
ior in Windows environment (Bayer et al., 2009).
The forensic investigator needs to recover this ev-
idence to ascertain further information about the
605 incident. Therefore, some studies focus on the re-
covery of deleted log files before conducting main
forensic analysis.
File recovery can be performed without any file
system metadata available. Exploiting this prop-
610 erty, Richard III and Roussev (2005) created the
9
 ACCEPTED MANUSCRIPT
Scalpel tool. This is a file carving tool that recov-
ers any files by analyzing the header and footer of
a chunked segment on the disk, which can then be
reconstructed to get the whole file. Subsequently,
615 Craiger (2005) proposed a technique to recover var-
ious digital evidence from Linux environment in-
cluding event logs. This technique uses standard
Linux command such as dd to recover a file based on
a particular signature. The benefit of Scalpel tool is
620 that it can recover the deleted file quickly and sup-
ports all major OS such as Windows, Linux, and
Mac OS X. Although the Craiger’s technique can
use Linux standard commands and tools, it does not
scale for large quantities of event log data (Craiger,
625 2005).
In the Windows environment, Murphey (2007a)
proposes a technique for automatic recovery and
repair of Windows NT5 (XP and 2003) event logs.
U
This method employs the Scalpel tool for recovery.
630 It then repairs the file by scanning for the trailer
signature and validates the result using LogParser
AN
tool (Microsoft, 2005). However, this approach has
not been extended to Windows NT6 (Vista) and
newer versions (Murphey, 2007a).
Furthermore, Schuster (2007) investigates Win-
M
635
dows Vista event logs and recovers them by parsing
the unique magic strings and block layout of a log
file. Every type of log file has these magic strings
D
in the header that differ from one to another, as-
640 sisting the recovery process. In addition, this work
offers a detailed description of Windows Vista logs.
TE
However, not all elements in the XML file can be de-
fined due to the unavailability of official documen-
tation from Microsoft (Schuster, 2007). Another
technique exists to retrieve the fragmented event
EP
645
log files automatically (Lou et al., 2009). It is able
to look for fragmented log files without metadata
by using a signature and the entropy difference be-
tween adjacent disk clusters. Similar to Murphey
C
650 (2007a), its main disadvantage is that it only cov-
ers Windows NT5 event logs. Another improve-
AC
ment can be made by defining a better fragment
boundary of a log file that saved in the disk. The
more accurate fragment detection will improve the
655 recovery results.
7. Main analysis of OS log investigation
This phase is the primary process of the foren-
sic investigation. Various types of examination are
conducted on the acquired evidence as discussed in
660 Section 6. The main objectives of this phase are to
detect a possible modification to OS logs, identify
and reconstruct the cause of incident by event cor-
relation and reconstruction, and anomaly detection
in log files. In addition, this section also addresses
665 event log retrieval and event log abstraction.
7.1. OS log retrieval
PT
In the case of digital forensics, retrieval deals with
how the investigator can save, search, or perform a
query on the event logs. This process can be clas-
RI
670 sified based on the type of storage used: 1) XML-
based; 2) database; and 3) live capture.
SC
7.1.1. XML-based log retrieval
Alink et al. (2006) present a mechanism called
XIRAF to manage and query digital evidence in-
cluding event logs from Windows. XIRAF extracts
675
the digital evidences from a forensic image and
converts them into XML (eXtensible Markup Lan-
guage) format. The investigator then performs in-
dexing and querying based on the XML database.
680 Another study parses Windows Vista event logs to
facilitate more effective analysis (Huang and Wu,
2009). The authors organize binary XML Windows
Vista event logs and convert these to a readable
XML-based structure. Another approach, namely
685 XLIVE, builds an XML-based structure to save
various event logs including OS logs and classify
them (Lee et al., 2010). This method then provides
a framework for automatic investigation. XLIVE
also supports live capture of digital evidence as dis-
690 cussed in the next subsection.
Although XIRAF (Alink et al., 2006) offers a
quite complete query platform, it does not provide
support for binary file indexing such as BLOB type
to be correlated with other log files. Another XML-
based method (Huang and Wu, 2009) still needs fur-
695
ther research for the unidentified data type in Win-
dows Vista event logs. In addition, XLIVE (Lee
et al., 2010) needs to be extended to non-Windows
environments.
700 A typical model for event log retrieval based on
an XML approach is given in Fig. 4. First, the OS
logs are preprocessed and the fields such as times-
tamp, process identifier, and the main message are
extracted. The log entries are then written into
705 an XML-based repository or database. The foren-
sic investigator can access the event logs via query
interface to examine a particular event or analyze
suspicious events via a log viewer.
10
 ACCEPTED MANUSCRIPT
Pre­process and  XML writer
field extraction
OS logs
Query interface
Forensic 
investigator
Log viewer
Figure 4: A typical model for OS log retrieval using XML-based approach
7.1.2. Log retrieval using database
The performance of searching for a particular
710
log message is important as a part of the anal-
ysis. Takahashi and Xiao (2008a,b) analyze the
complexity of event log retrieval in the searching
process. They compare and calculate the complex-
U
715 ity of a number of searching algorithms to retrieve
event logs in terms of big O notation. Awawdeh
AN
et al. (2013) suggest implementing recording mod-
ules for event logs and saving the processed logs
into a database. The investigator can then perform
720 a query to the database for further analysis.
M
Takahashi and Xiao (2008a) compare some search
algorithms for event logs. They report that the
work can still be improved by evaluating other tech-
niques as there are many such algorithms. The later
D
725 method (Awawdeh et al., 2013) only uses a small
portion of memory to process large logs. In spite
TE
of this advantage, the method can be extended to
other environments such as Windows 8, Linux, and
Mac OS X.
EP
730 7.1.3. Live capture of OS logs
Live capture means the analysis is conducted in
real time and there is no need to wait until an inci-
dent occurs. Choi et al. (2008) introduce live analy-
C
sis of digital evidence in Linux environments. They
735 provide a framework for real-time analysis, specifi-
AC
cally for evidence collection, forensic analysis such
as investigating running processes by a particular
user, and generating reports. This framework can
be improved by generalizing the capability to other
740 Linux distributions.
On the other hand, one can use XML-based
structures to perform live capture called XLIVE
(Lee et al., 2010) as discussed previous subsection.
XLIVE can both capture volatile and non-volatile
745 data including event logs. There are four main
modules in XLIVE: 1) type analyzer to detect the
XML­based repository 
or database
PT
XML parser
RI
type of digital evidence, 2) data collection module
to organize the evidence found, 3) data parsing
SC
and writing to manipulate the digital evidence into
750 XML structure, and 4) database and report mod-
ule. An agent-based approach (Awawdeh et al.,
2013) is another alternative for real-time evidence
collection. Its main module to get event logs is
Windows Event Watcher which gathers all event
755 logs generated by Windows such as application
logs, hardware event logs, and security logs.
Besides the aforementioned categories, there are
specific studies on the Windows operating system
760 for event log retrieval. In Talebi et al. (2015), a
deep analysis of Windows 8 event logs is presented
for the first time. This work explains a detailed
anatomy of Windows 8 event logs such as event
types, log format, and log structure. It also pro-
765 vides forensic analysis of an unauthorized access
attempt. Furthermore, the investigator can per-
form analyses in the Windows environments using
PowerShell. Barakat and Hadi (2016) demonstrate
PowerShell’s ability for evidence collection, extrac-
770 tion, and identifying various forensic artifacts in-
cluding event logs. The use of PowerShell brings
many advantages as it is a native tool supported
in Windows. The features from Barakat and Hadi
(2016) can be extended for newer Windows ver-
sions. A summary of event log retrieval publica-
775
tions is shown in Table 2.
7.2. Tamper detection of OS logs
The detection of modification is needed to vali-
date the integrity of event logs. It often becomes
780 a part of the same event log security process de-
scribed in Section 5.1. The difference with the pre-
vious phase is that the detection is performed after
an incident occurred. We divide the approach of
tamper detection into four main categories: 1) rule-
11
 ACCEPTED MANUSCRIPT

Table 2: A summary of key publications in OS log retrieval for forensic purposes

Method Publication Advantage
XML-based Alink et al. (2006) Provide a complete query
platform
Huang and Wu (2009) Parse raw file from
unallocated space
Lee et al. (2010) XML-based framework for
event log capture
Disadvantage
Do not provide support for binary file
indexing
There are some unidentified data types
Only support Windows environment

PT
Database Takahashi and Xiao (2008a) Review search algorithm for Needs to check for other search algorithms
event logs
Awawdeh et al. (2013) Small memory allocation to Only support Windows XP and Windows
process large logs 7
Live capture Choi et al. (2008) Framework for live capture of Only support Linux Fedora

RI
evidence
Lee et al. (2010) XML-based framework for Only support Windows environment
event log capture

SC
785 based; 2) cryptographic hashes; and 3) hardware- 820 this hash-based approach is that it is very fast be-
based. cause it uses an in-memory data structure with a
low overhead.
7.2.1. Rule-based tamper detection Accorsi (2011) implements this type of hash

U
Métayer et al. (2010) and Mazza et al. (2010) chaining method in a distributed architecture and
propose a framework that defines formal criteria of uses the public key cryptography to ensure only au-
AN
825

790 event log architectures that consider security and
can detect any malicious modifications. The formal
rules for log correctness and consistency are mod-
eled based on the B-method (Abrial, 1996). How-
M
thorized machines can send log entries to the de-
signed server. This method provides a complete
package of log security specifically integrity, tam-
per detection, and retrieval of the log entries. The

ever, the proposed architecture cannot handle in- 830 disadvantage of the cryptographic hash is when the
795 correct log entries that may exist (Mazza et al., attackers can successfully access the machine, the
2010). In other words, Mazza et al. (2010) assumes integrity of event logs will not be guaranteed any-
D

that log entries are always correct when appended more since they will break the log security architec-
to a log file for the first time. The modification ture.
detection is conducted after all log entries saved.
TE

800 Cho (2013) presents a method for investigators to
detect timestamp modification specifically in Win-
dows file systems. The paper provides a detailed
structure of a Windows journal file that saves the
EP
835 7.2.3. Hardware-based tamper detection
Detection of alteration can also be performed
in hardware-based secure storage for event logs
(Boeck et al., 2010; Borhan et al., 2012). For in-

operation sequences on the file system. The pro-
805 posed method then analyzes several types of times-
tamp tampering and creates the rules to detect the
forgery in various files. This work is the first use of
C
stance, AMD processor provides a feature called
840 Secure Virtual Machine (SVM) Trusted Platform
Module (TPM) that can run a special protected
code (Boeck et al., 2010). It also offers Secure

the NTFS journaling system for forensic purposes. Loader Block (SLB) to work with Legitimate Log-
The only limitation is when defining the rules for ging Client (LLC). This hardware technology pro-
AC

810 alteration attempts. If the rules have not defined,
the modification cannot be detected.
7.2.2. Tamper detection using cryptographic hashes
As an integral part of event log security, the cryp-
tographic hash approach can be used to detect the
815 modification of an event log. If a single log en-
try is removed or tampered with, it will break the
hash value chain because each log entry contributes
to the generation of the hash code (Schneier and
Kelsey, 1999; Etoh et al., 2010). The advantage of
845 tects the integrity of the OS logs and prevents tam-
pering operations.
As outlined in Section 5.1, software-based tech-
nique generates overhead as each log entry is sent
to the server. In log tamper detection using soft-
850 ware, further research is needed for evaluating the
sending mechanism. There is a possibility to send
some log entries in a period, not one-by-one, to the
server to make sending many log entries faster. On
the other hand, hardware-based approach gives an-
855 other insight into event log security compared with
12
 ACCEPTED MANUSCRIPT
those that only use software. However, the use of
hardware-supported techniques cannot prevent im-
personation attacks as discussed in (Boeck et al.,
2010; Borhan et al., 2012).
860 7.3. Event correlation and reconstruction
Correlation and reconstruction of the events are
closely related topics and accordingly are consid-
ered together here. Investigators need to correlate
two events, possibly from separate log files, in order
865 to reconstruct an attack. The classification of these
topics are: 1) rule-based; 2) database; 3) semantic
model; 4) tree or graph-based; 5) timestamp-based;
6) finite state machines; 7) virtual machines; and
8) live event reconstruction. Each category is de-
870 scribed below and a summary is given in Table 3.
7.3.1. Rule-based correlation and reconstruction
U
Simple Event Correlator (SEC) provides
lightweight and platform-independent event cor-
relation (Vaarandi, 2002a). It implements a
AN
875 rule-based method to identify possible attacks on
the system from event logs. However, SEC needs
many different rules for other application logs, so
the rules should be manually defined. Following
M
SEC, Abbott et al. (2006) provide automatic
880 recognition of events by creating logical matching
pattern saved in an XML file. Similar to SEC, this
D
technique depends on the pattern specifications
saved in the database.
To deal with massive log files, Herrerı́as and
TE
885 Gomez (2007) create a formal model for the events.
The method uses this formal model in the cor-
relation engine based on pre-condition and post-
condition rules. Although the paper offers auto-
EP
mated solutions, no experiment is presented in the
890 paper. Furthermore, to reconstruct the events au-
tomatically, Herrerı́as and Gómez (2010) use prede-
fined attack base rules and correlate events based
C
on the log properties. The author suggests further
improvement for adding more rules in the knowl-
AC
895 edge base.
In addition to the previous approaches, the in-
vestigator can run the correlation and filtering of
various logs including OS logs (Forte, 2004). In
this case, the filtering is related to extract and ar-
900 range event logs based on particular fields such as
by protocol or IP address. We can then use a top-
down or bottom-up approach to correlate events.
To deal with large log files, this method requires
some adaptation to distributed system platforms
905 (Forte, 2004).
7.3.2. Event correlation based on database
Since there can be multiple sources of logs in-
cluding OS logs, an investigator should unify those
sources, build a database, and run queries to corre-
910 late the events (Chen et al., 2003). Although it
gives flexibility through the query interface, this
method requires that the forensic investigator man-
PT
ually configure attack patterns in the database.
By using a database, the investigator can run an
915 event correlation automatically (Marrington et al.,
2007). One should normalize the Windows event
RI
logs before inserting them into the database. Next,
the investigator is able to discover and correlate
the event using an SQL query. Despite its auto-
SC
920 matic behavior, this approach does not consider the
timezone in timestamp analysis (Marrington et al.,
2007). In some cases, this will lead to inaccurate
event correlation. To address this issue, the forensic
investigators have to take a note both of the time of
925 examination and the timezone of investigated ma-
chine (Boyd and Forster, 2004). These timestamps
need to be synchronized later in the report.
7.3.3. Event correlation using semantic model
Schatz et al. (2004) found that the investigator
930 can detect a sequence of the events automatically
and semantically. This method constructs a se-
mantic domain model for OS logs based on web
ontology language. The authors combine the rule-
based method with semantic representation that
can provide contextual events for correlation. Am-
935
ato et al. (2017) support this idea by extracting the
digital evidence and representing it as a semantic
data model. The analysis utilizes reasoning and
the queries based on a semantic methodology. The
940 semantic approach enables more expressive way of
representing events in terms of subject-predicate-
object form and search-ability (Amato et al., 2017).
Although these approaches offer automated corre-
lation, they do not include standardized ontology
945 components (Schatz et al., 2004). Moreover, Am-
ato et al. (2017) do not provide any experimental
results.
7.3.4. Tree or graph-based event correlation and re-
construction
950 Wang and Daniels (2005) introduce a reasoning
framework to analyze event logs. They build a
graph-based structure for event logs, and employ
local and global reasoning to correlate events. They
also provide a method to reconstruct the attack sce-
955 nario from various event logs, including OS logs, to
13
 ACCEPTED MANUSCRIPT

Table 3: A summary of key publications in OS log correlation and reconstruction

Method Publication Advantage Disadvantage
Rule-based Vaarandi (2002a) Lightweight and Need many rules for other applications
platform-independent
Abbott et al. (2006) Automatic recognition of events Depend on the pattern specification in
database
Database Chen et al. (2003) Provide flexibility with query Manual recognition of pattern of attack
Marrington et al. (2007) Automatic correlation Not consider time zone in timestamp

PT
analysis
Semantic model Schatz et al. (2004) Automatic detection of event Not include standarized ontology
sequence components
Amato et al. (2017) The latest ontology approach Not supply experimental results
Tree or graph Wang and Daniels (2006) Automatic reasoning Focus on network packet and event logs

RI
as secondary
Arasteh et al. (2007) Tree-based formal model Need more efficient model checking for
proofing part
Timestamp- Gómez et al. (2005) Handle multiple logs from It is assumed that device and logs are

SC
based different devices not modified
Schatz et al. (2006) Manage various logs Assumed that time always synchronized
Finite state Gladyshev and Patel Automatic reconstruction Extend to general purposes event
machines (2004) reconstruction
Virtual machines Årnes et al. (2006) Snapshot of VM can be saved for In some aspects, it is slower than actual

U
later analysis machine
Live Olajide et al. (2009) Find the root cause of an Can be extended to other operating
reconstruction incident in real time systems
AN
support network forensic investigation (Wang and 7.3.5. Timestamp-based event reconstruction
Daniels, 2006). The model of event logs is graph
985 A timestamp can be extracted from a log entry
spectral and then the procedure extracts the attack
M

and can be an important factor for reconstruct-

scenario based on the suspicious graph structure. In
ing events. In the case of consolidating event logs
960 addition, it also supports large scale event logs. A
from various sources, Gómez et al. (2005) use Lam-
deeper investigation for event logs is need in Wang
port’s logical clock to model events from times-
D

and Daniels’ method since the OS logs act as a sec-

990 tamps found in different log sources. Meanwhile,
ondary object and the work primarily focuses on
Schatz et al. (2006) run an empirical study to ob-
the network traffic.
TE

serve temporal behavior from Windows-based do-

main controller logs and other sources like user’s
browser logs. Both methods (Gómez et al., 2005;
965 To accommodate multiple sources of log files in-
995 Schatz et al., 2006) are able to handle multiple logs
cluding OS logs, Arasteh et al. (2007) propose a
EP

from different devices. However, it is assumed that

tree-based data structure and analyze correlation
device and logs are not modified (Gómez et al.,
using algebraic terms. Another formal and uni-
2005) and the time is always synchronized (Schatz
fied verification model for event logs is presented
et al., 2006).
by Saleh et al. (2007). The event logs are mod-
C

970

eled based on logic for electronic commerce proto- 1000 Moreover, timestamps as a computer history
col called ADM logic and use a tree data struc- model can be extracted from various event logs to
AC

ture to query the properties. The authors chose
ADM logic because it accommodates a temporal,
975 dynamic, modal and linear logic. Saleh et al. (2007)
also model event logs as a tree, analyze using alge-
braic logic, and demonstrate the model implemen-
tation to Windows event logs. A tableau-based sys-
tem (Cleaveland, 1990) is used for event log verifica-
980 tion. However, both methods (Arasteh et al., 2007;
Saleh et al., 2007) require efficient checking for the
proofing part of the mathematical model to speed
up the analysis.
assist forensic analysis in event reconstruction (Car-
rier and Spafford, 2004). An unique approach was
presented by Koen and Olivier (2008). When log
1005 files are deleted by the attacker, the forensic investi-
gator can extract file timestamps to create relation-
ship between events. Despite being automatic, this
technique assumes that the file timestamp is not
modified by the application. This assumption be-
1010 comes a limitation because the attacker may modify
the timestamp in a particular file. Zhu et al. (2009)
show that the reconstruction of an event can be per-
14
 ACCEPTED MANUSCRIPT
formed based on the state of the operating system
as the state is saved based on a particular times-
1015 tamp. For instance, in Windows system state, a
timeline is built by extracting and comparing the
sequence of events in the saved system state. The
disadvantage of this approach is it demands mini-
mum one snapshot to be compared. In addition, an
1020 event may be removed between two snapshots so it
cannot be investigated after an incident occurred.
A high-level event is one that human can under-
stand such as “Connection from a USB stick”. In
event log files, such high-level events can be made
1025 up of many low-level events recorded as log en-
tries. To obtain a high-level event reconstruction,
the investigator can run an automatic analysis as
described in Hargreaves and Patterson (2012). The
proposed procedure extracts low-level timeline from
1030 various sources including Windows event logs and
U
then reconstructs high-level events using matching
rules. Despite its automated fashion, this proce-
dure only provides a limited timestamp extractor
AN
for some types of event logs such as browser logs
1035 and Windows XP logs. Additionally, there is a
time overhead when processing each log entry. To
reconstruct an event in a falsified logs, Tang and
M
Fidge (2010) propose two algorithms, specifically
A* search and a heuristic method. These algo-
1040 rithms quantify the steps to convert the predicted
D
attacker’s events to actual events found in the fal-
sified OS logs. They report a drawback that the
algorithm does not support large log files.
TE
To cope with cross-drive devices, Patterson and
1045 Hargreaves (2012) propose an automatic method
for timeline reconstruction. Similar to Hargreaves
and Patterson (2012), this method reconstructs
EP
high-level events from various devices. However, it
requires a more complex example in the experiment
1050 to prove the robustness of the method (Patterson
and Hargreaves, 2012).
C
As timestamp-based methods are a dense re-
search area, we provide a typical model for this
AC
approach in Fig. 5. First, the timestamps and
1055 event messages are extracted from various event
logs. Second, these logs can be correlated based on
a specific method or from attack rules repository.
The event logs can be from various sources espe-
cially OS logs (Gómez et al., 2005; Schatz et al.,
1060 2006), multiple computers (Hargreaves and Pat-
terson, 2012; Patterson and Hargreaves, 2012), or
Windows states (Zhu et al., 2009). The investigator
can check a reconstructed event and analyzes some
events of interest found via the timeline viewer.
1065 7.3.6. Event reconstruction based on finite state
machines
A proposal for reconstructing the event automat-
ically is presented in Gladyshev and Patel (2004).
The authors use a finite state machine (FSM) to
1070 model and reconstruct events by backtracking tran-
sitions in the FSM. FSM-based reconstruction is an
PT
automatic procedure and it can be extended to as-
sist general purpose event reconstruction.
This work is improved in James et al. (2009).
The authors improve the FSM for event reconstruc-
RI
1075
tion by converting the FSM model into a determin-
istic finite automaton (DFA). This improvement
solves the limitation of FSM’s high load computa-
SC
tion when backtracking each state of incident sce-
1080 nario. Although this method only supports simple
cases such as an investigation of network printer
logs as described in Gladyshev and Patel (2004),
this protocol can be implemented in OS logs.
7.3.7. Event reconstruction using virtual machines
1085 Another approach is to use the benefits of the vir-
tual machines for event reconstruction. For exam-
ple, we can use a virtual testbed to reconstruct an
attack where the main source is Linux logs (Årnes
et al., 2006). First, we need to build a virtualization
1090 architecture, replay the attack, clone the images,
and analyze the reconstruction. An improvement
of this method is presented in Årnes et al. (2007)
that involves building a testbed in a virtual ma-
chine and then reconstructing the event based on
1095 the replayed attacks.
The benefit of virtual machines is that the snap-
shot of the virtual machine can be saved for later
analysis (Årnes et al., 2006). While in Årnes et al.
(2007), the virtual testbed supplements the event
1100 reconstruction hypothesis to simulate different at-
tacks. However, the virtual machine approach is
slower than an actual machine in some aspects such
as booting, rebooting, and the creation of a foren-
sic image. Additionally, some attacks cannot be
1105 replayed in the virtual environment as they have a
feature to detect a virtual machine.
7.3.8. Live event reconstruction
Almost all of the aforementioned methods deal
with event logs after the attacks had occurred. This
1110 approach is called as post-mortem analysis. To re-
construct an event in real time, Olajide et al. (2009)
introduce automatic live event reconstruction to
support forensic investigation. This method corre-
lates a live memory analysis with various machine
15
 ACCEPTED MANUSCRIPT

Extract timestamps and events

OS logs
Correlate and reconstruct  Attack rules 
timestamp from: repository

PT
Various  Multiple  Several Windows 

RI
applications computers states

Event of interest or 
Timeline viewer

SC
malicious events

Figure 5: A typical model for event correlation and reconstruction with timestamp-based approach

U
1115 logs such as shell history and Linux logs. Further- In a Windows environment, the investigator can
more, live reconstruction enables us to find the root generate user profiles using time windows extracted
AN
cause of an incident in real time. Although the ex- from event logs and create event abstraction (Cor-
periment was only performed in Linux environment, ney et al., 2011). Event abstraction involves group-
it could be extended to other popular platform such 1150 ing of similar log entries. The anomaly is defined
1120 as Windows. as a different event from the trained normal user
M

profile. The possible improvement of this approach

7.4. Anomaly detection is to get better grouping of application processes
Anomaly detection is about identifying irregu- to increase the accuracy of user profiling (Corney
et al., 2011).
D

larities or suspicious activity in event logs. These 1155

irregularities can be further examined by the in- Schindler (2017) uses two Support Vector Ma-
vestigator. We group this topic into several cate- chines (SVM) to detect an anomaly in log data.
TE

1125

gories: 1) user profiling and machine learning; 2)
timestamp-based; and 3) event log clustering. A
summary of key publications in anomaly detection
for event log forensic investigation is given in Table
EP
The first SVM is for separating multiple prede-
fined classes from each other. The feature vec-
1160 tors are based on Windows logon, logoff, and fire-
wall logs. The second SVM is one-class SVM

1130 4. to classify previously discovered classes to nor-

mal and anomaly events. Moreover, Hu et al.
7.4.1. Anomaly detection based on user profiling (2017) proposed anomaly detection based on user
and machine learning 1165 activity. The method first tackles various formats
C

A normal activity profile of a particular user can from multi-source logs including OS logs by cre-
be used to discover anomaly. In 2002, Abraham and ating a metadata extraction to normalize the log
AC

1135 De Vel (2002) identify irregularities based on user
profiling from OS logs. The proposed method ap-
plies association rules to detect an unusual event.
The deviations will be found if the profiling from
raw event logs is different to the standard pro-
1140 file generated previously. In another study, Abra-
ham et al. (2002) use attribute-oriented induction
to generate a profile and then separate the outliers.
The attribute-oriented induction represents event
logs as a hierarchy and generalizes the attributes in
1145 a database by processing the hierarchy tree.
entries. After that, the method constructs user-
specific models to issue alerts for users whose event
1170 patterns are not similar to their patterns in the
training phase.
To give an illustration of profiling for anomaly
detection, we provide a typical model for this ap-
proach in Fig. 6. Event logs are first preprocessed
1175 to make them ready for analysis. A rule mining,
such as association rules (Abraham and De Vel,
2002) or attribute-oriented induction (Abraham
et al., 2002), is used to get base user profile. Addi-
16
 ACCEPTED MANUSCRIPT

Table 4: A summary of key publications in anomaly detection in event log

Method Publication Advantage Disadvantage
User profiling & Abraham and De Vel Build foundation for log Cannot handle multiple event logs
machine learning (2002) anomaly detection for forensic
purposes
Corney et al. (2011) Framework for anomaly Performance can be increased by
detection improving the grouping steps
Timestamp-based Marrington et al. (2009, Automatic analysis Detect inconsistencies without

PT
2011) providing correction
Thorpe and Ray (2012) Automatic detection Cannot detect user sessions in virtual
machine
Event log Vaarandi (2003) A lightweight clustering Many parameters required to be set
clustering method

RI
Studiawan et al. (2017) Automatic clustering and Do not consider semantic relationship
anomaly detection between events

tionally, a machine learning approach such as SVM
1180 (Schindler, 2017) is run in this step to generate
base profiles for existing users. The investigation
includes filtering profile, intra-profile analysis, and
SC
Each word in a log entry corresponds to an item-
1215 set and SLCT then analyzes using frequent itemset
mining to find the most common appearing words.
SLCT also employs density-based clustering, which

U
comparison of input log entries to the discovered identifies a dense region or group of similar words.
base profiles. This process will generate a report Afterward, SLCT defines an anomaly as the log en-
containing anomalies within the specific time pe- tries that do not fit well to any of the clusters found.
AN
1185 1220

riod. Although SLCT is not directly designed for foren-

sic purposes, it builds a foundation of cluster-based
7.4.2. Timestamp-based anomaly detection anomaly detection for event logs.
M

In terms of temporal inconsistency, Marrington
et al. (2009, 2011) examine this property in event
1190 logs. The timestamp in the Windows logs is mod-
eled using the Lamport relation and the method de-
D
Yen et al. (2013) created Beehive, a method that
1225 mining and extracting knowledge from log files au-
tomatically. Additionally, event logs considered are
Windows logs and security application logs. Bee-

tects both out of sequence and missing events. One hive extracts 15 features from log files and catego-
significance advantage of this technique is it can run rizes them into four groups, specifically destination-
TE

in an automated fashion. However, it detects in-
1195 consistencies without providing a recommendation
of correction in the event logs. This work can be ex-
panded to a non-Windows environment as it shows
EP
1230 based, host-based, policy-based, and traffic-based.
Beehive detects suspicious host behaviors using a
custom K-means clustering without specifying the
number of clusters in advance. An anomaly is de-

robustness in anomaly detection. fined as a cluster that deviates significantly from

The timestamp-based approach is also deployed 1235 others.
1200 in a virtual machine environment (Thorpe and Ray, A recent technique detects anomalies automati-
2012). With a similar Lamport model, the times- cally to support forensic investigations by cluster-
C

tamp is used to detect out of sequence or missing ing (Studiawan et al., 2017). The event log is rep-
events in virtual OS logs. Unfortunately, this ap- resented as a graph. The method then builds a
AC

proach cannot detect user sessions in a virtual ma-
1205 chine. As discussed above, we can also generalize
to different Windows platforms and analyze event
logs from various installed applications.
7.4.3. Anomaly detection based on event log clus-
tering
1210 Clustering-based event log analysis is a com-
monly used approach. Vaarandi (2003) initiated
this research domain by proposing a technique,
namely the Simple Log Clustering Tool (SLCT).
1240 parameter-free graph-based cluster and develops a
statistical score to detect anomalies automatically.
In spite of offering an automatic approach, this
method does not include the semantic relationship
between event messages as it only considers word
1245 frequency. Furthermore, the proposed method will
run slowly for large event logs.
To get the real-time analysis, there is an approach
that uses incremental clustering to detect anomalies
in Linux Ubuntu logs and bug-tracking application
1250 logs (Wurzenberger et al., 2017). This method of-
17
 ACCEPTED MANUSCRIPT
OS logs Event log  Rule mining or
pre­processing machine learning
New log entries
Figure 6: A typical model for OS log anomaly with profiling and machine learning approach
fers fast computation as it separates the training
and testing processes. Therefore, there is no need
to recalculate the cluster every time a new log entry
is created. One limitation found in this real-time
1255 approach is the transformation of log entry to the
Euclidean space can be sped up by changing the
U
granularity from character-based to word-based.
AN
7.5. Event log abstraction
M
Abstraction or signature of event logs is a tem-
1260 plate containing words and wild-cards represent-
ing all members in a group of event log entries
(Makanju et al., 2012). The event log signature
D
provides an abstraction so that the forensic investi-
gator will get a general insight or summary of event
TE
1265 logs.
The early works in event log abstraction are not
explicitly intended to be applied to forensic inves-
tigation (Vaarandi, 2003). However, there are two
EP
papers discussing deep learning for event log ab-
1270 straction in forensic analysis. Thaler et al. (2017a)
find a log signature using a supervised method with
the combination of a feed forward neural network
C
and long short-term memory (LSTM). Afterward,
a subsequent work by the same authors upgrade
AC
1275 the method to unsupervised approach using autoen-
coders with LSTM cells.
There is a disadvantage in Thaler et al. (2017a)
where the experiments were conducted on relatively
small datasets and it needs large event logs to
1280 demonstrate the generalization of the method. An-
other limitation found in Thaler et al. (2017b) is
that the proposed method requires a longer run-
time to train the model compared to the non-deep
learning methods (Vaarandi, 2003; Makanju et al.,
1285 2012).
Base profiles
­ Filtering
Anomaly 
­ Intra­profile analysis
report
PT
­ Compare input logs
  with base profiles
RI
8. Visualization of OS logs
SC
The next step after the main investigation is to
present the results of an analysis. The most com-
mon technique to support a presentation of event
1290 logs is using visualization. There are four common
types of visualization for event logs data: 1) foren-
sic timeline; 2) tree-based; and 3) graph-based.
8.1. Forensic timeline
Since almost all OS logs contain a timestamp,
1295 we can create a visualization based on this infor-
mation to view a timeline of the event. The pro-
posed tool by Olsson and Boldt (2009), named Cy-
berForensics TimeLab, provides a scanner to read
the source data to produce a chronological order of
1300 events. The main advantage of a timeline visualiza-
tion is that it uses an automatic approach, so there
is no user intervention. A caching mechanism is
used to handle large data to increase performance.
The only limitation is that the pattern search is yet
1305 to be automated.
Son and Lee (2011) recommended building foren-
sic timeline based on user behavior. The user ac-
tivities and respective time information are gener-
ated from various sources such as Windows event
logs, SetupAPI.log (containing USB device logs),
1310
browser logs, and instant messenger logs. The vi-
sualization displays input event logs based on the
timeline. The proposed tool uses an intuitive dis-
play as date and time are shown in x-axis and y-
1315 axis, respectively. In addition, each extracted event
has unique color for identification. Using this ap-
proach, it is possible to examine user behavior and
when it has occurred.
8.2. Tree-based log visualization
1320 Buchholz and Falk (2005) create a timeline edi-
tor from various event logs. Unlike the timeline ap-
proach (Olsson and Boldt, 2009), the visualization
18
 ACCEPTED MANUSCRIPT
is based on the tree data structure to accommo-
date events. The tool was called Zeitline (Falk and
1325 Buchholz, 2006) and it imports the events from var-
ious sources. Zeitline uses tree-based visualization
and query interface to assist the investigator. The
main benefit of Zeitline is it can provide both gen-
eral and detail event visualization based on a tree
1330 hierarchy. However, Zeitline will run slowly when
displaying large event logs due to GUI rendering in
Java (Buchholz and Falk, 2005). Moreover, it needs
manual intervention from the forensic investigator
when adding a particular timestamp.
1335 On the other hand, there is a framework for time-
line visualization (Inglot et al., 2012; Inglot and Liu,
2014). The author reviews the Zeitline tool (Buch-
holz and Falk, 2005), identifies its disadvantages,
and implements an improvement in user interface
1340 aspects for timeline visualization.
U
AN
8.3. Graph-based log visualization
Another approach to simplify event log analysis is
visualization via graphs (Schmerl et al., 2010). This
M
method builds virtual audit data spaces and con-
1345 structs interactive 3D visualization based on quan-
titative analysis of event interrelations. In our re-
D
cent work, we also employ graph-based visualiza-
tion to present anomaly detection in event logs
(Studiawan et al., 2017). Graph-based visualization
TE
1350 offers a big picture of anomaly and non-anomaly
event logs per cluster. However, it is unable to pro-
vide visualization in chronological order.
A new and most updated tool to visualize
EP
event log timeline is Timesketch (Google, 2018).
1355 It is a forensic tool from Google that provides
timeline analysis for forensic purposes. The main
advantage of Timesketch is that it enables forensic
C
investigators to collaborate to manage and examine
event logs at the same time. The supported inputs
AC
1360 are logs that formatted in CSV (Comma-separated
Value) or JSON (JavaScript Object Notation)
and logs that extracted from the log2timeline tool
(Metz, 2018c). Timesketch supports both tabular
and graph-based view for analyzed logs. The
1365 example of graph view of Timesketch is shown
in Fig. 7. In this visualization, an investigator
examines all users who logged in to a Windows
operating system when an application named
GROOVE.EXE was started.
1370
9. Post-process of OS log investigation
This phase deals with the evaluation of investi-
gation process. The forensic investigator needs to
review the model or framework used in the exami-
1375 nation. There are many frameworks available and
the investigator can choose which one is appropri-
PT
ate for our particular case. Do et al. (2014) propose
a framework for Windows event log forensics. This
framework contains the main step of forensic inves-
1380 tigation specifically for Windows event logs such as
RI
identification, preservation, analysis, and presenta-
tion. However, the framework needs to be extended
to newest versions of Windows. Futhermore, there
are frameworks for live analysis on Linux environ-
SC
1385 ment as presented in Choi et al. (2008); Lee et al.
(2010). These models offer a more complete ap-
proach from acquiring the evidence to the investi-
gation reports.
Like other digital evidence processes, the pro-
1390 cedure for examining event logs follows certain
phases. In other words, a general digital forensic
investigation framework can be applied to different
types of investigation including event logs (Carrier
and Spafford, 2006). For instance, one can build a
1395 computer history model from various event logs to
categorize forensic analysis in event reconstruction
(Carrier and Spafford, 2006).
Moreover, another approach is to perform a
multi-tier investigation framework (Beebe and
1400 Clark, 2005). This hierarchical structure enables
the investigator to see both the global and detailed
processes for each phase of the examination. To get
the global structure, the framework can collapse the
lower hierarchies and expand it again to get more
1405 details. Finally, a recent review of various inves-
tigation frameworks is presented in Agarwal and
Kothari (2015).
10. Tools for OS log forensics
There are many tools and software for event log
1410 forensics. We classify these tools into two cate-
gories: 1) general tools and 2) libraries. We de-
scribe each category as below.
10.1. General tools
As event logs are the part of the bigger digital evi-
1415 dence, specifically hard disk, the acquisition process
can be conducted by general forensic tools. The
three most popular general-purpose tools are En-
Case Forensic (OpenText, 2018), Forensic Toolkit
19
 ACCEPTED MANUSCRIPT
U SC
AN
Figure 7: Graph-based visualization of Windows event logs from Timesketch (Berggren, 2017)
M
(FTK) (AccessData, 2018), and Autopsy (Basis
1420 Technology, 2018). The first two tools are commer-
cially supported, while the later is an open source
project supported by the digital forensic commu-
D
nity.
In case of event log forensics, some researchers
TE
1425 share their code with the community. For example,
SEC (Simple Event Correlator), as discussed in Sec-
tion 7.3, is a tool for advanced event processing for
event log monitoring, event log forensics, or any
EP
other task involving event correlation (Vaarandi,
1430 2002a,b). Another tool by Gladyshev and Patel
(2004), EARL (Event Analysis and Reconstruction
in Lisp) performs event reconstruction based on
C
the finite state machine. In addition, Timesketch
(Google, 2018) and CyberForensics TimeLab (Ols-
AC
1435 son, 2012) have been discussed in Section 8.1. A
complete list of event log forensics tool reviewed
here is shown in Table 5.
PyFlag is a tool to simplify the process of log file
analysis and forensic investigations (Collett and Co-
1440 hen, 2008). PowerForensics is a tool that provides a
framework for hard drive forensic analysis including
stored event logs (Barakat and Hadi, 2016; Atkin-
son, 2018). PSRecon acquires data including event
logs from a remote Windows machine and delivers
1445 this data to the server (Foss, 2017). Kansa also uses
PT
RI
Windows Powershell to execute the agent modules
from multiple machines in a network (Hull, 2018).
This tool collects data including event logs for fur-
ther analysis. For event logs residing in a foren-
1450 sic memory images, Volatility tool (The Volatility
Foundation, 2018) provides a plugin called evtlogs
(Levy, 2013). This plugin is developed by Jamie
Levy and it can extract event logs from Windows
XP and 2003 memory images.
1455 EVTXtract (Ballenthin, 2018a) and FixEvt
(Murphey, 2007a) can recover and repair broken
Windows event logs due to unexpected crash. Also,
Microsoft has an official event log parser called Log-
Parser (Microsoft, 2005). This tool enables the
1460 forensic investigator to access the event logs and
the registry. Other advantages of LogParser are
that it can correlate events and supports network
traffic analysis. LogParser is also able to examine
Snort IDS logs and to provide SQL query feature
1465 for retrieving particular events.
A set of tools are provided by TZWorks for Win-
dows event log analysis namely evtx view (TZ-
Works, 2018c), evtwalk (TZWorks, 2018b), and
elmo (TZWorks, 2018a). evtwalk is a log parser
1470 for all versions of Windows. evtx view provides a
report for a particular category of events such as
credential changes or USB connections. The parser
20
 ACCEPTED MANUSCRIPT

Table 5: List of event log forensics tools

Tool Event log Language License
SEC (Vaarandi, 2002a,b) Custom Perl GNU GPL
EARL (Gladyshev and Patel, 2004; Custom Lisp GNU GPL
Gladyshev, 2006)
Timesketch (Google, 2018) Custom Python and TypeScript Apache
CyberForensics TimeLab (Olsson, 2012) Custom C# and Perl n/a
PyFlag (Collett and Cohen, 2008) Custom Python GNU GPL

PT
PowerForensics (Barakat and Hadi, Windows log C# and PowerShell MIT
2016; Atkinson, 2018)
PSRecon (Foss, 2017) Windows log PowerShell Apache
Kansa (Hull, 2018) Windows log PowerShell Apache

RI
Volatility evtlogs (Levy, 2013) Windows log Python GNU GPL
EVTXtract (Ballenthin, 2018a) Windows log Python Apache
FixEvt (Murphey, 2007a,b) Windows log n/a n/a
LogParser (Microsoft, 2005) Windows log n/a n/a

SC
evtx view (TZWorks, 2018c) Windows log C++ Proprietary
evtwalk (TZWorks, 2018b) Windows log C++ Proprietary
elmo (TZWorks, 2018a) Windows log C++ Proprietary
log2timeline (Metz, 2018c) Custom Python Apache
Event2Timeline (Chopitea, 2014) Windows log Python and JavaScript GNU GPL

U
AuditParser (Kazanciyan, 2013) Windows log Python Apache with commercial support
Splunk (Splunk Inc, 2018) Custom Python, Java, C#, Ruby, Apache with commercial support
PHP, and JavaScript
AN
GFI EventsManager (GFI Software, Custom n/a Proprietary and commercial
2018) support
ELM Enterprise Manager (TNT Windows log n/a Proprietary and commercial
Software, 2018) support
Assuria Log Manager (Assuria, 2018) Custom n/a Proprietary and commercial
M

support
Event Log Explorer (FSPro Labs, 2018) Windows log n/a Proprietary and commercial
support
EventLog Analyzer (ManageEngine, Custom n/a Proprietary and commercial
2018) support
D

Elasticsearch, Logstash, and Kibana Custom Ruby, Java, and Apache with commercial support
(ELK) (Elasticsearch, 2018) JavaScript
TE

engine for evtwalk and evtx view is the same. Fur- ular company with commercial support. In Ta-
thermore, elmo converts the raw event logs to be ble 5, only Elasticsearch, Logstash, Kibana stack
1475 inserted into SQLite database. From the database, 1495 (ELK) (Elasticsearch, 2018) provides the source
EP

the forensic investigator can correlate and recon-
struct the events. The input for elmo can be a live
Windows machine or a forensic image.
log2timeline provides a feature to extract times-
C
code, while Splunk (Splunk Inc, 2018) opens some
parts of the code’s tools. Other tools use propri-
etary licenses and do not disclose the code. These
tools include GFI EventsManager (GFI Software,

1480 tamps from event log files from an operating sys- 1500 2018), ELM Enterprise Manager (TNT Software,
tem and visualize them (Metz, 2018c). Similar with 2018), Assuria ALM-SIEM Assuria (2018), Event
AC

log2timeline, Event2Timeline also provides visual-
ization for Windows event logs (Chopitea, 2014).
However, Event2Timeline needs an external parser
1485 like Microsoft LogParser to preprocess the data. A
cyber security company called Mandiant provides
the AuditParser tool (Kazanciyan, 2013). Audit-
Parser is used to convert the XML output file from
other Mandiant tools into tab-delimited text files.
1490 These files contain many different types of evidence
including event logs and other Windows artifacts.
The remaining tools are provided by a partic-
Log Explorer (FSPro Labs, 2018), and EventLog
Analyzer (ManageEngine, 2018). In some cases, the
forensic investigator may choose open source alter-
1505 natives that can provide comparable performance
in event log examination. For example, Logstash
has powerful event filtering and conversion capabil-
ities (Vaarandi and Niziński, 2013).
10.2. Libraries
1510 The libraries for event log forensics are mainly for
Windows logs. Parse-Evtx (Schuster, 2007, 2011)
21
 ACCEPTED MANUSCRIPT
and python-evtx (Ballenthin, 2018b) are Windows
event log parsers written in Perl and Python, re-
spectively. Furthermore, the library for the legacy
1515 version of Windows event log format (.evt) and new
version based on XML structure (.evtx) is provided
by Metz (2018a) and libevtx (Metz, 2018b), respec-
tively.
These libraries have the main advantage that the
1520 investigator can focus on the content of event logs as
the parsing step is performed by the libraries. The
choice of the library can be based on the program-
ming language preferred by the investigator. One
limitation of these libraries is that none of them is
1525 dedicated to event logs on Linux-based operating
systems.
11. Public datasets for OS log forensics
U
The research community needs publicly available
datasets to support experiments and benchmark re-
AN
1530 sults for event log forensics research. There are a
few public datasets namely Digital Corpora, Digital
Forensic Research Workshop (DFRWS) Challenge,
Computer Forensic Reference Data Sets (CFReDS)
M
from NIST, the Honeynet Project, and SecRepo.
1535 Although these datasets contain various types of
digital evidence, such as picture files, documents,
and memory dump, we focus on datasets that
D
have event logs as the main object of investigation.
These datasets are summarized in Table 6.
TE
1540 As shown in Table 6, the datasets have various
ages from 2001 to 2018. The old datasets can be
used for educational purposes as the forensic case
and techniques are evolving year by year. To deal
EP
with new environments such as OS logs on the In-
1545 ternet of Things (IoT), the researchers can use the
new datasets from DFRWS 2018.
C
11.1. Digital Corpora
Garfinkel et al. (2009) argue that digital foren-
AC
sics need a standardized dataset or corpus to
1550 make the research reproducible. The pro-
posed dataset by Garfinkel (2018) is hosted in
http://digitalcorpora.org/. There are some types
of data such as cell phone dumps, disk images, files,
and network traffic dumps. In addition, they also
1555 provide some security incident scenarios so that re-
searchers can investigate this case study using var-
ious methods and tools.
The scenarios most closely related to event log
forensics are M57-Jean case (Garfinkel, 2008) and
1560 M57-Patents case (Garfinkel, 2009a). Although
these cases are fictional, the attacks are real and
need to be investigated properly as if they are real
cases. There are solution manuals for these two
scenarios for accredited lecturers and researchers.
1565 The M57-Jean case (Garfinkel, 2008) is about file
leakage in a startup company M57.Biz. The file
PT
contains the salaries and social security numbers of
all staff. The forensic investigator can examine the
disk image of one of the staff’s laptop and explain
1570 how the file was leaked. While the main focus in
RI
this scenario is a chat log, the investigation can cor-
relate it with Windows event logs.
The next case, M57-Patents (Garfinkel, 2009a),
SC
gives the researcher a challenge to solve three types
1575 of criminal activities involving illegal files, propri-
etary research exfiltration, and company eavesdrop-
ping, with the latter being most relevant to event
log forensics. In this scenario, one of the staff in-
stalled a keylogger on the CEO’s computer. The
investigator needs to find out the identity of that
1580
person and how they performed the eavesdropping.
There is also a disk image data namely nps-2009-
casper-rw (Garfinkel, 2009b). This image is an ext3
file system dump from a bootable USB. The user
1585 of this disk image browses several US Government
websites. This case can provide event logs, espe-
cially browser logs and various logs from a Linux
system, to be analyzed forensically.
11.2. Digital Forensic Research Workshop
1590 (DFRWS) Challenge
One of the most notable conferences in the dig-
ital forensics research area is the Digital Foren-
sic Research Workshop (DFRWS). This conference
not only provides a place to present research re-
1595 sults, but also issues an annual forensic challenge.
The challenges associated with event log foren-
sics are DFRWS Forensic Challenge 2008 (Geiger
et al., 2008), 2009 (Casey and Richard III, 2009),
and 2017-2018 (James, 2018). For each challenge,
1600 DFRWS has publicly posted the submitted solu-
tions.
The main case of the DFRWS Forensic Chal-
lenge 2008 (Geiger et al., 2008) is about investi-
gating unauthorized access to the company propri-
1605 etary information. We need to deal with a kernel
log or browsing log to reconstruct the event time-
line. Meanwhile, in DFRWS Forensic Challenge
2009 (Casey and Richard III, 2009), the researcher
has to examine authentication logs to trace the at-
1610 tacker. Other log files included are login and logout
22
 ACCEPTED MANUSCRIPT

Table 6: OS logs from public forensic case studies and datasets

Source Case study or dataset Event log Year
Digital Corpora M57-Jean (Garfinkel, 2008) Windows logs 2008
M57-Patents (Garfinkel, 2009a) Windows logs 2009
nps-2009-casper-rw (Garfinkel, 2009b) Linux logs 2009
Digital Forensic Research Workshop DFRWS Forensic Challenge 2008 (Geiger Linux logs 2008
(DFRWS) et al., 2008)
DFRWS Forensic Challenge 2009 (Casey and Linux logs 2009

PT
Richard III, 2009)
DFRWS Forensic Challenge 2018 (James, Linux logs 2018
2018)
Computer Forensic Reference Data Sets The Hacking Case (CFReDS, 2007) Windows logs 2007
(CFReDS)

RI
Data Leakage Case (CFReDS, 2015) Windows logs 2015
The Honeynet Project The Forensic Challenge 2001 (Dittrich, 2001) Linux logs 2001
Scan 34 2005 (Marty et al., 2010) Linux and 2005
application logs

SC
Challenge 5 of the Forensic Challenge 2010 Linux logs 2010
(Marty et al., 2010)
SecRepo (Sconzo, 2018) List of datasets Various event logs -

U
logs in wtmp file and command logs in a bash his- The second scenario, the Data Leakage Case
tory file. (CFReDS, 2015), focuses on the leakage of secret
AN
DFRWS challenge 2017-2018 case study is about 1645 proprietary technology from an international com-
a murder of a woman (James, 2018). The mur- pany. This case is more complicated than the first
1615 der was reported by victims husband. They lived one. Most event logs included in this scenario are
in an apartment. The investigator search through Windows event logs. From these log files, the inves-
M

an apartment and collected various IoT digital ev-
idence such as Raspberry Pi connected via HDMI
to TV, Amazon Echo device, and Google OnHub
wifi router. The log files containing useful informa-
D
tigator can gather the login and logout information,
1650 application logs, and reconstruct the event timeline.
11.4. The Honeynet Project

1620

tion for forensic analysis is located in Raspberry Pi

The Honeynet Project is a non-profit organiza-
with OSMC (Open Source Media Center) operating
tion that focuses on internet security especially hon-
TE

system installed. OSMC is a Debian-based operat-

eypot technology and digital forensics. This project
ing system, so the structure of logs are similar with
1655 offers a series of forensic challenges. The challenges
1625 other Linux distributions.
most related to event log forensics are Challenge 5
of the Forensic Challenge 2010 (Marty et al., 2010),
EP

11.3. Computer Forensic Reference Data Sets
(CFReDS) Project
NIST created a dataset to support digital foren-
sic research. There are various scenario that can be
C
Scan 34 2005 (Chuvakin, 2005), and the Foren-
sic Challenge 2001 (Dittrich, 2001). Solution files
1660 for each challenge are available from the Honeynet
Project’s website.

1630 investigated by the research community. However, The Forensic Challenge 2010 (Marty et al., 2010)
there are only two scenarios related to event log concerns a compromised Linux system. The direc-
AC

forensics. These are the Hacking Case (CFReDS,
2007) and Data Leakage Case (CFReDS, 2015).
There are solution manuals for these two case stud-
1635 ies so we can learn from the given scenario. The
Hacking Case (CFReDS, 2007) is about the inves-
tigation of an abandoned notebook that has been
used by a hacker. The investigator needs to check
Windows event logs to determine the last user to
1640 log in into the computer. Furthermore, there are
many chat logs in the forensic image that reveal
information about the attacker’s contacts.
tory /var/log/ has been imaged and the investiga-
1665 tor needs to analyze the brute-force attack recorded
in the event logs, especially authentication logs.
There is also an Apache access log that is related
to the attack. The challenge also requires that the
researchers construct a timeline of the incidents.
1670 Scan 34 2005 (Chuvakin, 2005) provides various
log files from a honeypot system such as Apache
logs, Linux syslogs, Snort NIDS logs, and iptables
firewall logs. There is a compromised system and
needs to be analyzed based on those logs. The
23
 ACCEPTED MANUSCRIPT
1675 investigator has to describe how the attacked was
launched. This case also demands more examina-
tion of time synchronization between log files.
The last challenge from the Honeynet Project is
an old case study from 2001 (Dittrich, 2001), but
1680 is nonetheless relevant to event log forensics. It
involves analyzing RedHat-based Linux syslog and
bash history command logs. The researcher investi-
gates the intrusion in a honeypot server and collects
information to identify the intruder. This challenge
1685 also requires that a timeline should be created to
describe the time of attack events.
11.5. SecRepo
SecRepo is a website that provides a list of the
various security-related datasets including event
1690 logs (Sconzo, 2018). The list is categorized based
on the platform such as network, system, and mal-
U
ware. There are a huge number of event logs namely
authentication logs from OS, honeypot logs, web
AN
server logs, FTP logs, and DNS logs. All data can
1695 be downloaded for free with a specific license such
as Creative Commons or Apache License. Other
datasets such as Digital Corpora and The Honeynet
M
Project are included in SecRepo.
This website does not provide any scenario or
1700 case study. Therefore, SecRepo does not give any
solution materials. Unlike other datasets, SecRepo
D
only provides various event logs and links to other
datasets related to computer security. However,
TE
SecRepo is very useful to get information about
1705 dataset lists and the researcher can go further to
the official website of the datasets.
EP
12. Open issues and future directions
Finally, we present the major open issues for each
phase in the digital forensic investigation of event
C
1710 logs. We also propose the future directions of this
research area.
AC
12.1. Pre-processing step as forensic readiness of
OS logs
The forensic readiness of an event log is criti-
1715 cal to support investigation when an incident oc-
curs. There is currently a trend towards network
architectures supported by virtual machines (Sato
and Yamauchi, 2012). In future, we suggest the
combination of cryptography, centralization, and
1720 hardware-supported architectures can improve the
security of event logs as described in Boeck et al.
(2010). Cryptography provides the security, cen-
tralization offers backup and easy management, and
a hardware-based approach will support software-
1725 based log security.
12.2. Acquisition of OS logs
The acquisition of data from recent hard disk
PT
technologies such as solid state drives (SSD) is
needed since existing techniques only deal with the
1730 traditional magnetic platter disk. SSD controller
RI
hides the disk operations such as compression and
garbage collection to the host OS. Therefore, the
data is inaccessible and only can be accessed via
the memory cells acquisition (Bonetti et al., 2013).
SC
1735 Bonetti et al. (2013) provides a methodology to test
forensic characteristics of SSD. This leads a chal-
lenge for recovery of the data stored in SSD. In ad-
dition, a more accurate method is needed to make
sure that event logs can be properly recovered. Re-
1740 covery tools usually only support a particular plat-
form. Therefore, it would be valuable to have a
generic tool that can run in any environment.
Another issue is about log rotation. Log rotation
is “closing a log file and opening a new log file when
1745 the first file is considered to be complete” (Kent and
Souppaya, 2006). When an incident occurred, the
investigator should first stop the rotation of log file
to avoid the loss of potential evidence. If the log
files are already missing, then a recovery process is
1750 needed.
12.3. Main analysis of OS log investigation
There are automatic methods for event log re-
trieval and event correlation or reconstruction to
assist the forensic investigator. However, there are
1755 a few methods for automatic anomaly detection in
the event logs. Along these lines, the present au-
thors have proposed a method to detect an anomaly
in the authentication logs from OS without any user
parameters (Studiawan et al., 2017).
1760 As event logs can become very large, big data
forensics has also become a major challenge in re-
cent years (Adedayo, 2016). In addition, cloud com-
puting creates a further challenge in certain aspects
such as ensuring the chain of custody, cloud log
1765 event correlation, and real-time cloud log visualiza-
tion (Khan et al., 2016).
The use of machine learning, especially the
emerging field of deep learning techniques, is also
a promising field in event log forensics. There are
1770 only two papers describing the use of deep learning
24
 ACCEPTED MANUSCRIPT

for event log abstraction to support a forensic exam-
ination (Thaler et al., 2017a,b). Moreover, Du et al.
(2017) proposed a deep learning technique to repre-
sent event logs as a natural language sequence and
1775 named the method DeepLog. The neural network
model used is Long Short-Term Memory (LSTM).
DeepLog trains the model from log patterns of nor-
12.5. Post-process of OS log investigation
There are many frameworks for digital forensic
investigation. The investigators can select models
1825 based on the needs of each forensic case. The exist-
ing tools should accommodate the various models
of investigations. At the moment, common foren-
sic tools only support a generic model. These fea-

mal activities, and then identify anomalies when log
patterns different from the model trained in an au-
1780 tomatic fashion. Although DeepLog is not intended
for forensic analysis explicitly, it can operate in real
PT
tures should be extended to accommodate a specific
1830 framework based on the needs of the case, poten-
tially employing a plugin-based architecture.

time so that it can handle new log patterns over
time. Accordingly, this is likely to be a promising
area for future work in event log forensics.
1835
RI
Another issue in post-process is anti-forensics.
As discussed in Garfinkel (2007), Windows log file
will execute regular expressions inserted in log en-
tries. This will make the operating system hang

SC
1785 Furthermore, the event log forensics is also crit-
and cannot record log entries. This attack was
ical to the security of Internet of Things (IoT) de-
demonstrated by Foster and Liu (2005) in Black-
vices (Watson and Dehghantanha, 2016). The ex-
hat Briefings 2005. To overcome this anti-forensics
isting frameworks, methods, and tools for forensics
technique the logging system needs to filter the log

U
are not designed for IoT as this is a new area in
1840 entries, so the logging system only run the legiti-
1790 information technology. Therefore, we may need
mate entries. In addition, one of various event log
existing approaches to be adapted to IoT forensics,
AN
security methods discussed in Section 5.1 should be
especially for event logs. Similar to the acquisition
implemented, with an assumption that the attacker
and preservation phases, most analysis techniques
does not have any physical access to the computer.
only support a particular platform. These methods
1845 The common nature to attack log files is to delete
should be designed to handle various environments
M

1795
them. Brett et al. (2017) have demonstrated how
or can be extended to suit other platforms. Gen-
to delete Windows 7 event logs with minimal suspi-
erally, the existing methods are implemented on a
ciousness to the investigators. The technique first
particular system such as Windows 7. However,
stops logging services and there is a window time for
D

these techniques could be recreated, redesign, and

1850 60 seconds before Windows restarts the services. In
1800 reimplemented for other operating systems.
this window time, the attacker can remove the log
TE

file at a binary level for several minutes. Brett et al.

12.4. Visualization of OS logs
(2017) also reported that this technique generates
Presentation of event logs contains two main as-
some errors to the system logs. Therefore, if the gap
pects, specifically visualization and preparation of
1855 between log entries and some of the error logs are
an investigation report. Event log visualization is
EP

suspicious, then the investigators need to pay more

1805 usually not intended for forensic purposes only, but
attention to this issue. As the anti-forensics tech-
is focused on security incident analysis. The dig-
nique continue developing, we also need to handle
ital forensic community should check the relevant
to create more sophisticated forensic analysis meth-
research areas in order to improve the features and
C

1860 ods.
advance event log visualization techniques. For ex-
1810 ample, the Zeitline tool (Buchholz and Falk, 2005)
AC

is not maintained anymore by its developers, but
this work is one of the most highly cited papers in
the field.
The only visualization tool for event log forensics
1815 that has active developers is log2timeline (Metz,
2018c). Additionally, a new tool named Timesketch
(Google, 2018) which enables collaborative time-
line analysis is actively maintained. Therefore, we
recommend forensic community to keep supporting
1820 these open source visualization tools to make event
log examination easy and accurate.
12.6. Tools for OS log forensics
There are a range of tools for event log parsing of
Windows logs. On the other hand, there are only a
few libraries to parse or analyze event logs on Unix-
1865 based operating systems. However, these libraries
are not integrated into a single library compared
to the Windows log parser (Metz, 2018a,b). The
parsing techniques mostly use regular expression or
Grok rules. One difficulty of event log parsing is
1870 that the configurations of event logs such as syslog
vary from one server to another and the system
25
 ACCEPTED MANUSCRIPT
administrator usually configures logging based on
their needs. However, there is a common pattern of
these Unix-based event logs.
1875 Therefore, a generic library, especially for Unix-
based OS, should be developed. Recent work by
Studiawan et al. (2018) propose nerlogparser tool
providing generic model to parse various semi-
structured log files such as OS logs. nerlogparser
1880 parses log files based on named entity recognition
(NER). NER is a mechanism to recognize named
entities from a text data. In event logs, nerlogparser
defines named entities as words or phrases contain-
ing common fields in a log entry such as timestamp,
1885 host name, or service name. Recognizing named en-
tities is equivalent to determining each field in a log
entry. The nerlogparser uses a deep learning tech-
nique namely bidirectional long short-term memory
networks to perform NER. However, nerlogparser
U
1890 still has a possible drawback when there is an input
from a particular log file that has a completely dif-
ferent log entry structure and nerlogparser cannot
AN
recognize the log format.
12.7. Public datasets for OS log forensics
M
1895 Most public datasets for event log forensics are
at least three years old. The research community
needs an up to date dataset to accommodate the
recent attack models (Grajeda et al., 2017). For
D
instance, there is a tool namely EviPlant to cre-
1900 ate digital forensics case study or images (Scanlon
TE
et al., 2017). Researchers can use this tool and
share the generated data with the digital forensics
community.
EP
13. Conclusion
1905 In this paper, we present a comprehensive sur-
vey of operating system (OS) log forensics research.
C
This work is structured based on each phase of a
digital forensic investigation framework. For each
AC
phase, we describe the existing methods in the lit-
1910 erature and identify both its advantages and dis-
advantages. We also provide a list of tools for con-
ducting OS log examination. Furthermore, publicly
available datasets are described in detail.
There are several open issues in the context of OS
1915 log forensics research. One of the main issues is to
encourage the research community to use standard
datasets so the performance of the proposed meth-
ods can be compared and evaluated against each
other. The use of open source tools also urgently
1920 needed to advance the state of the art techniques
for OS log investigation.
As time goes by, attackers use more sophisticated
and complicated techniques to evade the forensic
methods and tools. Therefore, more sophisticated
1925 methods are needed to identify and analyze such
attacks on computer systems.
PT
Acknowledgments
This work is supported by the Indonesia Lec-
RI
turer Scholarship (BUDI) from Indonesia Endow-
1930 ment Fund for Education (LPDP), Ministry of Fi-
nance, Republic of Indonesia.
SC
References
Abbott, J., Bell, J., Clark, A., De Vel, O., Mohay, G., 2006.
Automated recognition of event scenarios for digital foren-
1935 sics, in: Proceedings of the ACM Symposium on Applied
Computing, pp. 293–300.
Abraham, T., De Vel, O., 2002. Investigative profiling with
computer forensic log data and association rules, in: Pro-
ceedings of the IEEE International Conference on Data
1940 Mining, pp. 11–18.
Abraham, T., Kling, R., De Vel, O., 2002. Investigative
profile analysis with computer forensic log data using at-
tribute generalisation, in: Proceedings of the Australasian
Data Mining Workshop, pp. 17–27.
1945 Abrial, J.R., 1996. The B-book: Assigning programs to
meanings. Cambridge University Press.
AccessData, 2018. Forensic Toolkit (FTK).
https://accessdata.com/products-services/
forensic-toolkit-ftk.
1950 Accorsi, R., 2009a. Log data as digital evidence: What se-
cure logging protocols have to offer?, in: Proceedings of
the 33rd Annual IEEE International Computer Software
and Applications Conference, pp. 398–403.
Accorsi, R., 2009b. Safekeeping digital evidence with se-
1955 cure logging protocols: State of the art and challenges,
in: Proceedings of the 5th International Conference on
IT Security Incident Management and IT Forensics, pp.
94–110.
Accorsi, R., 2011. BBox: A distributed secure log archi-
1960 tecture, in: Proceedings of the 7th European Workshop
on Public Key Infrastructures, Services and Applications,
pp. 109–124.
Adedayo, O.M., 2016. Big data and digital forensics, in:
Proceedings of the IEEE International Conference on Cy-
1965 bercrime and Computer Forensic, pp. 1–7.
Agarwal, R., Kothari, S., 2015. Review of digital forensic in-
vestigation frameworks, in: Information Science and Ap-
plications, pp. 561–571.
Ahmad, A., Ruighaver, A., 2003. Improved event log-
1970 ging for security and forensics: Developing audit man-
agement infrastructure requirements, in: Proceedings of
the ISOneWorld.
Ahmad, A., Ruighaver, A., 2004. Towards identifying cri-
teria for the evidential weight of system event logs, in:
1975 Proceedings of the Australian Computer, Network and
Information Forensics Conference, pp. 40–47.
26
 ACCEPTED MANUSCRIPT
Alink, W., Bhoedjang, R.A.F., Boncz, P.A., de Vries, A.P.,
2006. XIRAF - XML-based indexing and querying for
digital forensics. Digital Investigation 3, Supplem, S50–
1980 S58. 2045 Borhan, N., Mahmod, R., Dehghantanha, A., 2012. A frame-
Almulla, S.A., Iraqi, Y., Jones, A., 2014. A state-of-the-art
review of cloud forensics. Journal of Digital Forensics,
Security and Law 9, 7–28.
Amato, F., Cozzolino, G., Mazzeo, A., Mazzocca, N., 2017.
Correlation of digital evidences in forensic investigation 2050
1985
through semantic technologies, in: Proceedings of the 31st
IEEE International Conference on Advanced Information
Networking and Applications Workshops, pp. 668–673.
Arasteh, A.R., Debbabi, M., Sakha, A., Saleh, M., 2007.
1990 Analyzing multiple logs for forensic evidence. Digital In- 2055
vestigation 4, Supplem, 82–91.
Årnes, A., Haas, P., Vigna, G., Kemmerer, R.A., 2006. Digi-
tal forensic reconstruction and the virtual security testbed
ViSe, in: Proceedings of the Detection of Intrusions and
1995 Malware and Vulnerability Assessment, pp. 144–163. 2060 Carrier, B.D., Spafford, E.H., 2006. Categories of digital
Årnes, A., Haas, P., Vigna, G., Kemmerer, R.A., 2007. Using
a virtual security testbed for digital forensic reconstruc-
tion. Journal in Computer Virology 2, 275–289.
Assuria, 2018. Assuria ALM-SIEM: SIEM, FIM and Enter-
U
2000 prise Log Management. https://assuria.com/products/ 2065
alm-siem/.
Atkinson, J., 2018. PowerForensics - PowerShell
AN
Digital Forensics. https://github.com/Invoke-IR/
PowerForensics.
2005 Awawdeh, S.A., Baggili, I., Marrington, A., Iqbal, F., 2013. 2070
Towards a unified agent-based approach for real time com-
puter forensic evidence collection, in: Proceedings of the
M
8th International Workshop on Systematic Approaches to
Digital Forensics Engineering, pp. 1–8.
2010 Ayrapetov, D., Ganapathi, A., Leung, L., 2002. Improving 2075 Chabot, Y., Bertaux, A., Nicolle, C., Kechadi, T., 2015.
the protection of logging systems. Technical Report. UC
D
Berkeley Computer Science, Berkeley, CA USA.
Ballenthin, W., 2018a. EVTXtract: A tool for recovering
and reconstructing fragments of EVTX log files. https:
TE
2015 //github.com/williballenthin/EVTXtract. 2080
Ballenthin, W., 2018b. python-evtx: Pure Python parser for
recent Windows event log files. URL: https://github.
com/williballenthin/python-evtx.
Barakat, A., Hadi, A., 2016. Windows forensic investiga-
tions using PowerForensics tool, in: Proceedings of the 2085
EP
2020
Cybersecurity and Cyberforensics Conference, pp. 41–47.
Basis Technology, 2018. Autopsy. https://www.autopsy.
com/.
Bayer, U., Habibi, I., Balzarotti, D., Kirda, E., Kruegel,
C., 2009. A view on current malware behaviors, in: Pro- 2090
C
2025
ceedings of the 2nd USENIX Conference on Large-scale
Exploits and Emergent Threats, pp. 1–8.
AC
Beebe, N.L., Clark, J.G., 2005. A hierarchical, objectives-
based framework for the digital investigations process.
2030 Digital Investigation 2, 147–167. 2095
Berggren, J., 2017. Thinking in graphs: Exploring
with Timesketch. https://medium.com/timesketch/
thinking-in-graphs-exploring-with-timesketch-84b79aecd8a6.Information Security and Assurance, pp. 421–426.
Boeck, B., Huemer, D., Tjoa, A.M., 2010. Towards
2035 more trustable log files for digital forensics by means of 2100
”Trusted Computing”, in: Proceedings of the 24th IEEE
International Conference on Advanced Information Net-
working and Applications, pp. 1020–1027.
Bonetti, G., Viglione, M., Frossi, A., Maggi, F., Zanero, S.,
2040 2013. A comprehensive black-box methodology for test- 2105 Corney, M., Mohay, G., Clark, A., 2011. Detection of anoma-
ing the forensic characteristics of solid-state drives, in:
27
Proceedings of the 29th Annual Computer Security Ap-
plications Conference, New Orleans, Louisiana, USA. pp.
269–278.
work of TPM, SVM and boot control for securing forensic
logs. International Journal of Computer Applications 50,
15–19.
Boyd, C., Forster, P., 2004. Time and date issues in forensic
computing—a case study. Digital Investigation 1, 18–23.
PT
Brett, E.S., Raymond, C.K.K., Sameera, M., Helen, A.,
2017. Windows 7 antiforensics: A review and a novel
approach. Journal of Forensic Sciences 62, 1054–1070.
Buchholz, F., Falk, C., 2005. Design and implementation of
RI
Zeitline: A forensic timeline editor, in: Proceedings of the
Digital Forensic Research Conference, pp. 1–7.
Carrier, B.D., Spafford, E.H., 2004. Defining event recon-
struction of digital crime scenes. J. Forensic Sci. 49,
JFS2004127–8.
SC
investigation analysis techniques based on the computer
history model. Digital Investigation 3, Supplem, S121–
S130.
Casey, E., 2011. Digital evidence and computer crime. 3rd.
ed., Academic Press.
Casey, E., Richard III, G.G., 2009. DFRWS Forensic Chal-
lenge 2009. http://old.dfrws.org/2009/challenge/
index.shtml.
CFReDS, 2007. Hacking Case from Computer Forensic Ref-
erence Data Sets (CFReDS). https://www.cfreds.nist.
gov/Hacking_Case.html.
CFReDS, 2015. Data Leakage Case from Computer Forensic
Reference Data Sets (CFReDS). https://www.cfreds.
nist.gov/data_leakage_case/data-leakage-case.html.
Event reconstruction: A state of the art, in: Handbook
of Research on Digital Crime, Cyberspace Security, and
Information Assurance, pp. 231–245.
Chen, K., Clark, A., De Vel, O., Mohay, G., 2003. ECF -
Event correlation for forensics, in: Proceedings of the 1st
Australian Computer Network and Information Forensics
Conference, pp. 1–10.
Cho, G.S., 2013. A computer forensic method for detecting
timestamp forgery in NTFS. Computers & Security 34,
36–46.
Choi, J., Savoldi, A., Gubian, P., Lee, S., Lee, S., 2008.
Live forensic analysis of a compromised linux system us-
ing LECT (Linux Evidence Collection Tool), in: Proceed-
ings of the 2nd International Conference on Information
Security and Assurance, pp. 231–236.
Chopitea, T., 2014. event2timeline: Simple Microsoft Win-
dows sessions event logs visualization. https://github.
com/certsocietegenerale/event2timeline.
Chou, B.H., Tatara, K., Sakuraba, T., Hori, Y., Sakurai,
K., 2008. A secure virtualized logging scheme for digi-
tal forensics in comparison with kernel module approach,
in: Proceedings of the 2nd International Conference on
Chuvakin, A., 2005. Scan 34 2005 from The Honeynet
Project. http://old.honeynet.org/scans/scan34/.
Cleaveland, R., 1990. Tableau-based model checking in the
propositional mu-calculus. Acta Informatica 27, 725–747.
Collett, D., Cohen, M., 2008. PyFlag: Forensic and log anal-
ysis GUI. https://sourceforge.net/projects/pyflag/.
lies from user profiles generated from system logs, in:
 ACCEPTED MANUSCRIPT

Proceedings of the 9th Australasian Information Security Forensic Challenge 2008. http://old.dfrws.org/2008/
Conference, pp. 23–32. challenge/index.shtml.
Craiger, P., 2005. Recovering digital evidence from Linux GFI Software, 2018. GFI EventsManager.
2110 systems, in: Proceedings of the IFIP International Con- 2175 https://www.gfi.com/products-and-solutions/
ference on Digital Forensics, pp. 233–244. network-security-solutions/gfi-eventsmanager.
Crosby, S., Wallach, D., 2009. Efficient data structures Gladyshev, P., 2006. EARL (Event Analysis and Re-
for tamper-evident logging, in: Proceedings of the 19th construction in Lisp). http://formalforensics.org/
USENIX Security Symposium, pp. 1–17. smforensics/earl/index.html.
Dittrich, D., 2001. The Honeynet Project’s Forensic Gladyshev, P., Patel, A., 2004. Finite state machine ap-

2115
Challenge 2001. http://old.honeynet.org/challenge/
index.html.
Do, Q., Martini, B., Looi, J., Wang, Y., Choo, K.k., 2014.
Windows event forensic process, in: Advances in Digital
PT
2180
proach to digital event reconstruction. Digital Investiga-
tion 1, 130 –149.
Gómez, R., Herrerias, J., Mata, E., 2005. Using Lamport’s
logical clocks to consolidate log files from different sources,

2120 Forensics X, pp. 87–100.
Du, M., Li, F., Zheng, G., Srikumar, V., 2017. DeepLog:
Anomaly detection and diagnosis from system logs
through deep learning, in: Proceedings of the 2017 ACM
SIGSAC Conference on Computer and Communications
RI
2185 in: Proceedings of the International Workshop on Inno-
vative Internet Community Systems, pp. 126–133.
Goodrich, M.T., Atallah, M.J., Tamassia, R., 2005. Indexing
information for data forensics, in: Proceedings of the 3rd
International Conference on Applied Cryptography and

2125 Security, ACM, Dallas, Texas, USA. pp. 1285–1298.
Elasticsearch, 2018. Elasticsearch, Logstash, and Kibana
(ELK): The open source elastic stack. https://www.
elastic.co/products.
Elyas, M., Ahmad, A., Maynard, S.B., Lonie, A., 2015. Digi-
SC
2190 Network Security, pp. 206–221.
Google, 2018. Timesketch: Collaborative forensic timeline
analysis. https://github.com/google/timesketch.
Grajeda, C., Breitinger, F., Baggili, I., 2017. Availability
of datasets for digital forensics – And what is missing.

U
2130 tal forensic readiness: Expert perspectives on a theoretical
framework. Computers & Security 52, 70 – 89.
Etoh, F., Takahashi, K., Hori, Y., Sakurai, K., 2010. Study
AN
of log file dispersion management method, in: Proceed-
ings of the 10th IEEE International Symposium on Ap-
2135 plications and the Internet, pp. 371–374.
European Commission, 2010. Standard on logging and mon-
itoring. Technical Report.
M
2195 Digital Investigation 22, S94–S105.
Hargreaves, C., Patterson, J., 2012. An automated timeline
reconstruction approach for digital forensic investigations.
Digital Investigation 9, Supplem, S69–S79.
Herrerı́as, J., Gomez, R., 2007. A log correlation model to
2200 support the evidence search process in a forensic investiga-
tion, in: Proceedings of the 2nd International Workshop
on Systematic Approaches to Digital Forensic Engineer-

Falk, C., Buchholz, F., 2006. Zeitline. https:// ing, pp. 31–39.
sourceforge.net/projects/zeitline/. Herrerı́as, J., Gómez, R., 2010. Log analysis towards an au-
2140 Farina, J., Scanlon, M., Le-Khac, N.A., Kechadi, M.T., 2015. 2205 tomated forensic diagnosis system, in: Proceedings of the
Overview of the forensic investigation of cloud services, 5th International Conference on Availability, Reliability,
D

in: Proceedings of the 10th International Conference on and Security, pp. 659–664.
Availability, Reliability and Security, pp. 556–565. Hu, Q., Tang, B., Lin, D., 2017. Anomalous user activity de-
Forte, D.V., 2004. The ”art” of log correlation: Tools and tection in enterprise multi-source logs, in: Proceedings of
TE

2145 techniques for correlating events and log files. Computer
Fraud and Security 2004, 15–17.
Foss, G., 2017. PSRecon: PowerShell incident response - live
forensic data acquisition. https://github.com/gfoss/
PSRecon.
Foster, J.C., Liu, V., 2005. Catch me, if you can ... Technical
EP
2210 the 2017 IEEE International Conference on Data Mining
Workshops, pp. 797–803.
Huang, X., Wu, S., 2009. Vista event log file parsing based
on XML technology, in: Proceedings of the 4th Interna-
tional Conference on Computer Science and Education,
pp. 1186–1190.

2150 2215
Report. Blackhat Briefings. Hull, D., 2018. Kansa: A Powershell incident response frame-
FSPro Labs, 2018. Event Log Explorer. https:// work. https://github.com/davehull/Kansa.
eventlogxp.com/. Ibrahim, N.M., Al-Nemrat, A., Jahankhani, H., Bashroush,
Garfinkel, S., 2007. Anti-forensics: Techniques, detection R., 2012. Sufficiency of Windows event log as evidence in
and countermeasures, in: Proceedings of the 2nd Interna- digital forensics, in: Proceedings of the Global Security,
C

2155 2220
tional Conference on i-Warfare and Security, pp. 77–84. Safety and Sustainability and e-Democracy, pp. 253–262.
Garfinkel, S., 2008. M57-Jean scenario. http:// Inglot, B., Liu, L., 2014. Enhanced timeline analysis for dig-
AC

digitalcorpora.org/corpora/scenarios/m57-jean. ital forensic investigations. Information Security Journal:

Garfinkel, S., 2009a. M57-Patents scenario. A Global Perspective 23, 32–44.
2160 http://digitalcorpora.org/corpora/scenarios/ 2225 Inglot, B., Liu, L., Antonopoulos, N., 2012. A framework for
m57-patents-scenario. enhanced timeline analysis in digital forensics, in: Pro-
Garfinkel, S., 2009b. nps-2009-casper-rw: An ceedings of the IEEE International Conference on Green
ext3 file system from a bootable USB. http: Computing and Communications, pp. 253–256.
//downloads.digitalcorpora.org/corpora/drives/ James, J., 2018. DFRWS Forensic Challenge 2017-2018.
2165 nps-2009-casper-rw/. 2230 https://jijames.github.io/DFRWS2018Challenge/.
Garfinkel, S., 2018. Digital Corpora: Producing the digital James, J., Gladyshev, P., Abdullah, M.T., Zhu, Y., 2009.
body. http://digitalcorpora.org/. Analysis of evidence using formal event reconstruction,
Garfinkel, S., Farrell, P., Roussev, V., Dinolt, G., 2009. in: Proceedings of the 1st International Conference on
Bringing science to digital forensics with standardized Digital Forensics and Cyber Crime, pp. 85–98.
2170 forensic corpora. Digital Investigation 6, S2–S11. 2235 Kazanciyan, R., 2013. AuditParser. https://github.com/
Geiger, M., Venema, W., Casey, E., 2008. DFRWS mandiant/AuditParser.

28
 ACCEPTED MANUSCRIPT
Kent, K., Souppaya, M., 2006. Guide to computer security
log management. Technical Report. National Institute of
Standards and Technology.
2240 Khan, S., Gani, A., Wahab, A.W.A., Bagiwa, M.A., Shiraz,
M., Khan, S.U., Buyya, R., Zomaya, A.Y., 2016. Cloud
log forensics: Foundations, state of the art, and future
directions. ACM Computing Surveys 49, 7:1–7:42.
Koen, R., Olivier, M., 2008. The use of file timestamps in
digital forensics, in: Proceedings of the Innovative Minds
2245
Conference.
Lazzez, A., Slimani, T., 2015. Forensics investigation of web
application security attacks. J. Computer Network and
Information Security 3, 10–17.
2250 Lee, S., Savoldi, A., Lim, K.S., Park, J.H., Lee, S., 2010.
A proposal for automating investigations in live forensics.
Computer Standards and Interfaces 32, 246–255.
Levy, J., 2013. evtlogs Volatility plugin. https:
//github.com/volatilityfoundation/volatility/
2255 blob/master/volatility/plugins/evtlogs.py.
Liao, Y.C., Langweg, H., 2014. Resource-based event recon-
struction of digital crime scenes, in: Proceedings of the
IEEE Joint Intelligence and Security Informatics Confer-
ence, pp. 129–136.
U
2260 Lin, C., Li, Z., Gao, C., 2009. Automated analysis of multi-
source logs for network forensics, in: Proceedings of the
1st International Workshop on Education Technology and
AN
Computer Science, pp. 660–664.
Lou, Y., Wang, P., Xu, M., Zheng, N., 2009. Automated
2265 event log file recovery based on content characters and
internal structure, in: Proceedings of the 1st International
Conference on Information Science and Engineering, pp.
M
4778–4781.
Makanju, A., Zincir-Heywood, A.N., Milios, E.E., 2012. A
2270 lightweight algorithm for message type extraction in sys-
tem application logs. IEEE Transactions on Knowledge
D
and Data Engineering 24, 1921–1936.
ManageEngine, 2018. EventLog Analyzer. https://www.
manageengine.com/products/eventlog/.
TE
2275 Marrington, A., Baggili, I., Mohay, G., Clark, A., 2011. CAT
detect (Computer Activity Timeline detection): A tool
for detecting inconsistency in computer activity timelines.
Digital Investigation 8, Supplem, S52–S61.
Marrington, A., Mohay, G., Clark, A., Morarji, H., 2007.
Event-based computer profiling for the forensic recon-
EP
2280
struction of computer activity, in: Proceedings of the
AusCERT Asia Pacific Information Technology Security
Conference, pp. 71–87.
Marrington, A., Mohay, G., Clark, A., Morarji, H., 2009.
Dealing with temporal inconsistency in automated com-
C
2285
puter forensic profiling. Technical Report. Queensland
University of Technology.
AC
Marty, R., Chuvakin, A., Tricaud, S., 2010. Challenge
5 of the Honeynet Project Forensic Challenge 2010 -
2290 Log Mysteries. http://honeynet.org/challenges/2010_
5_log_mysteries.
Mazza, E., Potet, M.L., Métayer, D.L., 2010. A formal
framework for specifying and analyzing liabilities using log
as digital evidence, in: Proceedings of the 13th Brazilian
2295 Symposium on Formal Methods, pp. 194–209.
Métayer, D.L., Mazza, E., Potet, M.L., 2010. Designing log
architectures for legal evidence, in: Proceedings of the 8th
IEEE International Conference on Software Engineering
and Formal Methods, pp. 156–165.
2300 Metz, J., 2018a. libevt: A library to access the Windows
event log (evt) format.
Metz, J., 2018b. libevtx: A library to access the Windows
XML event log (evtx) format.
Metz, J., 2018c. log2timeline: Super timeline all the things.
2305 https://github.com/log2timeline/plaso.
Microsoft, 2005. LogParser: A powerful, versatile tool
that provides universal query access to text-based
data. https://www.microsoft.com/en-us/download/
details.aspx?id=24659.
Mishra, A.K., Matta, P., Pilli, E.S., Joshi, R.C., 2012.
PT
2310
Cloud forensics: State-of-the-art and research challenges,
in: Proceedings of the International Symposium on Cloud
and Services Computing, pp. 164–170.
Monteiro, S.D.S., Erbacher, R.F., 2008. An authentication
RI
2315 and validation mechanism for analyzing syslogs forensi-
cally. ACM SIGOPS Operating Systems Review 42, 41.
Murphey, R., 2007a. Automated Windows event log foren-
sics. Digital Investigation 4, Supplem, S92–S100.
Murphey, R., 2007b. FixEvt: A tool for automating the
SC
2320 recovery and analysis of Windows NT5 event logs. http:
//murphey.org/fixevt.html.
Olajide, F., Savage, N., Ndzi, D., Al-Sinani, H., 2009. Foren-
sic live response and event reconstruction methods in
Linux systems, in: Proceedings of the Convergence of
2325 Telecommunications, Networking and Broadcasting, pp.
141–146.
Olsson, J., 2012. CyberForensics TimeLab. https://
github.com/jensolsson/CFTL/.
Olsson, J., Boldt, M., 2009. Computer forensic timeline visu-
2330 alization tool. Digital Investigation 6, Supplem, S78–S87.
One Identity, 2018. syslog-ng. https://www.syslog-ng.com/
products/open-source-log-management/.
OpenText, 2018. EnCase Forensic. https://www.
guidancesoftware.com/encase-forensic.
2335 Patterson, J., Hargreaves, C., 2012. The potential for cross-
drive analysis using automated digital forensic timelines,
in: Proceedings of the 6th International Conference on
Cybercrime Forensics Education and Training.
Pilli, E.S., Joshi, R.C., Niyogi, R., 2010. Network foren-
2340 sic frameworks: Survey and research challenges. Digital
Investigation 7, 14 – 27.
Ra, I., Park, T.K., 2009. A forensic logging system based on
a secure OS. International Journal of Computer Science
and Applications 6, 75–91.
Richard III, G.G., Roussev, V., 2005. Scalpel: A frugal,
2345
high performance file carver, in: Proceedings of the 2005
Digital Forensics Research Workshop, pp. 1–10.
Sahoo, P.K., Chottray, R.K., Pattnaiak, S., 2012. Research
issues on Windows event log. International Journal of
Computer Applications 41, 40–48.
2350
Saleh, M., Arasteh, A.R., Sakha, A., Debbabi, M., 2007.
Forensic analysis of logs: Modeling and verification.
Knowledge-Based Systems 20, 671–682.
Sato, M., Yamauchi, T., 2012. VMM-based log-tampering
2355 and loss detection scheme. Journal of Internet Technology
13, 655–666.
Scanlon, M., Du, X., Lillis, D., 2017. EviPlant: An efficient
digital forensic challenge creation, manipulation and dis-
tribution solution. Digital Investigation 20, S29–S36.
2360 Schatz, B., Mohay, G., Clark, A., 2004. Rich event represen-
tation for computer forensics, in: 5th Asia Pacific Indus-
trial Engineering and Management Systems Conference,
pp. 2.12.1–2.12.16.
Schatz, B., Mohay, G., Clark, A., 2006. A correlation method
2365 for establishing provenance of timestamps in digital evi-
dence. Digital Investigation 3, Supplem, S98–S107.
29
 ACCEPTED MANUSCRIPT
Schindler, T., 2017. Anomaly detection in log data using
graph databases and machine learning to defend advanced
persistent threats, in: Lecturer Notes in Informatics, pp.
2370 2371–2378.
Schmerl, S., Vogel, M., Rietz, R., König, H., 2010. Explo-
rative visualization of log data to support forensic anal-
ysis and signature development, in: Proceedings of the
5th International Workshop on Systematic Approaches to
Digital Forensic Engineering, pp. 109–118.
2375
Schneier, B., Kelsey, J., 1999. Secure audit logs to support
computer forensics. ACM Transactions on Information
and System Security 2, 159–176.
Schuster, A., 2007. Introducing the Microsoft Vista event log
2380 file format. Digital Investigation 4, Supplem, S65–S72.
Schuster, A., 2011. Parse-Evtx: Windows event
log parser library and tools collection for Perl.
URL: http://computer.forensikblog.de/en/2011/11/
evtx-parser-1-1-1.html.
2385 Sconzo, M., 2018. SecRepo.com: Security data samples
repository. http://www.secrepo.com/.
Sommer, P., 1997. Downloads, logs and captures: Evidence
from cyberspace. Journal of Financial Crime 5, 138–151.
Son, N., Lee, S., 2011. Forensic investigation method and
U
2390 tool based on the user behaviour, in: Proceedings of the
9th Australian Digital Forensics Conference, pp. 125–133.
Splunk Inc, 2018. Splunk tool. https://www.splunk.com/.
AN
Studiawan, H., Payne, C., Sohel, F., 2017. Graph clustering
and anomaly detection of access control log for forensic
2395 purposes. Digital Investigation 21, 76–87.
Studiawan, H., Sohel, F., Payne, C., 2018. Automatic log
parser to support forensic analysis, in: Proceedings of the
M
16th Australian Digital Forensics Conference, pp. 1–10.
Takahashi, D., Xiao, Y., 2008a. Complexity analysis of re-
2400 trieving knowledge from auditing log files for computer
and network forensics and accountability, in: Proceed-
D
ings of the IEEE International Conference on Communi-
cations, pp. 1474–1478.
Takahashi, D., Xiao, Y., 2008b. Retrieving knowledge from
TE
2405 auditing log-files for computer and network forensics and
accountability. Security and Communication Networks 1,
147–160.
Talebi, J., Dehghantanha, A., Mahmoud, R., 2015. Introduc-
ing and analysis of the Windows 8 event log for forensic
purposes, in: Computational Forensics. Lecture Notes in
EP
2410
Computer Science. volume 8915, pp. 145–162.
Tan, J., 2001. Forensic readiness. Technical Report. @stake
Inc.
Tang, M., Fidge, C.J., 2010. Reconstruction of falsified com-
puter logs for digital forensics investigations, in: Proceed-
C
2415
ings of the 8th Australasian Information Security Confer-
ence, pp. 12–21.
AC
Thaler, S., Menkonvski, V., Petković, M., 2017a. Towards a
neural language model for signature extraction from foren-
2420 sic logs, in: Proceedings of the 5th International Sympo-
sium on Digital Forensic and Security, pp. 1–6.
Thaler, S., Menkovski, V., Petković, M., 2017b. Towards
unsupervised signature extraction of forensic logs, in:
Proceedings of the 26th Benelux Conference on Machine
2425 Learning, pp. 154–159.
The Volatility Foundation, 2018. Volatility. http://www.
volatilityfoundation.org/.
Thorpe, S., Ray, I., 2012. Detecting temporal inconsistency
in virtual machine activity timelines. Journal of Informa-
2430 tion Assurance and Security 7, 24–31.
TNT Software, 2018. ELM Enterprise Manager. https:
//tntsoftware.com/solutions/event-log-management/.
TZWorks, 2018a. TZWorks event log message tables offline
(elmo). https://www.tzworks.net/prototype_page.php?
2435 proto_id=35.
TZWorks, 2018b. TZWorks event log parser (evt-
walk). https://www.tzworks.net/prototype_page.php?
proto_id=25.
TZWorks, 2018c. TZWorks Windows event log viewer
(evtx view). https://www.tzworks.net/prototype_
PT
2440
page.php?proto_id=4.
Vaarandi, R., 2002a. SEC - A lightweight event correlation
tool, in: IEEE Workshop on IP Operations and Manage-
ment, pp. 111–115.
RI
2445 Vaarandi, R., 2002b. SEC (Simple Event Correlator). http:
//simple-evcorr.github.io/.
Vaarandi, R., 2003. A data clustering algorithm for min-
ing patterns from event logs, in: Proceedings of the 2003
IEEE Workshop on IP Operations and Management, pp.
SC
2450 119–126.
Vaarandi, R., Niziński, P., 2013. Comparative analysis of
open-source log management solutions for security moni-
toring and network forensics, in: Proceedings of the Eu-
ropean Conference on Information Warfare and Security,
2455 pp. 278–287.
Van der Aalst, W.M.P., 2013. Business process manage-
ment: A comprehensive survey. ISRN Software Engineer-
ing 2013.
Wang, W., Daniels, T.E., 2005. Building evidence graphs for
2460 network forensics analysis, in: Proceedings of the Annual
Computer Security Applications Conference, pp. 254–264.
Wang, W., Daniels, T.E., 2006. Diffusion and graph spectral
methods for network forensic analysis, in: Proceedings of
the Workshop on New Security Paradigms, pp. 99–106.
2465 Watson, S., Dehghantanha, A., 2016. Digital forensics: the
missing piece of the Internet of Things promise. Computer
Fraud & Security 2016, 5–8.
Wettstein, G., Tweedie, S., Virtanen, J., Alderton, S.,
Schulze, M., 2018. syslogd. https://linux.die.net/man/
2470 8/syslogd.
Wurzenberger, M., Skopik, F., Landauer, M., Greitbauer, P.,
Fiedler, R., Kastner, W., 2017. Incremental clustering for
semi-supervised anomaly detection applied on log data,
in: Proceedings of the 12th International Conference on
Availability, Reliability and Security, ACM, Reggio Cal-
2475
abria, Italy. pp. 31:1–31:6.
Yen, T.F., Oprea, A., Onarlioglu, K., Leetham, T., Robert-
son, W., Juels, A., Kirda, E., 2013. Beehive: Large-scale
log analysis for detecting suspicious activity in enterprise
networks, in: Proceedings of the 29th Annual Computer
2480
Security Applications Conference, ACM, New Orleans,
Louisiana, USA. pp. 199–208.
Yusoff, Y., Ismail, R., Hassan, Z., 2011. Common phases
of computer forensics investigation models. International
2485 Journal of Computer Science and Information Technology
3, 17–31.
Zafar, F., Khan, A., Suhail, S., Ahmed, I., Hameed, K.,
Khan, H.M., Jabeen, F., Anjum, A., 2017. Trustworthy
data: A survey, taxonomy and future trends of secure
2490 provenance schemes. Journal of Network and Computer
Applications 94, 50 – 68.
Zhu, Y., James, J., Gladyshev, P., 2009. A comparative
methodology for the reconstruction of digital events using
Windows restore points. Digital Investigation 6, 8–15.
30