Applications Audit Checklist - en - v3

You might also like

Download as docx, pdf, or txt
Download as docx, pdf, or txt
You are on page 1of 13

Applications Audit Checklist

Ref Controls Objectives Verifications


  Input Controls    
1 Review and evaluate controls built As much as possible, transactions should perform Verify that invalid data is rejected or edited on entry. You will
into system transactions over the up-front validation and editing to ensure the need to understand the business function being supported by
input of data integrity of data before it is entered into the the system and the purpose and use of its various data
system’s files and databases. Invalid data in the elements. This likely will require discussion not only with the
system can result in costly errors. It is preferable
developers but also with the end users.
and much more cost-effective to catch a data Once you understand the purpose of the system and its data,
entry error prior to that data being entered into you can think through the various data integrity risks associated
and processed by the application. Otherwise, the with the application. In some cases, a code review may be
error may not be caught at all, may only be caught appropriate if the developers are available and the auditor is a
after it results in system disruption, or may only be
knowledgeable coder.
caught after time-consuming manual Poorly written, commented, or formatted code is often a red flag
reconciliation procedures, and so on that suggests that a deeper review is needed. If possible, obtain
access to a test version of the system that mirrors the
production environment and attempt to “break” the system by
entering invalid data to see whether it is accepted by the
application.
2 Determine the need for Error or exception reports allow any potential Discuss the application’s error and exception handling with the
error/exception reports related to dataintegrity problems to be reviewed and developer or administrator. Based on the results of the analysis
data integrity and evaluate corrected when it’s not feasible or practical to use from step 1, look for opportunities for additional data integrity
whether this need has been filled input controls to perform up-front validation of checks (which may not have been feasible to perform with
data entered into the system. For “hard” up-front input requirements). Again, discussions with the
example, although it may not be inherently wrong end users can be very helpful here. Ask them what sorts of
for an employee to enter 80 hours of overtime for reporting would be helpful for them in catching anomalies and
one week into a labor system, this sort of unusual errors. For any error and exception reports that do exist, look for
event should be captured on a report for review evidence that those reports are being regularly reviewed and
by the appropriate level appropriately handled
of management
  Interface Controls    
3 Review and evaluate the controls When an application passes and/or receives data Determine what interfaces exist with the system you are
in place over data feeds to and to or from other applications, controls need to be auditing, including data flowing into and out of the
from interfacing systems employed to ensure that the data is transmitted system. These interfaces could be in the form of realtime data
completely and accurately. Failure to do so can transmission or periodic transmission of data
result in costly errors and system disruption files via batch processes. Review system flow diagrams, review
system code, and interview the application
developer or administrator to obtain this information.
Once you have a feel for the interfaces that exist, determine
which controls are in place regarding those
interfaces through code review and interviews with the
application developer or administrator. Expect to see basic
controls such as those discussed in the following paragraphs.
Control totals from interface transmissions should be generated
and used by the system to determine whether the transmission
completed accurately. If a problem is found, reports should be
issued that notify the proper people of the problem.
Some examples of control totals that may be applicable are hash
totals (totals that have no inherent meaning, such as summing
all account numbers or employee numbers in a file being
transmitted), record counts, and total amounts (totals that do
have inherent meaning, such as summing the total sales entered
or salary paid in a file being transmitted). For example, prior to
transmission, the sending system might generate a count of all
records being sent. After transmission, the receiving system
could generate a count of all records received. Those two
numbers would then be compared. If they don’t match, it
would generate an error report, as this would indicate that some
records were not received accurately. Another type of control
total could flag missing record numbers when records are
transmitted in a sequential fashion.
All such methods are intended to detect instances when data
from the sending system is not correctly received. If these
controls do exist, review evidence that applicable error reports
are being regularly reviewed and acted upon.
The system should handle items that did not transmit
successfully in such a manner that reports and/or
processes enable these items to be resolved quickly and
appropriately, such as by placing them in a suspense file and
generating reports of all items in the suspense file.
Verify that any such suspense files and error reports are being
reviewed and acted upon.
Data files that contain interface source or target information
should be secured from unauthorized modifications. This may
mean appropriate authentication controls, authorization
controls, or encryption where necessary. Review the file security
of any applicable files.
When it is not feasible to use control totals to verify accurate
transmission of data, reconciliation reports
should be generated that allow users to compare what was on
one system with what was received on another system. If
applicable, review evidence that reconciliation reports are
regularly reviewed and acted upon.
Where applicable, data validation and editing, as described in
the “Input Controls” section of this
checklist, should be performed on data received from outside
systems. Error/exception reports should be generated that allow
any data integrity problems to be corrected, and those reports
should be regularly
reviewed.
4 If the same data is kept in multiple Storing the same data in multiple places can lead Determine, with the help of the application developer or
databases and/or systems, ensure to outof-sync conditions that result in system application administrator, where this sort of control is applicable
that periodic sync processes are errors. It can also have a negative impact on and review for its existence and effectiveness.Ideally, one
executed to detect any business decisions, as erroneous conclusions can database or data file should be designated as the “master” for
inconsistencies in the data be reached using inaccuratedata each data element, and other systems will reference the master
location as opposed to keeping a separate copy of the data. Even
if multiple copies of the data are kept, the location that
represents the master copy should be designated so that the
system can easily determine “who wins” in out-of-sync situations
and perform automated corrections
  Audit Trails and Security    
Monitoring
5 Review and evaluate the audit Audit trails (or audit logs) are useful for Review the application with the developer or administrator to
trails present in the system and troubleshooting and helping to track down ensure that information is captured when key data elements are
the controls over those audit possible breaches of your application changed and key activities are performed. This information
trails should include in most cases what activity was performed, the
original and new values of the data (in the event of a data
change), who performed the activity, and when it was
performed. This information should be kept in a secured log to
prevent unauthorized updates. The logs should be retained for a
reasonable period, such as three or six months, to aid
investigations into errors or inappropriate activities
6 Ensure that the system provides a This is important to verify that the transaction was Review the application with the developer or administrator and
means of tracing a transaction or fully processed and to pinpoint any errors or evaluate the existence of this ability.
piece of data from the beginning irregularities in the processing of that data Identify a sample of recent transactions and attempt to trace
to the end of the process enabled them through the system’s various processing steps
by
the system
7 Review and evaluate processes for If processes don’t exist for security monitoring and Interview the application administrator and review any relevant
monitoring and maintaining the maintenance, security holes could exist, and documentation to get an understanding of
state of security on the system security incidents could occur without anyone’s security practices for the application. This could include, for
knowledge example, routine scans for and remediation of known
vulnerabilities and/or alerts being sent and investigated when
key activities are performed within the application (triggering off
of audit logs kept within the system, as discussed in step 5).
Some level of monitoring is important, but the monitoring level
required should be consistent with the criticality of the system
and the inherent risk of the environment.
If security monitoring is performed, assess the frequency of the
monitoring and the quality with which it is performed. Review
recent results, and determine whether exceptions were
investigated and resolved.
If applicable for the application you are auditing, verify that
policies and procedures are in place to identify
when a security patch is available and to evaluate and apply
applicable patches. Ensure that all approved
patches are installed per your policy
  Account Management    
8 Ensure that the application Failure to authenticate users or just having a poor Review the application with the developer or administrator and
provides a mechanism that authentication scheme presents an open verify that appropriate authentication measures exist
authenticates users based, at a opportunity for curious users and malicious commensurate with the type of data on the application. For
minimum, on a unique identifier attackers to access your system example, two-factor authentication might be required in some
for each user and a confidential cases to authenticate users to sensitive applications or for end
password users accessing your applications from the Internet
9 Review and evaluate the The system’s security mechanism should allow for Employees should be given only the amount of access to the
application’s authorization each system user to be given a specific level of system necessary to perform their jobs. Review the application
mechanism to ensure that users access to the application’s data and transactions. with the developer or administrator, and verify this functionality
are not allowed to access any Without the ability to provide granular access in the application. In other words, it should be possible to specify
sensitive transactions based on user need, users will likely be granted which transactions and data sets or files a system user will be
or data without first being unnecessary levels of access able to access.
authorized by the system’s In general, it also should be possible to specify what level of
security mechanism access (such as display, update, and delete) the user will receive
to application resources
10 Ensure that the system’s The administrator user function should exist to Evaluate the use of the administrator function with the
security/authorization mechanism help administer users, data, and processes. This developer or application administrator. The user of this function
has an administrator function with account or functionality should be tightly should have the ability to add, delete, or modify user access to
appropriate controls and controlled in the application to prevent the application system and its resources.
functionality compromise and disruption of services to other The security mechanism should also provide the ability to
users control who has access to this administrator function.
Obtain a list of all employees who have been granted the
administrator access level and review each for appropriateness.
Also ensure that the system’s security mechanism provides the
system’s security administrator with the ability to view who has
access to the system and what level of access they have
11 Determine whether the security The application’s security mechanism should Verify with the developer or application administrator that
mechanism enables any applicable support granular controls over who can perform appropriate controls are in place. For example, if someone needs
approval processes what approval processes and then lock data that to approve journal entries before they can be passed on to the
has been formally approved from modification by general ledger, the system’s security mechanism should provide
a lower authority.Otherwise, a lower-authority or a means for defining who is authorized to perform this approval.
malicious user could modify or corrupt data in the Any data that has been through the approval process should be
system locked in order to prevent any further modifications.Interviews
with system users are a good way to help the auditor determine
the need for this sort of ability. It is critical that the auditor
understand not only the technical aspects of the application
being reviewed but also the business purpose
12 Review and evaluate processes for Users should have access granted and governed by Review processes for requesting and approving access to the
granting access to users. Ensure the application administrator(s) to prevent application. Ensure that these processes are documented and
that access is granted only when unauthorized access to areas outside the user’s that they require approval from a knowledgeable administrator
there is a legitimate business need intended scope. The application should have before application access is granted to a user. Select a sample of
controls in place, and the administrator(s) should users, and ensure that user access was approved appropriately.
have processes in place to prevent users from Verify that the authorization mechanism is working
having more access than is required for their roles. appropriately
This step embodies the concept of least privileged
access
13 Review processes for removing Poor deprovisioning processes may leave a user Verify that appropriate deprovisioning processes are in place
user access when it is no longer with inappropriate access to our application long with the developer and application administrator.
needed. Ensure that a mechanism after the access or authority should have been Review the administrator(s)’ processes for periodically reviewing
or process is in place that removed user access lists and validating that the access is still appropriate.
suspends user access on Be sure to look at both the application and the procedures
termination from the company or around the application to ensure that they are being followed
on a change of jobs within the and are capable of being followed as written. Automated
company suspension of accounts in the event of termination or job change
is preferable to processes that require manual intervention.
For applications that have been in “production” for some time,
select a sample of system users and validate that their access is
still appropriate. Alternatively, if possible, select a sample of
system users who have changed jobs or left the company and
ensure that their access has been removed
14 Verify that the application has The appropriateness of the password controls Verify appropriate password controls with the help of the
appropriate password controls. depends on the sensitivity of the data used within developer or the application administrator and by reviewing
Also, determine whether default the application. your company policy. For example, a threedigit PIN probably is
application account passwords Overly weak passwords make the application inappropriate for an application that stores credit card data, and
have been changed easier to compromise, and overly strong a 20-character password probably is overly paranoid for
passwords could place unnecessary overhead on someone trying to access his or her voicemail. If your company
usage of the system. policy requires periodic password changes (such as every 90
Many applications, particularly those that are days), ensure that the security mechanism requires users to
purchased, have default accounts with well-known change their passwords in alignment with that policy.
default passwords. Many of these default When appropriate, the security mechanism also should enforce
accounts are used for system administration and password composition standards such as the length and
therefore have elevated privileges. If those default required characters. Additionally, the security mechanism should
passwords are not changed, it is easy for an suspend user accounts after a certain number of consecutive
unauthorized user to access the application unsuccessful log-in attempts.
This is typically as low as 3 and can be as high as 25 depending
on the application, other forms of authentication required, and
the sensitivity of the data.
Determine whether default accounts and passwords exist with
the help of the developer or application administrator and by
review of system documentation and Internet research. If they
do exist, one of the easiest ways to determine whether they
have been changed is to attempt to log on using the default
accounts and passwords, but you’re likely better advised to ask
the application administrator to attempt to do so, as it might be
against company policy to attempt to log in using someone else’s
account
15 Ensure that users are Without timeout controls, an unauthorized user Review the application with the developer or administrator to
automatically logged off from the could obtain access to the application by accessing evaluate the existence of this ability
application after a certain period a logged-in workstation where the legitimate user
of inactivity didn’t log off and the application is still active
  Permissions Management    
16 Evaluate the use of encryption The need for encryption is determined most often Review the application with the developer or administrator to
techniques to protect application
by either policy, regulation, the sensitivity of the evaluate the existence of encryption where appropriate
data network, or the sensitivity of the data in the
application. Where possible, encryption
techniques should be used for passwords and
other confidential data that is sent across the
network. This prevents other people on the
network from “sniffing” and capturing this
information. For sensitive data, such as passwords,
encryption should also be considered when the
data is at rest (in storage). This is particularly
important for data that will be stored outside of
your company’s premises
17 Evaluate application developer In general, system developers should not be given Discuss this with the developer or administrator and evaluate
access to alter production data access to alter production data in order to the separation of duties between developers and business users
establish appropriate segregation of duties. Data
entry and alteration should generally be
performed by business users
  Software Change Controls    
18 Ensure that the application It should not be possible for developers to update Evaluate this capability with the developers and application
software cannot be changed production code directly. Your production code is administrator. Determine the location of the production code
without going through a standard your application, and it should be strictly and who has access to update that code.
checkout/staging/testing/approval controlled. Preferably, the code will be controlled by some sort of librarian
process after it is placed into Segregation of duties must be in place to ensure mechanism that provides granular control over how access to
production that all changes to the code are thoroughly the code is managed.
reviewed and tested. Proper software change controls will require that the code first
Without these checks and balances, untested or be checked out into a development environment, then checked
unintended changes could be made to your into a testing or staging environment, and only then checked
production application, severely damaging the back into the production environment. Determine whether this
system’s integrity and availability. Should a failure is the case.
in the application occur without enforced software In addition, ensure that the software change mechanism
change controls, it might be difficult or impossible requires sign-off before code will be moved into production. The
to track down the cause of the problem system should require that this sign-off be performed by
someone other than the person who developed or modified the
code. In addition, the software change mechanism should allow
for specific people to be authorized to perform sign-off on the
system’s programs. Review the people with this authorization
and ensure that the privilege is kept to a minimum.
Evaluate controls in place to prevent code from being modified
after it has been signed off on but before it has been moved to
production. Otherwise, developers will be able to bypass
approval processes.
Ensure that these controls are documented in a policy or
procedure and communicated to developers. Consider selecting
a sample of recent software changes and validating that all
controls are implemented and working as designed
19 Evaluate controls regarding code Strong software controls regarding code checkout Verify with the developers that the software change mechanism
checkout and versioning and versioning provide accountability, protect the requires developers to check out code that they want to modify.
integrity of the code, and have been shown to If another developer wants to modify the same code while it is
improve maintenance and reliability still checked out, he or she should be prevented from doing so.
Alternatively, the second developer could be warned of the
conflict but allowed to perform the checkout. In such a case, a
notification of the duplicate checkout should be sent
automatically to the original developer.
Ensure that the software change mechanism “versions” software
so that past versions of the code can be retrieved if necessary.
This allows an easy mechanism for backing out changes, should
issues be encountered
20 Evaluate controls regarding the Improperly tested code may have serious Determine whether the software change process requires
testing of application code before performance or vulnerability issues when placed evidence of testing (including security testing), code walk-
it is placed into a production into production with live data throughs, and adherence to software-development guidelines
environment (including guidelines for secure coding).
These should occur before the approver signs off on the code.
Testing of any software development or modifications should
take place in a test environment that mirrors the production
environment, using test data. Ensure that these requirements
are in place and documented. Pull a sample of recent software
changes and look for evidence that these processes have been
followed
21 Evaluate controls regarding batch Many applications execute programs (often called Work with the application developers and administrators to
scheduling “jobs”) in batch (offline) mode. For example, an understand what sort of batch processing is occurring and
accounts receivable application may have jobs review applicable controls. Following are examples of common
scheduled to run every night, and the application controls:• Ensure that the batch scheduling tool can establish
may acquire a feed of invoices and automatically predecessor/successor relationships and that this ability is used
apply payments to them.These functions are often where needed. Predecessor/successor relationships allow you to
performed by a series of jobs scheduled to run in establish a sequence of jobs, where one job cannot kick off until
sequence. If proper controls are not in place over another predetermined job successfully completes. This allows
the scheduling and monitoring of these jobs, it proper sequencing of processing.• Determine whether the tool
could result in inaccurate or failed processing allows for jobs to be monitored for successful completion and
has an alert mechanism in the event of unsuccessful completion.
This alert mechanism should be used to alert some sort of
central monitoring group, who in turn should have a contact and
escalation list.• The tool should provide the ability to control
who can sign off on and implement changes to job scheduling
and to job definitions (such as where the job is located, the
name of the job, the user ID that runs the job, how often the job
is scheduled, and so on). Review and evaluate who has the
ability to sign off on and implement changes to job scheduling
and job definitions. This ability should be limited.• For changes
to job scheduling and to job definitions, the tool should track
who made the change, who signed off on the change, when the
change was made, what was changed, and why the change was
made. The tool should also allow you to retrieve previous
versions of the schedule and of job definitions in the event of a
problem with any changes. Determine whether this is the case.•
Ensure that the tool allows you to perform exception date
processing. In other words, it should be able to accommodate
changes in the schedule due to holidays, leap years, and so on.•
Ensure that recovery procedures have been developed that will
allow for jobs that have ended abnormally to be restarted and
reprocessed
  Backup and Recovery    
22 Determine whether a business A BIA is performed to obtain input from the Through interviews with the application support personnel and
impact analysis (BIA) has been application’s business users regarding the impact end users, determine what sort of BIA, if any, has been
performed on the application to to the business in the event of an extended outage performed and review associated documentation. At a
establish backup and recovery of the application (such as in the event of a minimum, look for documented requirements regarding the
needs disaster). This drives the engineering of the application’s RTO (recovery time objective, which dictates how
application’s disaster recovery mechanisms quickly the system needs to be back up after a disaster) and RPO
(recovery point objective, which dictates how much data the
business can afford to lose in the event of a disaster)
23 Ensure that appropriate backup Failure to back up critical application data may Determine whether critical data and software are backed up
controls are in place severely disrupt business operations in the event periodically (for example, weekly full backups with daily
of a disaster (or possibly even a more common incremental backups for the data) and stored offsite in a secured
system outage), resulting in total loss of the location (with appropriate encryption protections applied to the
application and its data with no ability to recover offsite data). If cost-beneficial and appropriate, duplicate
it transaction records should be created and stored to allow
recovery of data files to the point of the last processed
transaction. Ensure that the backup schedule is in alignment
with the RPO and RTO established by the application’s users.
Also ensure that the application code is backed up and stored
offsite in a secured location, along with any tools necessary for
compiling and using the code
24 Ensure that appropriate recovery Recovery procedures and testing are necessary to Discuss this with the application administrator and appropriate
controls are in place ensure that the recovery process is understood personnel to ensure that detailed recovery procedures are
and that it functions operationally as intended documented that define what tasks are to be performed, who is
to perform those tasks, and the sequence in which they are to
be performed. Testing of the recovery from backup using the
documented recovery procedures should be performed
periodically.
Ensure that the recovery processes are in alignment with the
RTO established by the application’s users
  Data Retention and Classification    
and User
Involvement
25 Evaluate controls regarding the Data should be archived and retained in These requirements will vary based on the type of data and
application’s data retention accordance with business, tax, and legal should be acquired from the appropriate departments within
requirements. Failure to do so could result in your company. Ensure that inputs have been obtained from the
penalties and operational issues caused by the appropriate personnel (such as the business owner of the data in
inability to obtain needed data the application and the legal and tax departments) to determine
retention requirements. Evaluate the appropriateness of the
retention controls with the developers and application
administrator. Consider interviewing the business owner, legal
department, and/or tax department to validate the retention
requirements
26 Evaluate the controls regarding All application data should be assigned a business Identify the business application owner (and/or the business
data classification within the owner, and this owner should classify the data (for owner of the data contained within the application) and ask for
application example, public, internal use only, or confidential). evidence that the data has been classified according to your
This provides assurance that the data is being company’s data classification system. This classification should
protected in alignment with its sensitivity appear on any reports or transactions that display system data.
Also, determine whether the application’s access control
mechanisms are appropriate based on the classification
27 Evaluate overall user involvement Without appropriate user involvement and Interview the application’s users and support personnel to
and support for the application support, the application may not adequately determine what user involvement and support mechanisms
provide for user needs or appropriately support have been put in place. Following are examples:
the business • Review and evaluate the existence of a formal steering team
for the system. Generally, a steering team or some other form of
user committee should exist to approve and prioritize system
development and modifications.
• Ensure that changes to the functionality of the system are not
made without user testing and approval.
• A mechanism should be in place that allows system users and
developers to report and track system problems and to request
system changes.
• For significant applications, some form of helpdesk
functionality should exist to provide realtime help for user
questions and problems.
• Ensure that system documentation and training exist that
provides system users with adequate information to use the
application effectively in performing their jobs

You might also like