The document provides an applications audit checklist with controls, objectives, and verifications for input controls, error/exception reports related to data integrity, and interface controls. The checklist is intended to help auditors evaluate whether appropriate controls are in place to ensure the integrity of data entering and exiting the system. This includes controls to validate data on entry, generate error reports, and verify accurate transmission of data between interfacing systems through techniques like control totals and error reporting. The auditor should discuss controls with developers and users, review code, and test the system to evaluate data integrity.
The document provides an applications audit checklist with controls, objectives, and verifications for input controls, error/exception reports related to data integrity, and interface controls. The checklist is intended to help auditors evaluate whether appropriate controls are in place to ensure the integrity of data entering and exiting the system. This includes controls to validate data on entry, generate error reports, and verify accurate transmission of data between interfacing systems through techniques like control totals and error reporting. The auditor should discuss controls with developers and users, review code, and test the system to evaluate data integrity.
The document provides an applications audit checklist with controls, objectives, and verifications for input controls, error/exception reports related to data integrity, and interface controls. The checklist is intended to help auditors evaluate whether appropriate controls are in place to ensure the integrity of data entering and exiting the system. This includes controls to validate data on entry, generate error reports, and verify accurate transmission of data between interfacing systems through techniques like control totals and error reporting. The auditor should discuss controls with developers and users, review code, and test the system to evaluate data integrity.
The document provides an applications audit checklist with controls, objectives, and verifications for input controls, error/exception reports related to data integrity, and interface controls. The checklist is intended to help auditors evaluate whether appropriate controls are in place to ensure the integrity of data entering and exiting the system. This includes controls to validate data on entry, generate error reports, and verify accurate transmission of data between interfacing systems through techniques like control totals and error reporting. The auditor should discuss controls with developers and users, review code, and test the system to evaluate data integrity.
Input Controls 1 Review and evaluate controls built As much as possible, transactions should perform Verify that invalid data is rejected or edited on entry. You will into system transactions over the up-front validation and editing to ensure the need to understand the business function being supported by input of data integrity of data before it is entered into the the system and the purpose and use of its various data system’s files and databases. Invalid data in the elements. This likely will require discussion not only with the system can result in costly errors. It is preferable developers but also with the end users. and much more cost-effective to catch a data Once you understand the purpose of the system and its data, entry error prior to that data being entered into you can think through the various data integrity risks associated and processed by the application. Otherwise, the with the application. In some cases, a code review may be error may not be caught at all, may only be caught appropriate if the developers are available and the auditor is a after it results in system disruption, or may only be knowledgeable coder. caught after time-consuming manual Poorly written, commented, or formatted code is often a red flag reconciliation procedures, and so on that suggests that a deeper review is needed. If possible, obtain access to a test version of the system that mirrors the production environment and attempt to “break” the system by entering invalid data to see whether it is accepted by the application. 2 Determine the need for Error or exception reports allow any potential Discuss the application’s error and exception handling with the error/exception reports related to dataintegrity problems to be reviewed and developer or administrator. Based on the results of the analysis data integrity and evaluate corrected when it’s not feasible or practical to use from step 1, look for opportunities for additional data integrity whether this need has been filled input controls to perform up-front validation of checks (which may not have been feasible to perform with data entered into the system. For “hard” up-front input requirements). Again, discussions with the example, although it may not be inherently wrong end users can be very helpful here. Ask them what sorts of for an employee to enter 80 hours of overtime for reporting would be helpful for them in catching anomalies and one week into a labor system, this sort of unusual errors. For any error and exception reports that do exist, look for event should be captured on a report for review evidence that those reports are being regularly reviewed and by the appropriate level appropriately handled of management Interface Controls 3 Review and evaluate the controls When an application passes and/or receives data Determine what interfaces exist with the system you are in place over data feeds to and to or from other applications, controls need to be auditing, including data flowing into and out of the from interfacing systems employed to ensure that the data is transmitted system. These interfaces could be in the form of realtime data completely and accurately. Failure to do so can transmission or periodic transmission of data result in costly errors and system disruption files via batch processes. Review system flow diagrams, review system code, and interview the application developer or administrator to obtain this information. Once you have a feel for the interfaces that exist, determine which controls are in place regarding those interfaces through code review and interviews with the application developer or administrator. Expect to see basic controls such as those discussed in the following paragraphs. Control totals from interface transmissions should be generated and used by the system to determine whether the transmission completed accurately. If a problem is found, reports should be issued that notify the proper people of the problem. Some examples of control totals that may be applicable are hash totals (totals that have no inherent meaning, such as summing all account numbers or employee numbers in a file being transmitted), record counts, and total amounts (totals that do have inherent meaning, such as summing the total sales entered or salary paid in a file being transmitted). For example, prior to transmission, the sending system might generate a count of all records being sent. After transmission, the receiving system could generate a count of all records received. Those two numbers would then be compared. If they don’t match, it would generate an error report, as this would indicate that some records were not received accurately. Another type of control total could flag missing record numbers when records are transmitted in a sequential fashion. All such methods are intended to detect instances when data from the sending system is not correctly received. If these controls do exist, review evidence that applicable error reports are being regularly reviewed and acted upon. The system should handle items that did not transmit successfully in such a manner that reports and/or processes enable these items to be resolved quickly and appropriately, such as by placing them in a suspense file and generating reports of all items in the suspense file. Verify that any such suspense files and error reports are being reviewed and acted upon. Data files that contain interface source or target information should be secured from unauthorized modifications. This may mean appropriate authentication controls, authorization controls, or encryption where necessary. Review the file security of any applicable files. When it is not feasible to use control totals to verify accurate transmission of data, reconciliation reports should be generated that allow users to compare what was on one system with what was received on another system. If applicable, review evidence that reconciliation reports are regularly reviewed and acted upon. Where applicable, data validation and editing, as described in the “Input Controls” section of this checklist, should be performed on data received from outside systems. Error/exception reports should be generated that allow any data integrity problems to be corrected, and those reports should be regularly reviewed. 4 If the same data is kept in multiple Storing the same data in multiple places can lead Determine, with the help of the application developer or databases and/or systems, ensure to outof-sync conditions that result in system application administrator, where this sort of control is applicable that periodic sync processes are errors. It can also have a negative impact on and review for its existence and effectiveness.Ideally, one executed to detect any business decisions, as erroneous conclusions can database or data file should be designated as the “master” for inconsistencies in the data be reached using inaccuratedata each data element, and other systems will reference the master location as opposed to keeping a separate copy of the data. Even if multiple copies of the data are kept, the location that represents the master copy should be designated so that the system can easily determine “who wins” in out-of-sync situations and perform automated corrections Audit Trails and Security Monitoring 5 Review and evaluate the audit Audit trails (or audit logs) are useful for Review the application with the developer or administrator to trails present in the system and troubleshooting and helping to track down ensure that information is captured when key data elements are the controls over those audit possible breaches of your application changed and key activities are performed. This information trails should include in most cases what activity was performed, the original and new values of the data (in the event of a data change), who performed the activity, and when it was performed. This information should be kept in a secured log to prevent unauthorized updates. The logs should be retained for a reasonable period, such as three or six months, to aid investigations into errors or inappropriate activities 6 Ensure that the system provides a This is important to verify that the transaction was Review the application with the developer or administrator and means of tracing a transaction or fully processed and to pinpoint any errors or evaluate the existence of this ability. piece of data from the beginning irregularities in the processing of that data Identify a sample of recent transactions and attempt to trace to the end of the process enabled them through the system’s various processing steps by the system 7 Review and evaluate processes for If processes don’t exist for security monitoring and Interview the application administrator and review any relevant monitoring and maintaining the maintenance, security holes could exist, and documentation to get an understanding of state of security on the system security incidents could occur without anyone’s security practices for the application. This could include, for knowledge example, routine scans for and remediation of known vulnerabilities and/or alerts being sent and investigated when key activities are performed within the application (triggering off of audit logs kept within the system, as discussed in step 5). Some level of monitoring is important, but the monitoring level required should be consistent with the criticality of the system and the inherent risk of the environment. If security monitoring is performed, assess the frequency of the monitoring and the quality with which it is performed. Review recent results, and determine whether exceptions were investigated and resolved. If applicable for the application you are auditing, verify that policies and procedures are in place to identify when a security patch is available and to evaluate and apply applicable patches. Ensure that all approved patches are installed per your policy Account Management 8 Ensure that the application Failure to authenticate users or just having a poor Review the application with the developer or administrator and provides a mechanism that authentication scheme presents an open verify that appropriate authentication measures exist authenticates users based, at a opportunity for curious users and malicious commensurate with the type of data on the application. For minimum, on a unique identifier attackers to access your system example, two-factor authentication might be required in some for each user and a confidential cases to authenticate users to sensitive applications or for end password users accessing your applications from the Internet 9 Review and evaluate the The system’s security mechanism should allow for Employees should be given only the amount of access to the application’s authorization each system user to be given a specific level of system necessary to perform their jobs. Review the application mechanism to ensure that users access to the application’s data and transactions. with the developer or administrator, and verify this functionality are not allowed to access any Without the ability to provide granular access in the application. In other words, it should be possible to specify sensitive transactions based on user need, users will likely be granted which transactions and data sets or files a system user will be or data without first being unnecessary levels of access able to access. authorized by the system’s In general, it also should be possible to specify what level of security mechanism access (such as display, update, and delete) the user will receive to application resources 10 Ensure that the system’s The administrator user function should exist to Evaluate the use of the administrator function with the security/authorization mechanism help administer users, data, and processes. This developer or application administrator. The user of this function has an administrator function with account or functionality should be tightly should have the ability to add, delete, or modify user access to appropriate controls and controlled in the application to prevent the application system and its resources. functionality compromise and disruption of services to other The security mechanism should also provide the ability to users control who has access to this administrator function. Obtain a list of all employees who have been granted the administrator access level and review each for appropriateness. Also ensure that the system’s security mechanism provides the system’s security administrator with the ability to view who has access to the system and what level of access they have 11 Determine whether the security The application’s security mechanism should Verify with the developer or application administrator that mechanism enables any applicable support granular controls over who can perform appropriate controls are in place. For example, if someone needs approval processes what approval processes and then lock data that to approve journal entries before they can be passed on to the has been formally approved from modification by general ledger, the system’s security mechanism should provide a lower authority.Otherwise, a lower-authority or a means for defining who is authorized to perform this approval. malicious user could modify or corrupt data in the Any data that has been through the approval process should be system locked in order to prevent any further modifications.Interviews with system users are a good way to help the auditor determine the need for this sort of ability. It is critical that the auditor understand not only the technical aspects of the application being reviewed but also the business purpose 12 Review and evaluate processes for Users should have access granted and governed by Review processes for requesting and approving access to the granting access to users. Ensure the application administrator(s) to prevent application. Ensure that these processes are documented and that access is granted only when unauthorized access to areas outside the user’s that they require approval from a knowledgeable administrator there is a legitimate business need intended scope. The application should have before application access is granted to a user. Select a sample of controls in place, and the administrator(s) should users, and ensure that user access was approved appropriately. have processes in place to prevent users from Verify that the authorization mechanism is working having more access than is required for their roles. appropriately This step embodies the concept of least privileged access 13 Review processes for removing Poor deprovisioning processes may leave a user Verify that appropriate deprovisioning processes are in place user access when it is no longer with inappropriate access to our application long with the developer and application administrator. needed. Ensure that a mechanism after the access or authority should have been Review the administrator(s)’ processes for periodically reviewing or process is in place that removed user access lists and validating that the access is still appropriate. suspends user access on Be sure to look at both the application and the procedures termination from the company or around the application to ensure that they are being followed on a change of jobs within the and are capable of being followed as written. Automated company suspension of accounts in the event of termination or job change is preferable to processes that require manual intervention. For applications that have been in “production” for some time, select a sample of system users and validate that their access is still appropriate. Alternatively, if possible, select a sample of system users who have changed jobs or left the company and ensure that their access has been removed 14 Verify that the application has The appropriateness of the password controls Verify appropriate password controls with the help of the appropriate password controls. depends on the sensitivity of the data used within developer or the application administrator and by reviewing Also, determine whether default the application. your company policy. For example, a threedigit PIN probably is application account passwords Overly weak passwords make the application inappropriate for an application that stores credit card data, and have been changed easier to compromise, and overly strong a 20-character password probably is overly paranoid for passwords could place unnecessary overhead on someone trying to access his or her voicemail. If your company usage of the system. policy requires periodic password changes (such as every 90 Many applications, particularly those that are days), ensure that the security mechanism requires users to purchased, have default accounts with well-known change their passwords in alignment with that policy. default passwords. Many of these default When appropriate, the security mechanism also should enforce accounts are used for system administration and password composition standards such as the length and therefore have elevated privileges. If those default required characters. Additionally, the security mechanism should passwords are not changed, it is easy for an suspend user accounts after a certain number of consecutive unauthorized user to access the application unsuccessful log-in attempts. This is typically as low as 3 and can be as high as 25 depending on the application, other forms of authentication required, and the sensitivity of the data. Determine whether default accounts and passwords exist with the help of the developer or application administrator and by review of system documentation and Internet research. If they do exist, one of the easiest ways to determine whether they have been changed is to attempt to log on using the default accounts and passwords, but you’re likely better advised to ask the application administrator to attempt to do so, as it might be against company policy to attempt to log in using someone else’s account 15 Ensure that users are Without timeout controls, an unauthorized user Review the application with the developer or administrator to automatically logged off from the could obtain access to the application by accessing evaluate the existence of this ability application after a certain period a logged-in workstation where the legitimate user of inactivity didn’t log off and the application is still active Permissions Management 16 Evaluate the use of encryption The need for encryption is determined most often Review the application with the developer or administrator to techniques to protect application by either policy, regulation, the sensitivity of the evaluate the existence of encryption where appropriate data network, or the sensitivity of the data in the application. Where possible, encryption techniques should be used for passwords and other confidential data that is sent across the network. This prevents other people on the network from “sniffing” and capturing this information. For sensitive data, such as passwords, encryption should also be considered when the data is at rest (in storage). This is particularly important for data that will be stored outside of your company’s premises 17 Evaluate application developer In general, system developers should not be given Discuss this with the developer or administrator and evaluate access to alter production data access to alter production data in order to the separation of duties between developers and business users establish appropriate segregation of duties. Data entry and alteration should generally be performed by business users Software Change Controls 18 Ensure that the application It should not be possible for developers to update Evaluate this capability with the developers and application software cannot be changed production code directly. Your production code is administrator. Determine the location of the production code without going through a standard your application, and it should be strictly and who has access to update that code. checkout/staging/testing/approval controlled. Preferably, the code will be controlled by some sort of librarian process after it is placed into Segregation of duties must be in place to ensure mechanism that provides granular control over how access to production that all changes to the code are thoroughly the code is managed. reviewed and tested. Proper software change controls will require that the code first Without these checks and balances, untested or be checked out into a development environment, then checked unintended changes could be made to your into a testing or staging environment, and only then checked production application, severely damaging the back into the production environment. Determine whether this system’s integrity and availability. Should a failure is the case. in the application occur without enforced software In addition, ensure that the software change mechanism change controls, it might be difficult or impossible requires sign-off before code will be moved into production. The to track down the cause of the problem system should require that this sign-off be performed by someone other than the person who developed or modified the code. In addition, the software change mechanism should allow for specific people to be authorized to perform sign-off on the system’s programs. Review the people with this authorization and ensure that the privilege is kept to a minimum. Evaluate controls in place to prevent code from being modified after it has been signed off on but before it has been moved to production. Otherwise, developers will be able to bypass approval processes. Ensure that these controls are documented in a policy or procedure and communicated to developers. Consider selecting a sample of recent software changes and validating that all controls are implemented and working as designed 19 Evaluate controls regarding code Strong software controls regarding code checkout Verify with the developers that the software change mechanism checkout and versioning and versioning provide accountability, protect the requires developers to check out code that they want to modify. integrity of the code, and have been shown to If another developer wants to modify the same code while it is improve maintenance and reliability still checked out, he or she should be prevented from doing so. Alternatively, the second developer could be warned of the conflict but allowed to perform the checkout. In such a case, a notification of the duplicate checkout should be sent automatically to the original developer. Ensure that the software change mechanism “versions” software so that past versions of the code can be retrieved if necessary. This allows an easy mechanism for backing out changes, should issues be encountered 20 Evaluate controls regarding the Improperly tested code may have serious Determine whether the software change process requires testing of application code before performance or vulnerability issues when placed evidence of testing (including security testing), code walk- it is placed into a production into production with live data throughs, and adherence to software-development guidelines environment (including guidelines for secure coding). These should occur before the approver signs off on the code. Testing of any software development or modifications should take place in a test environment that mirrors the production environment, using test data. Ensure that these requirements are in place and documented. Pull a sample of recent software changes and look for evidence that these processes have been followed 21 Evaluate controls regarding batch Many applications execute programs (often called Work with the application developers and administrators to scheduling “jobs”) in batch (offline) mode. For example, an understand what sort of batch processing is occurring and accounts receivable application may have jobs review applicable controls. Following are examples of common scheduled to run every night, and the application controls:• Ensure that the batch scheduling tool can establish may acquire a feed of invoices and automatically predecessor/successor relationships and that this ability is used apply payments to them.These functions are often where needed. Predecessor/successor relationships allow you to performed by a series of jobs scheduled to run in establish a sequence of jobs, where one job cannot kick off until sequence. If proper controls are not in place over another predetermined job successfully completes. This allows the scheduling and monitoring of these jobs, it proper sequencing of processing.• Determine whether the tool could result in inaccurate or failed processing allows for jobs to be monitored for successful completion and has an alert mechanism in the event of unsuccessful completion. This alert mechanism should be used to alert some sort of central monitoring group, who in turn should have a contact and escalation list.• The tool should provide the ability to control who can sign off on and implement changes to job scheduling and to job definitions (such as where the job is located, the name of the job, the user ID that runs the job, how often the job is scheduled, and so on). Review and evaluate who has the ability to sign off on and implement changes to job scheduling and job definitions. This ability should be limited.• For changes to job scheduling and to job definitions, the tool should track who made the change, who signed off on the change, when the change was made, what was changed, and why the change was made. The tool should also allow you to retrieve previous versions of the schedule and of job definitions in the event of a problem with any changes. Determine whether this is the case.• Ensure that the tool allows you to perform exception date processing. In other words, it should be able to accommodate changes in the schedule due to holidays, leap years, and so on.• Ensure that recovery procedures have been developed that will allow for jobs that have ended abnormally to be restarted and reprocessed Backup and Recovery 22 Determine whether a business A BIA is performed to obtain input from the Through interviews with the application support personnel and impact analysis (BIA) has been application’s business users regarding the impact end users, determine what sort of BIA, if any, has been performed on the application to to the business in the event of an extended outage performed and review associated documentation. At a establish backup and recovery of the application (such as in the event of a minimum, look for documented requirements regarding the needs disaster). This drives the engineering of the application’s RTO (recovery time objective, which dictates how application’s disaster recovery mechanisms quickly the system needs to be back up after a disaster) and RPO (recovery point objective, which dictates how much data the business can afford to lose in the event of a disaster) 23 Ensure that appropriate backup Failure to back up critical application data may Determine whether critical data and software are backed up controls are in place severely disrupt business operations in the event periodically (for example, weekly full backups with daily of a disaster (or possibly even a more common incremental backups for the data) and stored offsite in a secured system outage), resulting in total loss of the location (with appropriate encryption protections applied to the application and its data with no ability to recover offsite data). If cost-beneficial and appropriate, duplicate it transaction records should be created and stored to allow recovery of data files to the point of the last processed transaction. Ensure that the backup schedule is in alignment with the RPO and RTO established by the application’s users. Also ensure that the application code is backed up and stored offsite in a secured location, along with any tools necessary for compiling and using the code 24 Ensure that appropriate recovery Recovery procedures and testing are necessary to Discuss this with the application administrator and appropriate controls are in place ensure that the recovery process is understood personnel to ensure that detailed recovery procedures are and that it functions operationally as intended documented that define what tasks are to be performed, who is to perform those tasks, and the sequence in which they are to be performed. Testing of the recovery from backup using the documented recovery procedures should be performed periodically. Ensure that the recovery processes are in alignment with the RTO established by the application’s users Data Retention and Classification and User Involvement 25 Evaluate controls regarding the Data should be archived and retained in These requirements will vary based on the type of data and application’s data retention accordance with business, tax, and legal should be acquired from the appropriate departments within requirements. Failure to do so could result in your company. Ensure that inputs have been obtained from the penalties and operational issues caused by the appropriate personnel (such as the business owner of the data in inability to obtain needed data the application and the legal and tax departments) to determine retention requirements. Evaluate the appropriateness of the retention controls with the developers and application administrator. Consider interviewing the business owner, legal department, and/or tax department to validate the retention requirements 26 Evaluate the controls regarding All application data should be assigned a business Identify the business application owner (and/or the business data classification within the owner, and this owner should classify the data (for owner of the data contained within the application) and ask for application example, public, internal use only, or confidential). evidence that the data has been classified according to your This provides assurance that the data is being company’s data classification system. This classification should protected in alignment with its sensitivity appear on any reports or transactions that display system data. Also, determine whether the application’s access control mechanisms are appropriate based on the classification 27 Evaluate overall user involvement Without appropriate user involvement and Interview the application’s users and support personnel to and support for the application support, the application may not adequately determine what user involvement and support mechanisms provide for user needs or appropriately support have been put in place. Following are examples: the business • Review and evaluate the existence of a formal steering team for the system. Generally, a steering team or some other form of user committee should exist to approve and prioritize system development and modifications. • Ensure that changes to the functionality of the system are not made without user testing and approval. • A mechanism should be in place that allows system users and developers to report and track system problems and to request system changes. • For significant applications, some form of helpdesk functionality should exist to provide realtime help for user questions and problems. • Ensure that system documentation and training exist that provides system users with adequate information to use the application effectively in performing their jobs