Professional Documents
Culture Documents
Auditing-Artificial-Intelligence Res Eng 1218 PDF
Auditing-Artificial-Intelligence Res Eng 1218 PDF
Auditing-Artificial-Intelligence Res Eng 1218 PDF
AUDITING
ARTIFICI AL
INTELLIGENCE
CONTENTS
4 Potential Impact of Artificial Intelligence
on Organizations
4 Why Should Auditors Care About AI?
4 /ChallengesfortheAuditor
6 /MappingCOBITtoStrategy:AVisual
RepresentationofHowtoApplyCOBIT ® 2019
intheAuditingofAI
8 /ChallengesandSolutionsfortheAIAuditor
9 Conclusion
10 Resources and References for Auditing AI
12 Acknowledgments
ABSTRACT
TherearemanypotentialchallengesforITauditorspreparingtoequipthemselvesto
auditartificialintelligence(AI).Butsolutionsdoexistthatcantransformchallengesinto
successes.Thiswhitepaperfocusesonwhatauditorsneedtoknowastheyprepareto
focusonAI.ItexploresthedefinitionofAI,describesthechallengesofauditingAI,and
discusseshowthecurrentversionofCOBIT® (COBIT® 2019)canbeleveragedtoauditAI.
Additionally,itidentifiesotherframeworksthatarealsorelevanttoday.Auditorswill
exploreinitialkeystosuccessfullyauditingAIanduncoverrelevantreferences.
inmanyways,bymanyexperts.WhereastheISACA ofAIinthatitfocusesonmachines’abilitytoreceiveasetof
researchteamhaselectedtoretainflexibilityinthe dataandlearnforthemselves,changingtheiralgorithmsas
definition,duetothetechnology’sever-changingscope neededastheylearnmoreabouttheinformationtheyare
andcontext,onegeneraldefinitioncanserveasan processing.3
indicatorastothenatureandpurposeofAI.Russelland
3
ofsuccessfullyachievingtheirgoals. 1 1
circumstances. Therefore, AI does not always operate based
Twootherconceptsmayalsobehelpfulinunderstanding
on a predefined set of rules.
AI:
Challenges for the Auditor
• AI may be envisioned as a large circle with several smaller
TracticaResearchexpectsAIsoftwarerevenuetogrow
circles within it. AI,whichismachinescarryingouttasksbased
fromUS$3.2billionin2016toUS$89.9billionby2025.4
4
onalgorithmsinan“intelligent”manner,2 isthelargecircle;
Withthesupportofadjacenttechnologies(suchascloud
2
other,morespecifictypesofAI,suchasmachinelearning,are
computingandstorage),AIhasemergedfromtheso-
called“AIwinter”of2010togarneruptoUS$40billionof
1
Russell,S.;P.Norvig;Artificial Intelligence: A Modern Approach (3rd Edition),Pearson,USA,2009,https://www.pearson.com/us/higher-
education/program/Russell-Artificial-Intelligence-A-Modern-Approach-3rd-Edition/PGM156683.html
2
Venkatesan,M.;“ArtificialIntelligencevs.MachineLearningvs.DeepLearning,”DataScienceCentral,7May2018,
https://www.datasciencecentral.com/profiles/blogs/artificial-intelligence-vs-machine-learning-vs-deep-learning
2
3
Ibid.
TracticaResearch,“ArtificialIntelligenceSoftwareMarkettoReach$89.8BillioninAnnualWorldwideRevenueby2025,”21December2017,
3
https://www.tractica.com/newsroom/press-releases/artificial-intelligence-software-market-to-reach-89-8-billion-in-annual-worldwide-revenue-by-2025
4
investmentcapital,atthesametimeproduction sourceofauditchallenges.However,thisassumesthat
deploymentshavebeenlimited. 5
traditionaltechnologyauditorsareresponsiblefor
auditingalgorithms.Thisisnotthecase.ITauditors
5
AI’srisehasbeenaccompaniedbythetraditionallagtime
shouldlookatthegovernanceofAIandtheintegration
betweenearlyadoptionandtheestablishmentofregulatory
amongsystems.Althoughthealgorithmsshouldbe
andcomplianceframeworks.Thereis,forexample,no
auditedbymodelspecialists,auditorshavingabasic
matureauditingframeworkinplacedetailingAI
understandingofthewouldbebeneficial.Infact,auditors
subprocesses,norarethereanyAI-specificregulations,
alreadydoso,usinginformationinregulationssuchasUS
standardsormandates.Clarkpioneeredthecross-industry
OfficeoftheComptrolleroftheCurrency(OCC)2011-12.
processfordatamining(CRISP-DM)frameworkinearly
2018,butindividualauditorsarechallengedwithhowto Therearealsoclaimsthechallengeisduetoalackof
performauditssuccessfullywhentherearevirtuallyno academicresearchandindustrypublicationsonthetopic.
widelyadoptedprecedentsforhandlingAIusecases.6 This,too,isinaccurate.Thereisaconsiderableamountof
research,butitishighlytechnicalandnottypicallyaimed
6
InadditiontoalackofexplicitauditstandardsaroundAI,
atthetraditionalauditor.Historically,traditionalITauditors
thereareadditionalchallengesimpactingtheaudit
havelookedatgovernanceandintegration,withoutdiving
process.Aspreviouslynoted,thedefinitionofAIis
deeplyintoalgorithms.
frequentlydebatedandtheITworld,includingauditors,
hasnotreachedacommondefinitionortaxonomyon MostenterpriseshavenotyetbeguntothinkabouthowAI
whichtospecifyasetofworld-classpractices. mayplayaroleintheirbusinesses,sotheyareunlikelyto
haveadocumentedplantoalignAIusecasestothe
Moreover,AIsystemsandsolutionsvarywidelyfromeach
businessortorecognizereturnonAIinvestments.
other,andthevastsetofexistingandemerging
However,iftheydodecidetoadoptAI,executiveswill
technologiesfoundationaltoAIarchitecturegivebirthto
demandclarityofahigherorderastheybegintheirefforts
complexsystems.Thiscomplexitypointstoahigh
todevelopaneffectiveAIstrategy.Becausethebusiness
likelihoodofuncertaintyaroundthescopeofAIwithinthe
caseandstrategydocumentsrepresenttypicalstarting
business.Despitethisuncertaintyinthebusiness,
pointsontheAIjourney,auditorswillbechallengedto
auditorsarefairlywellpositionedtotakeontheir
cascadedowntheCOBIT® 2019hierarchyfromthe
responsibilitiesrelativetoAI.Goodtechnologyauditors
strategictothetacticalpartsoftheaudit.
arealreadylikelytopossessenoughskilland
understandingtoeffectivelyassessAIintheenterprise. Insum,ITauditorsshouldnotgodownthepathof
overthinkingthechallengesofauditingAI.Reflectingon
Inaddition,thecomplexityofAIandtheshortageof
howtheyfirstauditedcloudcomputingorcybersecurity
qualifieddatascientistswillroutinelyleadtothe
shouldprovidethemwithausefulframeofreference.For
outsourcingofAIdevelopmentprojectstooneormore
example,itisunlikelytheyexaminedalltheprotocolsin
third-partyresources.Acoherentunderstandingof
depthandtestedthattheOpenSystemsInterconnection
enterpriseAIwillbedispersed—and,overtime,perhaps
(OSI)layer5implementationwasfunctioning
evenlost—acrosstiersofAIproviders.Thiswill
appropriately.Instead,withAI,aswiththosepreviousnew
subsequentlyincreasethechallengefortheAIauditor.
technologies,auditorswillfocusonthecontrolsand
WhiletherewillundoubtedlybechallengesforAIauditors governancestructuresthatareinplaceanddetermine
astheyrampupfortheirnewresponsibilities,thesituation thattheyareoperatingeffectively.Auditorscanprovide
isnotasdireasmightbeassumed.The“blackbox”effect someassurancebyfocusingonthebusinessandIT
oftenascribedtomachinelearningisoftencitedasa governanceaspects.
5
Bughin,J.;E.Hazan;S.Ramaswamy;M.Chui;T.Allas;P.Dahlstrom;N.Henke;M.Trench;“ArtificialIntelligence:TheNextDigitalFrontier?”
McKinseyGlobalInstitute,June2017,
5
https://www.mckinsey.com/~/media/McKinsey/Industries/Advanced%20Electronics/Our%20Insights/How%20artificial%20intelligence%20can
%20deliver%20real%20value%20to%20companies/MGI-Artificial-Intelligence-Discussion-paper.ashx
6
Clark,A.;“TheMachineLearningAudit—CRISP-DMFramework,”ISACA® Journal,vol.1,2018,https://www.isaca.org/Journal/archives/2018/
Volume-1/Pages/the-machine-learning-audit-crisp-dm-framework.aspx
6
Auditing of AI processwithinanorganization.
AstheapplicationofAIinthebusinessworldisstillinits ThereareseveralexamplesofriskrelatedtoAIstrategy:
earlystages,thereislimitedguidanceonhowtoapproach • LackofalignmentbetweenITplansandbusinessneeds
auditinganAIinitiativeforanorganization.Therefore,this • ITplansthatareinconsistentwiththeorganization’s
exampleleveragesISACA’sCOBIT® 2019frameworkasa expectationsorrequirements
startingpoint.TheCOBIT® 2019frameworkprovidesthe • ImpropertranslationofITtacticalplansfromtheITstrategic
auditorwithtools—includingprocessdescriptions,desired plans
outcomes,basepracticesandworkproductsacross • Ineffectivegovernancestructuresthatfailtoensure
virtuallyalltheITdomains—toenabletheauditorto accountabilityandresponsibilityforITprocessesrelatedtothe
provideassuranceovertheAIinitiativeforany AIfunction
organization.
Figure 1 highlightsseveralexamplesofprocesseswithin
Astartingpointforanauditofanorganization’sAIisto COBIT® 2019thatmayprovidehelpincompilingalistof
definethescopeandobjectivesoftheauditandconsider risksandcontrolsfortheAIinitiativewithinan
risktotheorganizationrelatedtotheAIinitiative.These organization.
EDM01—Ensured
Governance EDM02—Ensured EDM03—Ensured EDM04—Ensured EDM05—Ensured
Framework Setting Benefits Delivery Risk Optimization Resource Stakeholder
and Maintenance Optimization Engagement
MEA03—Managed
BAI08—Managed BAI09—Managed BAI10—Managed BAI11—Managed Compliance With
Knowledge Assets Configuration Projects External
Requirements
APO04 Managed Innovation
APO04.04 Assess the potential of emerging
technologies and innovative ideas.
APO04.06 Monitor the implementation and
use of innovation.
DSS01—Managed DSS02—Managed DSS05—Managed DSS06—Managed
Service Requests DSS03—Managed DSS04—Managed Security Business MEA04—Managed
Operations Problems Continuity Assurance
and Incidents Services Process Controls
DSS06providesamorein-depthexampleofhowthe informationtounderstandtherationalebehindeveryAI
auditorcanleverageCOBIT® 2019duringthecourseofan decisionmadewithintheorganization.TheDSS06.05
AIassurancereview. description(figure 2)follows:“Ensurethatbusiness
informationcanbetracedtoanoriginatingbusinessevent
DSS06Managed Business Process Controls includes
andassociatedwithaccountableparties.This
managementpracticeDSS06.05Ensure traceability and
discoverabilityprovidesassurancethatbusiness
accountability for information events,whichcouldbeused
informationisreliableandhasbeenprocessedin
toensureAIactivityaudittrailsprovidesufficient
accordancewithdefinedobjectives.”7 7
ProcessoutcomesinCOBIT2019arederivedfromthe 1 Capturesourceinformation,supportingevidenceandtherecord
practiceitself,andforDSS06.05,canbearticulatedas oftransactions.
“Businessinformationistracedtoanoriginatedbusiness 2 Defineretentionrequirements,basedonbusinessrequirements,
eventandisassociatedwithaccountableparties.” tomeetoperational,financialreportingandcomplianceneeds.
3 Disposeofsourceinformation,supportingevidenceandthe
ThefollowingactivitiesarelistedforDSS06.05:
recordoftransactionsinaccordancewiththeretentionpolicy.
Figure 3 showstheinputsandoutputsfromDSS06.05.
7
ISACA,COBIT® 2019 Framework: Governance and Management Objectives,USA,2018,http://www.isaca.org/COBIT/Pages/COBIT-2019-Framework-
Governance-and-Management-Objectives.aspx
7
C. Component: Information Flows and Items (see also Section 3.6) (cont.)
Management Practice Inputs Outputs
DSS06.04 Manage errors and exceptions. From Description Description To
Error reports and root Internal
cause analysis
Evidence of error MEA02.04
correction and
remediation
DSS06.05 Ensure traceability and accountability for Record of transactions Internal
information events.
Retention requirements Internal;
APO14.09
DSS06.06 Secure information assets. Reports of violations DSS05.03
Related Guidance (Standards, Frameworks, Compliance Requirements) Detailed Reference
National Institute of Standards and Technology Special Publication 3.1 Preparation (Task 10, 11): Inputs and Outputs
800-37, Revision 2, September 2017
Source:ISACA,COBIT ® 2019 Framework: Governance and Management Objectives,USA,2018
Auditsshouldevaluatetheworkproducts,retention
Challenges and Solutions for
the AI Auditor
requirementsandrecordsoftransactionaspartof
fieldworktesting.Criteriatheauditorwouldusefortesting
include,“DoesthedecisionmadebyAIseemappropriate, WhilethereareseveralpotentialchallengesforITauditors
giventhedecisioninputsandusecase?” preparingtoequipthemselvestoauditAI,solutionsdo
existthatcanconvertthechallengesintosuccesses.The
listinfigure 4 providesexamples.
FIGURE 4: ChallengesandSolutionsforAIAuditing
Thefollowinginformationexpandsonthekeystosuccess programming,datawarehousing,streamprocessingplatforms,
ofauditingAI: storage,computingclusters,computekernels,application
softwaretestinganddebugging,dataprocessandmodeling,
• Become informed about AI design and architecture to set
andcommercialoff-the-shelf(COTS)software.Fromaskills
proper scope. AIincludesalargesetoftechnologies,people
perspective,AIprojectsmayrequiredatascientists,data
andprocessesand,therefore,willrequiresignificantattentionto
engineers,dataarchitectsandprogrammerscapableinPython,
controls,policiesandgovernance.AIarchitecturemaycombine
R,Javaandmatrixlaboratory(MATLAB).8 8
8
8
Op cit Tractica
enterprisetechnologiesbutalsoinvolvesmultipleinternal existingframeworkscanbeadoptedtohandlemostofthe
teamsandexternalthirdparties.Internalstakeholdersinvolve existingAIusecasesthatwillbeencounteredinthefield.Also,
engineeringandsecurityteamsonthetechnicalsideand fromaregulatoryperspective,existingcharterssuchasthe
businessleadersengagedwiththeAIstrategy.Theuseofcloud UnitedStatesHealthInsurancePortabilityandAccountability
computingiswidespreadwithAIandimpliesthatthirdparties Act(HIPAA)andFairLendingActandtheEuropeanUnion’s
willcontrolpartoftheinfrastructure.Wherecloudcomputingis GeneralDataProtectionRegulation(GDPR)canbeadoptedto
used,forexample,auditorsmustaddressrisk(suchasvendor providelegalguidance.Theexistingframeworksandregulation
lock-inandpartitionedknowledge)differentlyfromtheon- (adaptedbyanauditorwhoknowstheAIlandscape)andlegal
premiseapplications. consultationwillsufficeuntilmorespecificAIstandardsare
deploymentofAI,enterprisestakeholdersmaybeuninformed TransparencyisanessentialaimfortheAIauditorduetothe
aboutitsuseandstrategy.AIauditorsmustbeproactiveto complexityoftheAIenvironment.Algorithmsrequiremultiple
addressAIconcernsandbeabletobreakdownandsimplify roundsoftuningbydatascientistsanddataengineers.Some
complexdesignsandissuesintotermsstakeholderscan enterprise-basedcommercialoff-the-shelfsolutionsmay
understand.Auditorsmustbeawareofthedifferentcontexts alreadycontaincomponentsofmachinelearning.Likewise,the
forAIdiscussionsandbeabletoadjustthelevelofthe auditingprocessmustensurevigilanceofcurrentandnewAI
conversationappropriately. developmentsandpromotecontinuousimprovementand
lackofnewframeworksspecifictoAIshouldnotbean infact,becomeatoolfortheAIauditor.
Conclusion
Artificialintelligencepromisestotransformmorethanjust socialenvironments.Initialdisruptionsbeingcausedby
thewayenterprisesdobusiness.Itwilltoucheverycorner emergingAItechnologieswillevolveandaffecttheway
ofsociety. peopleworkuntilmachinesactlikehumanbeingsforall
conceivablefunctions.Thisemergingphaseoffersagreat
Auditorsshouldaskthemselveswhetherorganizations
opportunityforthebusinessandinformationtechnology
andauditteamsarereadyforthetoughquestionsaround
communitytostepup,prepareandestablishsound
AIandtheapproachforauditingit.Thekeypointscovered
governancearoundauditingAI.COBIT® 2019,apowerful
inthiswhitepapercanbehelpfulingettingofftoa
andtime-testedmethodology,canbeleveragedtopave
successfulstart.
theway.
Despiteambiguityaroundaconceptualdefinition,AI
continuestoproliferateacrossbusiness,academicand
Bughin,J.;E.Hazan;S.Ramaswamy;M.Chui;T.Allas;P. TheInstituteofInternalAuditors,“GlobalPerspectivesand
Dahlstrom;N.Henke;M.Trench;“ArtificialIntelligence: InsightsSeries,ArtificialIntelligence—Considerationsfor
TheNextDigitalFrontier?”McKinseyGlobalInstitute,June theProfessionofInternalAuditing,”2017,
2017, https://na.theiia.org/periodicals/Public%20Documents/
https://www.mckinsey.com/~/media/McKinsey/Industrie GPI-Artificial-Intelligence.pdf
s/Advanced%20Electronics/Our%20Insights/How
TheInstituteofInternalAuditors,“GlobalPerspectivesand
%20artificial%20intelligence%20can%20deliver%20real
InsightsSeries,TheIIA’sArtificialIntelligenceAuditing
%20value%20to%20companies/MGI-Artificial-Intelligence-
Framework—PracticalApplications,PartA,”2017,
Discussion-paper.ashx
https://na.theiia.org/periodicals/Public%20Documents/
Clark,A.;“TheMachineLearningAudit—CRISP-DM GPI-Artificial-Intelligence-Part-II.pdf
Framework,”ISACA® Journal,vol.1,2018,
TheInstituteofInternalAuditors,“GlobalPerspectivesand
https://www.isaca.org/Journal/archives/2018/Volume-
InsightsSeries,TheIIA’sArtificialIntelligenceAuditing
1/Pages/the-machine-learning-audit-crisp-dm-
Framework—PracticalApplications,PartB,”2017,
framework.aspx
https://na.theiia.org/periodicals/Public%20Documents/
Conitzer,V.;W.Sinnot-Armstrong;J.S.Borg;Y.Deng;M. GPI-Artificial-Intelligence-Part-III.pdf
Kramer;“MoralDecisionMakingFrameworksforArtificial
TheInstituteofInternalAuditors,“ArtificialIntelligence:
Intelligence,”AssociationfortheAdvancementofArtificial
TheFutureforInternalAuditing,”Tone at the Top,
Intelligence,2017,
December2017
https://users.cs.duke.edu/~conitzer/moralAAAI17.pdf
InternalAuditFoundation,“RequestforProposals,
Cummings,M.L.;“ArtificialIntelligenceandtheFutureof
ArtificialIntelligenceResearchProject,”2017,
Warfare,”ChathamHouse,January2017,
https://na.theiia.org/iiarf/Public%20Documents/
https://www.chathamhouse.org/sites/default/files/public
RFP-Artificial-Intelligence.pdf
ations/research/2017-01-26-artificial-intelligence-future-
warfare-cummings-final.pdf
InternationalStandardsOrganization(ISO),“ISO/IEC Meek,T.;“HowHumansandAIWillSharetheAuditing
27000:2018(en),Informationtechnology—Security FunctionoftheFuture,”Forbes,10July2017,
techniques—Informationsecuritymanagementsystems— https://www.forbes.com/sites/workday/2017/07/10/
Overviewandvocabulary,” how-humans-and-ai-will-share-the-auditing-function-of-
https://www.iso.org/obp/ui/#iso:std:iso-iec:27000:ed-5:v1:e the-future/#63d3bf774fa1
Acknowledgments
ISACAwouldliketorecognize:
ClientSolutionsArchitect,USA CISA,CRISC,CISM,CGEIT,COBIT5
ImplementerandAssessor,CFE,CIPM,
Expert Reviewers
CIPT,CISSP,CITBCM,CPP,CSSLP,GCFA,
GCIA,GCIH,GSNA,ISSMP-ISSAP,PMP
Andrew Clark Merck&Co.,Inc.,Singapore
Gregory Touhill
CISM,CISSP
CyxteraFederalGroup,USA
Ted Wolff
CISA
Vanguard,Inc.,USA
Tichaona Zororo
CISA,CRISC,CISM,CGEIT,COBIT5
Assessor,CIA,CRMA
EGIT|EnterpriseGovernanceofIT(Pty)
Ltd,SouthAfrica
Theresa Grafenstine
ISACABoardChair,2017-2018
CISA,CRISC,CGEIT,CGAP,CGMA,CIA,
CISSP,CPA
Deloitte&ToucheLLP,USA
AboutISACA
1700E.GolfRoad,Suite400
Nearingits50thyear,ISACA® (isaca.org)isaglobalassociationhelping
Schaumburg,IL60173,USA
individualsandenterprisesachievethepositivepotentialoftechnology.
Technologypowerstoday’sworldandISACAequipsprofessionalswiththe
Phone: +1.847.660.5505
knowledge,credentials,educationandcommunitytoadvancetheircareers
andtransformtheirorganizations.ISACAleveragestheexpertiseofitshalf- Fax: +1.847.253.1755
millionengagedprofessionalsininformationandcybersecurity,governance,
assurance,riskandinnovation,aswellasitsenterpriseperformance Support: support.isaca.org
subsidiary,CMMI® Institute,tohelpadvanceinnovationthroughtechnology.
Website: www.isaca.org
ISACAhasapresenceinmorethan188countries,includingmorethan217
chaptersandofficesinboththeUnitedStatesandChina.
DISCLAIMER
Provide Feedback:
ISACAhasdesignedandcreatedAuditing Artificial Intelligence (the“Work”)
www.isaca.org/auditing-AI
primarilyasaneducationalresourceforprofessionals.ISACAmakesnoclaim
thatuseofanyoftheWorkwillassureasuccessfuloutcome.TheWork Participate in the ISACA Online
shouldnotbeconsideredinclusiveofallproperinformation,proceduresand Forums:
testsorexclusiveofotherinformation,proceduresandteststhatare https://engage.isaca.org/onlineforums
reasonablydirectedtoobtainingthesameresults.Indeterminingthepropriety
Twitter:
ofanyspecificinformation,procedureortest,professionalsshouldapplytheir
www.twitter.com/ISACANews
ownprofessionaljudgmenttothespecificcircumstancespresentedbythe
particularsystemsorinformationtechnologyenvironment. LinkedIn:
www.linkd.in/ISACAOfficial
RESERVATION OF RIGHTS
Facebook:
www.facebook.com/ISACAHQ
©2018ISACA.Allrightsreserved.
Instagram:
www.instagram.com/isacanews/