Professional Documents
Culture Documents
1 1100 H 4 MITs Migrations To Hana Enterprise Cloud
1 1100 H 4 MITs Migrations To Hana Enterprise Cloud
KevinLyons,FrankQuern,GarryZacheiss
IS&T,MassachusettsInstituteofTechnology
Session:H4
1
2016MassachusettsInstituteofTechnology,IS&T
Agenda
Background:SAP@MIT
WhyHANA?
ProjectStructure&Milestones
MIT/HECArchitecture&Integration
Testing&GoLive
LessonsLearned
2
2016MassachusettsInstituteofTechnology,IS&T
Background:SAP@MIT
September6,1996thejourneybeginson3.1with
corefinancials
April2007upgradetoECC6.0
December2015migrationofECC6.0,Ehp7toHANAin
theHANAEnterpriseCloud(HEC)
Eachandeveryyear,supportpacksandenhancement
packs
3
2016MassachusettsInstituteofTechnology,IS&T
Background:SAP@MIT
Alongthewayweimplemented
Procurement,assetaccounting,project
systems,sales&distribution,funds
management,inventorymanagement,
plantmaintenance,EHS,HR/Benefits/Payroll
includingselfservice,training&events,enterpriselearning,
toomanytoincludeall
SAPweb,MITsfirstportal
integratedwithSAP
Atlas,MITslatestportalfor
communityfacingadministrativeapps
4
2016MassachusettsInstituteofTechnology,IS&T
Didwementionthehardware?
StartedonTru64onUnix
MigratedtoSolarisaround2004
MigratedtoLinuxin2010
AspartofLinuxmigrationwevirtualizedourSAP
environment
5
2016MassachusettsInstituteofTechnology,IS&T
SAPintegrateswithalot
6
2016MassachusettsInstituteofTechnology,IS&T
SAPintegrateswithalot
SciQuest forprocurementcatalogs
Nimbusforbudgetadministration
Kuali Coeus forresearchadministration
Bannerandothersforstudentadministration
Adonisforgiftrecording
Kronos,Concur,OpenText,BSI,thedatawarehouse
SAP@MITalsosendsandreceivesmanyfeedsof
financialandnonfinancialdatato/fromexternal
partnersviaourcustommiddleware,theSAPDropbox
7
2016MassachusettsInstituteofTechnology,IS&T
Background:Conclusion
AllofthismakesSAP@MITacomplexinterplay
ofprocesses,systems,andtools,somestandard
andsomehomegrown,allembeddedindayto
daylifeatMIT
Andwewantedto
moveittothecloud
8
2016MassachusettsInstituteofTechnology,IS&T
WhyHANA?
9
2016MassachusettsInstituteofTechnology,IS&T
HANADB Motivations
LayfoundationforfuturemigrationtoS/4.
LeverageHANADataServicestoimproveData
Warehousing&Reportingenvironment workin
progress.
EliminatecomplexETLprocess/batchextractfrom
ECC.
Improveperformanceforremainingbatchworkloads.
10
2016MassachusettsInstituteofTechnology,IS&T
HANAEnterpriseCloud Motivations
Spaceiseverything.
Reduceonpremisesadministrativecomputingfootprint.
Prioritizelimiteddatacentercapacityforresearchcomputing
workloads.
Increasedresiliency:bicoastaldisasterrecoverystrategy.
Primarysite:Sterling,VA
Secondary:SantaClara,CA
TransferstandardizedandofftheshelfworkfromMITtovendorpartner:
rackandstack,hardwaremaintenance.
OSinstallation.
EnablestafftofocusonMITspecific,differentiatedneeds.
Integrationwithcampusidentitysystems(AuthN,AuthZ)
11
2016MassachusettsInstituteofTechnology,IS&T
ProjectStructure&Milestones
12
2016MassachusettsInstituteofTechnology,IS&T
ProjectStructure
Planning
Workshops to define what it is we want to accomplish
Replace Oracle with HANA
Migrate our on premise data center to SAPs HEC service
Code
remediation
Cycles
Support Packs
Organization
Traditional PM
Scrum Framework
13
2016MassachusettsInstituteofTechnology,IS&T
CodeRemediation
14
2016MassachusettsInstituteofTechnology,IS&T
TheProjectCycles
Cycle2 MigrateonpremisesdevelopmentenvironmenttoSAPsHANA
CloudEnterprise(HEC)
Cycle 3 Repeatmigrationofonpremises developmentenvironmentto
HECbutincludeyearendsupportpacksintheprocess
Cycle4 MigrateQA/Testenvironment toHEC
Cycle7 Executeproductioncutover
15
2016MassachusettsInstituteofTechnology,IS&T
WhereHANAintersectswithSupportPacks
Support Packs
We all know what they are, periodic releases of system changes and
corrections by SAP.
Rather than apply these support packs when released we collect the
Support Packs for a single installation at year end in an effort to
minimize business interruption.
The timing of our process is driven by SAPs release of its year end
support pack, typically in mid-October, which contains the base level
necessary for the successful execution of SAPs delivered year end tax
reporting functionality (AP and Payroll) and stages the system for
regulatory changes for the upcoming calendar year.
Can we leverage the Support Pack testing window to limit the projects
impact on business operations?
16
2016MassachusettsInstituteofTechnology,IS&T
ProjectMilestones
Theproposedprojectschedulehadfourmajormilestones:
1. DecisionforDecember Go/NoGodecisionafterourPOC
(Cycle1).By9/8/2015makeadecisionifMITsSAP
landscapecanbemigratedtoHECby12/14/2015
2. SupportPackDecision Go/NoGodecisionby10/2/2015
(afterCycle2)whethertoapplySupportPacksinHECoron
premiseforproductionuseafterDecember
3. SuccessfulmockcutoverbeforetheThanksgivingbreak
4. GoLivedateof12/14/2015
17
2016MassachusettsInstituteofTechnology,IS&T
Timeline
18
2016MassachusettsInstituteofTechnology,IS&T
ProjectStructure&Milestones
Whatsinaname?
Lettheteamdecide
Greekthemed
Scopas
Daedalus
Procrustes
Cloudthemed
Nimbus
Cumulus
Cirrus
Projectname:HANAtoOptimize(H2O)
19
2016MassachusettsInstituteofTechnology,IS&T
ProjectCharter
PROJECT IDENTIFICATION
ProjectType
ProjectName (Implementation,Upgrade,Internal,other)
HANAtoOptimize(H2O)ECC
MIT TechnicalMigration
Migration
Vision
StandupafunctioningHANAERPenvironmentinHECfromwhichMITmaytestfunctionalityasit
appliestoMITslongtermvisionofproviding aplatformasaservice.
Mission
TheH2OECCMigrationProjectTeamisResponsibleandEmpoweredtoMigrateMITsdatato
SAPsHANAEnterpriseCloud(HEC).
GuidingPrinciples
Technicalmigrationonlywithnofunctionalityadded.
MigrateDevtoHECforGo/NoGodecisionforaDecemberGoLive.
EvaluateSupportPacks(SPs)forGo/NoGodecisiontodetermineifSPswillbeappliedtothe
migratedsysteminHEC.
MigrateMITsfour(4)tiersenvironmentto HEC.
20
2016MassachusettsInstituteofTechnology,IS&T
ProjectStructure
Executive Sponsors
Steering Committee
Project Management
Project Team
21
2016MassachusettsInstituteofTechnology,IS&T
ProjectStructure
22
2016MassachusettsInstituteofTechnology,IS&T
ProjectStructure
23
2016MassachusettsInstituteofTechnology,IS&T
Toolsofthetrade
Wiki
SAPJam
MSProject
Dropbox
Scrumboards
Burndowncharts
RT
24
2016MassachusettsInstituteofTechnology,IS&T
MIT/HECArchitecture&Integration
25
2016MassachusettsInstituteofTechnology,IS&T
Networking
Strategy:extendMITnet toHECdatacenters.
AssigneddedicatedIPv4addressblockstoHEC:
18.4.128.0/24,18.4.130.0/24.
Twophaseapproachtoconnectivity:
InitialconnectivityviaIPSec VPN:abletodeployrapidly.
Dedicatednetworkconnections/peeringtobothSterling
andSantaClara:2x10Gbps
InstalledroutingequipmentinHECdatacenters:MITnet
California
HECenvironmentsappearaspartofMITsinternalnetworkto
otherapplications.
26
2016MassachusettsInstituteofTechnology,IS&T
DNS
Delegated.hec.mit.edudomaintoHEC.
4HECoperatedDNSservers,2persite.
.hec.mit.eduzonecontainsCNAMErecordspointedatcurrently
activedatacenter:
Sterling .stl.hec.mit.edu
SantaClara .sac.hec.mit.edu
DNSindirectionfacilitatesdisasterrecoveryprocess.
Regionspecificnamesneverexposedtoendusersorother
systems.
27
2016MassachusettsInstituteofTechnology,IS&T
Authentication
MITauthenticatesSAPGUIusersusingKerberos/GSSAPIviaSNC.
HECapplicationserversprovisionedwithcredentialsfromATHENA.MIT.EDU
Kerberosrealm.
RequiredextensivecoordinationwithHECtechnicalstaff:
AdditionalOSpackagesinstalled.
ChangestoOSconfiguration(/etc/krb5.conf).
ChangestoSAPinstanceprofiles.
Keylearnings:
Beveryexplicitaboutyourrequirementsinthisarea.
Bepreparedtodotheheavylifting.
28
2016MassachusettsInstituteofTechnology,IS&T
AncillarySystems
Outboundemail HECsystemssendviaMITsprimarySMTPrelay:outgoing.mit.edu
RequiredcoordinationwithHECteamsonPostfixconfiguration.
OpenText ArchiveServer migratedtoHECasIaaS.
HECmanagedHANADBinstancefordatabasebackend.
BSITaxFactory migratedtoHECasIaaS.
TransitionedfromOracletoSybaseASE.
ExploringuseofSAPApplicationManagementServices(AMS)forSybaseDBA
work.
SAPDropbox(custommiddleware) migratedtoHECasIaaS.
ProvidessharedstorageviaNFStoHECapplicationservers.
Requiredmostmodificationsforuseinnewenvironment.
AllotherMITmanagedsystems(customwebportal,centralauthorizationsDB,Data
Warehouse,etc.)remainonpremises.
ChangeswerelimitedtoconfigurationupdatesfornewenvironmentDNSnames
/ports.
29
2016MassachusettsInstituteofTechnology,IS&T
Miscellany
SAPGUI modifiedinstallertodistributeconfigurationfilesvia
HTTPfromcentrallocation.
Backups IaaSsystemsarebackedupasthoughtheywereon
premise.
Allthatnetworkbandwidthcomesinhandy.
Monitoring SupplementHECmonitoringwithMITsexisting
enterprisemonitoring.
NoOS/shellaccesstoHECmanagedsystems.
Monitorsystemsatnetwork(ping/nping)andapplication
(HTTP/HTTPS)levels.
30
2016MassachusettsInstituteofTechnology,IS&T
Testing&GoLive
31
2016MassachusettsInstituteofTechnology,IS&T
TestingApproach
Test Plans
Each team will determine a comprehensive approach, based on
past experience managing the SAP year-end support pack
process and similar large-scale application updates, to ensure the
integrity of all critical applications as well as testing all integration
methodologies in use across the MIT system.
32
2016MassachusettsInstituteofTechnology,IS&T
TestingApproach
Test Cases
Test cases from ALM/QC Repository will be leveraged where possible
Automated test cases will be used in Cycles 3 and 4 (HP QTP)
33
2016MassachusettsInstituteofTechnology,IS&T
Testing Cycles1 4
34
2016MassachusettsInstituteofTechnology,IS&T
Testing WhereH2OintersectswithSupportPacks
(Cycles3&4)
35
2016MassachusettsInstituteofTechnology,IS&T
TestingResults
36
2016MassachusettsInstituteofTechnology,IS&T
TestingResults
InfrastructureRelated 54 17 71
User/DataError 1 8 9
Duplicate 4 2 6
ProductionIssue
tobeaddressedafterSP 1 7 8
Totals 80 55 135
37
2016MassachusettsInstituteofTechnology,IS&T
GoLiveDecision
38
2016MassachusettsInstituteofTechnology,IS&T
Cutover
1. Preparation Work
2. System Isolation
3. Migration/Support Pack Application
i. Core cutover tasks
ii. HEC tasks
iii. High Availability tasks
iv. Disaster Recovery tasks
v. MIT web deployment tasks
4. Post Upgrade Processes (validations)
5. Go-live
39
2016MassachusettsInstituteofTechnology,IS&T
HECCutoverPlan
Timing:
BeganuptimeprocessingonDecember8th inthemorning
Begansystemshutdownat4pmonFridayDecember11th
WehavecommunicatedwithsystemownersofBanner,CSAIL,iLab,andeShipGlobalthat
theirAPIconnectionstoSAPwillbeimpactedbythisdowntime
BeganexporttoHECat~6pmFridaynight
SAPhadthesystemuntilnoononSaturdaywhenweperformedsecurityandsystem
verifications
SAPthenhadthesystemagaintill8pmSaturdaynight
MITcompletedtransportsandPart1oftheactionlogfrom8pmto~2AM
MITandSAPtestedHighAvailabilityandDisasterrecoveryfailoverfrom2AMto11AM
MITthencompletedactionlogitemsandbegantheprocessofreconnectingSAPtoallits
ancillarysystems
IS&TBAsandDevelopersonsiteperformedtestsassystemscomeonlinefromnoonto2pm
Businessvalidationsbeganat2pmonSundayDecember13th
ThecalltoapproveGoLiveplannedfor5pm.
40
2016MassachusettsInstituteofTechnology,IS&T
EndResult
Success!
MITisintheCloud
41
2016MassachusettsInstituteofTechnology,IS&T
Howdiditgo?
Oh,HEC!
LessonsLearned
42
2016MassachusettsInstituteofTechnology,IS&T
LessonsLearned
Doyourprojectplanningearly
Sponsors,steeringcommittee,stakeholders,
communicationsplans,recurringmeetings,all(and
wemeanall)projectmanagementprocesses
Doitearly,youllbetoobusytodoitonthefly
Engagethirdpartyrepresentationearly
Includeintegratedproductpartnersespecially
Weworkedextensivelywiththirdpartieslike
BSI,OpenText,BofA
43
2016MassachusettsInstituteofTechnology,IS&T
LessonsLearned
Identifyandorganizeyourteams
Planforworktransitions,theyregoingtobebusy
Weorganizedaroundproductteams
HR/Benefits/Payroll(Spartans)
Finance/Logistics/EHS(Poseidon)
Atlasportalandotherwebapps(Globetrotters)
Systemsadministration(Tectonics)
Identifyotherpotentialparticipants
Usethemwhenandasneeded
Haveacommunicationsdepartment/team?Usethemto
youradvantage!
44
2016MassachusettsInstituteofTechnology,IS&T
LessonsLearned
Identifyyoursystemstestlist,assignanowner,andtrackthem
45
2016MassachusettsInstituteofTechnology,IS&T
LessonsLearned
Plantotestallcustomizations
Doanycoderemediationasearlyaspossible
Testexecution
Documentyourtestresultsforeachcycle
Itmakesstatusreportingmucheasier
FullintegrationtestingnolaterthanUAT
Wetestedsomeintegrationsasearlyasunittesting
Identifyexistingproductionissuesseparatelyfromprojectissues
Dontaddtothescopeofthemigrationinflight
Dotrueendtoendlandscapetesting
Scheduleenoughtimetoanalyzeandremediateintegrationissues
46
2016MassachusettsInstituteofTechnology,IS&T
LessonsLearned
MPLS
Getitgoingearly,ittookus~70daysfromorderto
completion
DNS
Agreeonthemandconfigureasearlyaspossible
Email
AgreeearlyonHECvs.onpremisesSMTPrelay
IdentifycustomizationsorpotentialissuesaroundsystemIDs,
hardcodedservernames,URLs&IPaddresses
Firewallconfiguration
47
2016MassachusettsInstituteofTechnology,IS&T
LessonsLearned
OSlevelchangeswillimpactyourabilitytoresolveissues
HANADBcanbehavedifferently
Indexes,sorts,orderinwhichdataisreturnedcanbe
differentthanyoureusedto
SAPadminswonthavethesameaccessasbefore
Systemmonitoring
Proactivemonitoringofsystemsasearlyaspossible
2unscheduledoutagesduringtesting
48
2016MassachusettsInstituteofTechnology,IS&T
LessonsLearned
Golive
Theresalottodo,planinshiftsifnecessary
Communicateearlyandoftentoeveryone
HaveaSWATteam,escalationplans,theworks
Embedresourceswiththebusiness,theyreyoureyesandearsduring
golive
Makesureyoushutdownyouronpremisessystems completely
Finally,remember,stuffhappens
FedExlostthedisccopyofthedatabase,dobothbackups
WehadSAPGui issuesongoliveduetocentral*ini filechange,notSAP
systemissues,seekoutallreferencestolocalsaplogon.inifilesearly
49
2016MassachusettsInstituteofTechnology,IS&T
Whatstartedlikethis
InJuly
50
2016MassachusettsInstituteofTechnology,IS&T
Turnedintothis
byOctober
Anditwasonlyhalfdone
51
2016MassachusettsInstituteofTechnology,IS&T
Bythenumbers
Thenumberofstandupmeetingsheldduringthecourseoftheprojectbythe42
coreteammembersacrossfourscrumteams:Globetrotters(whichhandled
400+ everythingonthefrontend),Tectonics(infrastructure),Poseidon(finance,
logistics,andplantmaintenance),and,yes,Spartans(HR,payroll,andelearning).
ThenumberofcyclesthattheSAPHANAprojectwasdividedinto.Thesevencycles(Sounds
mythical,saidSpartansteamlead)were:
Cycle1 ProofofConcept
Cycle2 MigrateonpremisedevelopmentenvironmenttoSAPsHANACloudEnterprise(HEC)
Cycle3 RepeatmigrationonpremisedevelopmentenvironmenttoHECbutincludeyearendsupportpacksintheprocess
7 Cycle4 MigrateQA/TestenvironmenttoHEC
Cycle5 ExecutemigrationofmockcutoverofproductionenvironmenttoHEC
Cycle6 Executedisasterrecoverytest
Cycle7 Executeproductioncutover
ThebusiesttimeoftheprojectwasCycle4,duringtestingandmockcutover.Thiswasthemost
brutaltimebecausewehadfivesystemsthathadtobeupgradedandtestedsimultaneously,IS&T
projectlead.
ThenumberofMITcommunitymembersusingSAPgui clientsoftwarewhichneededtobeupgraded
2,154 tov7.40beforetheywouldbeabletoconnecttoSAPHANA.Thisrequiredrepeatedremindersto
userstoupgradebeforethefinalcutoverwascompletedandtimeranout.
52
2016MassachusettsInstituteofTechnology,IS&T
Bythenumberscontinued
ThenumberofstepsintheplanforthefinalcutovertoSAPHANA,whichincludedmore
84 than100subtasks.
ThefinalproductionmigrationcommencedlateintheafternoonofFriday,December
11thandwascompletedintimeforthestartofbusinessthefollowingMonday,
exactlyasplanned.
Thenumberofissuesthathadbeenidentifiedbygolive,thankstoexhaustivetestingduringearlier
cycles,including1,649testcasesrunduringCycle4alone.80oftheseissuessurfacedduringunit
135 testing,while55wereidentifiedduringsystemsintegrationtesting.Atthetimetheprojectwent
live,allbut5(lowpriority)issueshadbeenresolved,whichhelpedexplainthesmoothtransition
duringthefinalcutover.
Wedidatremendousamountofduediligencethatluckilycaught
1.7 everythingthatwassignificantbeforewewentintocutover,PMsaid.
Theamountofdata,interabytes(TB),(storedinMIT'sOracleSAPdatabase)
thatweremigrated.
AfterthemigrationtoHANA,thosesamedataonly tookup.95TB.
ThetotaldurationoftheH2Oprojectindays,fromkickoffonJuly13thtocompletionofproduction
155 cutoveronDecember14th.Thatdoesn'tincludepostprojectrecoverytimeforteammembers.
53
2016MassachusettsInstituteofTechnology,IS&T
Thankyou
54
2016MassachusettsInstituteofTechnology,IS&T