Download as pdf or txt
Download as pdf or txt
You are on page 1of 211

IT Infrastructure Operations Support Services

(ITIOSS) O&M and DM&E Task Order


Performance Work Statement & Quality
Assurance Service Plan

1
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Table of Contents
1. INTRODUCTION ......................................................................................................................................... 8
2. BACKGROUND ........................................................................................................................................... 8
3. TECHNOLOGY OVERVIEW .......................................................................................................................... 9
4. CONTRACT OVERVIEW ............................................................................................................................ 12

4.1 Period of Performance ................................................................................................................ 12


4.2 Contract Transition ..................................................................................................................... 12
4.3 Operations and Maintenance ..................................................................................................... 12
4.4 Contract Phase Out ..................................................................................................................... 13
5. BUSINESS PROCESS SUPPORT .................................................................................................................. 14

5.1 Program and Process Management Support .............................................................................. 14


5.1.1 Program and Process Management .................................................................................... 14
5.1.1.1 Overview ......................................................................................................................... 14
5.1.1.2 Objectives........................................................................................................................ 14
5.1.1.3 Scope of Services Supported........................................................................................... 14
5.1.1.4 Requirements .................................................................................................................. 15
5.1.1.5 Documentation and Reporting ....................................................................................... 18
5.2 Transition .................................................................................................................................... 19
5.2.1 Overview ............................................................................................................................. 19
5.2.2 Objectives............................................................................................................................ 19
5.2.3 Scope of Services Supported Under Transition .................................................................. 20
5.2.4 Requirements ...................................................................................................................... 20
5.2.5 Transition Deliverables ....................................................................................................... 21
5.2.6 Transition Service Level Agreement ................................................................................... 22
5.3 General Requirements ................................................................................................................ 22
5.3.1 Location of Documents and Records .................................................................................. 23
5.3.2 Cost Control ........................................................................................................................ 23
5.3.3 Contact List.......................................................................................................................... 23
5.3.4 Enterprise Architecture Compliance ................................................................................... 23
2
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
5.3.5 Audit and POA&M Support ................................................................................................. 23
5.3.5.1 Scope of Services Supported........................................................................................... 23
5.3.5.2 Requirements .................................................................................................................. 24
5.3.6 Application/Solution Development and Release Support Requirements .......................... 24
5.3.7 Maintenance and Outages .................................................................................................. 24
5.3.8 Inspection by Government Agencies .................................................................................. 24
5.3.9 Third Party Vendor Organizations....................................................................................... 25
5.3.10 Processes, Procedures, and Work Instructions................................................................... 25
5.3.11 Application, Service and System Diagrams and Schematics ............................................... 25
5.4 Service Level Agreements (SLAs) and Metrics ............................................................................ 26
6. IT INFRASTRUCTURE TECHNICAL SERVICES .............................................................................................. 26

6.1 General Overview ....................................................................................................................... 26


6.1.1 Scope of Services Supported ............................................................................................... 26
6.1.2 Requirements ...................................................................................................................... 26
6.2 End-User Services........................................................................................................................ 28
6.2.1 IT Service Desk .................................................................................................................... 28
6.2.1.1 Scope of Services Supported........................................................................................... 28
6.2.1.2 Requirements .................................................................................................................. 28
6.2.2 Site Support......................................................................................................................... 31
6.2.2.1 Scope of Services Supported........................................................................................... 31
6.2.2.2 Requirements .................................................................................................................. 33
6.3 Data Center Services ................................................................................................................... 35
6.3.1 Windows Server Administration and Support .................................................................... 35
6.3.1.1 Scope of Services Supported........................................................................................... 35
6.3.1.2 Requirements .................................................................................................................. 39
6.3.2 Windows Desktop Administration and Support, Software Packaging, and Software
Deployment......................................................................................................................................... 42
6.3.2.1 Scope of Services Supported........................................................................................... 42
6.3.2.2 Requirements .................................................................................................................. 44
6.3.3 Enterprise Identity Management Administration and Support .......................................... 46
6.3.3.1 Scope of Services Supported........................................................................................... 46
3
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.3.3.2 Requirements .................................................................................................................. 49
6.3.4 Office 365 and Messaging Administration and Support ..................................................... 51
6.3.4.1 Scope of Services Supported........................................................................................... 51
6.3.4.2 Requirements .................................................................................................................. 52
6.3.5 UNIX/LINUX Administration and Support ........................................................................... 55
6.3.5.1 Scope of Services Supported........................................................................................... 55
6.3.5.2 Requirements .................................................................................................................. 57
6.3.6 Virtualization Platforms Administration and Support ........................................................ 60
6.3.6.1 Scope of Services Supported........................................................................................... 60
6.3.6.2 Requirements .................................................................................................................. 65
6.3.7 Storage and Backup Administration and Support............................................................... 66
6.3.7.1 Scope of Services Supported........................................................................................... 66
6.3.7.2 Requirements .................................................................................................................. 69
6.3.8 Database Administration and Support ............................................................................... 71
6.3.8.1 Scope of Services Supported........................................................................................... 71
6.3.8.2 Requirements .................................................................................................................. 73
6.3.9 Web and Application Middleware Administration and Support ........................................ 76
6.3.9.1 Requirements .................................................................................................................. 78
6.4 Voice, Video, and Network Infrastructure Operations ............................................................... 80
6.4.1 Network Infrastructure support.......................................................................................... 81
6.4.1.1 Scope of Services Supported........................................................................................... 81
6.4.1.2 Requirements .................................................................................................................. 83
6.4.2 Telephony Infrastructure Support ...................................................................................... 86
6.4.2.1 Scope of Services Supported........................................................................................... 86
6.4.2.2 Requirements .................................................................................................................. 89
6.4.3 Network Operations Center ................................................................................................ 92
6.4.3.1 Scope of Services Supported........................................................................................... 92
6.4.3.2 Requirements .................................................................................................................. 95
6.5 IT Service Management (ITSM) and Infrastructure Monitoring and Reporting.......................... 97
6.5.1 ITSM Tool Support............................................................................................................... 97
6.5.1.1 Scope of Services Supported........................................................................................... 97
4
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.5.1.2 Requirements .................................................................................................................. 99
6.5.2 IT Service Catalog Support ................................................................................................ 100
6.5.2.1 Scope of Services Supported ......................................................................................... 100
6.5.2.2 Requirements ................................................................................................................ 102
6.5.3 IT Infrastructure and Application Availability, Capacity, and Performance Monitoring ... 103
6.5.3.1 Scope of Services Supported......................................................................................... 103
6.5.3.2 Requirements ................................................................................................................ 104
6.5.4 ITIOD Reporting and Dashboarding .................................................................................. 105
6.5.4.1 Scope of Services Supported......................................................................................... 105
6.5.4.2 Requirements ................................................................................................................ 106
6.5.5 Major Incident Management ............................................................................................ 107
6.5.5.1 Scope of Services Supported ......................................................................................... 107
6.5.5.2 Requirements ................................................................................................................ 115
6.5.6 Problem Management ...................................................................................................... 116
6.5.6.1 Scope of Services Supported ......................................................................................... 116
6.5.6.2 Requirements ................................................................................................................ 137
6.5.7 Risk Management ............................................................................................................. 138
6.5.7.1 Scope of Services Supported......................................................................................... 138
6.5.7.2 Requirements ................................................................................................................ 140
6.5.8 Change Management support .......................................................................................... 141
6.5.8.1 Scope of Services Supported......................................................................................... 141
6.5.8.2 Requirements ................................................................................................................ 141
6.5.9 Asset Management and Inventory.................................................................................... 142
6.5.9.1 Scope of Services Supported......................................................................................... 142
6.5.9.2 Requirements ................................................................................................................ 143
6.5.10 Configuration Management .............................................................................................. 144
6.5.10.1 Scope of Services Supported ..................................................................................... 144
6.5.10.2 Requirements ............................................................................................................ 147
6.6 IT Security Services.................................................................................................................... 148
6.6.1 IT Security Tools Support and Cybersecurity Incident Response...................................... 148
6.6.1.1 Scope of Services Supported......................................................................................... 148
5
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.6.1.2 Requirements ................................................................................................................ 150
6.6.2 IT Vulnerability Scanning and Reactive Vulnerability Management coordination ........... 153
6.6.2.1 Scope of Services Supported......................................................................................... 153
6.6.2.2 Requirements ................................................................................................................ 155
6.6.3 IT Security Controls Support ............................................................................................. 157
6.6.3.1 Scope of Services Supported......................................................................................... 157
6.6.3.2 Requirements ................................................................................................................ 157
6.7 Test Center Operations ............................................................................................................. 158
6.7.1 Scope of Services Supported............................................................................................. 159
6.7.2 Requirements .................................................................................................................... 160
6.8 Development, Modernization and Enhancements ................................................................... 163
6.8.1 Scope of Services Supported............................................................................................. 163
6.8.2 Requirements .................................................................................................................... 164
6.9 Disaster Recovery/Continuity of Operations Planning (COOP) and Testing Coordination and
Execution............................................................................................................................................... 167
6.9.1 Disaster Recovery/Continuity of Operations Planning (COOP) and Testing Coordination
(FFP) 168
6.9.1.1 Scope of Services Supported......................................................................................... 168
6.9.1.2 Requirements ................................................................................................................ 169
6.9.2 Disaster Recovery/Continuity of Operations Planning (COOP) and Testing Execution
(CPAF) 170
6.9.2.1 Scope of Services Supported......................................................................................... 170
6.9.2.2 Requirements ................................................................................................................ 170
6.10 Cloud Integration and Support ................................................................................................. 171
6.10.1 Overview ........................................................................................................................... 171
6.10.2 Objective ........................................................................................................................... 172
6.10.3 Scope of Services Supported............................................................................................. 173
APPENDIX A - QASP ........................................................................................................................................... 175
APPENDIX B - DELIVERABLES.............................................................................................................................. 176
APPENDIX C - LIST OF REQUIRED MEETINGS AND REPORTS ............................................................................... 185
APPENDIX D - PBGC NETWORK OVERVIEW DIAGRAM ....................................................................................... 189

6
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
APPENDIX E - REFERENCES ................................................................................................................................. 190
APPENDIX F - PBGC LOCATIONS ......................................................................................................................... 193
APPENDIX G - IT SERVICE SUPPORT GUIDELINES ................................................................................................ 196
APPENDIX H - IT INFRASTRUCTURE MAINTENANCE SCHEDULE .......................................................................... 197
APPENDIX I - OIG IT INFRASTRUCTURE SUMMARY ............................................................................................ 198

Background ........................................................................................................................................... 198


IT Infrastructure Summary .................................................................................................................... 198
APPENDIX J - IT SERVICE AND SUPPORT 2018 STATISTICAL SUMMARY .............................................................. 200
APPENDIX K - IT INFRASTRUCTURE TOOLS LIST .................................................................................................. 203
APPENDIX L - IT INFRASTRUCTURE PROGRAM REGISTRY AND ITIOD ROADMAP ............................................... 208

7
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
1. Introduction
This performance work statement (PWS) is for the delivery of professional services to support the
PBGC’S Information Technology (IT) Infrastructure. The PBGC IT Infrastructure Operations Support
Services (ITIOSS) contract provides a wide range of IT professional services supporting a diverse
technological environment. The list of service domains includes end-user services; data center
operations; voice, video, and network infrastructure operations; IT service management; infrastructure
monitoring and reporting; cybersecurity services; test center operations; development, modernization
and enhancements (DM&E); and disaster recovery/continuity of operations planning and testing.
Program management is also necessary to oversee all contracted tasks for the service areas. Two areas
of specific interest are cloud and mobile computing. In addition to limited task-specific, business-area
driven cloud instances, PBGC has recently adopted cloud on an enterprise-wide basis for electronic mail,
collaboration, office automation, individual user data storage and shared, unstructured data storage as
well as IT service management. Mobile computing initiatives include the expansion of office automation
to PBGC’s iPhones and the introduction of Surface Pro 4 tablets and enterprise wireless capabilities
established in 2018. Cybersecurity is a particularly critical service domain and ITIOSS provides both IT
security operations support as well as IT security analysis, ISSO support and audit support. These
supporting services are implemented as an integral part of the solution throughout the entire
enterprise.

2. Background
The Pension Benefit Guaranty Corporation (PBGC) protects the retirement income of more than 40
million American workers in nearly 24,000 private-sector defined benefit pension plans. A defined
benefit plan provides a specified monthly benefit at retirement, often based on a combination of salary
and years of service. PBGC was created by the Employee Retirement Income Security Act (ERISA) of 1974
to encourage the continuation and maintenance of private-sector defined benefit pension plans, provide
timely and uninterrupted payment of pension benefits, and keep pension insurance premiums at a
minimum. PBGC is not funded by general tax revenues. PBGC collects insurance premiums from
employers that sponsor insured pension plans, earns money from investments and receives funds from
pension plans it takes over. A department within PBGC’s Office of Information Technology, the IT
Infrastructure Operations Department (ITIOD) provides the mission-critical IT foundation for computing
services used and procured by PBGC. PBGC cannot accomplish its mission without automated tools and
business software. All other PBGC departments depend on this one for their IT support. PBGC’s Common
Security Controls under the IT Infrastructure Services General Support Systems (ITISGSS) are also
provided by this department. ITIOD is comprised of approximately 40 PBGC FTE and approximately 150
contractors. ITIOD stakeholders include a dedicated PBGC staff of approximately 2,250 government and
contracted personnel, 26,000 pension plan sponsors who pay premiums into the PGBC and over 1.5
million trusteed participants who, combined, receive over $5 billion dollars in benefits payments.

8
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
3. Technology Overview
The following information is provided as a high-level overview of PBGC’s IT infrastructure environments
and technology. Greater detail is provided in the succeeding delineations of the various service
domains.

The PBGC currently operates and maintains three separate yet highly interrelated environments within
its IT Infrastructure. Additionally, ITIOD provides limited support to the PBGC Office of Inspector
General (OIG), as detailed in Appendix I - OIG IT Infrastructure Summary. These are logically separate
infrastructures dedicated for (1) development, (2) testing, (3) production. Each is described below:
• The Common Development Environment (CDE) provides an area for development and
interoperability testing that is isolated from production and other environments. The CDE
provides the capability to develop major applications and infrastructure solutions, which
support PBGC’s changing business needs. The environment allows PBGC to take advantage
of emerging technologies and to improve the exchange of data with internal and external
customers. The CDE is currently comprised of two separate areas (CDE-T and CDE-I):
 The CDE-T is the “Team-Specific” area, used for code development, COTS
configuration and unit testing of development efforts.
 The CDE-I is the “Interoperability” area, used for various phases of testing
(functional, system, integration, performance, and deployment) of future
integrated releases. The CDE-I is further broken down into two sub-areas – “As Is”
and “To Be.” The “As Is” area reflects the current production environment and
is used for testing against applications that currently exist in production. This area
may be used for supporting production, emergency fixes, and point releases.
Currently this area also serves as the training environment for some custom
developed PBGC applications. The “To Be” area reflects the future production
environment and is used for testing against the applications that will be deployed
in the next major software release and limited training purposes.
• The Integration and Testing Center (ITC) environment has been established for deployment
verification, development shakedown, system integration verification and, user acceptance
testing (UAT). Vulnerability scanning is also conducted for releases in the ITC prior to
production release. Training can be performed in ITC when needed. Currently, testing (and
training) in the ITC is accessed by physically going to the facility located at 1275 K Street, but
over the next 6-12 months this is expected to become a virtual test center and access to it
will be achieved entirely using remote desktop connectivity
• The Production environment (PROD) hosts business application and infrastructure
operations including replication of software and data to the COOP site with similar
hardware.
 The disaster recovery (DR) / COOP environment provides redundancy for the
production environment and facilitates the continuation of business activities in the
event of a disruption to the production environment. Failover is to PBGC’s servers at
Wilmington, DE, which have similar cache boxes and web switches with server
health monitoring and failover.
9
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
 The DMZ environment provides the infrastructure for public-facing applications and
services. The DMZ includes its own Active Directory and has a presence at both the
HQW and the COOP sites.
 PBGC also has several applications and infrastructure services that are largely
supported externally and hosted off site including cloud and non-cloud managed
services

The application deployment process is initiated in the CDE by the contract development team. Solutions
are then deployed to the ITC, based on a deployment guide prepared by the contract development
team. This ITC deployment is a joint effort between the contract development team and the IT
Infrastructure O&M team. The contract development team is responsible for the communication and
coordination effort associated with the CDE, ITC and production deployments.

PBGC has more than 2,500 physical Windows workstations (desktops, laptops, and tablets) and more
than 500 virtual desktops all of which run Windows 10. PBGC is moving towards portable hardware, e.g.
MS Surface Pro 4, Latitude 5290 2-in-1, etc. PBGC also has approximately 1,000 GFE iPhone 6s devices
managed using InTune and providing users with access to PBGC’s Microsoft Office 365 tenant including
email. PBGC plans to upgrade these phones to the iPhone 8 plus model by the end of the 2018 calendar
year. PBGC has established a significant SharePoint capability and Office 365 (Government Community
Cloud E3 plan) as an enterprise collaboration tool. The majority of PBGC’s unstructured data resides on
Office 365.

In support of all environments, ITIOD leverages a Hewlett-Packard (HP) blade server infrastructure, using
VMWare for most systems. ITIOD takes a “virtualize first” approach whenever possible. Current server
operating systems are a mix of Windows, Redhat Linux, and Solaris (physical Sun servers).

The PBGC applications and databases are a mixture of custom, COTS and highly customized COTS.
Further, they have been designed, deployed and supported by multiple contractor teams, contracted for
by multiple PBGC front-line business areas. PBGC primarily uses Oracle and MS SQL databases and is
considering shifting its primary relational database services to MS SQL Server and other Microsoft Azure
cloud database offerings, over a timeline yet to be determined. Furthermore, PBGC is considering
shifting its many of its application middle-tier services from Oracle WebLogic and IIS to Azure and
Dynamics cloud-based services, also over a timeline yet to be determined.

PBGC maintains Storage Area Network (SAN) infrastructure to provide high volume and high-
performance data storage. PBGC primarily uses Brocade SAN switches connecting Hitachi storage
arrays. PBGC also uses Veritas NetBackup and Oracle Recovery Manager (RMAN) for backup and
recovery services.

ITIOD supports PBGC business functions through a number of core business applications, both custom
and COTS, Microsoft Office 365 E3 including SharePoint online, and numerous supporting applications
for office productivity including word processing, spreadsheet, graphics, email, collaboration, browser,
communications connectivity, etc.

10
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Core business applications presently use a mix of Web-based thin client World Wide Web Consortium
(W3C)-compliant browser-based n-tier applications, client/server and service-based architectures. All
current EA-compliant development is service- oriented with browser-based user interfaces.

PBGC supports secure remote access to networked resources via VPN on GFE devices. For non-GFE or
devices, PBGC also offers an RDP web proxy service using Pulse Secure.

PBGC uses ServiceNow for IT Service Desk interactions, incident management, problem management,
change management, asset management, configuration management and discovery. HP Service
Manager 9 service request and service catalog software is used to manage service and access requests.
PBGC plans to migrate these capabilities to the Service Now platform by the end of September 2019.
SailPoint Identity IQ suite will also be leveraged to fulfill access requests for applications that are not
Active Directory-integrated or federated to the extent possible.

PBGC uses HP Business Service Management (BSM) and HP Business Availability Center (BAC), Sitescope,
Real User Monitor (RUM), and Network Node Manager (NNMi) for monitoring the availability and
performance of IT infrastructure systems and applications.

PBGC uses the Microsoft Project On-line (referred to in PBGC as P3M) and Oracle/Primavera Project,
Program and Portfolio Management (PPM) system as a central repository for tracking and reporting
project data. Enterprise Architecture (EA) information is in the custom- developed EA Repository (EAR).

PBGC has selected Azure as its default IaaS and PaaS CSP and expects to shift more and more services to
this over the next several years.

11
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
4. Contract Overview
In addition to other requirements, this contract will also include following objectives:

1. In addition to operations and maintenance of the existing environment, this contract will include
the infrastructure systems engineering and deployment services to perform development,
modernization and enhancement work.

2. The performance of the contractor will be measured and evaluated based on Service Level
Agreements (SLAs) comprised of a set of service level metrics with clearly defined acceptable
levels of quality (AQL), which will be reviewed and adjusted as required quarterly.

3. The contractor shall be required to continuously identify, recommend, achieve, and report on
measurable results for on-going operations and shall specifically provide before and after
measurements that can quantify achievement of development results.

4. Gradually, under a phased approach and following the ITIOD roadmap, the current capital-
intensive, government-owned, locally hosted infrastructure will be transitioned to alternative
service delivery models.

4.1 Period of Performance


The estimated period of performance is:

Base period: 11/1/2019 to 10/31/2020.

Option Periods: Nine (9) 12-month performance periods subject to exercise by PBGC

4.2 Contract Transition


Upon Government approval of the Transition Plan, the Contractor will assume responsibility for the
operations and maintenance of the PBGC IT infrastructure. This phase will incorporate consolidation
and streamlining of IT assets and various other activities that are deemed necessary by the PBGC and
the Contractor to begin services under the new contract. The overarching objective of this phase is a low
risk and low impact (to end-users) transition as the Contractor assumes responsibilities under the new
ITIOSS program. Contract transition activities will include those activities defined in the Transition
section.

4.3 Operations and Maintenance


This phase consists of the on-going operations and maintenance of the ITIOSS environment after
contract startup and transition activities have been completed. In this phase, the service objectives and
requirements as defined in the Performance Work Statement (PWS) will be used to guide performance
and ensure compliance with the agreed upon requirements. Service Level Agreements (SLAs) will take
effect and on-going performance monitoring will be in place.

12
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
4.4 Contract Phase Out
The Contractor shall provide transition phase-out support and various activities to transition support to
the Federal Government or a third-party service provider at contract end-of-life. Contract phase-out
activities will include those activities defined in the Contract Phase-out deliverable due 90 days before
the end of any awarded period of the contract.

13
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
5. Business Process Support
This section of the PWS outlines those services and requirements that relate to the ancillary yet critical
business process that support ITIOD’s core mission, the provision of IT infrastructure services and tools.

5.1 Program and Process Management Support


The sections below describe the overview, objectives, scope, requirements, deliverables and SLAs of the
Program and Process Management Support services that the Contractor shall provide under this
solicitation.

5.1.1 Program and Process Management


5.1.1.1 Overview
The Contractor shall provide sufficient management to ensure that activities on the contract are
performed efficiently, accurately, on time, and in compliance with requirements of the contract.
Specifically, the Contractor shall designate key personnel to supervise staff assigned to this contract.
The Contractor shall ensure that appropriate performance reports are submitted outlining progress,
status, and any problems/issues encountered in the performance of the contract. The Contractor shall
require all subcontractors to furnish performance report data where there are critical or significant tasks
related to the prime contract.

5.1.1.2 Objectives
The contractor shall consider the following objectives when providing program and process
management services:

1) Ensure all IT infrastructure support services are provided timely, accurately, and in a quality
manner, in full compliance with all service level objectives and metrics
2) Cost of services are controlled, and the total cost of infrastructure asset ownership is lowered
on an annual basis via introduction of operational efficiencies
3) Development, modernization and enhancement projects are executed within initial cost and
schedule baselines, and are fully responsive to documented user and system requirements
4) Provide transparency to customers and end-users on service cost, performance, and satisfaction

5.1.1.3 Scope of Services Supported


The Contractor will perform program management to include overall management, planning,
accounting, tracking, reporting, and administrative support for ITIOSS. The Contractor's program
management staff will work in cooperation with PBGC management to ensure processes and practices
are defined, implemented, and evaluated on a regular basis in line with industry standard project
management and service delivery models. All program and process management methodologies
implemented and used by the Contractor shall be in compliance with the PBGC ITSLCM.

14
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
5.1.1.4 Requirements
The Contractor shall perform the following services for the Program and Process Management services
under this solicitation:

Reference Requirement
(PM-PM-xx)
PM-PM-01 Contractor shall be responsible for all technical training of Contractor staff, unless
otherwise directed by the Government. The Contractor shall provide technical
staffing proficient in the tools and technologies utilized, supported, planned, and
targeted under this contract. The contractor shall develop a set of experience and
skills required for each role, position, and function on the contract. The set of
experience and skills required for each role, position, and function on the contract
shall be reviewed and approved/accepted by the government prior to the experience
and skill definition to be considered as official for the services, roles, functions, and
support provided and performed on the contract. Additionally, in those cases where
hardware or software vendors require certified technicians to interact with their
products, the Contractor shall ensure that personnel are appropriately certified.
Appropriate certification documents shall be submitted to COR.

The Contractor shall establish a Technical Skill and Competency Maintenance


Framework and Plan for:
• Equipping contractor staff with requested technology and related skills and
competencies
• Identifying current technology, technology changing, and technology
standards
• Addressing deficiencies in skills and competencies in contractor staff
• Managing technical risks to the contract due to insufficient technical skills and
technical staff
• Addressing issues reported by the government with regards to technical skill
deficiencies and technology deficiencies
PM-PM-02 Contractor shall attend a Contract kickoff meeting within five (5) calendar days after
award. At this meeting the Contractor shall solicit government comments on all draft
deliverables submitted as part of the proposal and discuss any other contractual
and/or programmatic topics.
PM-PM-03 Contractor shall develop a Program Management Plan (PMP) with Government
approval that will be used to manage, track and evaluate the Contractor’s
performance. The Program Management Plan shall consist of risk mitigation, control
policies and procedures in accordance with standard industry practices for project
administration, execution and tracking. The Program Management Plan (PMP) shall
be presented as part of the original proposal. Modifications to the original PMP shall
be made after the initial project meetings, timelines, and tasks are finalized. This
revised PMP shall be considered the first deliverable and will serve as a baseline for
the program. The revised PMP shall be due thirty (30) calendar days after the
Contract Kick-off meeting. Updates or additions to specific tasks that affect the scope
15
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
of the plan shall be brought to the attention of the COR or Program Manager for
ITIOSS. The Contractor shall prepare and maintain the Integrated Master Schedule,
track Schedule Variance, and provide on-going updates to the Government. In
addition, the Contractor shall lead various meetings with key project stakeholders to
review performance.
PM-PM-04 Contractor shall report all identified risks whether under contractor or Government
control using PMI Risk Management standards. The Contractor will track and mitigate
risks associated with activities under the Contractor’s control and risks that the
Government directs the Contractor to monitor. The Contractor will define and
document a risk management approach for identifying, assessing (i.e., determining
the impact), mitigating, and monitoring risks and risk response actions no later than
the end of the Transition phase. The Risk Management Plan will be included in the
Contractor’s PMP.
PM-PM-05 Contractor shall evaluate, recommend and support a Government approved change
management approach that describes the policies, processes, products, roles and
responsibilities, reviews, and reports necessary for controlling and managing
proposed changes. The change management approach will include or address the
following content: requirements management, infrastructure management,
configuration management, incident management, and contract management. The
change management approach will integrate with the PBGC system life cycle
methodology (ITSLCM) and the Contractor’s management decision-making processes.
During Transition, the Contractor will observe, interface with, and work with the
incumbent and the Government to gain in-depth knowledge of the ITIOSS Program
change management processes and requirements. The Contractor’s Change
Management Plan will be based on the Contractor’s change management approach
and tailored to reflect the Contractor’s in-depth knowledge of the ITIOSS Program
change management processes and requirements. The approach shall include use of
supporting automated systems (tools) already in existence at PBGC. The Contractor’s
Change Management Plan will establish the Contractor’s processes, work products
and deliverables for documenting and controlling changes to the ITIOSS Program. The
change control process will consider risk, cost, schedule and quality impacts. The
Change Management Plan will be included in the Contractor’s PMP.
PM-PM-06 Contractor shall comply with PBGC’s ITSLCM and PMI best practices. The Contractor’s
project management shall provide the planning, direction, coordination, and control
necessary for effective and efficient accomplishment of all requirements contained in
the contract. This effort will be consistent with the base contract, referenced
documents, and contractor-developed and Government-approved plans, schedules,
and milestones. The Contractor shall include the use of PBGC’s automated project
management system as part of the Program Management Plan and all project
management activities. The Contractor shall ensure that all work performed under
this task is recorded in PBGC’s automated project management system, currently
P3M.
PM-PM-07 All Project Managers assigned by the Contractor to this contract shall be Project
Management Institute (PMI) certified
16
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PM-PM-08 As appropriate for each document, the Contractor will include the Government in the
conceptual planning of documents (e.g., purpose, objectives, audience, preliminary
outline, content, etc.), prepare draft versions of documents, seek and incorporate
Government’s comments on all versions of the document, and produce, disseminate,
and track the initial release and subsequent versions of the document. The
Contractor shall work collaboratively with the Government to ensure that all
deliverables and work products meet Government requirements.

The Contractor shall:


• Support, adhere to and make recommendations for improvement on the
PBGC Project Management Methodology in accordance with PMI and other
industry standards.
• Develop and maintain project management plans for each project in
accordance with PMI Project Management Methodology standards and PBGC
ITSLCM.
PM-PM-09 For that work, such as Development, modernization and enhancement, where EVM is
appropriate, the Contractor’s project management approach and methodology shall
incorporate Earned Value Management (EVM) through the use of PBGC’s Earned
Value Management System (EVMS), which currently is the P3M
PM-PM-10 Frameworks and quality models such as ISO9000, ISO20000 and the IT Infrastructure
Library (ITIL), provide a blueprint and a road map for improving processes and
procedures. Each framework and quality model have specific strengths in helping
meet business goals including the potential for cost reductions, increased customer
satisfaction and greater productivity. The Contractor shall assess PBGC’s current
service delivery maturity level and develop a plan to improve the PBGC IT service
delivery process to a level and scope approved by the Government. The Contractor
shall deliver the PBGC IT Infrastructure Service Delivery Maturity Plan by the end of
the base period. The Contractor-proposed Service Delivery Maturity Plan shall be
aligned with, but not be bound to, the IT Infrastructure Program Registry and
Roadmap and deliverables. Upon approval of the Service Delivery Maturity Plan, the
Contractor shall implement the Plan within the scope and schedule contained therein.
PM-PM-11 Acquisition Management services are the activities associated with the pricing,
evaluation (technical and costing), selection, acquisition assistance, and ongoing
management of new and upgraded IT assets (e.g., hardware, software, circuits, etc.).
The Contractor shall provide the appropriate acquisition assistance and tracking
procedures to support PBGC requirements including, but shall not be limited to:

• Integration with the PBGC Asset Management, Configuration Management,


and Service Desk processes.
• Installation and configuration of acquired components within established
SLAs.
• Streamlined proposal process for non-standard PBGC hardware, software, or
services

17
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PM-PM-12 PBGC considers effective and productive communications essential to a collaborative
relationship and to the success of the ITIOSS Program. Expectations on technical, cost,
schedule, performance and progress towards established expectations (or variances
from), must be communicated clearly and without ambiguity or delay. To this end,
the Contractor shall establish and maintain effective communication with the
Government. The Contractor will structure its ITIOSS support in a manner that
ensures that the Contractor’s goals and objectives are aligned with those of the
Government and reflect the attributes of a partnership through an open, customer-
oriented effort.

The Contractor shall communicate with the Government during all phases of the
contract and, at a minimum, take steps to understand the Government’s business and
technical issues, provide insight into issues and problems, recommend solutions for
issues, and recommend actions to maintain cost, schedule, quality, and technical
baselines. The Contractor shall have effective communications processes and plans to
identify, prepare, review, incorporate review comments, disseminate, and track
appropriate communication items (i.e., deliverables, work products, correspondence,
etc.) for the ITIOSS Program.

5.1.1.5 Documentation and Reporting


The Contractor shall be responsive to the development of reports and other document deliverables on a
regular and ad-hoc basis to assist in oversight, IT governance, performance monitoring, and quality
assurance/control of activities that impact the PBGC IT operations. Reports shall be made available
online to PBGC staff, as needed, in agreed to formats and timeframes as outlined in the RFP.

Documentation services are the activities associated with developing, revising, maintaining,
reproducing, and distributing up-to-date information in hard copy and/or electronic form across all
service areas. These documents shall be collectively referenced as the ITIOSS Standards, Procedures, and
Concept of Operations (CONOPS). The Contractor shall at a minimum:

• Provide documentation in agreed format to support activities throughout the life cycle of
services as specified in each service area.
• Follow PBGC’s Record Management policies and procedures.
• Maintain a system to centrally store, organize and distribute documentation.
• Document system specifications and configurations, e.g. interconnection topology,
configurations, and network diagrams).
• Document standard operating procedures (e.g., service desk, datacenter, network, boot,
failover, spool management, batch processing, backup, etc.).
• Document procedures, production and maintenance schedules, and job schedules, according to
PBGC policies and industry requirements.
• Document Meeting Minutes/Agendas: This document will be used to capture meeting agendas,
attendees, meeting highlights, meeting outcomes and meeting action items.

18
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Develop, maintain, and update all documentation on a regular basis

Refer to Appendix B – Deliverables and Appendix C- List of Required Meetings and Reports

5.2 Transition
The sections below describe the overview, objectives, scope, requirements, deliverables and SLAs of the
Transition services that the Contractor shall provide under this solicitation.

5.2.1 Overview
The Contractor shall submit a comprehensive Transition Plan in accordance with the PWS that shall
contain a detailed description of tasks, resources, schedule, assumptions, dependencies, risks and risk
mitigation plans associated with the transition.

Upon Government approval of the Transition Plan, the Contractor will accept and sustain ITIOSS
Services. This task involves the transition of the existing environment to an environment that will
support the core service areas as defined below. This transition phase will incorporate consolidation and
streamlining of IT assets and various other activities that are deemed necessary by the PBGC and the
Awarded Contractor to begin services under the new contract. This transition will involve the effort to
set up the needed infrastructure (if applicable) and installation and implementation of management
tools and agents, staffing plans and other preliminary activities needed to prepare for the start of the
contract. The overarching objective of this phase is a low risk and low impact (to end-users) transition as
the Awarded Contractor assumes responsibilities under the ITIOSS program. Additionally, the Contractor
shall specifically prepare for transition to an external, “rural-sourced” Service Desk akin to PBGC’s
current model or an external Service Desk of another model.

5.2.2 Objectives
The Contractor shall perform all services, tasks, and any other support activities required to transition
from the current version of the ITIOSS contract. The existing services under this contract are vital to
PBGC’s mission and must continue without interruption. The Contractor agrees to exercise best efforts
and cooperation to affect an orderly and efficient transition phase after award.

PBGC has identified the following key service objectives for transition in the table below:

Transition Objectives Description


Minimize End-User Concerns Promote the benefits of the contract to the PBGC user
and Impact community, and ensure end-users receive information about the
transition and the new services. Ensure end-user functionality
and service will not diminish and that the end-user’s experience
with transition will be as seamless as possible (low risk and low
impact).
Integrity & Accuracy Demonstrate end-to-end integrity and accuracy in conversion
approaches prior to execution. Leverage best practices and tools
to migrate cleanly assets, data, resources, and services to the new
contract.

19
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Adaptive Communication Adapt the Contractor’s transition approach to PBGC requirements
through the establishment of an adaptive communication strategy
and plan proper execution.
Institute Partnership Define and establish multi-level review and assessment meetings
Expectations and Collaboration with PBGC leadership to collaborate and ensure clear customer
expectations inclusive of published information, schedules, and
progress reports.

5.2.3 Scope of Services Supported Under Transition


The Contractor shall provide all required personnel, management oversight, tools, processes, and other
necessary resources to support fully support the transition from the current contractor. The Contractor
will provide support in all Services areas identified in the PWS.

5.2.4 Requirements
A startup transition period not to exceed sixty (60) days shall be required after final contract award for
the Contractor to conduct transition services. During the transition period, the Contractor shall
implement all activities necessary to establish a stable environment, where the Contractor assumes
operational control under PBGC’s Infrastructure Operations Department (ITIOD). Phase-in activities
include the finalization and formal adoption of ITIOSS contract SLAs, coordination and activation of the
business systems supporting the new ITIOSS environment and presentation of a Transition Readiness
Review (TRR) in which specific readiness criteria and status are presented. The Contractor’s transition
readiness includes satisfaction of a number of specific criteria including the establishment of:

1) Detailed transition plan, process and schedules coordinated with PBGC and incumbent
contractors for providing an orderly transition with following objectives:
a. Minimize the impacts on continuity of operations
b. Maintain communication with staff and affected communities
c. Identify key issues
d. Overcome barriers to transition
e. Perform due diligence to ensure that all transition activities are identified, negotiated
and completed during the Transition
f. Establish a transition management team capable of providing overall management and
logistical support of all transition activities
2) Contractor transition management team leaders on-board immediately after the award.
3) Required operational interface agreements (e.g., OLAs, Memoranda of Agreement (MOA)) with
incumbents prepared, reviewed, and signed.
4) Appropriate subcontractor and supplier agreements in place.
5) Qualified personnel identified, processed, and available at specific incremental turnover dates.
6) Applicable contractor management systems and support tools in place.
7) Status reporting requirements established, including applicable performance measurement
reporting requirements

20
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
8) PBGC and contractor agreement on existing systems, plans, procedures, forms, and instructions
to be used after transition.
9) Institutionalized knowledge of PBGC operations and technical requirements.
10) An appropriately sized external, “rural-sourced” Service Desk akin to PBGC’s current model or an
external Service Desk of another model

5.2.5 Transition Deliverables


Under this contract, the following Transition deliverables are required:

Deliverable Name Description/Details


Transition Plan Phasing in a new contract involves a number of activities. The transition
plan provides a method of identifying and tracking progress on each of the
activities associated with transition. As required in Sections L and M of the
Solicitation, the Contractor shall provide its high-level approach for
transition (Draft Transition Plan) with its proposal. After contract award,
the Contractor shall collaborate with the Government to further develop,
deliver, and implement a final transition plan. The transition plan shall, at
minimum, include following requirements. The Contractor may include
additional requirements for improving the transition plan.

Transition Plan shall include:


• Identifying functions, tasks, systems, and facilities required to
establish transition priorities and sequence.
• Determining specific functions and tasks that constitute “clean”
transition increments, resulting in clear contractor accountability.
• Aligning and integrating the schedules across teams to preclude any
impacts to primary mission requirements.
• Clearly defining and documenting each party’s responsibilities.
• Developing definitive transition readiness criteria for review with
PBGC.
• Establishing new contractor support systems and applications.
• Providing tasks, data, and schedules to PBGC that can be used for
issuing applicable direction to the incumbent contractors for phase-
down, coordination, and funding considerations.
• Providing end-user orientation and communication on services and
support procedures.
• Developing procedures, reference manuals, and operational guides
to support on-going operations and other pertinent information
needed to properly takeover services.
• Importing of historical data to contractor and/or Government
systems (if applicable).
• Implementing a detailed communication plan.
• Transferring hardware, hardware warranties, and software licenses
(if applicable).
21
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Transferring all necessary business and/or technical
documentation.
• Training to introduce Government customers, programs, and
technical staff to the tools, methodologies, and technical processes.
• Implementing new procedures and processes.
• Completing all required deliverables and SLA definitions.
• Providing practical approach/process for high retention of the
current staff
• All relevant information pertaining to the establishment of and
transition to a Service Desk capability not located on PBGC’s
premises.
• Obtain system passwords and change all passwords at the end of
transition. Submit all new passwords in the repositories maintained
by PBGC.
Transition Plan Review No later than 5 calendar days after award, the Contractor shall attend a
and Approval post-award conference with the CO, COR, other representatives of PBGC
and the incumbent contractor to discuss a plan for orderly transition from
the incumbent contractor to the new contractor to assume control and
administration of services. The Contractor shall amend/update the draft
transition plan (submitted with its proposal) based on the results of the
post-award conference and discovery activities, no later than 3 business
days after post award conference. The COR will review and approve the
amended transition plan no later than 3 business days after its submission
by the Contractor.
Awarded Contractor At the end of the period of performance for the awarded contract, the
Transition Out Plan Contractor shall assist the Government with planning and implementing a
complete transition of its current duties to the incoming support provider.
This shall include formal coordination with the Government and incoming
provider staff and management and shall include the delivery of copies of
existing policies, procedures, documentation, hardware (if applicable),
software (if applicable) and required metrics and statistics. As part of this
RFP response, the Contractor shall provide an Outgoing Transition Plan. The
COR will notify the contractor when the transition period is scheduled to
begin.

5.2.6 Transition Service Level Agreement


The Contractor shall propose a set of SLAs for transition period. These SLAs shall be aligned with
transition requirements, transition readiness and transition objectives identified in this section.

5.3 General Requirements


The Contractor shall meet the following set of General Requirements under this solicitation.

22
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
5.3.1 Location of Documents and Records
The Contractor shall publish all deliverables, reports, SOPs, Work Instructions and other required
documents on PBGC’s instance of SharePoint on-line according to applicable ITIOD processes and
procedures.

5.3.2 Cost Control


The Contractor shall use the PBGC project management tool, P3M, for cost control and cost
management and promote accurate program IT cost accounting, cost reporting and budgeting, and
facilitate cost-effective service provision.

5.3.3 Contact List


On no less than a monthly basis, the Contractor shall update the contact list, maintained on PBGC’s
instance of SharePoint on-line, of personnel to call in the event the Government needs to make contact
regarding the status of incident resolution and notify the COR upon completion. The list shall be
updated no later than the 5th business day of each month and include the name, telephone number,
email address, cellular telephone number and type of incidents or service area for which the person has
responsibility.

5.3.4 Enterprise Architecture Compliance


The Contractor shall adhere to and comply with the PBGC’s Enterprise Target Architecture (ETA) and EA
Standards. The Contractor shall also adhere to process and procedures for PBGC’s Technical Review
Board (TRB), (copy will be available in reading room) Product/Technology Selection and Project Reviews
according to the ITSLCM.

5.3.5 Audit and POA&M Support


5.3.5.1 Scope of Services Supported
The Contractor shall participate in audits of the PBGC systems. Participation consists of making available
operating procedures and controls, evidence of conformance with procedures and controls, execution of
all scripts to identify potential infrastructure vulnerabilities, and extracting data for analysis by the
auditors. Requests will generally have an assigned federal team lead who will provide the Contractor
with any clarifications required. The overall effort will be led by ITIOD’s Security Program Management
Office federal lead who will provide the Contractor with all information regarding the adequate
performance and workload requirements to perform all audit activities. Clarifications regarding any
issues during the audit shall be raised to the ITIOD Security Program Management Office federal lead.
The Contractor shall not send any official correspondence or transmit information relating to any audits
or Congressional inquiries without federal approval.

The Contractor shall participate in efforts to address IT security control deficiencies related to
infrastructure services the Contractor provides or systems the Contractor maintains, tracked through a
Plans of Action and Milestones (POA&Ms). The IT security control deficiencies may be identified
through an audit, control assessment, or by ITIOD staff during the course of normal operations or
internal control review. Participation consists of attending meetings, reviewing control requirements,
identifying gaps, identifying steps to address control deficiencies, taking steps to address control
23
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
deficiencies, and producing artifacts demonstrating the proper functioning of an IT security control.
POA&Ms will generally have an assigned federal team lead who will provide the Contractor with any
clarifications required. The overall effort will be led by ITIOD’s Security Program Management Office.

During the FY18 audit cycle, ITIOD received and process more than 240 data and meeting requests from
the auditors. In calendar year 2018, ITIOD worked on a total of 9 POA&Ms, completing and closing 2
POA&Ms and 63 POA&M milestones covering control families such as: Access Control, Configuration
Management, Identification and Authentication, Risk Assessment, System and Services Acquisition.

5.3.5.2 Requirements
The Contractor shall perform the following services for the Audit and POA&M Support services under
this solicitation:

Reference Requirement
(PM-AP-xx)
PM-AP-01 Contractor shall provide timely support for security audit-related data calls, reporting,
and presentations. Contractor will general have 1 week to complete an assigned
request.
PM-AP-02 Contractor shall provide support for POA&Ms and POA&M milestones including
attending meetings, reviewing control requirements, identifying gaps, identifying
steps to address control deficiencies, taking steps to address control deficiencies, and
producing artifacts demonstrating the proper functioning of an IT security control.
PM-AP-03 Contractor shall complete assigned POA&M milestones on time. See SLA section for
details on existing SLA measures, which define acceptable quality levels.

5.3.6 Application/Solution Development and Release Support Requirements


The Contractor shall participate in application/solution development Integrated Project Teams (IPT) as
needed providing feedback on information technology security, technical feasibility, compliance with IT
infrastructure standards and quality of project artifacts. This also extends to Infrastructure development
IPT. Infrastructure IPTs will include UAT Use Case building and testing activities. This will result in end
user acceptance of systems before infrastructure technology is approved for the production
environment.

5.3.7 Maintenance and Outages


The Contractor shall perform maintenance and other related activities that degrade or may degrade the
performance of PBGC’s computing environments, operating systems, databases and applications during
scheduled maintenance periods as outlined in Appendix H - IT Infrastructure Maintenance Schedule.

5.3.8 Inspection by Government Agencies


The Contractor shall provide access to and cooperate with Government personnel conducting official
inspections and surveys. Inspections will be made by agency representatives from, but shall not be
limited to, Property Inspectors, the Inspector General (IG), the Office of the Director, other offices in
PBGC such as the Workplace Solutions Department, as well as other Federal Government agencies such

24
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
as the Occupational Safety and Health Administration (OSHA), Environmental Protection Agency (EPA),
Government Accountability Office (GAO), and General Services Administration (GSA). For all
correspondence assistance relating to any audits and all Congressional inquiries, the Contractor shall not
actually draft any response. Input may be provided but the drafting of such responses will be done by
the Government. Additionally, the Contractor shall only communicate with other government agencies
through the COR.

5.3.9 Third Party Vendor Organizations


In some instances, the Contractor will need to interface with other contractor organizations performing
services in support of PBGC. Some examples include contractors performing specialized, one-time
projects and vendors providing software development. The Government will facilitate the initial contact,
as required, between the ITIOSS Contractor and these other contractor organizations. The Contractor
shall attempt to resolve any issues that may arise with other contractor organizations and shall verbally
notify the COR of unresolved issues in receiving support from or providing support to customers or
other contractors within two working hours from the occurrence of a dispute over any issue. The
Contractor shall notify the COR in writing of any unresolved issues within one business day. The terms of
this Contract shall govern the Contractor’s interaction with Third Party Vendor Organizations, and the
Contractor is not authorized to take actions not otherwise provided for under this Contract.

5.3.10 Processes, Procedures, and Work Instructions


The Contractor shall, within 60 calendar days after award of the Contract, review ITIOD processes,
procedures, and work instructions (WIs) for each of the Service Areas identified in this PWS. The
Contractor shall thoroughly review these documents with the contractor team responsible for its
operation, acceptance, and/or maintenance. Additionally, the Contractor shall receive, review,
comment upon and use operations documentation provided by other PBGC entities, such as applications
development groups, and other OIT functional teams to assist in the operation and maintenance of all
service areas identified in this PWS. The COR and PBGC technical subject matter experts will review the
processes, procedures, and WIs and provide written comments or approval to the Contractor within 10
business days following submission for federal review. The Contractor shall address Government
comments and deliver the final processes, procedures, and WIs within 5 business days. All the
processes, procedures, and WIs shall be the property of PBGC. The Contractor shall not include the
Contractor’s logo, name or Contractor’s disclosure to any deliverable document.

The Contractor shall maintain all processes, procedures, and WIs throughout the life of the Contract
including review, verification and update (if needed) no less than annually. The contractor will adhere to
the governance processes for processes, procedures, and WIs including change notification and federal
approval. The Contractor shall notify the COR at least 30 days in advance of any changes that may affect
contract cost and not implement these changes until receiving approval from the COR.

5.3.11 Application, Service and System Diagrams and Schematics


The Contractor shall maintain all relevant artifacts in support of the Operations and Maintenance of
PBGC’s infrastructure. This should include, but shall not be limited to: diagrams, schematics and
documents. Those artifacts that are the responsibility of application development will be provided by
25
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
the Government. The frequency of the updates of these artifacts can be found in the corresponding
service area requirement sections.

5.4 Service Level Agreements (SLAs) and Metrics


Refer to Attachment A- QASP for list all SLAs for this PWS.

6. IT Infrastructure Technical Services


6.1 General Overview
6.1.1 Scope of Services Supported
PBGC operates and maintains an enterprise network that supports logical access for approximately
2,250 users and provides connectivity between its Headquarters in Washington and several remote sites
including: the Kingstowne, VA alternate work site and call center site, the Wilmington, DE Continuity of
Operations (COOP) site, Field Benefit Administrator (FBA) and Plan Valuation Administration (PVA) sites,
and Actuarial sites, and to the State Street Corporation, PBGC’s current benefit paying agent.

This section and the sections that follow describe the scope and requirements that the Contractor shall
provide under this solicitation which include:

• End-User Services
• Data Center Services
• Voice, Video, and Network Infrastructure Operations
• IT Service Management (ITSM) and Infrastructure Monitoring and Reporting
• Security Services
• Test Center Operations
• Development, Modernization and Enhancements
• Disaster Recovery/Continuity of Operations Planning (COOP) and Testing
• Cloud Integration and Support

See Appendix J – IT Service and Support 2018 Statistical Summary for IT Service Desk interactions, tier 2
incidents, requests for information (RFIs), service/access requests, requests for change (RFCs), and RFC
tasks processed in calendar year 2018 for these services.

Additionally, PBGC provides limited but foundational support to PBGC’s Office of the Inspector General
(OIG). Complete details of that support can be found in Appendix I - OIG IT Infrastructure Summary.

6.1.2 Requirements
The contractor shall comply with all general requirements outlined in the following table:

Reference Requirement
(General-xx)

26
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
GENERAL-01 Contractor shall be in full compliance with and support the enforcement of all Federal
and PBGC policies and security controls
GENERAL-02 Contractor shall ensure that all work performed meets or exceeds acceptable quality
levels defined and shall provide evidence of having attained those levels of service.
See SLA section for details on existing SLA measures, which define acceptable quality
levels.
GENERAL-03 Contractor shall regularly sample, review, and document Information Technology
Service Management (ITSM) interaction, incident, and change quality levels and take
appropriate action, e.g. training, to rectify any deficiencies in ticket quality and
service delivery including failure to meet target performance metrics
GENERAL-04 Contractor shall ensure knowledge base articles (KBAs) or work instructions (WIs) are
established and maintained for common activities associated with interactions,
incidents, and changes and these are updated/certified no less than annually
including federal approval
GENERAL-05 Contractor shall fully implement, manage, and support all activities regarding incident
management according to PBGC policies. By using ITIL best practices, this should
include, but shall not be limited to:

• Ensure all IT operations-assigned incidents are addressed daily


• Ensure a high level of customer service
• Escalate and assign incidents requiring resolution by business application
support teams to the designated application support team provided by
separate contracts
• Escalate all unresolved incidents to the contractor and PBGC management
• Develop new, update existing, administer and report on customer service
surveys with PBGC approval.

See SLA section for details on associated SLA measures.


GENERAL-06 Contractor shall create Requests for Change (RFCs) and associated tasks and maintain
approved RFCs and associated tasks through their lifecycle in compliance with PBGC
IT Change Management Policies and Procedures including obtaining written approval
from an Application/Service Owner prior to implementing any change. Contractor
shall also ensure RFCs and RFC tasks are associated with the appropriate CIs.
Contractor shall also ensure a subject matter expert is present at the CAB to discuss a
major infrastructure change being deployed by the Contractor.
GENERAL-07 Contractor shall maintain passwords for privileged accounts, e.g. local system
emergency recovery accounts, privileged service accounts, database schema
accounts, etc., in privileged account management tool and use this tool to perform
administrative functions via brokered session or account check out
GENERAL-08 Contractor shall adhere to the NIST 800 series of special publications. Contractor shall
employ operational and technical NIST 800-53 controls for ensuring the
confidentiality, integrity and availability of PBGC information systems and networks

27
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
GENERAL-09 Contractor shall restrict the access to the IT Infrastructure General Support System to
authorized users only using a least privilege methodology and shall maintain data
integrity and prevent unauthorized use and release of PBGC information in
accordance with PBGC policy and procedures
GENERAL-10 Contractor shall revoke access based on the PBGC security guidelines, separation of
employment, evidence of dormant accounts, and other administrative reasons in
accordance with PBGC policy and procedures
GENERAL-11 Ensure all SLAs are met and reported as directed in this PWS

6.2 End-User Services


The sections below describe the scope and requirements of the End-User services that the Contractor
shall provide under this solicitation. Customer interaction is critical to excellent performance under this
contract. Thus, End-User services is a key service area. Much of how the customer views the success of
this contract will be dependent on how the Contractor interfaces with the end user, and how satisfied
the end users are with the IT services provided. End-User Services include the following:

• Providing IT Service Desk services


• Providing IT Site Support services

6.2.1 IT Service Desk


6.2.1.1 Scope of Services Supported
Provide a centralized service desk capability that provides a single point of contact (SPOC) for PBGC
employees and contractors to obtain IT service and support in a timely and consistent manner. The
PBGC Service Desk must support the following forms of user interaction:

• Telephone
• On-line ITSM Tool (Service Now) Chat
• On-line ITSM Tool (Service Now) ticket submission
• Email

Leverage and maintain the knowledge base module of the ITSM tool suite (currently Service Now) to
promote self-service, first call resolution, consistent support, and timely service restoration. Perform
basic account administration functions. See Appendix J – IT Service and Support 2018 Statistical
Summary for IT Service Desk interactions, tier 2 incidents, requests for information (RFIs), service/access
requests, requests for change (RFCs), and RFC tasks processed in calendar year 2018 for these services.
See Appendix K – IT Infrastructure Tools List for the software utilized to provides these services.

6.2.1.2 Requirements
The Contractor shall establish an IT Service Desk (as defined by Information Technology Infrastructure
Library (ITIL)), serving as the single customer-facing point of contact. The Service Desk will represent
ITIOD as the service provider to its end-users for all IT service and support including incidents, problem,
28
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
requests, advice, guidance and the rapid restoration of normal services, to meet service levels and
manage customer expectations in accordance with ITIL best practices and principles. The Contractor
shall propose either an external, “rural-sourced” Service Desk similar to PBGC’s current model or an
external Service Desk of another model. Except for a limited, on-site Site Support function, PBGC no
longer sees value in hosting a complete Service Desk on premises. Given past negative experience with
Service Desks staffed in the metropolitan DC area, the Service Desk model proposed will be an
evaluation factor and the evaluation will focus on cost, staff turnover and quality of service.

29
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
The contractor shall provide the IT Service Desk services outlined in the following table:

Reference Requirement
(EU-SD-xx)
EU-SD-01 Contractor shall identify a lead for the Service Desk area. This lead is required to serve
as the primary point of contact for all service desk related issues.
EU-SD-02 Contractor shall participate and support the following processes and functions using
ITIL best practices:
• Incident Management including Service Desk interactions and/or incidents
• Requests for Information (RFIs)
including compliance with PBGC-established processes in these functional areas
EU-SD-03 Contractor shall fully implement, manage, and support all activities regarding incident
management according to PBGC policies. By using ITIL best practices, this should
include, but shall not be limited to:

• Receive, respond, escalate, and resolve all IT Service Desk related calls,
emails, chat sessions, and on-line ticket submissions
• Establish a focal point for all customer communications and send PBGC
approved advisories and maintain the main Service Desk phone message, the
on-line IT Service Desk Portal (GetITAll), “Splash” screens, and intranet
content to communicate significant IT events and information, e.g. schedule
maintenance, unscheduled outages, etc. The Service Desk sends two (2) to
five (5) advisories in a typical week although more may be required if there
are significant outages to report. Updates to “Splash” screens and/or
intranet content occur less frequently; once or twice a month.

See SLA section for details on associated SLA measures.


EU-SD-04 Contractor shall, no less frequently than monthly, report on and review statistics on
most frequent Service Desk tickets to identify opportunities for creating and/or
promoting self-service.
EU-SD-05 Contractor shall work to reduce customer interactions by promoting self-service
through creation of new and refinement of existing on-line self-help/self-service
content and publicizing its availability via email, job-aids, offered training sessions,
integrated voice response (IVR) system (to be established in FY18/FY19), and open
houses events, etc.
EU-SD-06 Contractor shall, no less frequently than monthly, report on and review statistics on
most frequent Service Desk tickets requiring escalation to identify opportunities for
creating and/or promoting knowledge base articles and first call resolution.
EU-SD-07 Contractor shall work to increase customer interactions resolved by the Service Desk
(known as first call resolution) to drive down IT incident levels by drafting, publishing
and promoting knowledge base articles for common incident escalations that can be
addressed by IT Service Desk staff

30
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
EU-SD-08 Contractor shall perform basic account administration functions in accordance with
PBGC policy and procedures for systems and functions where automation is not
already in place including, but not limited to:
• Active Directory using Quest Active Roles Server:
o Updating user information, e.g. primary workstation, name, display
name, job title, associated contract, etc.
o Account unlocks and emergency/temporary disable/re-enable
o Creation of new distribution groups
o Generation of access reports
o Password Resets (including remote user verification when applicable
per PBGC policy)
o HSPD-12 temporary exemptions
• PBGC’s User Provision Tool (UPT)
o providing and removing access to target systems
o account separation
• Leapfile File Transfer
o Account creation upon request
o Account removal upon request or separation
EU-SD-09 Contractor shall establish and maintain, with updates no less frequently than
annually, IT service and support one or more job aids which provide a brief summary
of instructions or a checklist to ensure users know how to obtain IT Services and
Support, e.g.:
• General User – IT Support Job Aid
• General User – IT Requestor Job Aid
• IT Access/Request Approver Job Aid
• IT Access/Request Fulfiller Job Aid
EU-SD-10 Contractor shall assist customers with identifying the appropriate the IT service
catalog item/role to request and the most efficient way to request it, e.g. manual
submission, bulk submission, requests for control groups, etc., including ensuring
access requests are in compliance with approved governance
EU-SD-11 Contractor shall collaborate with PBGC by contributing, subject to Federal approval,
to the development and maintenance of Operating Level Agreements (OLAs) between
ITIOD Service Desk and all other IT functional areas throughout the life of the contract

6.2.2 Site Support


6.2.2.1 Scope of Services Supported
Provide a walk-up Site Support capability for PBGC employees and contractors to obtain IT service and
support in a timely and consistent manner for limited services: PIV card support, some types of mobile
phone support, asset pickup/drop-off, password pickup, etc.

Provide tier 2 site support to individual end users for IT issues that cannot be resolved by the Service
Desk, as well as support service requests. Typical site support activities include:

31
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Troubleshoot and repair or replace defective IT equipment (PC, laptop, monitor, cabling, IT
supported local and network printers, phone hand and head sets, mobile phones, etc.) as
needed
• Troubleshoot desktop operating systems including space cleanup, user profile repair or replace,
operating system re-image when needed, etc.
• Troubleshoot and reinstall desktop software including office automation, productivity tools, and
business application software as needed
• Troubleshoot mobile phone issues including device reset and re-enrollment and perform remote
wipes on mobile phones reported lost or stolen
• Relocate IT equipment in response to user relocation requests
• Install IT equipment for new user setups and remove upon user separation including
management of port security as needed/applicable
• Add/install additional equipment in response to user request, e.g. second monitor
• Enroll, configure, and provision mobile phones to end-users and remove upon separation
• Support IT equipment in conference and training rooms and provision additional IT equipment
to support specialized requirements upon request. In calendar year 2018, there were 38
requests for special conference room setups and 25 VTC setup requests.

This support is often provided remotely, using Skype for Business for screen sharing, but sometimes
requires physical desk-side visits to address certain hardware and software problems. See Appendix F -
PBGC Locations, including those requiring site support staff as well as planned changes to PBGC’s
facilities over the life of the contract. See Appendix G - IT Service Support Guidelines for impact,
urgency, and prioritization guidelines associated with IT service and support. See Appendix J – IT Service
and Support 2018 Statistical Summary for IT Service Desk interactions, tier 2 incidents, requests for
information (RFIs), service/access requests, requests for change (RFCs), and RFC tasks processed in
calendar year 2018 for these services. See Appendix K – IT Infrastructure Tools List for the software
utilized to provides these services.

Providing support network printers and multi-function devices is out of scope for this contract. This
support is provided by federal staff and the multi-function device lessor.

PBGC’s environment supported by site support consists of many components detailed in the tables that
follow:

Printer Summary

PBGC has approximately 136 network multi-function, printers, and plotters as well as 290 IT supported
local printers as outlined in the following table:

Model Network Local Total


Ricoh 4002SPG 70 70
Ricoh C4503 25 25
HP Color LaserJet M750 8 8

32
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
HP Color LaserJet Miscellaneous 13 13
HP B&W LaserJet Miscellaneous 18 18
EPSON SC-P20000 2 2
Brother HL-3170CDW 42 42
Brother HL-3170CDW 2 2
Brother DCP-L2550DW 161 161
HP LaserJet P2035 85 85
Total 136 290 426
ITIOD is currently wrapping up the process of standardizing the network printer fleet through our
existing enterprise printer lease contract, currently with Ricoh, and will be phasing out a few more of the
non-Ricoh printers. PBGC has more than 75 local printers, mostly HP LaserJet P2015 or 2055 that are
not supported and will be phased out on move to the new headquarters building.

6.2.2.2 Requirements
The Contractor shall provide a walk-up Site Support capability for PBGC employees and contractors, for
those services requiring in-person response that cannot resolved by the IT Service Desk, as well as
support for service requests. This involves providing the end-users, from the PBGC inventory of
government furnished equipment, with a standard configuration desktop or laptop (depending on job
function, location, and/or end-user preference) and any approved peripherals, standard PBGC–approved
COTS office automation and productivity software (e.g., MS Office), and access to any authorized PBGC
COTS and custom-developed applications and resources as required.

The contractor shall provide the Site Support services outlined in the following table:

Reference Requirement
(EU-SS-xx)
EU-SS-01 Contractor shall identify a lead for the Site Support area. This lead is required to serve
as the primary point of contact for all site support related issues.
EU-SS-02 Contractor shall fully implement, manage, and support all activities regarding site
support incident management according to PBGC policies. By using ITIL best practices,
this should include:

• Receive, respond, escalate, and resolve all Site Support walk-up requests
• Provide support for desktop and laptop hardware and operating systems
including basic network connectivity and peripheral support
• Provide support for IT supported, centrally managed (COTS/GOTS)
applications on end-user workstations, e.g. Microsoft Office, Adobe Reader,
Adobe Pro, CRM, Specturm, CFS, PPS, Comprizon, etc.
• Provide support for IT provided phone handsets and headsets and associated
connectivity
• Provide support for IT provided and managed mobile devices, e.g. smart
phones
• Provide support for IT supported local printers

33
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
See SLA section for details on associated SLA measures.
EU-SS-03 Contractor shall provide a walk-up Site Support capability for PBGC employees and
contractors to obtain IT service for services that can only be handled face-to-face, e.g.
PIV card testing, mapping and PIN resets; mobile phone support, asset pickup/drop-
off, password pickup, facilitating new employee/contractor training conducted in the
same space, etc.

Contractor shall offer this service at the HQ campus. See Appendix F – PBGC
Locations, for sites beyond HQ requiring site support staff now as well as planned
changes to PBGC’s facilities and associated services over the life of the contract.
EU-SS-04 Contractor shall provide services to install, move, add, and change (IMAC) IT
equipment for end-users including desktops, laptops, monitors, cabling, IT supported
local and network printers, IT supported phone hand and headsets, mobile phones,
etc., in accordance with PBGC procedures and using PBGC approved security
configuration baselines and associated images when available.

Contractor shall ensure that all changes resulting from IMAC activity are properly and
accurately recorded in PBGC’s asset management system.

See SLA section for details on associated SLA measures.


EU-SS-05 Contractor shall enroll, configure, and provision mobile phones to end-users and
remove and reset upon separation
EU-SS-06 Contractor shall support IT equipment in conference and training rooms and provision
additional IT equipment to support specialized requirements upon request
EU-SS-07 Contractor shall perform basic account and PIV card administration functions in
accordance with PBGC policy and procedures including, but not limited to:
• Active Directory using Quest Active Roles Server or ADUC:
o Maintaining an accurate relationship between users and their primary
computers
o Mapping PIV cards to Active Directory User Accounts after verifying
mandatory training requirements have been met (including users at
remote locations or off-site)
• PIV Card management:
o Perform PIN resets using light activation stations

EU-SS-08 Contractor shall facilitate new employee/contractor training conducted at the HQ


campus as required by PBGC policies and procedures. Approximately 430 new
employee/contractors onboarded at the PBGC’s HQ campus in calendar year 2018.
Onboarding is typically limited to a single day (Monday) each week.

34
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.3 Data Center Services
The sections below describe the scope and requirements of the Data Center services that the Contractor
shall provide under this solicitation. Providing stable, reliable, secure, optimally performing, and highly
available systems and service is critical to enable the accomplishment of the agency mission and as such,
are critical to excellent performance under this contract. Thus, Data Center services is a key service area.
Much of how the customer views the success of this contract will be dependent on how well the
Contractor administers and supports PBGC’s data center environment and platforms, and how satisfied
ITIOD staff members are with the IT services provided. Data Center services include the following:

• Windows Server Administration and Support


• Windows Desktop Administration and Support, Software Packaging, and Software Deployment
• Enterprise Identity Management Administration and Support
• Office 365 and Messaging Administration and Support
• UNIX/LINUX Server Administration and Support
• Virtualization Platforms Administration and Support
• Storage and Backup Administration and Support
• Database Administration and Support
• Web and Middleware Administration and Support

The on-premise IT equipment, e.g. servers, SAN, etc., that supports Data Center services is largely
located at PBGC’s HQW location today. The on-premise IT equipment that supports PBGC’s disaster
recovery capability is currently located at PBGC’s Wilmington (WIL) facility. PBGC plans to move most of
the on-premise IT equipment supporting the Data Center services to co-located data centers over the
next several years. Please refer to Appendix F – PBGC Locations for a tentative timeline for this
transition.

6.3.1 Windows Server Administration and Support


6.3.1.1 Scope of Services Supported
Provide tier 2 support for incidents relating to Windows servers or requiring administrative access to
Windows servers that cannot be resolved by an End-User services team as well as support service
requests. Typical Windows server administration and support activities include:

• Monitor, troubleshoot and repair or replace defective IT equipment (stand-alone servers, blade
infrastructure and servers, and associated components) as needed
• Monitor, troubleshoot, and repair Windows Server operating systems including space cleanup,
event log analysis, role and feature reconfiguration and reinstall, operating system re-image
when needed, etc.
• Troubleshoot and reinstall application software including office automation, productivity tools,
infrastructure software, and business application software as needed on Windows servers
• Maintain up-to-date physical and virtual Windows Server operating system images/templates
• Provision and configure new physical and virtual servers as requested including SAN connectivity

35
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Provision and configure new blade infrastructure, e.g. chassis, out of band management
modules, connectivity modules (virtual connect flex fabrics), blade servers and associated blade
profiles, etc.
• Perform initial installation and configuration of operating system roles and features, application
software including office automation, productivity tools, infrastructure software, and business
application software as needed on Windows servers
• Remove/decommission physical and virtual servers as requested
• Perform Windows Active Directory account administration for privileged and service accounts
• Monitor, troubleshoot, and repair Windows Server Active Directory and associated services
• Create and modify Active Directory group policy objects as required to ensure compliance with
Windows operating system baselines as well as to achieve desired operational configuration and
user experience
• Deploy patches monthly to Windows servers using patch deployment tool, e.g. BigFix
• Address operating system and software vulnerabilities detected on Windows servers during
monthly vulnerability scans
• Configure and administer NTFS file systems and Windows file shares
• Configure and administer Microsoft Windows Certificate Authorities and associated services
• Configure, monitor, troubleshoot, and repair high availability and disaster recovery capabilities
pertaining to Windows Server and associated services including, but not limited to Windows
clustering, distributed file system replication (DFS-R), failover procedures, etc.
• Support internal and external IP address management and name resolution services by updating
IPAM DNS records
• Maintain server racks
• Establish and maintain work instructions
• Escalate to and work collaboratively with 3rd party vendors on hardware and software issues

Support for Windows Servers and IT equipment in the PBGC data centers is typically handled using
remote management software, e.g. RDP, vCenter console, SSH, Powershell, ILO, OA, etc., but does
occasionally require physical visits to address hardware and software problems. See Appendix F - PBGC
Locations for PBGC locations, including planned changes to PBGC’s facilities and data centers over the
life of the contract. See Appendix G - IT Service Support Guidelines for impact, urgency, and
prioritization guidelines associated with IT service and support. See Appendix J - IT Service and Support
2018 Statistical Summary for IT Service Desk interactions, tier 2 incidents, requests for information
(RFIs), service/access requests, requests for change (RFCs), and RFC tasks processed in calendar year
2018 for these services. See Appendix K - IT Infrastructure Tools List for the software utilized to provides
these services.

PBGC’s Windows Server environment consists of many components detailed in the tables that follow:

Active Directory Summary

36
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PBGC has 4 separate and independent Windows 2008 R2 Active Directory Forests servicing the various
PBGC computing environments.

Domain Name Function DC Count Forest Name


ent.pbgc.gov Production empty forest root 4 ent.pbgc.gov
prod.ent.pbgc.gov Primary Production domain for PBGC 4
users and resources
oig.ent.pbgc.gov Primary domain for PBGC OIG users 4
and resources
dmz.pbgc.gov Domain for managing and 2 dmz.pbgc.gov
administering Windows devices in
PBGC’s DMZ
dev.pbgc.gov Development environment empty 2 dev.pbgc.gov
forest root
dpn.dev.pbgc.gov Primary domain for PBGC developers 2
and resources
cdei.dev.pbgc.gov Domain for emulating, via SSO, 2
production PBGC users for
applications under development in the
development environment. The
majority of user and group objects are
a copy of prod.ent.pbgc.gov.
itcp.pbgc.gov Test environment empty forest root 2 itcp.pbgc.gov
itcpc.itcp.pbgc.gov Primary domain for PBGC testers and 2
resources; also used for emulating, via
SSO, production PBGC users during
user acceptance testing (UAT) for
application releases. The majority of
user and group objects are a copy of
prod.ent.pbgc.gov.
An upgrade to Windows 2012 R2 or Windows 2016 Active Directory is planned for FY19. The version
chosen may be constrained by compatibility with PBGC’s Oracle Internet Directory and Access
Management services or management agent compatibility. PBGC also plans to provision domain
controllers for the development, test, and production environments on Azure by the end of calendar
year 2019.

Windows Server Summary

PBGC has approximately 629 Windows servers, with more than 75% of them being virtual. The following
table breaks them down by physical/virtual and environment:

Operating System Physical/Virtual PROD COOP DMZ DEV TEST Total


Count
Windows Server 2008 R2 Physical 20 15 1 36

37
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Virtual 21 11 4 42 22 100
Physical 37 4 74 1 116
Windows Server 2012 R2
Virtual 120 27 23 126 76 372
Windows Server 2016 Physical 0
Virtual 3 3
Windows Server 2019 Physical 0
Virtual 2 2
Total Count 198 57 27 248 99 629
An upgrade to Windows Server 2016/2019 is in progress and will be largely completed in calendar year
2019 to the extent possible.

Windows Physical Server Breakdown by Model

PBGC has approximately 152 physical Windows servers, of which almost all are HP Proliant servers and
the majority of which are HP Proliant blade servers. The following table breaks them down by model and
operating system:

Hardware Model Vendor Windows Windows Total Count


2008 R2 2012 R2
Express5800/R120f- NEC 2 2
2M [N8100-2225F]
Precision Rack 7910 Dell 4 4
ProLiant BL460c G1 HP 2 2
ProLiant BL460c G6 HP 8 16 24
ProLiant BL460c G7 HP 2 7 9
ProLiant BL460c G8 HP 8 30 38
ProLiant BL460c G9 HP 16 16
ProLiant BL460c G10 HP 19 19
ProLiant BL680c G5 HP 2 2
ProLiant DL360 G5 HP 1 1
ProLiant DL360 G6 HP 2 2
ProLiant DL360 G7 HP 4 5 9
ProLiant DL360 G8 HP 3 3
ProLiant DL360 G9 HP 11 11
ProLiant DL360 G10 HP 1 1
ProLiant DL370 G6 HP 1 1
ProLiant DL380 G5 HP 5 5
ProLiant DL380 G7 HP 1 1
ProLiant DL380 G9 HP 2 2
Total Count 36 116 152

38
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
An upgrade from HP Proliant Generation 5, 6 and 7 servers to HP Proliant Generation 10 servers is
planned for calendar year 2019 as part of the Windows Server 2016/2019 upgrade project.

EnableIT Service Requests

The following table provides the count for EnableIT service requests processed in Calendar Year 2018 for
Windows servers:

Service Request Category Total


Requests
Server: New Server 79
Server: Request Additional Memory or 25
CPU

6.3.1.2 Requirements
The Contractor shall provide incident response and resolution for issues relating to Windows servers or
requiring administrative access to Windows servers that cannot be resolved by an End-User services
team as well as support service requests and requests for change.

The contractor shall provide the Windows Server administration and support services outlined in the
following table:

Reference Requirement
(DC-WS-xx)
DC-WS-01 Contractor shall identify a lead for the Windows Server area. This lead is required to
serve as the primary point of contact for all Windows Server related issues.
DC-WS -02 Contractor shall provide and maintain a fully functional, optimally performing, secure
Windows Server infrastructure in all PBGC computing environments. This shall
include, but is not limited to:

• Identifying and addressing performance bottlenecks


• Performing capacity planning and management including allocation, resizing,
and reconfiguration of Windows servers and associated services as needed
• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
DC-WS-03 Contractor shall fully implement, manage, and support all incident management
activities regarding Windows servers and the associated services they provide

39
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
according to PBGC policies. By using ITIL best practices, this should include, but shall
not be limited to:

• Provide support for Windows server hardware and operating systems


including basic network and SAN connectivity
• Provide support for blade infrastructure hardware including network and SAN
connectivity as well as blade server profiles
• Provide support for IT supported applications and services deployed on
Windows servers including, but not limited to Active Directory, Group Policy,
IIS, NTFS, RDP, business applications deployed on Windows servers, etc.

See SLA section for details on associated SLA measures.


DC-WS-04 Contractor shall fully implement, manage, and support all change management
activities regarding Windows servers and the associated services they provide
according to PBGC policies. By using ITIL best practices, this should include, but shall
not be limited to:

• Provide installation and configuration of Windows server hardware and


operating systems including basic network and SAN connectivity or
decommission when no longer required
• Provide installation and configuration of blade infrastructure hardware
including network and SAN connectivity as well as blade server profiles or
decommission when lo longer required
• Provide installation and configuration of IT supported applications and
services deployed on Windows servers including, but not limited to Active
Directory, Group Policy, IIS, NTFS, RDP, business applications deployed on
Windows servers, etc. or remove when lo longer required
• Apply applicable security patches at least monthly and install applicable
hardware firmware updates at least quarterly
• Address operating system and software vulnerabilities detected on Windows
servers during monthly vulnerability scans
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices

See SLA section for details on associated SLA measures.


DC-WS-05 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New Windows servers


• Server: Request Additional Memory or CPU (Windows Servers)

See SLA section for details on associated SLA measures.

40
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DC-WS-06 Contractor shall maintain up-to-date physical and virtual Windows server operating
system images and templates for supported Windows Server operating system
versions that are in compliance with PBGC-approved security baselines and that
include PBGC approved security patches within 60 days of production security patch
approval, i.e. updated no less infrequently than every other month
DC-WS-07 Contractor shall administer and support PBGC’s internal certificate authority and
associated certificates according to PBGC policy and procedures
DC-WS-08 Contractor shall administer and support NTFS file systems and Windows file shares
according to PBGC policy and procedures including securing to standards and
performing quota management
DC-WS-09 By the end of January of each year, contractor shall:
• Upgrade each Java Platform (JRE/JDK) instance on Windows servers to the
identified target version established the previous January unless risk accepted
• Identify and communicate the target Java platform version for the following
January
DC-WS-10 Contractor shall perform Windows account administration functions in accordance
with PBGC policy and procedures for systems and functions where automation is not
already in place including, but not limited to:
• Active Directory using Quest Active Roles Server:
o Account unlocks and emergency/temporary disable/re-enable for
privileged and service accounts
o Password Resets (including remote user verification when applicable
per PBGC policy) for privileged and service accounts
o Account removal for privileged and service accounts upon request or
separation
DC-WS-11 Contractor shall maintain server racks to include, but not limited to:
• ensuring front and rear doors can be closed and are closed
• ensuring patch cables are of suitable length, are free from entanglement, are
patched to the appropriate port based on applicable PBGC standards, and are
routed neatly to the patch panel
• ensuring power to redundant power supplies is properly distributed between
PDUs to minimize downtime due to a single component failure
• maintaining Visio diagrams of the contents of each rack with
updates/verification no less frequently than every 6 months
• ensuring equipment not in use is powered down, clearly labeled, and
removed from the rack within 90 days of decommission
DC-WS-12 Contractor shall ensure all Windows accounts supporting the Windows server
environment, e.g. local Windows administrator; Windows service accounts; etc., are
changed periodically in accordance with PBGC policy and procedures and the
passwords are stored for emergency use
DC-WS-13 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the Windows servers and the associated services

41
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.3.2 Windows Desktop Administration and Support, Software Packaging, and Software
Deployment
6.3.2.1 Scope of Services Supported
Provide tier 2 support for incidents relating to Windows Desktop Administration and Support, Software
Packaging, and Software Deployment or incidents requiring administrative access to Windows desktops
and laptops that cannot be resolved by an End-User services team as well as support service requests.
Typical Windows Desktop Administration and Support, Software Packaging, and Software Deployment
activities include:

• Monitor, troubleshoot, and address issues pertaining to the collective set of Windows enterprise
desktop operating systems including event log analysis, reconfiguration, etc.
• Package and deploy application software including office automation, productivity tools,
infrastructure software, and business application software as needed on Windows desktops,
laptops, and general purpose remote desktop services servers
• Create and distribute shortcuts for IT supported web-based applications
• Maintain up-to-date physical and virtual Windows enterprise desktop operating system
images/templates
• Monitor, troubleshoot, and repair software deployments
• Administer and support software packaging and software deployment tools
• Create and modify Active Directory group policy objects as required to ensure compliance with
Windows enterprise desktop operating system baselines as well as to achieve desired
operational configuration and user experience, e.g. adding trusted sites
• Deploy patches monthly to Windows desktops and laptops utilizing patch deployment tool, e.g.
BigFix
• Address operating system and software vulnerabilities detected on Windows workstations
during monthly vulnerability scans
• Escalate to and work collaboratively with 3rd party vendors on hardware and software issues
• Establish and maintain work instructions

See Appendix G - IT Service Support Guidelines for impact, urgency, and prioritization guidelines
associated with IT service and support. See Appendix J - IT Service and Support 2018 Statistical
Summary for tier 2 incidents, requests for information (RFIs), service/access requests, requests for
change (RFCs), and RFC tasks processed in calendar year 2018 for these services. See Appendix K – IT
Infrastructure Tools List for the software utilized to provides these services.

PBGC’s Windows Desktop Administration and Support, Software Packaging, and Software Deployment
environment consists of many components detailed in the tables that follow:

Windows Workstation Summary

PBGC has approximately 3,087 Windows enterprise workstations, all (100%) of them running Windows
10 and more than 83% being physical. The following table breaks them down by OS, physical/virtual and
environment:
42
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Operating System Physical/Virtual Production Development Test Total
Count
Windows 10 Physical 2,504 15 51 2,570
Windows 10 Virtual 234 277 6 517
Total Count 2,738 292 57 3,087
PBGC has been moving towards portable computers and reduction of users with two physical devices.
This includes the recent replacement of physical workstations dedicated to application development and
testing with virtual desktops. Virtual desktops have also been deployed to support off-site actuaries
without government furnished equipment (GFE).

Windows Physical Workstation Breakdown by Model

PBGC has approximately 2,570 physical Windows workstations, of which about 49% are portable devices
which are concentrated at PBGC’s HQ campus. Currently, PBGC is deploying one of three models based
on user location and user preference: MS Surface Pro 4 (HQ campus default), Dell Latitude E7450 (for
mobile power users who prefer it to SP4), and Dell Precision T1700 (for FBA sites, shared spaces, and
users who don’t want a SP4). These three models make up more than 93% of deployed inventory.
PBGC is currently evaluating the Dell Latitude 5290 2-in-1 as the successor to the SP4. PBGC is
considering deployment of laptops/tablets at its remote sites and eliminating use of desktops in
conference rooms. The following table breaks down all PBGC physical workstations by model and
operating system:

Hardware Model Vendor Total


Count
Other Laptops/Tablets (under Misc 13
evaluation)
Latitude 6430U Dell 25
Latitude E6440 Dell 12
Latitude E7450 Dell 166
OptiPlex 780 Dell 120
Precision T1700 Dell 1191
Surface Pro 4 Microsoft 1043
Total Count 2570
PBGC is working towards allocating only a single physical workstation per user.

Remote Access (Terminal) Servers:

PBGC maintains a pool of physical Windows remote access (terminal) servers at both the headquarters
and disaster recovery sites which can provide PBGC users with access to the majority of PBGC
applications remotely during normal operating conditions (in support of telework), during a pandemic,
or should PBGC have a need to operate out of our disaster recovery site as follows:

Location Server Count

43
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
HQW 5

WIL (COOP) 7

These servers are currently running Windows 2008 R2 but are scheduled for hardware upgrades (BL460
G10) and operating system modernization (to Windows 2016 or 2019) in calendar year 2018.

EnableIT Service Requests

The following table provides the count for EnableIT service requests processed in Calendar Year 2018

Service Request Category Total


Requests
Software: JRE Update 3
Software: Packaging 69

6.3.2.2 Requirements
The Contractor shall provide incident response and resolution for issues relating to Windows Desktop
Administration and Support, Software Packaging, and Software Deployment or incidents requiring
administrative access to Windows desktops and laptops that cannot be resolved by an End-User services
team as well as support service requests and requests for change.

The contractor shall provide the Windows desktop imaging and software deployment support services
outlined in the following table:

Reference Requirement
(DC-WD-xx)
DC-WD-01 Contractor shall identify a lead for the Windows Desktop Administration and Support,
Software Packaging, and Software Deployment area. This lead is required to serve as
the primary point of contact for all Windows desktop imaging and software packaging
and deployment related issues.
DC-WD-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
Windows Desktop and Software Packaging and Deployment infrastructure in all PBGC
computing environments. This shall include, but is not limited to:

•Identifying and addressing performance bottlenecks


•Performing capacity planning and management including allocation, resizing,
and reconfiguration of Windows servers and associated services as needed
• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
Coordinating performance of work by vendors as required and in accordance with
PBGC security policies, vendor warranties and maintenance contracts
44
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DC-WD-03 Contractor shall fully implement, manage, and support all incident management
activities regarding Windows Desktop Administration, Software Packaging, and
Software Deployment according to PBGC policies. By using ITIL best practices, this
should include, but shall not be limited to:

• Provide support for the collective set of Windows enterprise desktop


operating systems and general purpose remote desktop services servers
• Provide support for software packages and software deployments including
IT-supported office automation, productivity tools, infrastructure software,
and business application software deployed on Windows desktops, laptops,
and general purpose remote desktop services servers

See SLA section for details on associated SLA measures.


DC-WD-04 Contractor shall fully implement, manage, and support all change management
activities regarding Windows Desktop Administration, Software Packaging, and
Software Deployment according to PBGC policies. By using ITIL best practices, this
should include, but shall not be limited to:

• Provide installation and configuration of the collective set of Windows


desktop and laptop hardware and enterprise desktop operating systems
• Build software packages and perform software deployments for IT-supported
office automation, productivity tools, infrastructure software, and business
application software deployed on Windows desktops, laptops, and general
purpose remote desktop services servers
• Apply applicable security patches at least monthly
• Address operating system and software vulnerabilities detected on Windows
desktops and laptops during monthly vulnerability scans
• Apply necessary configuration changes to ensure compliance with Windows
enterprise desktop operating system baselines as well as to achieve desired
operational configuration and user experience, e.g. adding trusted sites, and
industry best practices

See SLA section for details on associated SLA measures.


DC-WD-05 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New Software Packages/Deployments

See SLA section for details on associated SLA measures.


DC-WD-06 Contractor shall maintain up-to-date physical and virtual Windows Enterprise
(desktop) operating system images and templates for supported Windows desktop
operating system versions that are in compliance with PBGC-approved security
baselines and that include PBGC approved security patches within 60 days of
45
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
production security patch approval, i.e. updated no less infrequently than every other
month
DC-WD-07 By the end of January of each year, contractor shall:
• Upgrade each Java Platform (JRE/JDK) instance on Windows workstations
(desktops, laptops, and tablets) to the identified target version established
the previous January unless risk accepted
• Identify and communicate the target Java platform version for the following
January
DC-WD-08 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the Windows enterprise desktops and the associated services

6.3.3 Enterprise Identity Management Administration and Support


6.3.3.1 Scope of Services Supported
Provide tier 2 support for incidents relating to Enterprise Identity Management administration and
support as well as support service requests. Typical Enterprise Identity Management administration and
support activities include:

• Monitor, troubleshoot, and address issues pertaining to the collective set of Enterprise Identity
and Privileged Account Management tools, scripts, and databases
• Configure Enterprise Identity and Privileged Account Management tools, scripts, and databases
to ensure accounts and access are managed in compliance with PBGC policies and procedures
• Configure new application security access roles in enterprise identity management tool(s) to
support request, approval, and automated fulfillment
• Update existing application security access roles in enterprise identity management tool(s) that
support request, approval, and automated fulfillment to address changing requirements as
identified
• Conduct annual certification/validation of existing application security access roles in enterprise
identity management tool(s) that support request, approval, and automated fulfillment
• Conduct annual account and access recertification for accounts and access managed by PBGC’s
suite of enterprise identity management tools
• Review reports and alerts to ensure all access has been properly authorized for accounts and
access managed by PBGC’s suite of enterprise identity management tools and revoke and create
security event upon detection of unauthorized access
• Federate new cloud-based systems with existing on-premise identity stores
• Establish, monitor, and maintain connectivity between multiple PBGC systems, e.g. HR,
procurements, etc.
• Escalate to and work collaboratively with 3rd party vendors on hardware and software issues
• Establish and maintain work instructions

See Appendix G - IT Service Support Guidelines for impact, urgency, and prioritization guidelines
associated with IT service and support. See Appendix J - IT Service and Support 2018 Statistical

46
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Summary for tier 2 incidents, requests for information (RFIs), service/access requests, requests for
change (RFCs), and RFC tasks processed in calendar year 2018 for these services. See Appendix K – IT
Infrastructure Tools List for the software utilized to provides these services.

PBGC’s Enterprise Identity Management environment consists of many components detailed in the
tables that follow:

Active Directory Production User Summary

The following table details the current number of Active Directory user accounts in PBGC’s primary
production user and resource domain (prod.ent.pbgc.gov) broken down by function:

Account Type Count of Users


Regular User 2,157
AP User 146
Service Account 283
Shared Mailbox 469
See Appendix I - OIG IT Infrastructure Summary for details about their environment which is largely
administered and supported by OIG staff.

Production Account Summary beyond AD

The following table details the current number of user accounts (outside of production AD) in PBGC’s
production environment broken down by platform and type:

Platform Account Type PROD COOP DMZ FBA Total


Count

DMZ Active Generic - Recovery 1 1


Directory Account

Service Account 14 14

Individual - AP User 27 27

Individual - Regular 1 1
User

Database - MS SQL Generic - Recovery 4 38 42


Server Account

Service Account 2 40 42

47
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Database - Oracle Individual - Regular 232 232
User

Service Account 1896 1896

Network Device Generic - Recovery 61 12 5 12 90


Account

Service Account 125 20 7 45 197

Security Device Generic - Recovery 9 9


Account

Server - RHEL Generic - Recovery 95 48 9 152


Account

Service Account 268 143 16 427

Server - Solaris Generic - Recovery 14 8 22


Account

Service Account 35 20 55

Server - Windows Generic - Recovery 187 50 19 10 266


Account

Service Account 45 3 (20) 4 2 54


(170)

Total Count 2973 382 103 69 3527

PBGC is actively working to standardize its identity and access management on Active Directory and to
reduce dependencies on OAM and OID. Where Active Directory cannot be used, SailPoint will be
configured to manage accounts and access.

PBGC has implemented a Privileged Account Management tool, CyberArk , to provide central access
control for privileged access to PBGC systems. This includes storing privileged credentials and brokering
and recording sessions requiring privileged access. Currently, CyberArk is used to broker privileged
sessions to Windows, RHEL, Unix, Cisco, and Security devices using Service Accounts or Generic –
Recovery accounts instead of Individual – Elevated Accounts. The use of CyberArk to manage privileged
sessions will be expanded to include other platforms during calendar year 2019.

PBGC is in the process of implementing the service request module of Service Now which will be used as
a front-end request platform for automated fulfillment of access requests using SailPoint LifeCycle
Manager. This is expected to be in place by September 2019.

48
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.3.3.2 Requirements
The Contractor shall provide incident response and resolution for issues relating to Enterprise Identity
Management administration and support as well as support service requests and requests for change.

The contractor shall provide the enterprise identity management support services outlined in the
following table:

Reference Requirement
(DC-EI-xx)
DC-EI-01 Contractor shall identify a lead for the Enterprise Identity Management area. This
lead is required to serve as the primary point of contact for all enterprise identity
management related issues.
DC-EI-02 Contractor shall provide and maintain a fully functional, optimally performing
enterprise identity management infrastructure to support all PBGC computing
environments. This shall include, but is not limited to:

• Identifying and addressing performance bottlenecks


• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
DC-EI-03 Contractor shall fully implement, manage, and support all incident management
activities regarding Enterprise Identity Management services according to PBGC
policies. By using ITIL best practices, this should include, but shall not be limited to:

• Provide support for the collective set of Enterprise Identity and Privileged
Account Management tools, scripts, and databases

See SLA section for details on associated SLA measures.


DC-EI-04 Contractor shall fully implement, manage, and support all change management
activities regarding Enterprise Identity Management services according to PBGC
policies. By using ITIL best practices, this should include, but shall not be limited to:

• Provide installation and configuration of the collective set of Enterprise


Identity and Privileged Account Management tools, scripts, and databases
• Apply necessary configuration changes to achieve identity and access
management objectives including compliance with PBGC policies and
procedures as well as to achieve desired operational configuration and user
experience

See SLA section for details on associated SLA measures.

49
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DC-EI-05 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New Service Catalog Items

See SLA section for details on associated SLA measures.


DC-EI-06 Contractor shall maintain application security access roles in enterprise identity
management tool(s) to support request, approval, and automated fulfillment
including additions, modifications, and removals
DC-EI-07 Contractor shall conduct an annual certification/validation of existing application
security access roles in enterprise identity management tool(s) that support request,
approval, and automated fulfillment to ensure they are current and accurate
DC-EI-08 Contractor shall conduct an annual account and access recertification for accounts
and access managed by PBGC’s suite of enterprise identity management tools
DC-EI-09 Contractor shall review reports no less frequently than monthly, as well real-time
alerts within 1 business day, to ensure all access has been properly authorized and
revoke access upon detection of unauthorized access and created associated security
event
DC-EI-10 Contractor shall maintain the privileged account management system and associated
safes containing passwords for privileged accounts, e.g. local system emergency
recovery accounts, privileged service accounts, database schema accounts, etc.
facilitating session brokering where possible and account check out otherwise
DC-EI-11 Contractor shall ensure Active Directory (AD) user, computer, and group objects are
maintained in compliance with PBGC defined standards by limiting direct access to
update AD, by configuring and utilizing Quest ActiveRoles Server (ARS) for AD
administration to the extent possible, by regularly performing ARS automation health
checks, and by periodic compliance monitoring and reporting
DC-EI-12 Contractor shall process manual and establish, monitor and maintain automated data
feeds in support of enterprise identity management including but not limited to:
• Bi-weekly HR data feed including staff report and organization/supervisor
report and maintenance of the IT Approval Hierarchy list in SharePoint
• Daily feed from procurement system
• Periodic feeds to/from personnel security system
• Feeds to/from badging system
• Configuration management systems/tools, e.g. mAppIT (SharePoint), uCMDB,
etc.
DC-EI-13 Contractor shall work with 3rd party vendors as necessary to federate new cloud-
based systems with existing on-premise identity stores
DC-EI-14 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the collective set of Enterprise Identity and Privileged Account Management
tools, scripts, and databases

50
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.3.4 Office 365 and Messaging Administration and Support
6.3.4.1 Scope of Services Supported
Provide tier 2 support for incidents relating to PBGC’s Microsoft Office 365 E3 tenant including Intune
Mobile Device Management (MDM) and Advanced Threat Protection (ATP) as well as PBGC’s on-premise
IronPort email hygiene, relay, and data loss prevention appliances and associated services. Provide
fulfillment for service requests. Typical Office 365 and Messaging administration and support activities
include:

• Monitor, troubleshoot, and address issues pertaining to PBGC’s Microsoft Office 365 E3 tenant
including Intune Mobile Device Management (MDM) and Advanced Threat Protection (ATP) as
well as PBGC’s on-premise IronPort email hygiene, relay, and data loss prevention appliances
and associated services, maintenance tools, and scripts
• Configure PBGC’s Microsoft Office 365 E3 tenant including Intune Mobile Device Management
(MDM) and Advanced Threat Protection (ATP) as well as PBGC’s on-premise IronPort email
hygiene, relay, and data loss prevention appliances and associated services to maximize
availability, optimize performance, and ensure compliance with PBGC governance
• Maintain and administer PBGC’s Microsoft Office 365 E3 tenant including Intune Mobile Device
Management (MDM) and Advanced Threat Protection (ATP) including, but not limited to:
o Performing litigation holds and sensitive data collection requests
o Providing reports on message transport utilizing message tracking
o Updating hub transport rules
o Updating email retention policies
o Creating and removing shared mailboxes
o Delegating access to or transferring email or OneDrive contents as required
o Periodically running a script to copy all email content from separated and deprovisoned
users to a shared mailbox to preserve email according to PBGC’s record schedule and
remove license
o Troubleshooting/resolving identity synchronization issues with Azure AD
o Monitoring Microsoft’s Office 365 IP and URL list and coordinate associated PBGC
network updates to ensure connectivity
• Maintain and administer PBGC’s on-premise IronPort email hygiene, relay, and data loss
prevention appliances including, but not limited to:
o Updating blacklists and whitelists including adding and removing email domains and
individual email addresses as required
o Updating authorized relay list including adding and removing IPs
o Performing periodic device configuration back-ups no less than monthly
o Examining and acting on outbound messages quarantined by the DLP engine in
accordance with PBGC policy and procedures
• Enable and support email integration with SharePoint on-line and perform other business
process automation using Think Automation, e.g. automate upload to SharePoint on-line reports
distributed via email
51
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Enable and support email-enabled/integrated applications as well as MDAemon Mail Server for
application-based mail services that cannot leverage Office 365
• Escalate and work collaboratively with 3rd party vendors on issues
• Establish and maintain work instructions

See Appendix G - IT Service Support Guidelines for impact, urgency, and prioritization guidelines
associated with IT service and support. See Appendix J - IT Service and Support 2018 Statistical
Summary for tier 2 incidents, requests for information (RFIs), service/access requests, requests for
change (RFCs), and RFC tasks processed in calendar year 2018 for these services. See Appendix K – IT
Infrastructure Tools List for the software utilized to provides these services.

PBGC’s Office 365 and Messaging administration and support environment consists primarily of the
following:

• Microsoft Office 365 E3 tenant including Intune Mobile Device Management (MDM) and
Advanced Threat Protection (ATP). PBGC has 2 tenants, one for testing which is licensed for 10
users and on for production use which is licensed for 2,400 users. PBGC Active Directory users
and groups are synchronized to Azure AD utilizing Azure Active Directory Synchronization and
user authentication is federated utilizing Active Directory Federation Services (ADFS). PBGC has
approximately 500 distribution groups, more than 14,000 mail-enabled security groups primarily
for SharePoint access management, and 450 shared mailboxes in addition to its regular 2,250
mailbox-enabled users.
• PBGC maintains on-premise IronPort appliance for email hygiene, relay, and data loss
prevention. Mail is routed between the internet and PBGC’s Microsoft Office 365 tenant
through the IronPort mail gateways located at PBGC’s headquarters and disaster recovery data
centers.
• PBGC maintains a single instance of the MDAemon Mail Server for application-based mail
services that cannot leverage Office 365. This solution hosts approximately 10 application IMAP
mailboxes.

6.3.4.2 Requirements
The contractor shall provide the Office 365 and messaging support services outlined in the following
table:

Reference Requirement
(DC-MS-xx)
DC-MS-01 Contractor shall identify a lead for the Office 365 and Messaging area. This lead is
required to serve as the primary point of contact for all Office 365 and messaging
related issues.
DC-MS-02 Contractor shall support and maintain a fully functional, optimally performing on-
premise messaging infrastructure, e.g. mail hygiene appliances, and infrastructure
services that support connectivity and integration with the cloud-based Office 365.
This shall include, but is not limited to:

52
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Identifying and addressing performance bottlenecks
• Using monitoring tools and Microsoft-provided health reports to proactively
plan and manage infrastructure resources to maximize system and service
availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
DC-MS-03 Contractor shall fully implement, manage, and support all incident management
activities regarding Office 365 and Messaging services according to PBGC policies. By
using ITIL best practices, this should include, but shall not be limited to:

• Provide support for Microsoft Office 365 E3 tenant including Intune Mobile
Device Management (MDM) and Advanced Threat Protection (ATP)
• Provide support for PBGC’s on-premise IronPort email hygiene, relay, and
data loss prevention appliances

See SLA section for details on associated SLA measures.


DC-MS-04 Contractor shall fully implement, manage, and support all change management
activities regarding Office 365 and Messaging services according to PBGC policies. By
using ITIL best practices, this should include, but shall not be limited to:

• Provide integration with and configuration of Microsoft Office 365 E3 tenant


including Intune Mobile Device Management (MDM) and Advanced Threat
Protection (ATP)
• Provide configuration for PBGC’s on-premise IronPort email hygiene, relay,
and data loss prevention appliances
• Apply necessary configuration changes to achieve Office 365 and Messaging
objectives including compliance with PBGC policies and procedures as well as
to achieve desired operational configuration and user experience and align
with industry best practices

See SLA section for details on associated SLA measures.


DC-MS-05 Contractor shall fulfill all approved IT Security requests (in SecureIT) in accordance
with PBGC procedures and timelines for the following:

• Block email sender

See SLA section for details on associated SLA measures.


DC-MS-06 Contractor shall maintain and administer PBGC’s Microsoft Office 365 E3 tenant
including Intune Mobile Device Management (MDM) and Advanced Threat Protection
(ATP) including, but not limited to:
• Performing litigation holds and sensitive data collection requests
• Providing reports on message transport utilizing message tracking

53
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Updating hub transport rules
• Updating email retention policies
• Creating and removing shared mailboxes
• Delegating access to or transferring email or OneDrive contents as required
• Periodically run a script to copy all email content from separated and
deprovisoned users to a shared mailbox to preserve email according to
PBGC’s record schedule and remove license
• Troubleshoot/resolve identity synchronization issues with Azure AD
• Monitor Microsoft’s Office 365 IP and URL list and coordinate associated
PBGC network updates to ensure connectivity
DC-MS-07 Contractor shall maintain and administer PBGC’s on-premise IronPort email hygiene,
relay, and data loss prevention appliances including, but not limited to:
• Updating blacklists and whitelists including adding and removing email
domains and individual email addresses as required
• Updating authorized relay list including adding and removing IPs
• Perform periodic device configuration backups no less than monthly
• Examine and act on outbound messages quarantined by the DLP engine in
accordance with PBGC policy and procedures
DC-MS-08 Contractor shall enable and support email integration with SharePoint on-line and
perform other business process automation using Think Automation
DC-MS-09 Contractor shall enable and support email-enabled/integrated applications as well as
MDAemon Mail Server for application-based mail services that cannot leverage Office
365
DC-MS-10 Contractor shall maintain passwords for Office 365 and messaging privileged
accounts, e.g. tenant accounts, local system emergency recovery accounts, privileged
service accounts, etc. in privileged account management tool and utilize this tool to
perform administrative functions via brokered session or account check out
DC-MS-11 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the Microsoft Office 365 E3 tenant including Intune Mobile Device
Management (MDM) and Advanced Threat Protection (ATP) as well as PBGC’s on-
premise IronPort email hygiene, relay, and data loss prevention appliances

54
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.3.5 UNIX/LINUX Administration and Support
6.3.5.1 Scope of Services Supported
Provide tier 2 support for incidents relating to UNIX/LINUX servers or requiring administrative access to
Windows servers that cannot be resolved by an End-User services team as well as support service
requests. Typical UNIX/LINUX server administration and support activities include:

• Monitor, troubleshoot and repair or replace defective IT equipment (stand-alone servers, blade
infrastructure and servers, and associated components) as needed
• Monitor, troubleshoot, and repair UNIX/LINUX Server operating systems including space
cleanup, event log analysis, system daemon and package reconfiguration and reinstall, operating
system re-image when needed, etc.
• Troubleshoot and reinstall software including operating system daemons, RPM packages,
infrastructure software and business application software as needed on UNIX/LINUX servers,
e.g. SSSD, SFTP, Splunk, SAMBA, SAS, Oracle Web Logic middleware, Oracle e business suite, etc.
• Maintain up-to-date physical and virtual UNIX/LINUX Server operating system images/templates
• Provision and configure new physical and virtual servers as requested including SAN connectivity
• Perform initial installation and configuration of operating system daemons, RPM packages,
infrastructure software and business application software as needed on UNIX/LINUX servers
• Remove/decommission physical and virtual servers as requested
• Perform Active Directory account administration for privileged and service accounts supporting
the UNIX/LINUX environment using Quest ActiveRoles Server
• Deploy patches monthly to UNIX/LINUX servers utilizing patch deployment tool, e.g. BigFix
• Address operating system and software vulnerabilities detected on UNIX/LINUX servers during
monthly vulnerability scans
• Configure and administer UNIX/LINUX file systems and file shares
• Install and configure SFTP services
• Develop and maintain custom shell scripts to automate routine activities and support file
transfers
• Schedule automated tasks utilizing Cron
• Configure, monitor, troubleshoot, and repair high availability and disaster recovery capabilities
pertaining to UNIX/LINUX Servers and associated services including, but not limited to Veritas
clustering, failover procedures, etc.
• Support internal and external IP address management and name resolution services by updating
IPAM DNS records
• Maintain server racks
• Establish and maintain work instructions
• Escalate and work collaboratively with 3rd party vendors on hardware and software issues

Support for UNIX/LINUX Servers and IT equipment in the PBGC data centers is typically handled using
remote management software and protocols, e.g. vCenter console, SSH, Powershell, ILO, OA, etc., but
does occasionally require physical visits to address certain hardware and software problems. See
55
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix F - PBGC Locations for PBGC locations, including planned changes to PBGC’s facilities and data
centers over the life of the contract. See Appendix G - IT Service Support Guidelines for impact,
urgency, and prioritization guidelines associated with IT service and support. See Appendix J - IT Service
and Support 2018 Statistical Summary for tier 2 incidents, requests for information (RFIs), service/access
requests, requests for change (RFCs), and RFC tasks processed in calendar year 2018 for these services.
See Appendix K – IT Infrastructure Tools List for the software utilized to provides these services.

PBGC’s UNIX/LINUX Server environment consists of many components detailed in the tables that follow:

LINUX Server Summary

PBGC has approximately 361 RHEL servers, with more than 85% of them being virtual. The following
table breaks them down by physical/virtual and environment:

Operating System Physical/Virtual PROD COOP DMZ DEV TEST Total


Count
Linux Red Hat Virtual 43 26 3 95 42 209
Enterprise Server 6.10
Linux Red Hat Physical 9 5 9 6 29
Enterprise Server 6.10
Linux Red Hat Virtual 11 5 16
Enterprise Server 7.5
Linux Red Hat Physical 1 1 2
Enterprise Server 7.5
Linux Red Hat Virtual 14 9 5 47 17 92
Enterprise Server 7.6
Linux Red Hat Physical 11 2 13
Enterprise Server 7.6
Total Count 89 46 8 153 65 361
An upgrade from RHEL 6.x to 7.x has been initiated, however, business application dependencies on
Oracle Weblogic and Fusion Middleware 11g have left several legacy systems in place. The majority, but
not all, of these application dependencies are expected to be address by March 2020.

LINUX Physical Server Breakdown by Model

PBGC has approximately 44 physical RHEL servers, of which almost all are HP Proliant servers and the
majority of which are HP Proliant blade servers. The following table breaks them down by model and
operating system:

Hardware Model Vendor RHEL 6.x RHEL 7.x Total Count


ProLiant BL460c G7 HP 4 1 5
ProLiant BL460c G8 HP 25 3 28
ProLiant DL360 G5 HP 1 1

56
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
ProLiant DL360 G6 HP 1 1
ProLiant DL360 G8 HP 3 3
ProLiant DL360 G9 HP 3 3
ProLiant DL380 G5 HP 2 2
ProLiant DL380 G8 HP 1 1
Total Count 29 15 44
An upgrade from HP Proliant Generation 5, 6 and 7 servers to HP Proliant Generation 10 servers is
planned for FY19 as part of the RHEL 7 and Oracle Fusion Middleware upgrades.

UNIX Server Summary

PBGC currently has 3 physical domains on 3 physical Oracle Solaris servers. These servers are all running
Oracle Solaris 11 (5.11):

Hardware Physical Physical Virtual


Model Server Domain Machines
Count Count (Ldom)
T5-8 (5.11) 3 3 29

EnableIT Service Requests

The following table provides the count for EnableIT service requests processed in Calendar Year 2018 for
LINUX/UNIX servers:

Service Request Category Total


Requests
Server: New Server 44
Server: Request Additional Memory or 26
CPU

6.3.5.2 Requirements
The Contractor shall provide incident response and resolution for issues relating to UNIX/LINUX servers
or requiring administrative access to UNIX/LINUX servers that cannot be resolved by an End-User
services team as well as support service requests and requests for change.

The contractor shall provide the UNIX and LINUX support services outlined in the following table:

Reference Requirement
(DC-UX-xx)
DC-UX-01 Contractor shall identify a lead for the UNIX/LINUX area. This lead is required to serve
as the primary point of contact for all UNIX and LINUX related issues.

57
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DC-UX-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
UNIX/LINUX Server infrastructure in all PBGC computing environments. This shall
include, but is not limited to:

• Identifying and addressing performance bottlenecks


• Performing capacity planning and management including allocation, resizing,
and reconfiguration of UNIX/LINUX Servers and associated services as needed
• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
DC-UX-03 Contractor shall fully implement, manage, and support all incident management
activities regarding UNIX/LINUX servers and the associated services they provide
according to PBGC policies. By using ITIL best practices, this should include, but shall
not be limited to:

• Provide support for UNIX/LINUX server hardware and operating systems


including basic network and SAN connectivity
• Provide support for IT supported applications and services deployed on
UNIX/LINUX servers including, but not limited to SSSD, SFTP, Splunk, SAMBA,
SAS, Oracle Web Logic middleware, Oracle e business suite, etc.

See SLA section for details on associated SLA measures.


DC-UX-04 Contractor shall fully implement, manage, and support all change management
activities regarding UNIX/LINUX servers and the associated services they provide
according to PBGC policies. By using ITIL best practices, this should include, but shall
not be limited to:

• Provide installation and configuration of UNIX/LINUX server hardware and


operating systems including basic network and SAN connectivity or
decommission when no longer required
• Provide installation and configuration of IT supported applications and
services deployed on UNIX/LINUX servers including, but not limited to SSSD,
SFTP, Splunk, SAMBA, SAS, Oracle Web Logic middleware, Oracle e business
suite, etc. or remove when no longer required
• Apply applicable security patches at least monthly to UNIX/LINUX servers that
do not host Oracle software and services and at least quarterly for
UNIX/LINUX servers that do host Oracle software and services. Install
applicable hardware firmware updates at least quarterly.
• Address operating system and software vulnerabilities detected on
UNIX/LINUX servers during monthly vulnerability scans
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices
58
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
See SLA section for details on associated SLA measures.
DC-UX-05 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New Unix or Linux Servers


• Server: Request Additional Memory or CPU (UNIX/LINUX Servers)

See SLA section for details on associated SLA measures.


DC-UX-06 Contractor shall maintain up-to-date physical and virtual LINUX (RHEL) Server
operating system images and templates for PBGC supported LINUX Server operating
system versions that are in compliance with PBGC-approved security baselines and
that include PBGC approved security patches within 60 days of production security
patch approval, i.e. updated no less infrequently than every other month
DC-UX-07 Contractor shall perform UNIX/LINUX account administration functions in accordance
with PBGC policy and procedures for systems and functions where automation is not
already in place including, but not limited to:
• Active Directory using Quest Active Roles Server:
o Account unlocks and emergency/temporary disable/re-enable for
privileged and service accounts that support the UNIX/LINUX Server
environment and associated services
o Password Resets (including remote user verification when applicable
per PBGC policy) for privileged and service accounts that support the
UNIX/LINUX Server environment and associated services
DC-UX-08 Contractor shall maintain server racks to include, but not limited to:
• ensuring front and rear doors can be closed and are closed
• ensuring patch cables are of suitable length, are free from entanglement, are
patched the appropriate port based on applicable PBGC standards, and are
routed neatly to patch panel
• ensuring power to redundant power supplies is properly distributed between
PDUs to minimize downtime due to a single component failure
• maintaining Visio diagrams of the contents of each rack with
updates/verification no less than every 6 months
ensuring equipment not in use is powered down, clearly labeled, and removed from
the rack within 90 days of decommission
DC-UX-09 Contractor shall ensure all UNIX/LINUX accounts supporting the UNIX/LINUX server
environment, e.g. local UNIX/LINUX root account; UNIX/LINUX service accounts; etc.,
are changed periodically in accordance with PBGC policy and procedures and the
passwords are stored for emergency use
DC-UX-10 Contractor shall provide installation and maintenance support for SFTP services to
provide secure file access and transfer mechanisms over SSH tunnels
DC-UX-11 Contractor shall provide installation and configuration support for Samba file services
59
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DC-UX-12 Contractor shall create/maintain shell scripts as needed to:
• Automate file transfers
• Automate aspects of computer maintenance
• Startup applications on reboots
DC-UX-13 By the end of January of each year, contractor shall:
• Upgrade each Java Platform (JRE/JDK) instance on Linux and UNIX servers to
the identified target version established the previous January unless risk
accepted
• Identify and communicate the target Java platform version for the following
January
DC-UX-14 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the UNIX/LINUX server environment and the associated services

6.3.6 Virtualization Platforms Administration and Support


6.3.6.1 Scope of Services Supported
Provide tier 2 support for incidents relating to PBGC’s Virtualization Platform(s) or requiring
administrative access to PBGC’s Virtualization Platform(s) that cannot be resolved by an End-User
services team as well as support service requests. Typical Virtualization Platform(s) administration and
support activities include:

• Monitor, troubleshoot, and repair Virtualization Platform server operating systems, clusters,
datastores, etc. including space cleanup, event log analysis, feature reconfiguration and
reinstall, operating system re-image when needed, etc.
• Provision and configure new physical servers and load and configure hypervisor software in
compliance with PBC security baselines as requested including SAN connectivity
• Remove/decommission physical servers providing virtualization services as requested
• Deploy patches no less than quarterly to virtualization platform utilizing virtualization platform
patch deployment tool
• Address operating system and software vulnerabilities detected on PBGC’s Virtualization
Platform(s) during monthly vulnerability scans
• Configure and administer virtualization platform data stores
• Migrate physical servers to virtual servers when technically feasible
• Configure, monitor, troubleshoot, and repair high availability and disaster recovery capabilities
pertaining to PBGC’s virtualization platforms and associated services including, but not limited
to VMware clustering, VMware high availability, VMware Distributed Resource Scheduler (DRS),
etc.
• Establish and maintain work instructions
• Escalate and work collaboratively with 3rd party vendors on hardware and software issues

Support for PBGC’s Virtualization Platform(s) and the associated IT equipment in the PBGC data centers
is typically handled using remote management software, e.g. vCenter console, SSH, Powershell, ILO, OA,

60
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
etc., but does occasionally require a physical visit to address certain hardware and software problems.
See Appendix F - PBGC Locations for PBGC locations, including planned changes to PBGC’s facilities and
data centers over the life of the contract. See Appendix G - IT Service Support Guidelines for impact,
urgency, and prioritization guidelines associated with IT service and support. See Appendix J - IT Service
and Support 2018 Statistical Summary for tier 2 incidents, requests for information (RFIs), service/access
requests, requests for change (RFCs), and RFC tasks processed in calendar year 2018 for these services.
See Appendix K – IT Infrastructure Tools List for the software utilized to provides these services.

PBGC’s Virtualization environment is comprised primarily of VMware software running on HP hardware.


PBGC’s version of VMware ESXi, vCenter, View, Horizon, etc. is largely at version 6.5. The following
tables provide additional details about PBGC’s Virtualization environment:

VMware Cluster Summary

PBGC currently has 18 VMware clusters supporting more than 750 virtual servers (Windows and RHEL),
several virtual appliances, and more than 500 virtual Windows enterprise desktops. The following table
summarizes their function and notes the number of ESXi hosts comprising each cluster as well as the
number of virtual machines, datastores, and networks it supports:

Cluster Name Function Number Number Number of Number of


of Hosts of VMs Datastores Networks
(in use)
HQW_MGMT Small cluster to host 2 22 4 7
vCenter, its SQL
database server, and a
few other virtual
servers that are
essential to operate
the remainder of the
Headquarters PBGC
environment, e.g. AD
domain controllers
HQWPROD_CiscoPhone Primary Cisco 5 10 4 1
proprietary cluster
used to host Unified
Communications and
UCCX servers running
on BE7H-M5-K9
hardware
HQWPROD_LowCore Small cluster 2 0 4 10
comprised of ESXi
servers with a small
number of processor
cores to host

61
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
production virtual
machines that run
expensive software
licensed by core
processor count to
control costs, e.g.
Oracle Web Center;
slated for
decommission
HQWPROD_RHEL Primary cluster to host 5 74 10 13
production RHEL
virtual servers
HQWPROD_RHEL_12c Primary cluster to host 4 0 5 14
production RHEL
virtual servers
supporting 12c
WebLogic middleware
HQWPROD_VDI Primary cluster to host 6 236 10 8
production VDI for
Windows enterprise
desktops supporting
off-site staff, e.g.
actuaries, IT Service
Desk, etc.
HQWPROD_Windows Primary cluster to host 10 152 20 13
production Windows
virtual servers
HQWTCO_ITCVDI Small cluster to host 2 10 4 16
VDI for Windows
enterprise desktops
supporting user
acceptance testing in
the ITC which will
ultimately replace a
physical lab consisting
of ~75 desktops
HQWTCO_LowCore Small cluster 2 9 4 15
comprised of ESXi
servers with a small
number of processor
cores to host
development and test
virtual machines that
run expensive
62
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
software licensed by
core processor count
to control costs, e.g.
Oracle Web Center
HQWTCO_RHEL Primary cluster to host 5 145 12 15
Development and Test
RHEL virtual servers
HQWTCO_RHEL-12c Primary cluster to host 3 26 12 15
Development and Test
RHEL virtual servers
supporting 12c
WebLogic middleware
HQWTCO_RHEL-BI Primary cluster to host 3 3 6 15
Development and Test
RHEL virtual servers
supporting BI
HQWTCO_VDI Primary cluster to host 9 279 11 15
Development and Test
VDI for Windows
enterprise desktops
primarily supporting
application
development and
testing
HQWTCO_Windows Primary cluster to host 10 296 26 21
Development and Test
Windows virtual
servers
WIL_MGMT Small cluster to host 2 22 4 7
vCenter, its SQL
database server, and a
few other virtual
servers that are
essential in the event
PBGC must operate
from its DR site
WILPROD_RHEL Primary cluster to host 4 41 10 11
RHEL virtual servers
used in the event
PBGC must operate
from its DR site
WILPROD_Windows Primary cluster to host 3 50 10 11
Windows virtual
servers used in the
63
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
event PBGC must
operate from its DR
site
WILPROD_CiscoPhone Cisco proprietary 5 14 16 1
cluster used to host
Unified
Communications and
UCCX servers running
on BE7H-M5-K9
hardware at DR site
Total Counts 82 1,389

VMware Physical Server Breakdown by Model

PBGC has approximately 79 physical VMware ESXi servers supporting 12 VMware clusters of which are
all are HP Proliant blade servers. The following table breaks them down by model and cluster:

Cluster Name HP BL460c G8 HP BL460c G10 Total


HQW_MGMT 2 2
HQWPROD_LowCore 2 2
HQWPROD_RHEL 5 5
HQWPROD_RHEL_12c 4 4
HQWPROD_VDI 6 6
HQWPROD_Windows 10 10
HQWTCO_ITCVDI 2 2
HQWTCO_LowCore 2 2
HQWTCO_RHEL 5 5
HQWTCO_RHEL-12c 2 1 3
HQWTCO_RHEL-BI 3 3
HQWTCO_VDI 9 9
HQWTCO_Windows 10 10
WIL_MGMT 2 2
WILPROD_RHEL 2 2 4
WILPROD_Windows 3 3
Total Count 22 50 72
An upgrade from HP Proliant Generation 7 and 8 VMware servers to HP Proliant Generation 10 servers is
long underway with expected completion by the end of FY19. There are also two clusters,
HQWPROD_CiscoPhone and WILPROD_CiscoPhone, that are Cisco proprietary used to host Cisco UC
comprised of BE7H-M5-K9 servers.

64
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.3.6.2 Requirements
The Contractor shall provide incident response and resolution for issues relating to PBGC’s Virtualization
Platform(s) or requiring administrative access to PBGC’s Virtualization Platform(s) that cannot be
resolved by an End-User services team as well as support service requests and requests for change. The
contractor shall provide the Virtualization Platforms administration and support services outlined in the
following table:

Reference Requirement
(DC-VM-xx)
DC-VM-01 Contractor shall identify a lead for the Virtualization area. This lead is required to
serve as the primary point of contact for all virtualization related issues.
DC-VM-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
Virtualization Platform infrastructure in all PBGC computing environments. This shall
include, but is not limited to:

• Identifying and addressing performance bottlenecks


• Performing capacity planning and management including allocation, resizing,
and reconfiguration of Virtualization Platform servers, i.e. hypervisor servers
and associated services as needed
• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
DC-VM-03 Contractor shall fully implement, manage, and support all incident management
activities regarding PBGC’s Virtualization Platform according to PBGC policies. By
using ITIL best practices, this should include, but shall not be limited to:

• Provide support for Virtualization Platform server operating systems, clusters,


datastores, and Virtual Desktop Infrastructure (VDI)

See SLA section for details on associated SLA measures.


DC-VM-04 Contractor shall fully implement, manage, and support all change management
activities regarding PBGC’s Virtualization Platform according to PBGC policies. By
using ITIL best practices, this should include, but shall not be limited to:

• Provision and configure new physical servers and load and configure
hypervisor software in compliance with PBC security baselines as requested
including SAN connectivity
• Provide installation and configuration of virtualization platform management
tools, e.g. vCenter, Horizon View, etc.
• Provision resources to virtual servers and workstations as required to ensure
optimal operation including CPUs, memory, disk drives/space, etc.
• Apply applicable security patches at least quarterly
65
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Address operating system and software vulnerabilities detected on PBGC’s
Virtualization Platform during monthly vulnerability scans
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices

See SLA section for details on associated SLA measures.


DC-VM-05 Contractor shall provision and manage access to virtual desktops utilized in support of
off-site users and secondary developer workstations
DC-VM-06 Contractor shall configure and administer virtualization platform data stores
DC-VM-07 Contractor shall migrate physical servers to virtual servers when technically feasible
DC-VM-08 Contractor shall configure, monitor, troubleshoot, and repair high availability and
disaster recovery capabilities pertaining to PBGC’s virtualization platforms and
associated services including, but not limited to VMware clustering, VMware high
availability, VMware Distributed Resource Scheduler (DRS), etc.

DC-VM-09 Contractor shall ensure all accounts supporting the Virtualization Platform
environment, e.g. local VMware ESXi administrator; Active Directory service accounts;
etc., are changed periodically in accordance with PBGC policy and procedures and the
passwords are stored for emergency use
DC-VM-10 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain PBGC’s Virtual Platform(s)

6.3.7 Storage and Backup Administration and Support


6.3.7.1 Scope of Services Supported
Provide tier 2 support for incidents relating to enterprise storage and backup administration and that
cannot be resolved by an End-User services team as well as support service requests. Typical Storage
and Backup administration and support activities include:

• Management of storage systems including provisioning and capacity planning


• Configure and support enterprise storage protocols including Fiber Channel, iSCI, FCoE, CIFS, NFS
protocols and hardware which use these products
• Troubleshoot storage access issues
• Monitor faults, performance, and capacity issues
• Perform zoning activities on Brocade SAN fabric switches including multi-pathing configuration
• Manage RAID groups, Host groups and LUN allocation
• Support database refreshes from Production to development and test environments
• Perform nightly differential backups, weekly full backups, and ad-hoc restoration from backups
for PBGC platforms including, but not limited to:
o Windows Server
o Unix (Solaris)
o Linux (RHEL)
o VMware vCenter
o MS SQL Server Windows
66
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
o Oracle Databases using RMAN
• Identify, troubleshoot, and resolve backup/recovery issues in a timely manner
• Configure, monitor, and maintain backup/recovery software and hardware so that it complies
with specified security guidelines
• Apply software upgrades, security patches and planned maintenance of the enterprise storage
and backup environment
• Diagnose, solve and provide root cause analysis for specialized storage and backup hardware
and software related issues
• Provide reports regarding storage and backup capacity and performance
• Configure, monitor, troubleshoot, and repair high availability and disaster recovery capabilities
pertaining to Storage and Backup infrastructure and associated services including, but not
limited to clustering, RAID groups, redundant systems, multi-pathing, replication, snapshots,
failover procedures, etc.
• Migrate data and services to new storage and backup infrastructure services

Support for storage and backup equipment in the PBGC data centers is typically handled using remote
management software and protocols, e.g. https, SSH, Powershell, ILO, OA, etc., but does occasionally
require physical visits to address certain hardware and software problems. See Appendix F - PBGC
Locations for PBGC locations, including planned changes to PBGC’s facilities and data centers over the
life of the contract. See Appendix G - IT Service Support Guidelines for impact, urgency, and
prioritization guidelines associated with IT service and support. See Appendix J - IT Service and Support
2018 Statistical Summary for tier 2 incidents, requests for information (RFIs), service/access requests,
requests for change (RFCs), and RFC tasks processed in calendar year 2018 for these services. See
Appendix K – IT Infrastructure Tools List for the software utilized to provides these services.

PBGC’s current storage infrastructure primarily consists of Brocade DCX8510-4 (Primary data center) and
6520 (DR data center) fibre channel switches and Hitachi Virtual Storage (HVS) platform G-900 (HQ) and
G-700 (COOP) series storage arrays. PBGC is currently migrating data and services from Hitachi HUS and
NetApp Fabric-Attached Storage (FAS) arrays to the VSP platform and is largely done. The total usable
capacity of PBGC’s VSP SAN arrays is over one exabyte and considering all PBGC SAN arrays, capacity
currently amounts to more than 2 exabytes broken down as follows as of March 2019:

Storage Array Usable Utilized Functions


Model Capacity Space
(TBs) (TBs)
Hitachi Virtual 794 420 Supports Primary Data Center equipment/services:
Storage Platform • Production UNIX/Oracle/flashback – 51 TBs
(VSP) G900 • Dev/Test UNIX/Oracle/flashback – 120 TBs
• Production VMware server cluster datastores
– 78 TBs
• Dev/Test VMware server cluster datastores –
88 TBs

67
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Production VMware VDI cluster datastores –
24 TBs
• Dev/Test VMware VDI cluster datastores – 36
TBs
• Production MS SQL Server storage – 9 TBs
• Production Windows Server storage – 12 TBs
• Dev/Test Windows Server storage – 0 TBs
• Production LINUX Server storage – 2 TBs
• Dev/Test LINUX Server storage – 0 TBs
Hitachi Virtual 355 109 Supports Secondary (DR/COOP) Data Center
Storage Platform equipment/services:
(VSP) G700 • COOP UNIX/Oracle/flashback – 0 TBs –
migration is in progress
• COOP VMware server cluster datastores – 44
TBs
• COOP MS SQL Server storage – 0 TBs
• COOP Windows Server storage – 38 TBs
• COOP LINUX Server storage – 27 TBs
Hitachi Unified 130 13.5 Supports lower-tier, legacy storage needs at HQ:
Storage (HUS) • Windows server storage – 3 TBs
150 • Legacy Backup Catalog – 10.5 TBs
Hitachi Unified 130 3 Supports lower-tier, legacy storage needs at DR site:
Storage (HUS) • Windows server storage – 3 TBs
150
Hitachi Unified 464 0 Supported higher-tier storage at HQ
Storage (HUS) VM
Hitachi Unified 93 19 Supports higher-tier, legacy storage needs at DR site:
Storage (HUS) VM • DR UNIX/Oracle – 19 TBs
NetApp FAS8040 295 115 Supports higher-tier, legacy storage needs at HQ:
• Production file shares – 38 TBs
• Production IPS file system – 28 TBs
• Production NFS and CIFS shares – 9 TBs
• Dev/Test file shares – 20 TBs
• Dev/Test NFS and CIFS shares – 17 TBs
• NetApp – 3 TBs
Migration to G900 is in progress
NetApp FAS2554 80 63 Supports higher-tier, legacy storage needs at DR site:
• DR file shares replicated from HQ – 34 TBs
• DR IPS file system – 28 TBs
• DR NFS and CIFS shares – 1 TB
Migration to G700 is in progress

68
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Veritas NetBackup software is predominantly used to support backup and restore operations. A master
server controls backup of all production data at HQ and remote locations. All non-production data is
backed up to a pure-disk based Veritas NetBackup 5230 appliance. All production data is backed up to a
3 pure-disk based Veritas NetBackup 5240 appliances (1 masters, 2 media) appliances and replicated to
a DR site to a mirror set of appliances using NetBackup’s Auto Image Replication (AIR). PBGC plans to
move some of its older backup data to Azure storage in the next year. In the interim, PBGC may make
use of its SAN arrays to house this data if it exceeds are capacity. Backup data from the development
environments are retained for 90 days, while the agency’s retention policies for production data require
that data be kept for a maximum of seven years. The agency has a continuing requirement to store tape
backup media on-site at HQ until 2025 to support data restore requests. The legacy backup medium is
LTO tapes which are bar-coded before being loaded into IBM TS3310 tape library, but these are rarely if
ever used.

EnableIT Service Requests

The following table provides the count for EnableIT service requests processed in Calendar Year 2018

Service Request Category Total


Requests
Storage: Storage Allocation 268
Oracle: Database Refresh 107

6.3.7.2 Requirements
The contractor shall provide the storage and backup infrastructure support services outlined in the
following table:

Reference Requirement
(DC-SB-xx)
DC-SB-01 Contractor shall identify a lead for the storage and backup area. This lead is required
to serve as the primary point of contact for all storage and backup related issues.
DC-SB-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
Storage and Backup infrastructure in all PBGC computing environments. This shall
include, but is not limited to:

• Identifying and addressing performance bottlenecks


• Performing capacity planning and management including allocation, resizing,
and reconfiguration of storage and backup infrastructure and associated
services as needed
• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts

69
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DC-SB-03 Contractor shall fully implement, manage, and support all incident management
activities related to Storage and Backup infrastructure and associated services
according to PBGC policies and procedures. By using ITIL best practices, this should
include, but shall not be limited to:
• Troubleshoot storage access issues
• Monitor faults, performance, and capacity issues
• Identify, troubleshoot, and resolve backup/recovery issues
• Diagnose, solve and provide root cause analysis for specialized storage and
backup hardware and software related issues

See SLA section for details on associated SLA measures.


DC-SB-04 Contractor shall fully implement, manage, and support all change management
activities regarding PBGC’s Storage and Backup infrastructure and associated services
according to PBGC policies. By using ITIL best practices, this should include, but shall
not be limited to:

• Provision storage to hosts including zoning, masking, management of RAID


groups, Host groups, LUN allocation, and configuration of multi-pathing
• Support database refreshes from Production to development and test
environments
• Migrate data and services to new storage and backup infrastructure services
• Apply applicable security patches at least quarterly
• Address system and software vulnerabilities detected on PBGC’s
Virtualization Platform during monthly vulnerability scans
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices

See SLA section for details on associated SLA measures.


DC-SB-05 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New Storage Allocation

See SLA section for details on associated SLA measures.


DC-SB-06 Contractor shall monitor and improve capacity utilization by decommissioning unused
devices/LUNs on multiple servers, and performing Storage Capacity, Storage
Performance audits
DC-SB-07 Contractor shall implement and maintain backup and restoration capabilities for all
data, applications and component configurations in accordance with the agency’s
approved backup, restoration, and data retention policies
DC-SB-08 Contractor shall ensure sufficient backup and recovery controls to provide reasonable
assurance that the datacenter will be able to recover from loss or destruction of data-
processing facilities, hardware, software, or data
70
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DC-SB-09 Contractor shall review the backup success or failure status of backup jobs daily and
take timely action to identify and resolve exceptions
DC-SB-10 Contractor shall establish and maintain documentation regarding the enterprise
storage and backup environment including:
• Enterprise storage and backup system diagrams
• Capacity Utilization and Availability Report
DC-SB-11 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the storage and backup environment and the associated services

6.3.8 Database Administration and Support


6.3.8.1 Scope of Services Supported
Provide tier 2 support for incidents relating to enterprise relational database administration and support
that cannot be resolved by an End-User services team as well as support service requests. Provide
database administration support to create, maintain, and manage several enterprise-wide databases
that are used by various COTS and custom applications across the agency. This support also includes
providing operational database assistance and support to the applications teams as needed to
implement and maintain applications systems.

Typical database administration and support activities include:

• Installation, organization, storage, management, administration and efficient retrieval and


storage of data for database management systems (DBMS)
• Database administration on all existing and new database instances that support agency’s
custom applications
• Perform optimization of database to increase performance and make recommendations for
improving database performance, and implement approved recommendations
• Identify and resolve system errors and assist at troubleshooting application program errors
• Deploy and integrate new applications, troubleshoot problems, and install upgrades
• Design and create database structures
• Prepare database environment(s) for application deployments and migrate data across
environments
• Perform data masking to enforce controls to protect sensitive data per PBGC policies and
procedures
• Develop and/or maintain up-to-date documents showing the configuration of all databases
• Create and test backups of data, provide data cleansing services, verify data integrity,
implement access controls
• Creation/modification of custom tables, indexes, tablespaces and other DDL requests
• Performance tuning, capacity planning, and architecture planning
• Perform database reorganizations as required to assist performance and ensure maximum
uptime of the databases
• Provide technical support to application services teams on matters related to the databases
• Enforce and maintain database constraints to ensure integrity of the databases
71
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Troubleshoot problems regarding the databases, applications and development tools
• Perform review, revise and implement security policies and access rights to database servers
and database
• Automate database management and backup tasks
• Configure, monitor, troubleshoot, and repair high availability and disaster recovery capabilities
pertaining to enterprise relational Database infrastructure systems and associated services
including, but not limited to Windows clustering, database mirroring, database replication,
failover procedures, etc.
• Establish and maintain work instructions
• Escalate and work collaboratively with 3rd party vendors on software issues

Support for databases in the PBGC data centers is typically handled using remote management software
and protocols, e.g. https, SSH, Powershell, ILO, OA, etc., but does occasionally require physical visits to
address certain hardware and software problems. See Appendix F - PBGC Locations for PBGC locations,
including planned changes to PBGC’s facilities and data centers over the life of the contract. See
Appendix G - IT Service Support Guidelines for impact, urgency, and prioritization guidelines associated
with IT service and support. See Appendix J - IT Service and Support 2018 Statistical Summary for tier 2
incidents, requests for information (RFIs), service/access requests, requests for change (RFCs), and RFC
tasks processed in calendar year 2018 for these services. See Appendix K - IT Infrastructure Tools List for
the software utilized to provides these services.

PBGC’s current enterprise relational database infrastructure is heterogeneous in nature and primarily
consists of Oracle Relational Database Management Systems (RDBMS), and Microsoft SQL Server.
Historically PBGC predominantly used Oracle RDBMS, but more recently has been moving towards
Microsoft SQL Server for its relational database system and PBGC is considering use of PostgreSQL in the
near future. Being able to provide any needed PostgreSQL support is a requirement of this contract.
Data masking is performed utilizing Dataguise.

The following table provides a recent summary of PBGC’s enterprise relational database inventory
spread across approximately 29 Oracle servers and 31 Microsoft SQL servers:

Database Database Count Size


Oracle 11g (11.2.x) Dev: 19 Dev: 6,700 GBs
Test: 8 Test: 3,700 GBs
Prod: 8 Prod: 2,735 GBs
COOP: 6 COOP: 1,773 GBs
Total: 41 Total: 14,908 GBs
Oracle 12c (12.1.x) Dev: 34 Dev: 28,300 GBs
Test: 11 Test: 7,900 GBs
Prod: 7 Prod: 4,551GBs
COOP: 5 COOP: 4,356 GBs
Total: 57 Total: 45,107 GBs
Oracle 12c (12.2.x) Dev: 48 Dev: 16,200 GBs

72
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Test: 18 Test: 4,100 GBs
Prod: 21 Prod: 5,028 GBs
COOP: 10 COOP: 3,500 GBs
Total: 97 Total: 28,828 GBs
Microsoft SQL Server 2012 Dev:17 Dev: 148 .07 GBs
Test: 2 Test: 24.93 GBs
Prod:10 Prod: 32.72 GBs
COOP: 2 COOP: 31.60 GBs
Total: 31 Total: 237.32 GBs
Microsoft SQL Server 2014 Dev: 13 Dev: 64.25 GBs
Test: 13 Test: 69.97 GBs
Prod: 12 Prod: 29.04 GBs
COOP: None COOP: 0 GBs
Total: 38 Total: 163.26 GBs
Microsoft SQL Server 2016 Dev: 26 Dev: 126.40 GBs
Test: 10 Test: 30.21 GBs
Prod: 21 Prod: 291.81 GBs
COOP: 1 COOP: 8.39 GB
Total: 58 Total: 492.81GBs
Microsoft SQL Server 2017 Dev: 66 Dev: 1138.31GBs
Test: 17 Test: 203.74 GBs
Prod: 35 Prod: 587.29 GBs
COOP: 2 COOP: 18.55 GBs
Total: 120 Total: 1,947.89 GBs
PostgreSQL Planned; none Planned; none presently
presently

EnableIT Service Requests

The following table provides the count for EnableIT service requests processed in Calendar Year 2018:

Service Request Category Total


Requests
Oracle: Database Refresh 107
Oracle: Data-Fix/Script 2,382
Oracle: New Database 5
Oracle: New Database Role 14
SQL Server: New Database 13

6.3.8.2 Requirements
For each database platform referred to above, the contractor shall provide the database management
support services outlined in the following table:
73
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Reference Requirement
(DC-DB-xx)
DC-DB-01 Contractor shall identify a lead for the Database Management area. This lead is
required to serve as the primary point of contact for all relational database
management related issues.
DC-DB-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
enterprise relational Database infrastructure in all PBGC computing environments.
This shall include, but is not limited to:

• Identifying and addressing performance bottlenecks


• Performing capacity planning and management including allocation, resizing,
and reconfiguration of relational database management systems and
associated services as needed
• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
DC-DB-03 Contractor shall fully implement, manage, and support all incident management
activities regarding enterprise relational Database infrastructure and associated
services according to PBGC policies. By using ITIL best practices, this should include,
but shall not be limited to:

• Provide support for IT supported applications and services deployed to


PBGC’s enterprise relational database management systems including, but
not limited to Oracle, Microsoft SQL Server, PostgreSQL, etc.

See SLA section for details on associated SLA measures.


DC-DB-04 Contractor shall fully implement, manage, and support all change management
activities regarding enterprise relational Database infrastructure and associated
services they provide according to PBGC policies. By using ITIL best practices, this
should include, but shall not be limited to:

• Provide installation and configuration of database components such as


databases, schemas, tables, clusters, indexes, views, sequences, roles,
packages and procedures
• Provide installation and configuration of for IT supported applications and
services deployed to PBGC’s enterprise relational database management
systems including, but not limited to Oracle, Microsoft SQL Server,
PostgreSQL, etc. or remove when lo longer required
• Perform database data fixes
• Perform database refreshes to include data masking where applicable
• Perform database restores to include data masking where applicable
• Perform database reorganizations
74
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Enable replication for disaster recovery
• Apply applicable security patches at least quarterly
• Address software vulnerabilities detected on enterprise relational Database
infrastructure during monthly vulnerability scans
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices

See SLA section for details on associated SLA measures.


DC-DB-05 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New Database
• Database Fixes (updates)
• Database Refresh
• Database Restore
• New Database Role

See SLA section for details on associated SLA measures.


DC-DB-06 Contractor shall enforce and maintain database constraints to ensure integrity of the
database and administer all database objects, including tables, clusters, indexes,
views, sequences, packages and procedures
DC-DB-07 Contractor shall perform enterprise relational Database account administration
functions in accordance with PBGC policy and procedures for systems and functions
where automation is not already in place including, but not limited to:
• Oracle, Microsoft SQL, and PostgreSQL:
o Regular User, Privileged and Service Account creation and
modification including role assignments
o Account unlocks and emergency/temporary disable/re-enable
o Password Resets (including remote user verification when applicable
per PBGC policy)
o Account removal upon request or separation
DC-DB-08 Contractor shall manage sharing of database resources amongst applications
DC-DB-09 Contractor shall ensure all accounts supporting the enterprise relational Database
infrastructure environment, e.g. service (schema) accounts; privileged accounts; etc.,
are changed periodically in accordance with PBGC policy and procedures and the
passwords are stored for emergency use
DC-DB-10 Contractor shall plan and implement backup and recovery of databases in accordance
with established PBGC procedures and standards
DC-DB-11 Contractor shall leverage PBGC-owned database management tools, e.g. Oracle
Enterprise Manager to automate repetitive tasks whenever possible. When not
possible, contractor shall create and manage scripts needed to automate repetitive
tasks.

75
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DC-DB-12 Contractor shall copy data between environments as required to support application
development and testing and while doing so, contractor will enforce controls to
protect sensitive data per PBGC policies and procedures including data masking
DC-DB-13 Contractor shall participate in planning for the releases of new COTS/GOTS
applications to ensure that any new product usage or release upgrade takes place
with minimal impact
DC-DB-14 Contractor shall deploy database updates as required for the release of new
COTS/GOTS applications to ensure that any new product usage or release upgrade
takes place with minimal impact
DC-DB-15 Contractor shall monitor and improve capacity utilization by decommissioning unused
databases
DC-DB-16 Contractor shall establish and maintain documentation regarding the enterprise
database management environment including:
• An inventory of all databases broken down by environment, server, and
database version
DC-DB-17 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the enterprise relational Database infrastructure and the associated services
DC-DB-18 Contractor shall install, configure and support a PostgreSQL relational database
management system including support for high availability, backup and recovery, and
disaster recovery. Contractor shall establish and maintain work instructions to
support this database platform.

6.3.9 Web and Application Middleware Administration and Support


Provide tier 2 support for incidents relating to enterprise Web and Application Middleware
administration and support that cannot be resolved by an End-User services team as well as support
service requests. Provide Web and Application Middleware support to create, maintain, and manage
several enterprise-wide web and application and middleware platforms that are used by various COTS
and custom applications across the agency. This support also includes providing operational Web and
Application Middleware assistance and support to the applications teams as needed to implement and
maintain applications systems.

Typical enterprise Web and Application Middleware administration and support activities include:

• Installation, configuration, management, administration and optimization of Web and


Application Middleware infrastructure
• Create and manage connections to databases and application interfaces including those that
interact with external entities
• Optimize configuration for Java Virtual Machines (JVMs)
• Provide application code deployment services in support of COTS/GOTS application releases
• Apply applicable security patches

76
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Address software vulnerabilities detected on enterprise infrastructure during monthly
vulnerability scans on Web and Application Middleware
• Apply necessary configuration changes to ensure compliance with PBGC security baselines and
industry best practices

• Identify and resolve system errors and assist at troubleshooting application program errors
• Deploy and integrate new applications, troubleshoot problems, and install upgrades
• Prepare Web and Application Middleware infrastructure environment(s) for application
deployments
• Develop and/or maintain up-to-date documents showing the inventory and configuration of all
Web and Application Middleware infrastructure
• Performance tuning, capacity planning, and architecture planning
• Provide technical support to application services teams on matters related to the Web and
Application Middleware infrastructure
• Automate Web and Application Middleware infrastructure management and backup tasks
• Configure, monitor, troubleshoot, and repair high availability and disaster recovery capabilities
pertaining to enterprise Web and Application Middleware infrastructure systems and associated
services including, but not limited to clustering, network load balancing, failover procedures,
etc.
• Establish and maintain work instructions
• Escalate and work collaboratively with 3rd party vendors on software issues

Support for web and application middleware running in the PBGC data centers is typically handled using
remote management software and protocols, e.g. https, SSH, Powershell, ILO, OA, etc., but does
occasionally require physical visits to address certain hardware and software problems. See Appendix F
- PBGC Locations for PBGC locations, including planned changes to PBGC’s facilities and data centers
over the life of the contract. See Appendix G - IT Service Support Guidelines for impact, urgency, and
prioritization guidelines associated with IT service and support. See Appendix J - IT Service and Support
2018 Statistical Summary for tier 2 incidents, requests for information (RFIs), service/access requests,
requests for change (RFCs), and RFC tasks processed in calendar year 2018 for these services. See
Appendix K - IT Infrastructure Tools List for the software utilized to provides these services.

PBGC’s current enterprise Web and Application Middleware infrastructure is heterogeneous in nature
and primarily consists of Oracle WebLogic, Oracle SOA suite, Oracle BPEL, Oracle forms and reports,
Microsoft IIS and .Net software, and PBGC’s Oracle E Business Suite applications including CRM and
CFS/PPS Oracle. PBGC is exploring use of Microsoft Azure and Dynamics for many of its business
functions in the future. PBGC may also consider Apache Tomcat for its web and middleware platform.

The following table provides a summary of PBGC’s web application middleware platform instances as of
March 2019:

Web/Middleware Development Test Production COOP (DR) Total


Instances Instances Instances Instances

77
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Oracle WebLogic vers. 11g, 12c 494 165 127 93 879
Microsoft IIS 7.5 14 7 18 3 42
Microsoft IIS 8.5 41 22 42 9 114
Apache 68 24 16 108
Tomcat 5 1 2 3
Tomcat 6 2 4 6
Tomcat 7 12 10 12 4 38
Tomcat 8 16 6 9 1 32
Tomcat (Other) 3 2 5 10
JBoss 3 3
NGINX Web Server 1.12 1 1
Grand Total 648 239 239 110 1236

6.3.9.1 Requirements
For each web and middleware platform noted above, the contractor shall provide the support services
outlined in the following table:

Reference Requirement
(DC-WM-xx)
DC-WM-01 Contractor shall identify a lead for the Web and Application Middleware area. This
lead is required to serve as the primary point of contact for all web and middleware
related issues.
DC-WM-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
enterprise Web and Application Middleware infrastructure in all PBGC computing
environments. This shall include, but is not limited to:

• Identifying and addressing performance bottlenecks


• Performing capacity planning and management including allocation, resizing,
and reconfiguration of web and application middleware and associated
services as needed
• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
DC-WM-03 Contractor shall fully implement, manage, and support all incident management
activities regarding enterprise Web and Application Middleware infrastructure and
associated services according to PBGC policies. By using ITIL best practices, this should
include, but shall not be limited to:

78
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Provide support for IT supported applications and services deployed to
PBGC’s enterprise Web and Application Middleware infrastructure systems
including, but not limited to Oracle WebLogic, Oracle SOA suite, Oracle BPEL,
Oracle forms and reports, Microsoft IIS and .Net software, Apache Tomcat,
PBGC’s Oracle E Business Suite applications including CRM and CFS/PPS
(Financials), etc.

See SLA section for details on associated SLA measures.


DC-WM-04 Contractor shall fully implement, manage, and support all change management
activities regarding enterprise Web and Application Middleware infrastructure and
associated services they provide according to PBGC policies. By using ITIL best
practices, this should include, but shall not be limited to:

• Provide installation and configuration of Web and Application Middleware


components services deployed to PBGC’s enterprise Web and Application
Middleware infrastructure systems including, but not limited to Oracle
WebLogic, Oracle SOA suite, Oracle BPEL, Oracle forms and reports, Microsoft
IIS and .Net software, Apache Tomcat, PBGC’s Oracle E Business Suite
applications including CRM and CFS/PPS (Financials), etc. or remove when lo
longer required
• Create and manage connections to databases and application interfaces
including those that interact with external entities
• Optimize configuration for Java Virtual Machines (JVMs)
• Provide application code deployment services in support of COTS/GOTS
application releases
• Apply applicable security patches at least quarterly
• Address software vulnerabilities detected on enterprise infrastructure during
monthly vulnerability scans on Web and Application Middleware
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices

See SLA section for details on associated SLA measures.


DC-WM-05 Contractor shall design, implement, and support middleware clusters that meet the
high-availability and scalability requirements of mission critical applications
DC-WM-06 Contractor shall manage sharing of Web and Application Middleware infrastructure
resources amongst applications
DC-WM-07 Contractor shall ensure all accounts supporting the enterprise Web and Application
Middleware infrastructure environment, e.g. service accounts; privileged accounts;
etc., are changed periodically in accordance with PBGC policy and procedures and the
passwords are stored for emergency use
DC-WM-08 Contractor shall leverage PBGC-owned management tools, e.g. Oracle Enterprise
Manager, to automate repetitive tasks whenever possible. When not possible,
contractor shall create and manage scripts needed to automate repetitive tasks.

79
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DC-WM-09 Contractor shall perform application deployments from controlled sources, e.g. PVCS
per PBGC policies and procedures
DC-WM-10 Contractor shall participate in planning for the releases of new COTS/GOTS
applications to ensure that any new product usage or release upgrade takes place
with minimal impact
DC-WM-11 Contractor shall establish and maintain documentation regarding the enterprise Web
and Application Middleware infrastructure environment including:
• An inventory of all Web and Application Middleware infrastructure software
installations/instances broken down by environment, server, and software
version
DC-WM-12 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the Web and Application Middleware infrastructure environment and the
associated services
DC-WM-13 Contractor shall install, configure and support an Apache Tomcat web and middle
infrastructure including support for high availability, backup and recovery, and
disaster recovery. Contractor shall establish and maintain work instructions to
support this platform.

6.4 Voice, Video, and Network Infrastructure Operations


The sections below describe the scope and requirements of the Voice, Video and Network Infrastructure
Operations services that the Contractor shall provide under this solicitation. Providing stable, reliable,
secure, optimally performing, and highly available systems and service is critical to enable the
accomplishment of the agency mission and as such, are critical to excellent performance under this
contract. Thus, Voice, Video and Network Infrastructure Operations is a key service area. Much of how
the customer views the success of this contract will be dependent on how well the Contractor
administers and supports PBGC’s voice, video and network infrastructure, and how satisfied ITIOD staff
members are with the IT services provided. The Voice, Video and Network Infrastructure services
include the following:

• Network Infrastructure Support


• Telephony Infrastructure Support
• Network Operations Center

The Voice, Video and Network Infrastructure equipment, e.g. phones, routers switches, firewalls, etc.,
are located at PBGC’s HQW Campus as well as Field Benefit Administration FBA) locations. The on-
premise IT equipment that supports PBGC’s disaster recovery capability is currently located at PBGC’s
Wilmington (WIL) facility. PBGC plans to move most of the on-premise IT equipment supporting the
Data Center services to co-located data centers over the next several years. The co-located data centers
and the current and future HQ campuses will be connected via 10 GB Wave backbone replacing the

80
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
existing 1 GB backbone. Please refer to Appendix F - PBGC Locations for a tentative timeline for this
transition.

6.4.1 Network Infrastructure support


6.4.1.1 Scope of Services Supported
Provide tier 2 support for incidents relating to Network Infrastructure requiring administrative access to
Network equipment that cannot be resolved by an End-User services team as well as support service
requests. Typical Network administration and support activities include:

• Troubleshoot customer network connectivity issues (LAN, WAN, VPN, and Wireless)
• Support internal and external IP address management and name resolution services by creating,
updating, and removing IP address management (IPAM) DNS records
• Establish and maintain work instructions
• Escalate and work collaboratively with 3rd party vendors on hardware and software issues
• Escalate and work collaboratively with external network providers to resolve circuit
degradations and outages
• Maintain current IOS (internetwork Operating System) versions including startup and running
configurations for all network devices
• Manage internal/external hardware firewalls and secure web gateways.

Support for Network Infrastructure equipment is typically handled using remote management software,
e.g. SSH, but does occasionally require physical visits to address certain hardware and software
problems. See Appendix F - PBGC Locations for PBGC locations, including planned changes to PBGC’s
facilities and data centers over the life of the contract. See Appendix G - IT Service Support Guidelines
for impact, urgency, and prioritization guidelines associated with IT service and support. See Appendix J
- IT Service and Support 2018 Statistical Summary for tier 2 incidents, requests for information (RFIs),
service/access requests, requests for change (RFCs), and RFC tasks processed in calendar year 2018 for
these services. See Appendix K - IT Infrastructure Tools List for the software utilized to provides these
services. See Appendix D – PBGC Network Overview Diagram for more information on network
connectivity between sites and with the internet.

PBGC’s Network Infrastructure consists of many components detailed in the tables that follow:

Network Infrastructure Equipment Summary

The following table breaks down PBGC’s network infrastructure equipment by model:

Device Function Count


Cisco Nexus 7018 Core Data Center Switches/Routers 3
Cisco Nexus 7010 Core Data Center Aggregation Switches 2
Cisco Catalyst 9400 Floor distribution Switches 14
Cisco Catalyst 9300 Floor distribution Switches 19

81
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Cisco Catalyst 3850 Floor distribution Switches 2
Cisco Catalyst 3750 Floor distribution Switches 5
Cisco Catalyst 6509 Floor distribution Switches 1
Cisco Catalyst 6504 Floor distribution Switches 2
Cisco 4451-X router Vendor Managed Routers for L3/Presidio MPLS 8
WAN
Cisco ARS1002 Vendor Managed Routers for AT&T backbone 3
Cisco ARS1002 Vendor Managed Routers for Verizon TIC 2
Brocade Serverlron ADX Network load balancer 10
1000F
Infoblox Trinzic 1410 Infoblox IPAM Appliance - Grid Master and Grid 5
Master Candidates (DNS, DHCP, IPAM)
Infoblox Trinzic 810 Infoblox IPAM Appliance - DNS resolution and 4
forwarding in the DMZ
Infoblox Reporter 1400 Infoblox IPAM Appliance – Reporting 4
Cisco Wireless Controller Wireless Controller for Access points 3
5520
Cisco Wireless AP3800 Wireless Access points 109
Spectracom NetClock NTP Network Time Appliance 3
9843
PBGC is nearing completion of its effort to replace legacy floor distribution switches, e.g. 3750, 650x, in
the table above with Cisco 9410s and 23 x Cisco 9300s that will support PoE for VoIP. This work is
planned to be completed by May 2019. Network IT infrastructure equipment that supports remote
access or IT security functions is not included in the table above and will be made available in PBGC’s
reading room.

Wireless Summary

PBGC maintains several corporate Wi-Fi networks throughout its facilities (with end-users) administered
under this contract as follows:

Function Details
Corporate WiFi network for GFE computers Provides full access to PBGC’s production
network mimicking wired access. Access to
this network is controlled by Cisco ISE via AD
authentication.
WiFi network for GFE mobile phones Provides temporary access to the internet.
Access to this network is controlled by InTune
profile. Cisco Umbrella services is used to
restrict access to appropriate sites while on
this network.
Guest WiFi network Provides temporary access to the internet. A
PBGC user must contact the Service Desk to
arrange for a temporary guest account for
82
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
their guest. Cisco Umbrella services is used to
restrict access to appropriate sites while on
this network.
Corporate WiFi network for OIG’s GFE Provides full access to PBGC’s OIG production
computers end-user network mimicking wired access.
Access to this network is controlled by Cisco
ISE via AD authentication.

IPAM and DNS Summary

PBGC uses a split DNS configuration in which separate DNS servers are provided for internal and
external networks as a means of security and privacy management. The following table provides a
summary of external DNS which is hosted externally, but administered under this contract:

Record Type Record Count


A 65
CNAME 5
MX 5

The following table provides a summary of internal DNS from PBGC’s Infoblox IPAM system which is
supported and administered under this contract:

Record Type Count


Managed subnets 191
A Record 3,734
Host Record 4,575
A records are utilized for DHCP enabled hosts, e.g. workstations and Host Records are utilized for
statically configured hosts, e.g. servers.

6.4.1.2 Requirements
The contractor shall provide the network infrastructure support services outlined in the following table:

Reference Requirement
(NIO-NI-xx)
NIO-NI-01 Contractor shall identify a lead for the Network Infrastructure area. This lead is
required to serve as the primary point of contact for all network infrastructure related
issues.
NIO-NI-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
enterprise network infrastructure in all PBGC computing environments. This shall
include, but is not limited to:

• Identifying and addressing performance bottlenecks

83
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Performing capacity planning and management including allocation and
reconfiguration of network infrastructure and associated services as needed
• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
NIO-NI-03 Contractor shall fully implement, manage, and support all incident management
activities regarding Network infrastructure according to PBGC policies. By using ITIL
best practices, this should include, but shall not be limited to:

• Provide support for network infrastructure hardware and associated


operating systems and local wired and wireless, wide-area, and internet
network connectivity
• Provide support for IPAM including IP assignment, DHCP, and name
resolution services
• Troubleshoot and resolve issues with new or existing network, VOIP
connections to PBGC data and voice infrastructure
• Troubleshoot and resolve backbone, WAN, and TIC service issues and outages
including coordinating with vendors to troubleshoot managed routers and
activation of alternate paths to maintain network availability
• Troubleshoot and resolve network routing issues including issues with core
and WAN routers as well as internal and external firewalls including activation
of alternate paths to maintain network availability
• Troubleshoot and resolve basic network connectivity issues including issues
with port security, cabling, floor distribution switches, WAPs, Cisco ISE, etc.
• Collaborate, troubleshoot and help resolve application availability or
performance issues by reviewing network infrastructure for faults,
performing packet capture and analysis, etc. and addressing any issues with
packet loss, hardware load balancers, etc.
• Troubleshoot and resolve remote access issues including VPN and web-based
RDP proxy services

See SLA section for details on associated SLA measures.


NIO-NI-04 Contractor shall fully implement, manage, and support all change management
activities regarding Network Infrastructure devices and software and the associated
services they provide according to PBGC policies. By using ITIL best practices, this
should include, but shall not be limited to:

• Provide installation and configuration of network infrastructure hardware and


associated operating systems and local wired and wireless, wide-area, and
internet network connectivity or decommission when no longer required

84
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Apply applicable security patches at least monthly and install applicable
hardware firmware updates at least quarterly
• Address operating system and software vulnerabilities detected on network
infrastructure during monthly vulnerability scans
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices
• Process approved modifications to PBGC’s firewall rule set and distributed
access control lists
• Establish and maintain site-to-site VPN connectivity as needed to support
secure network connectivity with other federal agencies and service
providers
• Process approved modifications to PBGC’s web proxy black and white lists
• Provision new subnets or remove those no longer in use
• Update network load balancers to support new or changes to existing
applications

See SLA section for details on associated SLA measures.


NIO-NI-05 Contractor shall fulfill all approved IT Security requests (in SecureIT) in accordance
with PBGC procedures and timelines for the following:
• SOE ACL/Firewall Change
• URL to Blacklist
• URL to Whitelist

See SLA section for details on associated SLA measures.


NIO-NI-06 Contractor shall maintain and update the network management software, e.g. Cisco
Prime, and ensure that it contains a copy of the current IOS (internetwork Operating
System), Startup and Running Configurations for all network devices, e.g. routers,
firewalls, switches, proxies, gateways for emergency restore.
NIO-NI-07 Contractor shall review, no less than weekly, the IPAM system for all active,
responsive IP addresses on the PBGC network that have no associated DNS name and
identify the associated device. If the device is determined to be authorized (by
associated approved RFC or as directed by the Government), Contractor shall
establish the appropriate DNS record. If the device is determined to be unauthorized
(by lack of associated RFC or as directed by the Government), Contractor shall open a
security event and remove the device from the network immediately. Contractor
shall log all aspects of this process.
NIO-NI-08 Contractor shall maintain communication racks to include, but not limited to:

• ensuring front and rear doors can be closed and are closed
• ensuring patch cables are of suitable length, are free from entanglement, are
patched the appropriate port based on applicable PBGC standards, and are
routed neatly to the patch panel

85
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• ensuring power to redundant power supplies is properly distributed between
PDUs to minimize downtime due to a single component failure
• maintaining Visio diagrams of the contents of each rack with
updates/verification no less frequently than every 6 months
• ensuring equipment not in use is powered down, clearly labeled, and
removed from the rack within 90 days of decommission
• ensuring all cables are clearly labeled for purposes of troubleshooting and
inspection
NIO-NI-09 Contractor shall ensure all local accounts supporting the network infrastructure, e.g.
the built-in admin account, are changed periodically in accordance with PBGC policy
and procedures and the passwords are stored for emergency use
NIO-NI-10 Contractor shall establish, maintain and post an inventory of virtual IP addresses to
include, at a minimum the following information regarding network load balancing
and clustering in the environment: VIP address, fully qualified DNS name,
balanced/clustered devices, application/service, date established, RFC#
NIO-NI-11 Contractor shall maintain a list of PBGC subnets in the IPAM system with details to
include subnet function and whether it is managed by PBGC or by a network carrier.
Contractor shall ensure that updates to this list are communicated to the
configuration management team and the vulnerability scanning team.
NIO-NI-12 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the network infrastructure and the associated services

6.4.2 Telephony Infrastructure Support


6.4.2.1 Scope of Services Supported
Provide tier 2 support for incidents relating to Telephony infrastructure that cannot be resolved by an
End-User services team as well as support service requests. Typical Telephony infrastructure
administration and support activities include:

• Monitor, troubleshoot and repair or replace defective Telephony infrastructure and video
hardware and software as needed
• Relocate IT phone equipment in response to user relocation requests including updating
connectivity and update database containing relationships between phone numbers/extensions,
users and locations
• Install IT phone equipment for new user setups and remove upon user separation including
updating connectivity and update database containing relationships between phone
numbers/extensions, users and locations as well as CommView call accounting system
• Monitor, troubleshoot and resolve voice traffic routing issues between PBGC’s HQ campus,
Customer Call Center, and FBA/PVA locations
• Monitor telephony call statistics to detect anomalies in system performance and manage
underlying systems supporting call metrics and assist with call reporting issues and run reports
when issues arise

86
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Provision Polycom personal conference bridges for new user setups and remove upon user
separation
• Provide support to users and organizations that have requested voice/video conferencing
assistance
• Deploy patches, firmware updates, and software updates to the Telephony Infrastructure
• Maintain racks that contain telephony equipment
• Coordinate message updates on PBGC’s IVR
• Administer accounts on telephony infrastructure
• Establish and maintain work instructions
• Escalate and work collaboratively with 3rd party vendors on hardware and software issues

Support for Telephony Infrastructure equipment in the PBGC data centers is typically handled utilizing
remote management software, e.g. RDP, SSH, but does occasional require a physical visit to address
hardware and certain software problems. See Appendix F - PBGC Locations for PBGC locations, including
planned changes to PBGC’s facilities and data centers over the life of the contract. See Appendix G - IT
Service Support Guidelines for impact, urgency, and prioritization guidelines associated with IT service
and support. See Appendix J - IT Service and Support 2018 Statistical Summary for tier 2 incidents,
requests for information (RFIs), service/access requests, requests for change (RFCs), and RFC tasks
processed in calendar year 2018 for these services. See Appendix K - IT Infrastructure Tools List for the
software utilized to provides these services.

PBGC’s Telephony Infrastructure consists of many components detailed in the tables that follow:

Telephony Infrastructure Equipment Summary

PBGC is currently in the process of replacing its legacy telephony infrastructure at all PBGC locations
including the FBA sites by the end of June 2019. All legacy PBX and phones will be replaced with the
latest supported Cisco VoIP (Cisco UC and Voicemail) and call center (Cisco UCCX) technology. This effort
will also introduce enterprise FAX and the InformaCast emergency notification system. The Contractor
is expected to support all aspects of the new VoIP based telephony infrastructure. This effort is
expected to eliminate the NEC PBXs, NEC GNAV, NEC Voice Mail systems, NEC Quework, Nortel IVR,
Mutare ENS, and Qfiniti Etalk. The following table breaks down PBGC’s current telephony infrastructure
equipment by model for the existing system and the system being deployed in June 2019:

Device Function Count New Solution


Polycom RMX 4000 Conference bridge 1 Cisco WebEx
Audio only.
NEC SV9500 IP PBX Corporate Telephony System 3 Cisco UC
NEC SV8300 IP PBX Corporate Telephony System 5 Cisco UC
NEC Dterm Series 1 Digital and Voice Over IP (VOIP) desk phones 2200 Cisco 8800
Series Phones
NEC UM8700 Corporate Voice Mail 4 Cisco Unity

87
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Cisco Catalyst 3750 VOIP Switches 20 Cisco catalyst
9000 series
Switches
NEC Global Navigator Management information system that records 2 Cisco Finesse
(GNAV) the activity of calls, tracks the performance of
agents and coordinates the scheduling of
personnel
Nortel MPS500 Interactive Voice Response (IVR) 2 Cisco UCCX
NEC QueWorx Provides Customer Contact Center with a range 3 Cisco UCCX
of customer-focused applications
Qfiniti Etalk Call quality monitoring system 1 Cisco Calabrio
Mutare ENS Emergency Notification System 1 Cisco
Informacast
@Comm CommView Call Accounting and Reporting System 1 Latest
Commview
version

88
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
The following table breaks down PBGC’s target telephony infrastructure equipment by model for the
new system being deployed in June 2019:

Device Function Count

Cisco 7000H Cisco UC Virtual Appliances (Telephony Core 6


Infrastructure Hardware – Runs on VMWare)

Cisco 7000M Cisco UC Virtual Appliances (Telephony Core 4


Infrastructure Hardware – Runs on VMWare)

Cisco UC Version 12.X Unified Communications Software Licensed to


3000 Users

Cisco 8800 Series Phones Digital and Voice Over IP (VOIP) desk phones 2500

Cisco Unity Version 12.X Corporate Voice Mail

Cisco Catalyst 9000 Series VOIP and Floor Distribution Switches 20


Switches

Cisco Finesse Version 11 - Call Center Agent Interface Software Up to 400


12.X Agents

Cisco UCCX Version 11 - 12.X Unified Contact Center Express – Call Center Up to 400
Agent, Interactive Voice Response (IVR) Agents

Cisco Calabrio Version 11 - Integrated solution for call recording, quality Up to 400
12.X assurance, workforce management, analytics Agents
and reporting.

Cisco Informacast Version 11 Emergency Notification System 3000-5000


– 12X Users

Imagicle - StoneFax Server Software-based IP Fax Server that virtualizes 3000-5000


Version 2.X fax management Users

All Cisco UC/UCCX solutions run on two VMware clusters, HQWPROD_CiscoPhone and
WILPROD_CiscoPhone, that are Cisco proprietary used to host Cisco UC comprised of BE7H-M5-K9
servers.

6.4.2.2 Requirements
The contractor shall provide the Telephony infrastructure support services outlined in the following
table:

89
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Reference Requirement
(NIO-TI-xx)
NIO-TI-01 Contractor shall identify a lead for the Telephony Infrastructure area. This lead is
required to serve as the primary point of contact for all Telephony infrastructure
related issues.
NIO-TI-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
enterprise Telephony infrastructure in all PBGC computing environments. This shall
include, but is not limited to:
• Identifying and addressing performance bottlenecks
• Performing capacity planning and management including allocation and
reconfiguration of telephony infrastructure and associated services as needed
• Using monitoring tools to proactively plan and manage infrastructure
resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
NIO-TI-03 Contractor shall fully implement, manage, and support all incident management
activities regarding Telephony infrastructure according to PBGC policies. By using ITIL
best practices, this should include, but shall not be limited to:

• Provide support for Telephony infrastructure hardware and software as well


as analog, digital and VoIP network connectivity
• Troubleshoot and resolve issues with new or existing network, VoIP
connections to PBGC Telephony infrastructure
• Troubleshoot and resolve Telephony service issues and outages including
coordinating with vendors to troubleshoot and activate alternate paths to
maintain voice and video availability
• Troubleshoot and resolve basic voice and video connectivity issues including
issues with cabling, faulty equipment, connectivity problems with external
callers, etc.

See SLA section for details on associated SLA measures.


NIO-TI-04 Contractor shall fully implement, manage, and support all change management
activities regarding telephony servers and the associated services they provide
according to PBGC policies. By using ITIL best practices, this should include, but shall
not be limited to:
• Provide installation and configuration of Telephony infrastructure hardware
and software or decommission when no longer required
• Apply applicable security patches at least monthly and install applicable
hardware firmware updates at least quarterly
• Address operating system and software vulnerabilities detected on Telephony
infrastructure during monthly vulnerability scans

90
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices

See SLA section for details on associated SLA measures.


NIO-TI-05 Contractor shall complete daily morning checks on all voice/video equipment and
report any issues to Contractor and Federal Lead prior to the Daily Operations
Standup Meeting
NIO-TI-06 Contractor shall provide administrative, management, monitoring, and support for all
PBGC IT infrastructure components supporting voice and video services based on
industry best practices. This should include, but shall not be limited to:
• Relocate IT phone equipment in response to user relocation requests
including updating connectivity and update systems to ensure relationships
between phone numbers/extensions, users and locations are properly
documented
• Install IT phone equipment for new user setups and remove upon user
separation including updating connectivity and update systems to ensure
relationships between phone numbers/extensions, users and locations are
properly documented
• Manage call routing
• Assist with call forwarding requests within the corporate phone system
• Manage greeting recording and voice traffic routing for the Interactive Voice
Response (IVR) Service
• Assist with call reporting issues
• Fulfill requests for Voice/Video conferences from all organizations. Ensure
each conference voice/video conference bridge and/or equipment has been
tested prior to conference and troubleshoot any issues that arise.
NIO-TI-07 Contractor shall provision personal conference bridges for new user setups and
remove upon user separation
NIO-TI-08 Contractor shall maintain telephony infrastructure racks to include, but not limited to:

• ensuring front and rear doors can be closed and are closed
• ensuring patch cables are of suitable length, are free from entanglement, are
patched the appropriate port based on applicable PBGC standards, and are
routed neatly to the patch panel
• ensuring power to redundant power supplies is properly distributed between
PDUs to minimize downtime due to a single component failure
• maintaining Visio diagrams of the contents of each rack with
updates/verification no less frequently than every 6 months
• ensuring equipment not in use is powered down, clearly labeled, and
removed from the rack within 90 days of decommission
• ensuring all cables are clearly labeled for purposes of troubleshooting and
inspection

91
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
NIO-TI-09 Contractor shall perform telephony infrastructure account administration functions in
accordance with PBGC policy and procedures for systems and functions where
automation is not already in place
NIO-TI-10 Contractor shall ensure that the configuration of all telephony infrastructure
equipment and associated data is backed to the corporate backup system up no less
than monthly
NIO-NI-11 Contractor shall ensure all local administrative accounts supporting the Telephony
infrastructure, e.g. the built-in admin account, are changed periodically in accordance
with PBGC policy and procedures and the passwords are stored for emergency use
NIO-NI-12 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the telephony infrastructure and the associated services

6.4.3 Network Operations Center


6.4.3.1 Scope of Services Supported
Staff a Network Operations Center (NOC) at the PBGC Headquarters location to support continuous,
uninterrupted (24 hours per day, 365 days per year) monitoring of PBGC’s IT infrastructure as well as
provide limited IT service and support services. Typical Network Operation Center administration and
support activities include:

• Monitoring PBGC’s IT infrastructure supporting all environments and all locations


• Reporting and tracking system outages and problems through resolution
• Monitoring the HVAC, PDU and UPS equipment in PBGC’s Data Centers and LAN closets
• Monitoring temperatures, humidity and air flow, as necessary to maintain specified parameters
required for optimum hardware operation in PBGC’s Data Centers and LAN closets
• Perform physical inspection of server and communications racks and environmental equipment
in PBGC’s HQ data center and LAN closets and log and communicate any identified issues
• Maintain a computer room security log for emergency password and hard key check-out in
accordance with PBGC policies and procedures
• Provide escort as needed for vendors performing maintenance
• Coordinate and monitor electrical, cabling, and HVAC services
• Maintain a shift log detailing all activities
• Create and send out the Significant Outage Report daily by 5am
• Provide end-user customer support outside of normal business hours
• Perform repeatable tasks in support of other IT infrastructure service areas, e.g. imaging
workstations, assisting with asset management activities, etc.

PBGC’s Network Operations Center monitors the following facilities and facilities equipment:

Item Count Details

92
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Data Center 2 HQW (1) – The NOC staff are currently located at this location in room
L721A.
WIL (1)
LAN Closets 24 BUC (1) – This facility is located away from HQ, so the NOC does not
conduct a walkthrough of the LAN closet. They use other monitoring tools
to verify equipment status (NNMi, SiteScan and Temp guard for
environmental)
DOR (1)- This facility is located away from HQ, so the NOC does not
conduct a walkthrough of the LAN closet. They use other monitoring tools
to verify equipment status (NNMi, SiteScan and Temp guard for
environmental)
HQB (2) – The NOC conducts a walkthrough of the LAN closet twice
during an 8-hour shift to conduct visual inspection of network and
telephony. The results are captured in the shift pass down log
HQI (1) - The NOC conducts a walkthrough of the LAN closet twice during
an 8-hour shift to conduct visual inspection of network and telephony.
The results are captured in the shift pass down log
HQW (12) - The NOC conducts a walkthrough of the LAN closet twice
during an 8-hour shift to conduct visual inspection of network and
telephony. The results are captured in the shift pass down log
KIN (2) - This facility is located away from HQ, so the NOC does not
conduct a walkthrough of the LAN closet. They use other monitoring tools
to verify equipment status (NNMi, SiteScan and Temp guard for
environmental)
QUI (1) - This facility is located away from HQ, so the NOC does not
conduct a walkthrough of the LAN closet. They use other monitoring tools
to verify equipment status (NNMi, SiteScan and Temp guard for
environmental)
EUC (2) - This facility is located away from HQ, so the NOC does not
conduct a walkthrough of the LAN closet. They use other monitoring tools
to verify equipment status (NNMi, SiteScan and Temp guard for
environmental)
WIL (2) - This facility is located away from HQ, so the NOC does not
conduct a walkthrough of the LAN closet. They use other monitoring tools
to verify equipment status (NNMi, SiteScan and Temp guard for
environmental)
Phone 24 BUC (1)
Closets DOR (1)
EUC (2)
HQB (2)
HQI (1)
HQW (12)
KIN (2)
QUI (1)
93
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
WIL (2)

Data Center 12 HQW (7) – The NOC monitors the status (temperature, humidity and any
AC Units alarms) of these unit by using SiteScan. The NOC conducts a walkthrough
of the Data Center twice during an 8-hour shift to conduct visual
inspection of the AC units and record temperature and humidly reading
from thermometers through out the Data Center.
WIL (5) – The AC units are located on the first and second floor data
centers. The only equipment on the first floor is the Power Distribution
Unit (PDU).
LAN Closets 22 EUC (1) - WSD has the maintenance contract for the AC unit. The NOC will
AC Units monitor the status via sitescan and will coordinate with LAN admin and
notify WSD if service is required after hours or over a weekend.
DOR (2) - WSD has the maintenance contract for the AC unit. The NOC
will monitor the status via sitescan and will coordinate with LAN admin
and notify WSD if service is required after hours or over a weekend.
HQB (2) – ITIOD has the maintenance contract for the AC unit. The NOC
will monitor the status via sitescan and visual inspection twice during the
8 hour shift and will coordinate with the provider if service is required.
HQI (1) - WSD has the maintenance contract for the AC unit. The NOC will
monitor the status via sitescan and will coordinate with LAN admin and
notify WSD if service is required after hours or over a weekend.
HQW (14) - ITIOD has the maintenance contract for the AC unit. The NOC
will monitor the status via sitescan and visual inspection twice during the
8-hour shift and will coordinate with the provider if service is required.
KIN (2) - ITIOD has the maintenance contract for the AC unit. The NOC will
monitor the status via sitescan and visual inspection twice during the 8-
hour shift and will coordinate with the provider if service is required.
Data Center 2 KIN (1) – The property management at Kingstowne has the support
Generators contract for the preventative maintenance of the generators a long with
Work Place Solution (WSD). WSD will schedule the maintenance with the
vendor during normal business hours. In the event of a power outage in
which the generator fails to operate after hours, the NOC has instructions
on who needs to be contacted and will facilitate the call.
WIL (1) - The property management at Wilmington has the support
contract for the preventative maintenance of the generators a long with
Work Place Solution (WSD). WSD will schedule the maintenance with the
vendor during normal business hours. In the event of a power outage in
which the generator fails to operate after hours, the NOC has instructions
on who needs to be contacted and will facilitate the call.

PBGC plans to move most of the on-premise IT equipment supporting the Data Center services to co-
located data centers over the next several years. Please refer to Appendix F – PBGC Locations for a

94
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
tentative timeline for this transition. Also, a Privileged Account Management tool, CyberArk, is being
deployed at PBGC (FY17/FY18) to store privileged credentials and broker and record sessions requiring
privileged access. These initiatives will eliminate the vast majority of required services noted in the
requirements for the Network Operations Center that follows. Once these initiatives are completed,
PBGC no longer sees value in hosting a complete Network Operations Center on premises. The
Contractor shall propose how to best meet any remaining requirements.

6.4.3.2 Requirements
The contractor shall provide Network Operations Center support outlined in the following table:

Reference Requirement
(NIO-NO-xx)
NIO-NO-01 Contractor shall identify a lead for the Network Operation Center. This lead is
required to serve as the primary point of contact for all network operation center
related issues and activities.
NIO-NO-02 Contractor shall provide and maintain a fully functional, optimally performing IT
Infrastructure by providing ongoing monitoring through the Network Operations
Center. This shall include, but is not limited to:

• Using monitoring tools to identify issues and coordinate resolution


• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
NIO-NO-03 Contractor shall fully implement, manage, and support all incident management
activities regarding Network Operations Center according to PBGC policies. By using
ITIL best practices, this should include, but shall not be limited to:

• Provide support for on data center facility infrastructure and associated


monitoring tools
• Troubleshoot and resolve backbone, WAN, and TIC service issues and outages
including coordinating with vendors to troubleshoot managed routers and
activation of alternate paths to maintain network availability
• Collaborate, troubleshoot and Provide “smart hand” support to O&M staff to
assist with verifying equipment status (Power) trace cabling or reboot
equipment as instructed

See SLA section for details on associated SLA measures.


NIO-NO-04 Contractor shall fully implement, manage, and support all change management
activities regarding Network Operation Center and the associated services they
provide according to PBGC policies. By using ITIL best practices, this should include,
but shall not be limited to:

• Assist with installation, configuration and decommissioning of network,


infrastructure hardware
95
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Coordinate maintenance and changes in PBGC’s data centers for data center
facility infrastructure, e.g. HVAC, power, and other environmentals
• Apply applicable security patches and applicable hardware firmware updates
at least quarterly to data center facility infrastructure
• Address operating system and software vulnerabilities detected on data
center facility infrastructure during monthly vulnerability scans
See SLA section for details on associated SLA measures.
NIO-NO-05 Contractor shall monitor the data center facility temperatures, humidity and air flow,
as necessary to maintain specified parameters required for optimum hardware
operation. The Network Operations Center shall escalate within the contract and to
the PBGC incident manager within 15 minutes of discovery of any problems with
environmental equipment including, but not limited to:

• Problems with the uninterruptible power supply (UPS) and power distribution
units
• Problems with AC units or with data center-wide or area specific temperate
issues
• Problems with fire detection/suppression equipment, heat exchanger, water
pumps, etc.
NIO-NO-06 Contractor shall staff a Network Operations Center (NOC) at the PBGC Headquarters
location to support continuous, uninterrupted (24 hours per day, 365 days per year)
monitoring of PBGC’s IT infrastructure as well as provide limited IT service and
support services including, but not limited to:
• Maintaining computer room security sign in log to include Password and hard
key check-out in accordance with PBGC policies and procedures
• Provide escort as needed for vendors performing maintenance or cleaning of
the data center (floors) or other designated locations
• Conduct a visual inspection of all infrastructure equipment racks every 4
hours to identify equipment with fault lights and escalate to the appropriate
contract staff
• Respond to monitoring tool e-mail notifications to coordinate
troubleshooting and resolution, e.g. HP Site Scope
• Monitor local area (LAN) and wide area network (WAN) availability via
performance monitoring and topology software, e.g. HP Network Node
Monitor (NNMI)
• Facilitate troubleshooting and circuit restoration with external circuit
providers by monitor external circuit provider portals for scheduled and or
emergency maintenance which may impact network availability
• Maintain a shift log to capture routine checks as well as outages or anomalies
that need to be escalated or passed to the other shift
• Provide end-user customer support outside of normal business hours
• Perform repeatable tasks in support of other IT infrastructure service areas,
e.g. imaging workstations, assisting with asset management activities, etc.

96
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
NIO-NO-07 Contractor shall ensure all local accounts supporting the data center facility
infrastructure including HVAC, power, and other environmentals; e.g. the built-in
admin account; are changed periodically in accordance with PBGC policy and
procedures and the passwords are stored for emergency use
NIO-NO-09 Contractor shall establish and maintain a periodic task list for the services provided by
the NOC

6.5 IT Service Management (ITSM) and Infrastructure Monitoring and Reporting


The sections below describe the scope and requirements of the IT Service Management (ITSM) and
Infrastructure Monitoring and Reporting services that the Contractor shall provide under this
solicitation. Providing current, vendor-supported, optimally performing, highly available compliant
ITSM, IT infrastructure monitoring, and reporting systems and service is critical to maintain optimally
performing and highly available infrastructure services and thus enable the accomplishment of the
agency mission. As such, these services are critical to excellent performance under this contract and
make this a key service area. Much of how the customer views the success of this contract will be
dependent on how well the Contractor performs ITSM, IT infrastructure monitoring and reporting, and
how satisfied ITIOD staff members are with the IT services provided. The Contractor is expected to
provide ITSM, IT infrastructure monitoring and reporting services for all IT infrastructure technology
noted in this PWS.

The ITSM and the Infrastructure Monitoring and Reporting services include the following:

• ITSM Tool Support


• IT Service Catalog Support
• IT Infrastructure and Application Availability, Capacity, and Performance Monitoring
• ITIOD Reporting and Dashboarding
• Major Incident Management
• Problem Management
• Risk Management
• Change Management
• Asset Management and Inventory
• Configuration Management

6.5.1 ITSM Tool Support


6.5.1.1 Scope of Services Supported
Provide tier 2 support for incidents relating to PBGC’s ServiceNow instances; specifically, the IT Service
Management (ITSM) module as well as PBGC’s on-premise ServiceNow MID server which facilitates
communication and movement of data between the ServiceNow platform and external applications,
data sources, and services including configuration item discovery. Provide fulfillment for service
requests. Typical ITSM tool administration and support activities include:
97
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Monitor, troubleshoot, and address issues pertaining to PBGC’s ServiceNow instances;
specifically, the ITSM module as well as PBGC’s on-premise ServiceNow MID server and
associated services, maintenance tools, and scripts
• Configure PBGC’s ServiceNow instances; specifically, the ITSM module as well as PBGC’s on-
premise ServiceNow MID server and associated services to address changing business
requirements, maximize availability, optimize performance, and ensure compliance with PBGC
governance including, but not limited to:
o Service Desk interaction and incident management
o Problem management
o Change and Release management
o Request management
o Asset and Cost management
o Configuration management including discovery
o Knowledge management
o Reports and Dashboards
o Service Level management
o Surveys and Assessments
o Access management
• Collect requirements and then configure and customize PBGC’s ITSM implementation on the
ServiceNow platform including, but not limited to:
o Documenting user stories
o Developing mockups for forms, workflow, access and privacy controls, etc.
o Documenting the proposed data model
o Collaborating with and presenting to ITIOD stakeholders on the development of
enhancements and bug fixes to PBGC’s implementation of the ServiceNow ITSM
modules
o Development, testing, and delivery of enhancements and bug fixes to PBGC’s
implementation of the ServiceNow ITSM modules
• Enable and support integration with on-premise data through the MID server
• Escalate and work collaboratively with ServiceNow vendor on issues
• Establish and maintain work instructions

See Appendix G - IT Service Support Guidelines for impact, urgency, and prioritization guidelines
associated with IT service and support. See Appendix J - IT Service and Support 2018 Statistical
Summary for tier 2 incidents, requests for information (RFIs), service/access requests, requests for
change (RFCs), and RFC tasks processed in calendar year 2018 for these services although these apply to
HP Service Manager 9, PBGC’s previous ITSM tool. See Appendix K – IT Infrastructure Tools List for the
software utilized to provides these services.

PBGC’s ITSM tool and associated services administration and support environment consists primarily of
the following:

98
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• PBGC’s 3 instances of ServiceNow instances; specifically, the ITSM module as well as PBGC’s on-
premise ServiceNow MID server and associated services. PBGC’s 3 instances are licensed for
2,400 users and 750 servers/nodes for discovery. PBGC Active Directory users and select groups
are synchronized to ServiceNow utilizing LDAP and the ServiceNow MID servers and user
authentication is federated utilizing Active Directory Federation Services (ADFS).

6.5.1.2 Requirements
The contractor shall provide the ITSM tool support services outlined in the following table:

Reference Requirement
(SM-SN-xx)
SM-SN-01 Contractor shall identify a lead for the IT Service Management (ITSM) tool support
area. This lead is required to serve as the primary point of contact for all ITSM tool
related issues and enhancement requests.
SM-SN-02 Contractor shall support and maintain a fully functional, optimally performing on-
premise infrastructure supporting ITSM, e.g. ServiceNow MID server, and
infrastructure services that support connectivity and integration with PBGC’s cloud-
based ServiceNow instances. This shall include, but is not limited to:

• Identifying and addressing performance bottlenecks


• Using monitoring tools and ServiceNow-provided health reports to
proactively plan and manage infrastructure resources to maximize system
and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by ServiceNow as required and in
accordance with PBGC Security policies, vendor warranties and maintenance
contracts
SM-SN-03 Contractor shall fully implement, manage, and support all incident management
activities regarding PBGC’s instances of ServiceNow; specifically, the ITSM module as
well as PBGC’s on-premise ServiceNow MID server according to PBGC policies. By
using ITIL best practices, this should include, but shall not be limited to:

• Provide support for PBGC’s ServiceNow instances; specifically, the ITSM


module as well as PBGC’s on-premise ServiceNow MID server

See SLA section for details on associated SLA measures.


SM-SN-04 Contractor shall fully implement, manage, and support all change management
activities regarding ITSM tool services according to PBGC policies. By using ITIL best
practices, this should include, but shall not be limited to:

• Provide integration with and configuration of PBGC’s ServiceNow instances;


specifically, the ITSM module as well as PBGC’s on-premise ServiceNow MID
server

99
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Apply necessary configuration changes to achieve ITSM objectives including
compliance with PBGC policies and procedures as well as to achieve desired
operational configuration and user experience and align with industry best
practices. This shall include staging update sets to the appropriate OBGC
code repository for promotion to PBGC’s production instance of ServiceNow.

See SLA section for details on associated SLA measures.


SM-SN-05 Contractor shall manage requests for modification and enhancements to PBGC’s ITSM
implementation on the ServiceNow platform and resolve reported defects including,
but not limited to:
• Documenting requirements utilizing detailed, unambiguous user stories
• Developing design summaries for modification and enhancements to PBGC’s
ITSM implementation on the ServiceNow platform including, but not limited
to:
o mockups for forms, workflow diagrams, access and privacy controls,
etc.
o proposed data model
o level of complexity and effort and ServiceNow technologies to be
utilized, e.g. tables, workflow, KBAs, widgets, etc.
• Manage, track, and publish progress on development efforts through agile
sprints and present updates to ITSM tool service/application owners and
ITIOD stakeholders no less than monthly
SM-SN-06 Contractor shall develop and maintain required on-line forms for all actions needed
to maintain the IT Service Catalog, e.g. change ownership, reroute requests, request
for updates, etc.
SM-SN-07 Contractor shall evaluate and test new ITSM functionality when deployed by
ServiceNow and assess impact on PBGC and report to federal point of contact
SM-SN-08 Contractor shall create and maintain knowledge base articles (KBAs) for the IT Service
Desk to answer frequently asked questions and train IT Service Desk staff to ensure
they are prepared to answer basic questions regarding ITSM
SM-SN-09 Contractor shall maintain passwords for the ITSM tool accounts, e.g. cloud admin
accounts, local system emergency recovery accounts, etc. in privileged account
management tool and utilize this tool to perform administrative functions via
brokered session or account check out
SM-SN-10 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain PBGC’s ServiceNow instances; specifically, the ITSM module as well as
PBGC’s on-premise ServiceNow MID servers

6.5.2 IT Service Catalog Support


6.5.2.1 Scope of Services Supported
Maintain a centralized IT service catalog or catalogs of IT systems, resources, and services that can be
requested by PBGC users to obtain IT service in a timely and consistent manner. Provide support for the
service catalog tool(s) to include tracking of and implementation of tool enhancements and bug fixes.
100
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
For end-user service requests, PBGC currently leverages the service request catalog and service request
modules of the HP Service Manager 9 tool, referred to as “GetITAccess”, to support this capability, but is
planning to migrate, prior to the start of this contract, this capability to the request module of Service
Now and SailPoint IdentityIQ LifeCycle Manager in FY19 and decommission Service Manager 9. The vast
majority of items and roles in PBGC’s service catalog are used to request and provide access to IT
applications and resources. The majority of these items/roles are published and fulfilled automatically
utilizing Active Directory groups and associated group memberships. The current service catalog is
organized into a hierarchical structure as follows including the count for each level:

Level Name Description and Examples Count


1 Category A high-level category grouping the type of system or resource 15
access being requested; examples:
• Access: Application
• Access: SharePoint Site
2 Subcategory A high-level subcategory grouping the type of system or resource 249
access being requested, often by organization; examples:
• Applications: ITIOD
• SharePoint Sites: ITIOD
3 Item The system/resource being requested used to group roles; 3,184
examples:
• ServiceNow ITSM Access
• ITIOD SOP and Work Instruction Site
4 Role The specific role being requested/provided which includes the 24,522 1
system/resource and the desired level of access; examples:
• APPS_ServiceNow_ITSM_ITIL_ProcessUser (Grants edit
access to create and update interaction, incident and,
problem and change tickets within ServiceNow)
• SP-Teams-ITIOD-WorkIT-V (Visitor Access to ITIOD SOP and
Work Instruction Site)
This hierarchy may change slightly as part of the migration to ServiceNow/SaiPoint.

In addition to GetITAccess, PBGC has a SharePoint site called EnableIT to provide for request and
approval of requests for IT infrastructure resource and services such as:

• New Servers
• New Software Packages/Deployments
• New Databases
• Database Fixes (updates)

1
Includes only those “dynamic” roles generated via Active Directory group. There are approximately 10 “custom
developed” service catalog items that have custom forms and workflows to address more complicated service
requests like: new employee setup, employee separation, equipment relocation, direct Oracle access, etc.
Requests pertaining to accounts and access will move to ServiceNow/SailPoint solution. Requests for equipment
relocation will also move to Service Now.
101
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Database Refresh
• Database Restore
• New Database Role
• New Service Catalog Items
• New Storage Allocation
• New ServiceNow CMDB CI
• New Report
• New System or Application Monitoring
• New SharePoint site
• New SharePoint library with unique permissions
• New SharePoint content development
• Existing SharePoint content enhancement

EnableIT Service Requests

The following table provides the count for EnableIT service requests processed in Calendar Year 2018

Service Request Category Total


Requests
Service Catalog: New Catalog Item 187

6.5.2.2 Requirements
The Contractor shall maintain PBGC’s IT Service catalogs, centralized catalogs of systems, resources, and
services that can be requested by PBGC users, including, but not limited to:

• publishing any additional items/roles/IT infrastructure services as required/requested


• unpublishing/removing any items/roles/IT infrastructure services that are no longer needed as
required/requested
• renaming items/roles/IT infrastructure services as required/requested
• changing primary and alternate approvers for items/roles/IT infrastructure services as
required/requested
• identifying items/roles/IT infrastructure services missing required information and obtain and
update or escalate to federal counterpart for assistance in data collection

The contractor shall provide support the IT Service Catalog tools and associated support services
outlined in the following table:

Reference Requirement
(SM-SC-xx)
SM-SC-01 Contractor shall identify a lead for IT Service Catalog support. This lead is required to
serve as the primary point of contact for all IT Service Catalog support.

102
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
SM-SC-02 Contractor shall maintain the IT Service Catalogs, centralized catalogs of systems;
resources; and services that can be requested by PBGC users, including, but not
limited to:
• publishing any additional items/roles/IT infrastructure services as
required/requested
• unpublishing/removing any items/roles/IT infrastructure services that are no
longer needed as required/requested
• renaming items/roles/IT infrastructure services as required/requested
• changing primary and alternate approvers for items/roles/IT infrastructure
services as required/requested
• No less than monthly, identifying items/roles/IT infrastructure services
missing required information and obtain and update or escalate to federal
counterpart for assistance in data collection
SM-SC-03 Contractor shall reroute all service requests to the appropriate approver upon
request when in accordance with PBGC policy and procedures and alternatively
communicate to customer when such a reroute is not authorized
SM-SC-04 Contractor shall identify required on-line forms for all actions needed to maintain the
IT Service Catalog, e.g. change ownership, reroute requests, request for updates, etc.
SM-SC-05 Contractor shall ensure access requests are in compliance with approved governance
and serve as liaison for project teams and end-users to IT Customer & Operations
Services (ITCOS) division should questions regarding access control process and
procedures arise including access to Development and Test environments
SM-SC-06 Contractor shall provide automated reminders for IT service requests pending
approval or fulfillment to ensure all such requests are approved, fulfilled, and closed
out properly and on time and shall escalate to the federal service catalog manager
any request that has been pending approval or fulfillment for more than 30 days.
SM-SC-07 Contractor shall assist customers with identifying the appropriate the IT service
catalog item/role to request and the most efficient way to request it, e.g. manual
submission, bulk submission, requests for control groups, etc.
SM-SC-08 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New Service Catalog Items

See SLA section for details on associated SLA measures.

6.5.3 IT Infrastructure and Application Availability, Capacity, and Performance Monitoring


6.5.3.1 Scope of Services Supported
Provide support for IT infrastructure and application availability, capacity, and performance monitoring
to help ensure PBGC’s infrastructure and supported applications are fully functional and performing
optimally in all PBGC computing environments and to facilitate identification of issues when they aren’t.

103
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PBGC currently uses Network Node Manager (NNMi) and SiteScope for monitoring the status/availability
of the PBGC infrastructure. PBGC utilizes Microfocus Application Performance Management including
Synthetic monitoring, and Real User Monitor (RUM) for monitoring the availability and performance of
IT infrastructure systems and applications. Typical IT Infrastructure and Application Availability,
Capacity, and Performance Monitoring administration and support activities include:

• Monitor, troubleshoot, and address issues pertaining to PBGC’s IT Infrastructure and Application
Availability, Capacity, and Performance Monitoring tools; e.g. HP/MicroFocus NNMi, SiteScope,
BSM, BAC, etc.
• Configure PBGC’s IT Infrastructure and Application Availability, Capacity, and Performance
Monitoring tools, including adding and remove devices to the monitoring tools and configuring
application and system performance monitoring

See Appendix F - PBGC Locations for PBGC locations, including planned changes to PBGC’s facilities and
data centers over the life of the contract. See Appendix G - IT Service Support Guidelines for impact,
urgency, and prioritization guidelines associated with IT service and support. See Appendix J - IT Service
and Support 2018 Statistical Summary for tier 2 incidents, requests for information (RFIs), service/access
requests, requests for change (RFCs), and RFC tasks processed in calendar year 2018 for these services.
See Appendix K - IT Infrastructure Tools List for a comprehensive list of the software utilized to provides
these services.

6.5.3.2 Requirements
The contractor shall provide the IT infrastructure and application availability, capacity, and performance
monitoring services outlined in the following table:

Reference Requirement
(SM-AA-xx)
SM-AA-01 Contractor shall identify a lead for IT Infrastructure and Application Availability,
Capacity, and Performance Monitoring. This lead is required to serve as the primary
point of contact for all IT Infrastructure and Application Availability, Capacity, and
Performance Monitoring.
SM-AA-02 Contractor shall provide and maintain a fully functional, optimally performing IT
Infrastructure by providing ongoing monitoring capabilities. This shall include, but is
not limited to:

• Monitoring system and application availability


• Monitoring system and application performance
• Monitoring infrastructure capacity
SM-AA-03 Contractor shall configure PBGC’s infrastructure monitoring tools to monitor new
servers within 7 days of network connectivity and will remove decommissioned
servers from monitoring tools with 7 days of removal from the network

104
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
SM-AA-04 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New System or Application Monitoring

See SLA section for details on associated SLA measures.


SM-AA-05 Contractor shall configure and provide ongoing support for synthetic transactions
and/or real user monitoring for all new applications and infrastructure solutions
introduced at PBGC
SM-AA-06 Contractor shall manage requests for new IT Infrastructure and Application
Performance Monitoring solutions and enhancements or updates to existing
Performance Monitoring solutions including, but not limited to:
• Documenting requirements
• Developing design summaries including level of complexity and effort and
monitoring technologies to be utilized, e.g. synthetic transactions, real user
monitoring, etc.
• Manage, track, and publish progress on development efforts and present
updates to customer no less than monthly
SM-AA-07 Contractor shall update synthetic transactions and/or real user monitoring in
conjunction with application and infrastructure solution updates necessitating
changes to existing monitoring configurations and provide ongoing support as needed
SM-AA-08 Contractor shall maintain passwords for IT Infrastructure and Application Availability,
Capacity, and Performance Monitoring privileged accounts, e.g. service accounts,
local system emergency recovery accounts, etc. in privileged account management
tool and utilize this tool to perform administrative functions via brokered session or
account check out
SM-AA-09 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the IT Infrastructure and Application Availability, Capacity, and Performance
Monitoring infrastructure

6.5.4 ITIOD Reporting and Dashboarding


6.5.4.1 Scope of Services Supported
Provide ongoing Reporting and Dashboarding about IT infrastructure in addition to providing support for
PBGC’s enterprise-wide reporting and business intelligence tools; currently Oracle BI Publisher 11.x and
Tableau 10.x. Leverage disparate data sources and reporting capabilities to meet ITIOD reporting
requirements, e.g. ServiceNow reporting capabilities and data to report on IT Service Management,
HP/Microfocus tools and data to report on IT infrastructure and application availability; capacity; and
performance, Splunk to report on event-based data, Active Directory and SailPoint for reporting on
identity and access management, and SharePoint on-line for all kinds of miscellaneous data. Typical
ITIOD Reporting and Dashboarding administration and support activities include:

105
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Generate and compile Contract required reports and deliverables
• Generate reports to support SLAs
• Collect requirements for and then develop new reports with federal approval
• Modify existing reports with federal approval
• Configure reporting tools
• Post reports to central location and/or distribute via email

See Appendix B – Deliverables and Appendix C - List of Required Meetings and Reports for more
information on required deliverable and reports that are generated as part of these services. See
Appendix K - IT Infrastructure Tools List for a comprehensive list of the software utilized to provides
these services.

6.5.4.2 Requirements
The contractor shall provide the ITIOD reporting and dashboarding services outlined in the following
table:

Reference Requirement
(SM-RD-xx)
SM-RD-01 Contractor shall identify a lead for ITIOD Reporting and Dashboarding. This lead is
required to serve as the primary point of contact for all ITIOD Reporting and
Dashboarding.
SM-RD-02 Contractor shall provide and maintain a fully functional, optimally performing IT
Infrastructure by providing ongoing reporting and associated capabilities. This shall
include, but is not limited to generation of:

• Daily operations and executive summary reports


• ITSM reports
• Identity and Access Management reports
• System and application availability reports
• System and application performance reports
• Infrastructure capacity reports
• Asset and Configuration Management reports
• SLA Reports
SM-RD-03 Contractor shall manage requests for new IT Infrastructure reports and
enhancements or updates to existing reports including, but not limited to:
• Documenting requirements
• Developing level of complexity and effort and reporting tools technologies to
be utilized, e.g. BI Publisher, Tableau, ServiceNow, etc.
• Manage, track, and publish progress on development efforts and present
updates to customer no less than every other week

106
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
SM-RD-04 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New Report

See SLA section for details on associated SLA measures.


SM-RD-05 Contractor shall ensure all reports are delivered to an approved, centralized
repository, e.g. SharePoint and/or distributed via email to a distribution list
SM-RD-06 Contractor shall maintain and publish a list of all ITIOD reports and dashboards
including the following information at a minimum:
• Report Name
• Report Description
• Schedule
• Data Source(s)
• Report Destination
• Federal and Contractor PoCs
SM-RD-07 Contractor shall maintain passwords for IT Infrastructure Reporting and Dashboarding
privileged accounts, e.g. service accounts, local system emergency recovery accounts,
etc. in privileged account management tool and utilize this tool to perform
administrative functions via brokered session or account check out
SM-RD-08 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the IT Infrastructure Reporting and Dashboarding infrastructure

6.5.5 Major Incident Management


6.5.5.1 Scope of Services Supported
Maintain and participate in PBGC’s Major Incident Management process which formally establishes a
consistent and disciplined strategy to managing significant Information Technology (IT) issues resulting
in ongoing significant adverse impact to PBGC users and/or IT systems with these expected outcomes:

• An established standard approach to classifying and responding to IT issues resulting in


significant ongoing adverse impact to PBGC users and/or IT systems

PBGC utilizes the ITSM module of ServiceNow to track relate Service Desk interactions and incidents to
issues with an ongoing significant adverse impact to PBGC users and/or IT systems. For all such events,
PBGC requires the Contractor to produce an After-Action Report (AAR) within the weeks following the
major incident. Typical Major Incident Management activities include:

• Identify and escalate issues within the program and to federal incident manager
• Assess impact of issues
• Communicate with impacted users and with ITIOD federal stakeholders
• Develop recommended strategies to work around identified issues
• Conduct root cause analysis and resolve issues through PBGC’s change control process
107
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Open tickets with 3rd party vendors supporting IT infrastructure hardware, software and service
issues
• Develop and obtain approval for AAR
• Address any recommendations identified in AAR with federal approval

The following table summarizes the Major Incidents in the 2018 calendar year. PBGC transitioned to
ServiceNow as its operational system for incident management on January 19th. The following table is of
data between January 19th and December 31st, 2018.

Incident Opened Title (Short Notes


Number Description)
INC0101158 2018-01-29 MASTER TICKET: Packaged a new version of Symantec and it is
09:21:54 VPN - Users are not being pushed thru SCCM to the specific who were
able to log into VPN experiencing VPN issue when they attach to the
local network
INC0111423 2018-04-11 MASTER TICKET: This was a high-level network issue with Verizon,
08:26:43 Network - PBGC and has been resolved. Everything was restored
internet connection to relay as of 1300
is unavailable.
INC0112455 2018-04-17 Master Ticket - Corrected the configuration on the port-channel
14:35:22 Network: no for the VLANs. After doing so, services were now
connection issues reachable.
INC0114411 2018-05-01 MASTER TICKEt: Rolled Back OAM Upgrade
11:36:22 CCS, Spectrum, UPT,
IPS - Users receiving
OAM Access error
INC0115341 2018-05-08 Miami Switch Moved all connections from switch 3 into all the
09:02:37 Failure remainder switch ports
INC0119255 2018-05-30 MASTER TICKET: Successfully resolved the issue that the users
08:12:21 Cannot remote to were unable to access their VDI machine.
VDI machine
INC0120388 2018-06-05 MASTER TICKET - ADFS servers were rebooted and the issues were
09:59:14 SharePoint - USers resolved. problem ticket PRB0001115 was
are recievng opened to validate with Microsoft as to why the
Authentication issue occured
errors

108
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
INC0123276 2018-06-22 Master ticket - Emailed user letting him know this issue has been
11:59:45 Sharepoint - getting resolved. Pasting email below.
Access Denied error
message
The issue has been resolved. A notice to the
entire PBGC was just sent out stating:

`````````````````````````````````````````````````````````````````````
``````````````````````````````````
Impact: Users accessing PBGC
Connect/SharePoint sites may have gotten an
Access Denied message for one or more sites.
What's Happening: Technical teams identified
and resolved the issue that was causing
intermittent Access Denied messages when trying
to access some SharePoint sites.
Requested User Action: If you received an Access
Denied message when accessing a SharePoint
site, please close and reopen your browser
before accessing the site again.

`````````````````````````````````````````````````````````````````````
````````````````````````````````````

Please close any browser windows you might


have open, and try to get to your sites, and you
should have no more issue.

109
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
INC0123641 2018-06-26 Master Ticket - resolved by workaround instruction:
06:41:06 Outlook - Users are
getting an error Good Afternoon
when accessing
outlook client We have been investigating the outlook issue and
believe we have found a workaround. If you are
still having an issue with outlook, can you please
try these steps below to fix the outlook issue? Let
me know if you have any questions or concerns:

1. Close Outlook if Open


2. Run Machine Policy from the Configuration
Manager Action Tab

3. Open Software Center


4. Select PBGC Script –
DisableADALatopWAMOverride Icon

5. Select Install

6. Reopen Outlook and it should work.

INC0128383 2018-07-25 Master Ticket - CCS, Databases crashed unexpectedly due to a


15:11:40 CMS, IPS, MyPPA , connection issue. DBA Team has restarted the
Comprizon impacted databases (Genesis, BFIPROD,
applications are BPMPROD, COMPROD & CAS). Restored services
down to following applications Spectrum, MyPAA, IPS,
CMS, CCS, COMPRIZON.
INC0129120 2018-07-31 Master Ticket - Microsoft has confirmed that a recent Office 365
07:32:59 outlook - prompting configuration update was causing the observed
for log in issue. O365 Service Engineering deployed a fix
that has corrected the configuration issue and
restored functionality. Customers should close
out of outlook and reopen it. Service Desk
Advisory Restored has been sent out.
INC0129567 2018-08-01 Master ticket - Oracle database and application were started up
12:55:02 spectrum - can't and verified by 6:52pm 08/01/2018. Please refer
reach this page. to notes by Cesar below at 3:01am on
08/02/2018 for details.
INC0130381 2018-08-06 Master Ticket - PBGC's external phone service is now available..
14:51:09 PBGC Phones are Power was lost to the lower level pbx room
not working switch. Systems are now back up.

110
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
INC0132393 2018-08-20 Master Ticket - The databases were restarted and service has
09:39:17 PROD - Three (3) been restored.
production
databases crashed -
COMPROD,
PMVPRD &
OEMPROD
INC0134419 2018-08-31 MASTER TICKET: Services have been restored.
09:05:39 CRM - Users are
reporting the
application is very
slow
INC0134587 2018-09-01 cor0prd1wdfp01 WAN service was restored by rebooting the
12:29:55 host unreachable Switches and L3 NID.
due to
cor0ent1crtr01.ent.
pbgc.gov being
unreachable
INC0134713 2018-09-04 Master Ticket - Technical teams worked with Microsoft, whom
11:24:52 Office365/Sharepoi identified an issue on the vendor side. Microsoft
nt Issue has confirmed that the issue has been resolved
and connectivity restored.
INC0134971 2018-09-05 Link from NMC continues to monitor the circuit after brief
12:42:44 Wilmington to bounced for stability, contacted service provider
Headquarters down. for support. AT&T immediately tested the circuit
and found it was a brief hit – came clear with
test, closing the ticket since the outage was
within the threshold.
INC0134982 2018-09-05 MASTER TICKET - Outlook and Skype thrttole was caused by an
13:32:56 outlook - error update which has since been rolled back by the
message "throttled" vendor Microsoft.

The issue is now resolved and users can continue


to work by restarting Outlook Client.
INC0135537 2018-09-10 Master: BPEL The replicas.prop file was corrupted. It had to be
08:22:31 Outage renamed and then the service restarted, so that
BPEL SOA could create a new file. This resolved
the issue.
INC0138067 2018-09-25 Master ticket - CRM Cleared the FRA space to release the space and
07:40:03 error message - the CRMPROD database was back to normal.
Internal Server Error Application team verified the CRM application
and no issues found.

111
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
INC0139195 2018-10-01 MASTER TICKET - Restarting CAS DB resolved the issue.
10:50:35 CAS - users
reporting issues
with CRM, IPS and
Spectrum
INC0145149 2018-11-06 Master ticket - Java The certificate was renewed, the DRS file was
08:10:55 application error - updated and deployed to all workstations in all
Applciation Blocked environments. A BigFix fixlet has been developed
by Deployment Rule to push this file when necessary.
Set
INC0147828 2018-11-21 Master Ticket - Gigamon recommendations have been followed
13:21:56 Network Latency and UDLD has been set to be bypassed on the
and Applications Gigamon, all interfaces are back up
down
INC0149492 2018-12-03 Master Ticket - Determine Symantec Endpoint was scanning
14:26:20 Oracle Access several servers which caused overutilization
Manager Operation which translated to latency on several
Error applications. Security team will investigate on
why the scan was being run as this was not
scheduled in policy during business hours.

112
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
The following diagram depicts the workflow for PBGC’s major incident response process:

113
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
114
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.5.5.2 Requirements
The contractor shall provide the services outlined in the following table in supporting major IT
infrastructure incidents:

Reference Requirement
(SM-MI-xx)
SM-MI-01 Contractor shall identify a lead for Major Incident Response. This lead is required to
serve as the primary point of contact for the Major Incident Response process as well
as lead the Contractor’s response to major incidents when available.
SM-MI-02 The Contractor shall ensure that all IT major incidents are tracked and managed in a
consistent manner in accordance with PBGC’s Major Incident Response procedures to
include, but not limited to assessing scope and impact of issues, establishing
workarounds, corresponding with and obtaining guidance from 3rd party support
vendors, addressing root cause of the problem, and communication to impacted
customers and amongst subject matter experts
SM-MI-03 The Contractor shall provide a recovery method to restore service to the user(s)
affected by a major incident as quickly as possible. The Contractor shall perform root
cause analysis and, if it will restore service more quickly than any other approach,
develop and implement a permanent fix during the course of major incident handling.
SM-MI-04 The Contractor shall determine if the work to resolve major incidents is dependent
upon resources outside this contract, such as another vendor or an OEM. If that is
the case, the contractor, upon learning of the dependency, shall:
• Immediately request support from the vendor.
• Notify the PBGC Federal Incident Manager of the dependency including the
ticket number.
• Provide hourly (or time period agreed upon by the federal incident manager)
updates on the status of the ticket with the external vendor or lack thereof.
• Immediately be available to act on any vendor recommendations.
• Receive, respond, escalate, and resolve all IT major incidents in an expedient
manner.
See SLA section for details on existing major incident SLA measure.
SM-MI-05 The Contractor shall assess incidents and make recommendations to the Federal
Incident Manager and the COR regarding declaration of an issue as a “Major
Incident”. Once the Federal Incident Manager determines that the incident is major,
the Contractor shall immediately implement the major incident handling process
SM-MI-06 The Contractor shall at a minimum implement the following Major Incident Handling
procedures:
• Contact the Network Operations Center (NOC), contact the Contractor
Incident Manager, contact the Federal Incident Manager, open an incident
bridge, open an incident ticket, notify the on-call Subject Matter Expert, and
send a SOR or Advisory with Federal approval.
• Work with the Major Incident Management Team to resolve the incident, to
include creating a Request for Change (RFC) where necessary. The Contractor
shall follow the Change Management process for all RFCs.
115
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Update Federal IM and/or Operations Manager every half hour or as
requested
Once the incident is closed, the Contractor shall request Federal approval to close the
incident, send an ending SOR or Advisory noting the incident as closed.
SM-MI-07 The Contractor shall support the Federal Incident Manager in ensuring effective and
efficient resolution of major incidents and ensuring that communications related to
the incident are consistent and concise.
SM-MI-08 The Contractor shall produce an “After Action” Report within 14 days following the
conclusion of a major incident. See Appendix B – Deliverables for more information
on this deliverable.

6.5.6 Problem Management


6.5.6.1 Scope of Services Supported
Provide support for PBGC’s ITIL-based Problem Management process which formally establishes a
consistent and disciplined strategy to managing Information Technology (IT) problems with these
expected outcomes:

• An established standard approach to managing, tracking and reporting IT related problems


• An established formal escalation path for IT related problems

PBGC utilizes the ITSM module of ServiceNow to track IT infrastructure problems. Typical Problem
Management activities include:

• Identify and track problems associated with activities under the Contractor’s control and
problems that the Government directs the Contractor to monitor
• Develop recommended strategies to resolve or workaround identified problems
• Document correspondence with 3rd party vendors supporting IT infrastructure hardware,
software and service issues

As of 3/11/2019, ITIOD had 63 open problems as follows:

Problem Title (Short Description Priority Assignment State


Number Description) Group
PRB0001 Symantec - Not ITIOD Security Operations is not able 4 - Low ITIOD- Open
339 able to login to login with AD credentials to OPSSECURIT
with AD Symantec Endpoint Protection web Y-IC
credentials to console. We have reached out to
Symantec Symantec support to troubleshoot
Endpoint the issue (case number 28614526).
Protection web
console

116
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Checkpoint - Checkpoint logs are not being 3- ITIOD- Open
338 Logs are not ingested into Splunk due to Modera OPSSECURIT
being ingested incompatibility issues between te Y-IC
into Splunk PBGC's Checkpoint modules and
Splunk. For any security
investigations in regards to USB
usage on PBGC workstations, the
ITIOD Security Operations team is
limited to the local logs that are only
retained for 120 days within the
Checkpoint management console.
Any security event involving
Checkpoint that occurred prior to
120 days will not be searchable.
PRB0001 Serena - PVCS Support, 4 - Low ITIOD- Open
334 VM 861 I am looking for an (old) solution to OPSCMTOOL
Vulnerabilities removing the PVCS VM security S-IC
vulnerability find for the below
numbers. I believe in the past we
turn off SSL in IIS or turned some
option off. Can you help with the
steps required to remove these
security vulnerability finding. See
attachment for listings.

PRB0001 NetBackup - Multiple VM backup failures 4 - Low ITIOD- Kno


333 Backup failures following the VM upgrade to VMFS6. OPSSTORAG wn
on VM clients Troubleshooting in progress E-IC Error
[Incident:
190305-
001394
PRB0001 CyberArk - We have two Incident tickets for the 3- ITIOD- Pend
332 Network Windows team. That we're unable Modera OPSNETWOR ing
Devices not to access any of the FBA and te K-IC Chan
connecting via Wilmington switches using Cyber ge
CyberArk Ark.

INC0162834
INC0162892

117
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Veritas case - Plan of action has been devised. 4 - Low ITIOD- Open
331 190305- Waiting for approval to restore the OPSSTORAG
001504 - database from this morning backup. E-IC
unfinished Then inform Veritas.
OpsCenter
database
defragmentatio
n
PRB0001 Microsoft - Issue: Skype for business app. Need 4 - Low ITIOD- Open
330 Able to modify to lock down the application from to OPSWINDO
Phone Number prevent users from changing their WS-IC
settings in phones numbers. Users can currently
Skype change their Phone information on
the app causing the their contact
information to change.

Vendor ticket created 13330048


with Microsoft.
PRB0001 Micro Focus - We are unable to run any test scripts 2 - High ITIOD- Open
324 Unable to run form UFT 14. Test scripts are not OPSTCOTOO
test scripts in responding on selecting the run LS-IC
UFT. button in UFT. On running the test
scripts, UFT is expected to show
pause/stop buttons.
We see pause and stop buttons
missing on selecting the run button,
please see screenshot 1 in attached
word document. We are unable to
stop or pause UFT background
process from the task manager. See
the last screenshot in the word
document attached.

Attached are the log files associated


and screenshots. Please feel free to
contact us if you have any questions.

Opened a critical ticket with Micro


Focus. SD02418582.

118
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Acunetix - Web We learned that a scan of PBGC.gov 4 - Low ITIOD- Open
323 Application web site that is scheduled to run OPSSECURIT
Scans from 10pm through 6am 4 days a Y-IC
performed week was scanning the portions of
with Acunetix the PBGC.gov web-site daily. That is,
appear to not on the second day of scanning it did
be pauseable not resume from where it ended on
the first day but appeared to start
from the beginning again on the
second day of scanning. We have
reached out to the vendor for
assistance
PRB0001 '-Veritas case validation of old master server SAN 4 - Low ITIOD- Kno
320 190227- storage migration to new Hitachi - OPSSTORAG wn
001634 - INC0162981 E-IC Error
validation of
old master
server SAN
storage
migration to
new Hitachi -
INC0162981
PRB0001 Veritas case Image expiration report shows 4 - Low ITIOD- Pend
319 190227- images whose expiration dates have OPSSTORAG ing
000128 - Image passed/ INC0162978 E-IC Chan
expiration ge
report shows
images whose
expiration
dates have
passed/
INC0162978
PRB0001 Veritas - Veritas case 190226-001178 4 - Low ITIOD- Pend
318 Appliance 5230 OPSSTORAG ing
firmware E-IC Chan
upgrade ge
PRB0001 Veritas - Veritas - 190222-001296 Sev3 4 - Low ITIOD- Kno
317 190222- Vulnerability remediations OPSSTORAG wn
001296 Sev3 E-IC Error
Vulnerability
remediations

119
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Veritas - Sev3 Sev3 Security alert - Nessus plugin 4 - Low ITIOD- Pend
316 Security alert - Veritas case 190226-001178 OPSSTORAG ing
Nessus plugin E-IC Chan
ge

PRB0001 Infoblox - DNS 4 - Low ITIOD- Open


314 issues within OPSNETWOR
Domain K-IC
Controllers
PRB0001 AutoSupport Call has been made to replace the 4 - Low ITIOD- Kno
312 Appliance faulty part tomorrow - 02-19-2019 OPSSTORAG wn
Case: 190216- E-IC Error
000150
NetBackup
5240
VTAS0003251
PRB0001 Requested Requested Microfocus for HP UFT 4 - Low ITIOD- Open
307 Microfocus for latest stable release information. OPSTCOTOO
HP UFT latest LS-IC
stable release Problem ticket is created to track the
information - vendor ticket.
SD02403846 System does not allow us to create a
problem ticket without an related
incident ticket number. Using a HP
UFT incident ticket from the past.
PRB0001 ServiceNow - Workflow issue on Dev after London 4 - Low ITIOD- Open
305 Workflow issue upgrade. The link for "Show OPSSERVICE
on Dev after workflow" is not visible for some NOW-IC
London admin users on Change request.
upgrade When a Change is submitted and
moved from Change Logging to Build
and Test there are no Approvers
associated to the Change. This
behavior is on and off and happens
to only some admin users.
We also noticed Service Catalog base
workflow is missing.

120
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 ServiceNow - SOR tab missing for Incident view in 4 - Low ITIOD- Open
304 SOR tab Development environment after OPSSERVICE
missing from London upgrade. Case has been NOW-IC
the Incident opened with vendor for resolution.
view in
Development
environment
PRB0001 Verritas - Not able to access java console on 4 - Low ITIOD- Open
300 Receiving error old master server- hqw0prod1bak01 OPSSTORAG
when trying to Receving TomCat certifcate error E-IC
access the
NetBackup java
console.
PRB0001 Unable to close Environment - Production. 1- ITIOD- Pend
299 test script as On trying to run and close test Critical OPSTCOTOO ing
HP scripts, with various test status (pass, LS-IC Chan
ALM/Quality no run, block) on trying to close test ge
Center script. HP ALM/QC does not respond.
becomes Vendor ticket no - SD02396238
unresponsive. -
SD02396238
PRB0001 Microsoft - Please open a ticket for the Windows 4 - Low ITIOD- Open
295 Skype Sharing team due to several users OPSWINDO
Issues experiencing issues sharing their WS-IC
screen and being dropped from
Skype for Business meetings.

We have a few (if not several)


Incidents that we believe are being
caused by some incorrect network
configurations uncovered during a
Microsoft NPA Review. Although
these config changes might not be
the root cause, we need to raise the
priority of this and expedite the
changes necessary

121
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Oracle - one one oam sever in prod failed , which 4 - Low ITIOD- Open
289 oam server caused multiple applications outage OPSDBA-IC
failed today.

Based on the oam log on wlm08,


oam_server application failed to be
started due to coherence
communication timeout although the
managed server was up. I don't know
why we suddenly got coherence
communication issue. I verified the
network connection, but did't find
issue. A SR (3-19255338541 : one
oam server in a cluster failed) has
been opened with oracle support.

122
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Hitachi - Failed The issue is two ldev devices(4402 4 - Low ITIOD- Kno
288 to install two and 4431) failed to be installed. OPSSTORAG wn
ldev devices on E-IC Error
hqw0cdi0sdb1 One of them on hqw0cdi0sdb12/22.
2/22 and 14 The other on hqw0cdi0sdb14.

I0/HORCM/usr/bin/raidcom add
device_grp -device_grp_name
hqw0cdi0sdb12_5A_6A
hqw0cdi0sdb12_4402 -ldev_i
raidcom: [EX_CMDRJE] An order to
the control/command device was
rejected
It was rejected due to SKEY=0x05,
ASC=0x26, ASCQ=0x00,
SSB=0x2E20,0x0000 on
Serial#(445111)
CAUSE : LDEV is not installed.
[root@hqw0cdt1rcat01 bin]#
/HORCM/usr/bin/raidcom add
device_grp -device_grp_name
hqw0cdi0sdb14_5A_6A
hqw0cdi0sdb14_4431 -ldev_id 4431 -
s 445111 -I0
raidcom: [EX_CMDRJE] An order to
the control/command device was
rejected
It was rejected due to SKEY=0x05,
ASC=0x26, ASCQ=0x00,
SSB=0x2E20,0x0000 on
Serial#(445111)
CAUSE : LDEV is not installed.
PRB0001 Cisco - Vendor Vendor Engagement to Obtain Script 3- ITIOD- Open
281 Engagement to To Address Backlog Issue in IronPort. Modera OPSWINDO
Obtain Script te WS-IC
To Address Related to higher level event which
Backlog Issue caused email latency .
in IronPort

123
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 SharePoint - A vendor ticket (CS-258) has been 3- ITIOD- Open
277 Repeated created with the 3rd party Modera OPSSHAREP
alerts on disk SharePoint online class tagging tool te OINT-IC
space caused Concept Search.
by concept
search Issue: Repeated alerts on disk space
database. caused by concept search database.
Services need to be restarted
constantly. Tagging SharePoint
online items does not exhibit
expected behavior. Need health
check prior to version upgrade.
PRB0001 CyberArk - The CyberArk ticket is case # 3- ITIOD- Open
271 CyberArk 00370893 Modera OPSMEPROJ
connections te ECTS-IC
are dropping Losing connections to several TCO
hosts while working actively on
them. Most recent connection
dropped from
oracle@hqw0cdt0sdb15.

Another issue I noticed yesterday, I


lost all items saved under "My
Views" tab in CyberArk.
PRB0001 Microsoft - Agents receiving an Error Message 3- ITIOD- Kno
269 Error Message When Making Changes in InTune that Modera OPSPHONE- wn
When Making you do not have enough permissions. te IC Error
Changes in This happens even though the agents
InTune have the correct permissions
assigned. No configuration changes
have been made within InTune

Permissions have not been changed


to restrict access within intune with
agents.
PRB0001 NetApp Working with NetApp to identify 3- ITIOD- Open
268 Engagement - issues related to IPS performance Modera OPSSTORAG
IPS issues. te E-IC
Performance
Issues/

124
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 VM Ware - ESXi the ESXi Host 3- ITIOD- Open
265 Host and VMs "hqw0prd1evpi36.ent.pbgc.gov" and Modera OPSWINDO
Loss of VMs (running on this ESX Host) lost te WS-IC
Connection connection from vCenter
from vCenter.
Ticket to be created with VMware to
identify Root Cause.
PRB0001 SharePoint - Issue: User are having intermittent 4 - Low ITIOD- Pend
262 Users are issues with opening the library, OPSSHAREP ing
getting errors saving, searching and editing excel OINT-IC Chan
in CC Internal spreadsheets in this library. ge
Reports site
Library: CC Internal Reports
URL:
https://pbgcgov.sharepoint.com/tea
ms/PSD/CCD/CCC/CCInternalReports
/

Troubleshooting: Technicians are


aware of the issue. Users have excel
files with multiple links to other excel
sheets in other directories. This is
causing an issue. A solution is being
investigated.

KB Document(s): MASTER TICKET


(Requested by James Jefferson)
Verified Contact Information: Yes
Offered Ticket number: Yes
PRB0001 VertivCo - Site Site Scan-SS Web issues post change 3- ITIOD- Open
261 Scan-SS Web Modera OPSWINDO
issues post May need to reboot the server to te WS-IC
change resolve the issue.

Working with vendor on this and


Problem ticket to be opened.

Working with vendor VertivCo.


PRB0001 Veritas - move move archive logs policy from 4 - Low ITIOD- Open
259 archive logs med01 to med02 OPSSTORAG
policy from E-IC
med01 to
med02

125
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Veritas - The The OpsCenter vxpmdb.db 4 - Low ITIOD- Kno
258 OpsCenter database file was overwritten. Case OPSSTORAG wn
database file opened to Veritas to determine how E-IC Error
was to restore it.
overwritten.
Veritas case #181227-001203 on
how best to restore the Opscenter
database that was deleted due /var
partition getting full.
PRB0001 Hewlett QC freezes with certain commands: 4 - Low ITIOD- Kno
253 Packard - HP INC0151524 - QC Keeps Freezing OPSTCOTOO wn
ALM QC v1120 (Tiffany) LS-IC Error
Freezes - - INC0151588 - QC freezing when
Vendor navigating to requirements section
number: (Teresa)
SD02364889 - INC0151884 - change the status of
a test script from a test set takes
more than 5 mins (Teresa)

Contacted Vendor with vendor ticket


#: SD02364889 and SD02369126

126
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Image Change retention policy for backups 4 - Low ITIOD- Kno
252 expiration - - and expire older backup images OPSSTORAG wn
Related to E-IC Error
Change - Related to INC0152139- and RFC
retention #CHG-0012192
policy for
backups and Expiring images as documented in
expire older RFC #CHG-0012192
backup images
1) please implement the image
expirations ASAP, starting with those
on MSDP01 and let me know when
done.
2) Who is going to be on the call with
Veritas besides Bashir? I am going to
be out of pocket for a couple of
hours. Please make sure that we
have the right person on the call
supporting Bashir.
3) Please make sure that we have
Veritas tell us what the right way is
to back up our Oracle 12c data based
on our current environment. Do not
tell them how we want to do it. Ask
them how they recommend doing it,
and summarize the plan for the team
after the call.
PRB0001 RedHat - Recently as a part of patching 4 - Low ITIOD- Open
251 Servers is hqw0prd1rcma61/62, OPSUNIXLIN
unable to boot alx0prd1rnes01 update to new kernel UX-IC
after upgrade version 3.10.0-957.1.3.el7.x86_64,
to new kernel After patching servers not able to
version boot in new kernel. Currently servers
are online on old kernel Version
3.10.0-862.14.4.el7.x86_64.
PRB0001 Microsoft - Users are reporting network drive 4 - Low ITIOD- Kno
249 W:\Drive Mapping issues with W:\Drive which OPSWINDO wn
Mapping issue is impacting Archive access every WS-IC Error
in Wilmington day.
Users have to click on the W:\drive
several times every morning after
they login to reconnect. (see
attached)

127
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Move the 4 - Low ITIOD- Kno
248 policies OPSSTORAG wn
attached from E-IC Error
med01 to
med02
PRB0001 Veritas - VMware full server restore fails. 3- ITIOD- Kno
244 VMware full Veritas case - 181210-001948 Modera OPSSTORAG wn
server restore te E-IC Error
fails
PRB0001 Micro Focus - I am running into an issue with the 4 - Low ITIOD- Pend
241 PVCS VM Client client when trying to login after OPSCMTOOL ing
- Unable to project has been converted from S-IC Chan
Login After VLOGIN to LDAP. User get two errors ge
Ldap Change 'Error reading Config file and cannot
access LDAP Password . See
Attachments for more detail.

There are Lock.lck-arc files (8).


Error message references this line
417:

LDAPINFOFILE = "L:/CAS-
Suite/pvcsldap.ini"

I get the following error after


selecting the PVCS VM L:\CAS-Suite
project but before entering
credentials. Prod LDAP user/pwd.

128
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Symantec - ITIOD Security Operations is working 4 - Low ITIOD- Open
238 Investigate with Symantec support in order to OPSSECURIT
Symantec AV determine if there was any AV Y-IC
scanning on scanning occurring on the weblogic
Weblogic servers on Monday December 3
Servers around 2:30pm EST.
PRB0001 Splunk - Increase in Physical Memory Usage 4 - Low ITIOD- Pend
234 Increase in on 3 indexers upon upgrade of OPSSECURIT ing
Physical Enterprise Security. This is a known Y-IC Chan
Memory Usage issue. Issue has being reported to ge
upon Upgrade Vendor.
of Enterprise
Security

129
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Infoblox - Users Desktop - Users connecting to 3- ITIOD- Open
232 connecting to remote.pbgc.gov getting -"unable to Modera OPSNETWOR
remote.pbgc.g resolve" host when selecting My PC te K-IC
ov getting - link.
"unable to
resolve" host Previously we used the below to
when selecting release the IP if previously connected
My PC link. to WiFi.
Open up command prompt as admin
and type:
ipconfig – will give you the active IP
address on the PC
nslookup "IP Address" – this will
resolve to the host name in DNS
nslookup "host name" – this will
resolve the IP address

Wi-Fi IP address range is


10.0.10.0/24 to 10.0.11.0/24

If client PC is pointing to a Wi-Fi


address when it's plugged in you will
need to do ipconfig /release on the
hardwire. Then connected the PC to
the wireless network and open up
command prompt and run as admin
"ipconfig /release wi-fi" that will
remove the stale host from DNS
server.

This has not been a consistant fix,


there have been instances where the
Network team will need to resolve
the issue via Infoblox.
PRB0001 HP SM9 - Repeated occurrence of index 3- ITIOD- Open
230 Repeated GetIT corruption of GetIT catalog. Modera OPSSERVICE
index te MANAGER-IC
corruption.

130
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Gigamon - We experienced a network issue 3- ITIOD- Open
229 Network which caused severe latency on the Modera OPSPHONE-
Latency and PBGC Network. We isolated the issue te IC
Applications on the interface between AG2 and
down the Gigamon. This interface has been
shut down to resolve the latency
issue. Vendor case will need to be
opened with Gigamon to further
investigate the issue.
PRB0001 Infoblox - Several users when trying to access 3- ITIOD- Kno
228 Issues logging remote.pbgc.gov will get an error. Modera OPSNETWOR wn
into This is an issue with Infoblox and IP te K-IC Error
remote.pbgc.g host names not being released from
ov - IP host Wi-Fi subnet. This has been tested a
names not verified.
being released
from WiFi
subnet
PRB0001 Failed update Issues updating the firmware on a 3- ITIOD- Pend
210 of the component on one of the blade Modera OPSWINDO ing
Vcflexfabric chassis which is causing a te WS-IC Chan
vulnerability finding for Open SSH. ge
PRB0001 Surface Pro - Issue: Users are experiencing several 3- ITIOD- Kno
207 Systems are different issues on their Surface Pros. Modera OPSSITESUP wn
getting an te PORT-IC Error
operating error 1. Blue Screen of death
- BSOD 2. System is "Studdering"
3. Randomly will shut down.
4. system freezes

Troubleshooting: Have User Install


the firmware listed in Software
Center. If this does not fix it site
ticket to Site Support.

KB Document(s):MASTER TICKET -
Per Site Support
Verified Contact Information: Yes
Offered Ticket number: Yes

131
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Intact - Asset Following migration of AST43PRD 3- ITIOD- Open
198 Manager CG4 database from M8000 to T5-8, the Modera OPSSERVICE
integration Asset Manager to CG4 integration te MANAGER-IC
scripts not scripts failed. The scenario executes
running as when run manually. It keeps failing
expected. when run as a service. Produces
Stopping error: API error 57094232: '???'
shortly after
restart.
PRB0001 VMware - Jim Edwards is working with VMware 4 - Low ITIOD- Open
191 Setup vMotion Technical support to figure out the OPSWINDO
across configuration that would work with WS-IC
HQW/WIL sites PBGC VMWare current setup.
- ISO getting
help for
RFC0010127
PRB0001 DataGuise - DataGuise software is masking data 4 - Low ITIOD- Open
182 Data Masking with numerous inconsistencies after OPSDBA-IC
Software Issues Oracle 12c patch upgrade.
Data masking timing and status are
different between the software user
interface and the database log.
Documented masking times from the
old version has doubled after the 12c
patch upgrade.
Masking software is overall slow
while performing masking.

132
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Service Issue: In researching the Inactive COR 3 - ITIOD- Open
180 Manager9 - report we found some issues in SM 9 Modera OPSSERVICE
ID's are still . Lis Fortune-Williams has 2 different te MANAGER-IC
valid that ID's Requests are being made udder
should not the wrong ID. Both IDs are showing
be/Duplicate up as active.
IDs under user
name Need to check the LDAP feed or the
way the report is pulled in SM9 to
correctly identify that way a users
account status is conveyed to SM9.

Troubleshooting: Reports are being


pulled to compare active users within
ARS and SM9. But need to address
this on a long term basis.

KB Document(s): No KB Referenced
(Ticket Per Mike Skov)
Verified Contact Information: Yes
Offered Ticket number: Yes
PRB0001 Microsoft - When using RDP to connect to VDI 4 - Low ITIOD- Open
169 RDP - The machines and desktop computers, OPSNETWOR
connection has the remote session loses with a K-IC
been lost. message saying "The connection has
been lost. Attempting to
reconnected to your session...
Connection attempt: 1 of 20. See
attached.
PRB0001 Oracle - PROD - PROD - Master ticket - spectrum - 4 - Low ITIOD- Open
159 Master ticket - Users cannot access Spectrum OPSDBA-IC
spectrum - Application.
Users cannot Spectrum application created an
access overwhelming number of
Spectrum transactions that overloaded SDB12,
Application OHS03, OHS04 and crippled the
LDOM server bringing down all
production databases and
applications.

133
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Oracle - ASM Oracle 12c databases on the new 4 - Low ITIOD- Open
144 Instance lost Solaris 11 Server (SDB12) OPSDBA-IC
communication unexpectedly crashed due to a
with Oracle 12c communication issue the ASM
Databases. Instance.
DBA Team opened Ticket with the
vendor to urgently look into the
issue.
PRB0001 Bluecoat - No The issue with PBGC's Blue Coat's 4 - Low ITIOD- Kno
123 Maintenance version SGOS 6.5.10.4. The lack of OPSSECURIT wn
support support for CBC and ECDHE cipher Y-IC Error
contract for suites is preventing PBGC users from
Bluecoat proxy accessing FAITAS and DAU websites
on PBGC network via the Blue Coat
web proxies. Without a maintenance
support contract, PBGC cannot
obtain vendor's technical support nor
upgrade to the latest OS for Blue
Coat.

Impact: PBGC users cannot access


FAITAS.army.mil
Learn.day.mil
PRB0001 HP - Unable to Unable to upgrade a primary HP 4 - Low ITIOD- Pend
117 upgrade a Virtual Connect Firmware from 4.5 to OPSWINDO ing
primary HP 4.8 on HP Enclosure WS-IC Chan
Virtual Connect "wil0prd1ahpc03-oa1.ent.pbgc.gov" ge
Firmware from
4.5 to 4.8 on
HP Enclosure
"wil0prd1ahpc
03-
oa1.ent.pbgc.g
ov"

134
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 Oracle 12c Oracle 12c Platform Interface Issues 4 - Low ITIOD- Open
107 Platform Post Upgrade OPSDBA-IC
Interface Issues

First, we need to confirm we have


accounted for all known issues and
then prioritize. I don't claim to have
full knowledge, but this is what I
know about…

1. ARS in ITC is unable to establish a


connection to the SM9 schema – this
means that Autofulfill (to ITC) is not
working and some ARS functions are
not working – This is a critical fix;
likely Separation of Environments
related.
2. TACS in ITC is not working – this
means software developers/testers
are unable to check out test accounts
in order to validate their code under
various user role and access levels.
This is a critical fix; likely Separation
of Environments related.
3. ARS in DPN is working, sort of, but
probably not in a way that is
sustainable or tolerant of changes to
the operational status of the
database cluster. This is an important
fix, but can be deferred to prioritized
(1) and (2) above.
4. TACS in DPN is apparently working,
though we should check this
thoroughly giving the concerns listed
in (3).
5. ARS (Production) queries to the
SM9 schema are much slower. This is
an important fix as the slow
performance impacts numerous daily
actions including the build of the
service catalog.

135
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
PRB0001 ServiceNow - Email notification is not triggered 4 - Low ITIOD- Pend
077 Change from ServiceNow to the Change OPSSERVICE ing
Coordinators Coordinators after clicking "Request NOW-IC Chan
do not receive Approval to Deploy" ge
my email
notification
when clicking
"Request
Approval to
Deploy"
PRB0001 HPSM9 - Active ref ticket -sd254463, Q077179-001 4 - Low ITIOD- Open
019 Directory - tel number - 202-326-4600, 7456 OPSSERVICE
Autofullfiller pc number - pc0028759 MANAGER-IC
did not add
accesses user was approved for pd-sh drive
(full)
the autofullfiller did not add it to her
account. user states it was approve a
couple of weeks ago and was never
added to her profile
PRB0001 HP SM9 - Some Line items which are approved and 3- ITIOD- Open
009 sequenced line predecessors are complete are Modera OPSSERVICE
items hung in sometimes not move from requested te MANAGER-IC
'requested' to ordered status. This is especially
status. common for the iPhone fulfillment
sequence. (Sequenced line items not
showing up- SM9)
PRB0001 HPSM9 - GetIT The synopsis is that on occasion, 4 - Low ITIOD- Kno
002 returns an GetIT returns an error when OPSSERVICE wn
error when attempting to submit a request for MANAGER-IC Error
attempting to another user. The work around is to
submit a restart GetIT web services.
request for Problem ticket recreated from
another user existing PM in legacy Service
Manager 9 system. Legacy ticket PM
276).

PBGC transitioned to ServiceNow as its operational system for problem management on January 19th.
The following table is of data between January 19th and December 31st, 2018 and includes problem
tickets “opened” in ServiceNow as a transfer from the legacy system.

Problem Action Count


Opened New Problem 135

136
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Closed Existing Problem 97
6.5.6.2 Requirements
The contractor shall provide the IT problem management services outlined in the following table:

Reference Requirement
(SM-PM-xx)
SM-PM-01 Contractor shall identify a lead for IT Problem Management support. This lead is
required to serve as the primary point of contact for all IT Problem Management
support.
SM-PM-02 The Contractor shall ensure that all IT problems are tracked and managed in a
consistent manner in accordance with PBGC’s Problem Management Policies and
Procedures to include, but not limited to documenting scope and impact of issues,
established workarounds, correspondence with and guidance from 3rd party support
vendors, root cause of the problem, and communication to impacted customers
SM-PM-03 The Contractor shall review all open problems at least weekly and report progress
until the problem is resolved
SM-PM-04 Contractor shall open a problem ticket each time a 3rd party infrastructure hardware
or software vendor is contacted for service
SM-PM-05 The Contractor shall gather and record initial information from the source for the
reported problem, and perform the following actions pertaining to Problem
Management:
• Review incidents for indicators of eligibility for Problem Management
including any that are escalated to a vendor
• Perform incident matching to identify recurring incidents
• Review incidents for linked incidents
• Review alerts from applications and vendors for Known Errors
• Compare potential problems to existing problems
SM-PM-06 The Contractor shall apply criteria related to business impact and urgency to both
prioritize problems against each other, and to assist in decisions related to application
of resources for problem resolution
SM-PM-07 The Contractor shall provide a recovery method to restore service to the user(s)
affected by the incident as quickly as possible. The Contractor shall perform root
cause analysis and develop and implement a permanent fix. The overriding objective
is to restore service to the user as quickly as possible.
SM-PM-08 The Contractor shall create a Known Error Record. This shall involve but not be
limited to:
• Verifying satisfaction with the workaround
• Creating the Known Error Record in the tool
• Transferring the problem description to the Known Error Record
• Relating all relative incidents to the Known Error
• Incrementing incident count against the Known Error
Notifying the Service Desk that a new Known Error Record has been established

137
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
SM-PM-09 The Contractor shall correct the root cause of the problem, including but not limited
to resolve the root cause of the Problem in compliance with IT change management
process; and updating the Problem Record/Known Error Record with the solution.
SM-PM-10 The Contractor shall close out the Problem record to include, but not limited to:
• Documenting Change Record information in the Know Error Database
• Verifying the following fields are properly filled out in the ITSM tool:
o Problem categorization
o Problem prioritization
o Resolution activities
o Root cause analysis
• Updating the ticket as “Closed”

6.5.7 Risk Management


6.5.7.1 Scope of Services Supported
Provide support for PBGC’s Risk Management Process which formally establishes a consistent and
disciplined strategy to managing Information Technology (IT) Risks with these expected outcomes:

• The ability to consistently leverage formal up-to-date risk management practices.


• An established standard approach to managing, tracking and reporting IT related risks.
• An established formal escalation path for IT related risks.

PBGC utilizes Microsoft Project on-line to track IT risks. Typical Risk Management activities include:

• Identify and track risks associated with activities under the Contractor’s control and risks that
the Government directs the Contractor to monitor
• Develop recommended risk mitigation strategies for identified risks

As of 3/25/201, ITIOD had the following open risks:


Risk OMB Prob Impact Exposu Prioritization Response Risk Response Strategy
Category . re
Any user in PBGC Security 2 3 6 Moderate Transfer/Share Mitigate: Secure BPEL Processes, Add
production Message level confidentiality using
environment can WS-Security, Consider using Oracle
access Web Services Manager to protect and
ApprovalEngine secure web services
Service with a
browser and can
initiate Business
Process Engineering
Language (BPEL)
process. The state of
the backend system
or database can be
accessed/altered
which could consist
PII.

138
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
The ITIOSS Oracle Data/Info 3 4 12 Moderately Transfer/Share The current risk response is
Database High acceptance. Adding an ITIOD Federal
Administration team Approver does not mitigate the risk
provides a "data-fix" since ITIOD personnel typically do not
service to aid the have the necessary domain-specific
application support knowledge to evaluate these changes.
teams by correcting Proper risk mitigation requires
data errors in establishment of an auditable
production application/service owner review and
databases directly, approval process for data fixes, which
rather than through is not available currently. An effort to
officially released implement a service/application owner
application software approval list and management process
user interfaces. is currently underway and can
eventually be leveraged to support
accurate approval routing for data
fixes in the future.
Scrutinizer, which is Reliability Of 1 3 3 Extremely Reduce Currently in the process of finalizing
End of Life (EOL), is Systems Low - Minor Probability Netflow configuration on Cisco Prime.
used to collect Once the configuration has been
network traffic / completed, this risk can be closed since
NetFlow Prime can provide this capability.
information. If the
Risk is not mitigated
or another solution
identified, then
troubleshooting
traffic flow will be
challenging.
Verizon MTIPS Reliability Of 4 2 8 Moderate Reduce No impact thus far. There are multiple
Circuit reduction; Systems Probability options that can be accomplished if
This has the the Circuit Reduction impacts PBGC
potential of affecting business functions. PBGC plans to
all users of PBGC acquire larger circuits during its
internet based transition to EIS as well as acquiring
services including Express Route connectivity to Office
website access such 365.
as PBGC, myPAA,
myPBA, OIG, as well
as teleworking.
PBGC’s installed Reliability Of 2 3 6 Moderate Reduce Have already moved incident and
version of Service Systems Probability change functions to Service Now.
Manager is 9.35. This Working on moving request
version is end-of- management to ServiceNow and
support-life SailPoint. Confirmed vendor support
beginning for the current version that does not
November, 2017. As include bug fixes. Can upgrade to 9.5x
this software is the if a major issue surfaces.
ticketing system
behind the ITIOD
GetIT Access Service
Catalog, an inability
to obtain full Tier 4
support in the event
of a serious outage
could result in longer
service restoration
times and an
extended period
when PBGC
employees were
139
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
unable to request,
approve, and
provision IT Support
services and IT
system access.
Checkpoint logs are Security 5 2 10 Moderate Accept
not being ingested
into Splunk and the
logs are only being
retained locally in
the management
console for 120 days.
Replication of data Data/Info 2 3 6 Moderate Accept Work with the vendor to identify ways
for the SAN of increasing replication throughput or
Migration Project is recover data from backup.
running
slow. Estimated
replication
completion is
approximately 4
weeks. Getting data
on old LUNS will be
quite cumbersome, if
not impossible.
In the absence of a Reliability Of 2 3 6 Moderate Accept A dedicated circuit will be established
deployed solution, Systems to the new ITIOSS TO1 winner.
insufficient data
transport
information and
modeling is available
to guarantee PBGC’s
new Telephony
solution over PBGC’s
existing VPN
infrastructure will be
sufficient to support
continued ITIOD
service desk
operations.

The following table summarizes risks opened and closed in the 2018 calendar year:

Risk Action Count


Opened New Risk 2
Closed Existing Risk 6

6.5.7.2 Requirements
The contractor shall provide the IT infrastructure risk management services outlined in the following
table:

Reference Requirement
(SM-RM-xx)
SM-RM-01 Contractor shall identify a lead for IT Risk Management support. This lead is required
to serve as the primary point of contact for all IT Risk Management support.
140
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
SM-RM-02 The Contractor shall proactively identify, prioritize, and assess the likelihood and
potential impact of IT Risks, and develop and recommend options for mitigating those
risks
SM-RM-03 The Contractor shall ensure that all ITIOD Risks are tracked and managed in a
consistent manner in accordance with PBGC’s Risk Management Policies and
Procedures to include, but not limited to documenting probability, potential impact,
mitigation strategies, and escalation depending upon the rating of the risk
SM-RM-04 The Contractor shall review all open risks at least monthly and report progress until
the risk is resolved

6.5.8 Change Management support


6.5.8.1 Scope of Services Supported
Provide support for PBGC’s IT change management process including assisting with preparing for and
facilitating the weekly Change Advisory Board. PBGC utilizes ServiceNow ITSM module to track and
manage IT changes. Typical Change Management support activities include:

• Serve as CAB secretary outlined in the Change Advisory Board Charter including, but not limited
to:
o Preparing and distributes CAB meeting documentation including agenda and meeting
minutes
o Tracking and following-up on action items from the CAB or change controls needing
required information
• Reconcile proposed changes with the master release schedule and the infrastructure calendar
• Establish and maintain knowledge base articles (KBAs) for the IT Change Management process

See Appendix J - IT Service and Support 2018 Statistical Summary for requests for change (RFCs), and
RFC tasks processed in calendar year 2018 for these services.

6.5.8.2 Requirements
The contractor shall provide the IT infrastructure change management services outlined in the following
table:

Reference Requirement
(SM-CM-xx)
SM-CM-01 Contractor shall identify a lead for the IT Change Management support. This lead is
required to serve as the primary point of contact for all IT Change Management
support related issues
SM-CM-02 Contractor shall comply with and support the PBGC Change Management process
that is based on the ITIL 3.0 framework and associated best practices.

141
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
SM-CM-03 Contractor shall perform the duties of the CAB secretary outlined in the Change
Advisory Board Charter including, but not limited to:
• Prepares and distributes CAB meeting documentation
• Documents, posts and distributes the board meeting minutes to all board
members and attendees
• Prepares CAB agenda based on the PBGC CAB request approved items within
ITSM tool and CAB Chair direction
• Ensures required room is reserved and necessary equipment prepared for
CAB meetings
• Prepares minutes that include discussions and decisions from the CAB,
obtains federal approval of same, and posts to appropriate on-line site
• Distributes CAB action items to the respective Federal RFC Requestors
• Follows-up on each action item and briefs the CAB Chair on their status
SM-CM-04 The Contractor shall conduct reviews of all proposed changes against the Master
Release Schedule and the Infrastructure calendar prior and advise the business
change manager of any conflicts
SM-CM-05 Contractor shall create and maintain knowledge base articles (KBAs) on PBGC’s IT
change management process to answer frequently asked questions and train IT
Service Desk staff to ensure they are prepared to answer basic questions regarding
PBGC’s IT change management process

6.5.9 Asset Management and Inventory


6.5.9.1 Scope of Services Supported
Provide support for PBGC’s IT Assessment Management process, a collection of activities focused on
integrating the physical, technological, contractual, and financial aspects of IT hardware and software
assets for controlling inventory that is purchased and used and managing the life cycle of each asset
from planning to disposal. Typical Asset Management activities include:

• Asset intake including the following:


o Entering high-level contract (purchase order) information
o Decomposing the bill of material from the new contract and creating associated assets
in the ITSM tool
o In collaboration with subject matter expert, receiving and inspecting IT equipment
and/or software, assigning/attaching bar code, and associating with asset record in ITSM
tool or updating inventory of consumables (parts and supplies) if applicable
• Asset deployment which may include preparing assets for use, shipping assets, and updating
location and ownership information in the ITSM tool
• Asset lifecycle management including tracking assets through changes in location and ownership
including collecting assets upon user separation
• Asset retirement and disposal including degaussing if applicable
• Managing consumables including provisioning and requesting

142
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Monitor deployed assets (CIs) to ensure compliance with PBGC policy and notify on/report to
the government on non-compliance, e.g. mobile phones that haven’t checked in to MDM in 90
days, workstations that haven’t been on the network for 30 days, etc.
• Develop and generate reports on a regular and ad-hoc basis about hardware assets and
software licenses and consumables in the environment or in inventory
• Conduct licensing audits by reviewing software deployments relative to software licenses
• Conduct comprehensive annual inventory and reconciliation

See Appendix J - IT Service and Support 2018 Statistical Summary for requests for change (RFCs), and
RFC tasks processed in calendar year 2018 for these services. See Appendix B – Deliverables and
Appendix C - List of Required Meetings and Reports for more information on required deliverable and
reports that are generated as part of these services. See Appendix K - IT Infrastructure Tools List for a
comprehensive list of the software utilized to provides these services.

PBGC’s Asset Management administration and support is currently performed utilizing ServiceNow’s
asset management module. Inventory tasks are currently automated and mobilized utilizing CG4 bar
code scanners. PBGC has more than 7,000 tracked assets.

PBGC will maintain purchase responsibilities for assets and other related property such as parts and
supplies. PBGC will perform this function in procurement systems which are outside the scope of this
contract. These purchase responsibilities include the payment of invoices and necessary budgeting
activities. PBGC will also control the overarching property and asset management policies. The
contractor shall maintain and update documentation on asset and property management lifecycle
procedures and documents related to the asset system, administrative and operational procedures.

6.5.9.2 Requirements
The Contractor shall maintain accurate records and accountability for all Government Furnished
Property (GFP) including hardware and software assets, related licenses, warranty and maintenance
contracts, and parts and supplies, throughout the item life cycle. The contractor shall provide the IT
asset management services outlined in the following table:

Reference Requirement
(SM-AM-xx)
SM-AM-01 Contractor shall identify a lead for IT Asset Management support. This lead is required
to serve as the primary point of contact for all IT Asset Management support related
issues and activities
SM-AM-02 The contractor shall create each order of asset(s) and property within the asset
management system in preparation for the delivery and receipt of PBGC’s equipment,
parts and supplies, or software purchased
SM-AM-03 Contractor shall prepare, track, and document all IT Assets in accordance with ITIL
best practices and PBGC policy and procedures including preparing assets for use,
shipping assets, and updating location and ownership information in the ITSM tool

143
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
SM-AM-04 Contractor shall record any movement and/or updates to asset location or
assignments in PBGC’s IT Asset Management System within 3 business days of any
change
SM-AM-05 With the approval of COR, Contractor shall coordinate the disposal of Government
property that is no longer needed in accordance with the requirements of PBGC
Directives, NIST Standards, and the General Services Administration (GSA) including
degaussing if applicable
SM-AM-06 The Contractor shall perform a “wall to wall” annual physical inventory. The
Contractor shall conduct random and periodic inventories to verify accuracy and
accountability of asset management procedures and recording in the asset system.
SM-AM-07 The Contractor shall maintain hardware warranty maintenance records and
implement the provisions of the agreements when Government property fails to
perform in accordance with the specifications of the manufacturer
SM-AM-08 The Contractor shall be responsible for tracking “end-of-life” of PBGC hardware and
software assets and provide the COR and other designated PBGC federal staff with a
recommended schedule for the replacement or upgrade of those assets
SM-AM-09 The Contractor shall conduct licensing audits by reviewing software deployments
relative to software licenses at least annually for all software recorded in the ITSM
tool
SM-AM-10 The Contractor shall monitor deployed assets (CIs) to ensure compliance with PBGC
policy and notify on/report to the government on non-compliance
SM-AM-11 The Contractor shall monitor consumables (parts and supplies) and request
replenishment within one business day of reaching the minimum inventory threshold
of any consumable item. The contractor shall send daily reminders to the federal
asset manager, the COR, and the ITA&TRM division manager beginning one week
after reaching the minimum inventory threshold of any consumable item.
SM-AM-12 Contractor shall establish and maintain a periodic task list to maintain the IT Asset
Management system and associated data

6.5.10 Configuration Management


6.5.10.1 Scope of Services Supported
Provide support for PBGC’s IT Configuration Management process, a collection of activities focused on
establishing and maintaining the integrity of products and systems, through control of the processes for
initializing, changing, and monitoring the configurations of those products and systems. Typical
Configuration Management support activities include:

• Add new Configuration Items (CIs) that can’t be automatically discovered and added to the
Configuration Management Database (CMDB)
• Configure and maintain the CI model in the CMDB
• Configure rules for mapping discovered CIs to the appropriate CI classification and populating
required attributes

144
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Synchronize and reconcile PBGC’s application list/CIs across disparate systems, e.g. DOJ Cyber
Security Assessment and Management (CSAM); used for storing information about PBGC’s
FISMA systems, ServiceNow CMDB; used to discover and manage PBGC’s Cis, and mAppIT;
PBGC’s centralized application list
• Develop and generate reports on a regular and ad-hoc basis about hardware, software, and CIs
in the environment
• Define relationships between CIs
• Monitor for and notify upon detection of unauthorized hardware and software installation
• Establish and maintain knowledge base articles (KBAs) for the IT Configuration Management
process

See Appendix J - IT Service and Support 2018 Statistical Summary for requests for change (RFCs), and
RFC tasks processed in calendar year 2018 for these services. See Appendix B – Deliverables and
Appendix C - List of Required Meetings and Reports for more information on required deliverable and
reports that are generated as part of these services. See Appendix K - IT Infrastructure Tools List for a
comprehensive list of the software utilized to provides these services.

PBGC’s Configuration Management tools and associated services administration and support
environment consists primarily of the following:

Configuration Management Tools

• PBGC currently utilizes the configuration management and discovery modules in ServiceNow
• PBGC’s Configuration Management program integrates the ServiceNow CMDB data with PBGC’s
ServiceNow asset management, it’s centralized application list, mAppIT, and its instance of
CSAM.

Configuration Management Model/Scope

The scope of managed infrastructure CIs, known as CI Types, that are tracked by the PBGC infrastructure
CM process are currently (as of March 2019) as follows:

• FISMA System
• PBGC Developed Application
• COTS Application
• Workstation – Windows
• Laptop - Windows
• Mobile phone
• Database – MS SQL Server
• Database - Oracle
• Server - Windows
• Server – RHEL

145
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Server – Solaris
• Server – ESXi
• Storage Array
• Backup Device
• Network Attached Storage
• Printer – Multi-Function Device (MFD)
• Printer – non-MFD (networked)
• Video Conferencing
• Video Streaming
• Telephony Device
• Router
• Network Switch
• Network Load Balancer
• Firewall
• Virtual Private Network (VPN) Concentrator
• Storage Switch
• Network Tap
• Remote Access Device
• Security Device
• Network Management Device
• Environmental Monitoring Device
• Blade Device
• Uninterruptible Power Supply (UPS)
• Virtual Local Area Network (VLAN) / Internet Protocol (IP) Subnet
• IP Address

The degree of CM process that PBGC applies to managing a CI is based on the following factors:

• Criticality based on risk as a function of impact and likelihood


• Technical cost
• Feasibility

EnableIT Service Requests

The following table provides the count for EnableIT service requests processed in Calendar Year 2018

Service Request Category Total


Requests
CMDB (CI Request) 5

146
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.5.10.2Requirements
The contractor shall provide the IT configuration management services outlined in the following table:

Reference Requirement
(SM-CF-xx)
SM-CF-01 Contractor shall identify a lead for IT Configuration Management support. This lead is
required to serve as the primary point of contact for all IT Configuration Management
support related issues
SM-CF-02 The Contractor shall provide and be responsible for PBGC Configuration Management
processes that is based on the ITIL 3.0 framework and associated best practices.
Contractor shall be responsible for managing PBGC configuration management plans,
processes, and procedures support, including identification, control, status
accounting, and reviews and audits according to PBGC policy.
SM-CF-03 Contractor shall maintain the CMDB including ensuring relationships between CIs are
defined such that, at a minimum, the following reports can be produced at any point
in time:
• A report of all CIs (dedicated or shared) associated with a FISMA system
• A report of all CIs (dedicated or shared) associated with an application
• A report of impacted applications can be generated in the event of a failure
of any one CI
• A report of all CIs of a specific (or any) CI type with all associated attributes
according to PBGC’s CI model
SM-CF-04 Contractor shall fulfill all approved service requests for IT infrastructure resource and
services (in EnableIT) in accordance with PBGC procedures and timelines for the
following:

• New CMDB CI

See SLA section for details on associated SLA measures.


SM-CF-05 Contractor shall ensure application CIs are synchronized between mAppIT, the
ServiceNow CMDB, and CSAM monthly and will notify the government of any
discrepancies and work the government to reconcile
SM-CF-06 Contractor shall monitor for and notify upon detection of unauthorized hardware and
software installation daily
SM-CF-07 Contractor shall create and maintain knowledge base articles (KBAs) on PBGC’s IT
configuration management process to answer frequently asked questions and train IT
Service Desk staff to ensure they are prepared to answer basic questions regarding
PBGC’s IT configuration management process
SM-CF-08 Contractor shall maintain passwords for IT Configuration Management privileged
accounts, e.g. service accounts, local system emergency recovery accounts, etc. in
privileged account management tool and utilize this tool to perform administrative
functions via brokered session or account check out
SM-CF-09 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the IT Configuration Management infrastructure
147
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.6 IT Security Services
The sections below describe the scope and requirements of the IT Security services that the Contractor
shall provide under this solicitation. The IT Security services provided will enable PBGC to defend
against cyber threats and ensure that PBGC is able to execute mission objectives and conduct daily
business and as such, are critical to excellent performance under this contract. Thus, IT Security services
is a key service area. Much of how the customer views the success of this contract will be dependent on
how well the Contractor executes the IT Security services under this contract and how satisfied ITIOD
staff members are with the IT services provided. In an increasingly interconnected world, cyber security
threats are outpacing the limited resources of reactively focused infrastructure management
operations. IT Security services are activities that are specifically focused on leveraging technology to
predict risks and preempt threats as well as react to security incidents upon detection. IT Security
services include the following:

• IT Security Tools and Cybersecurity Incident Response


• IT Vulnerability Scanning and Reactive Vulnerability Management coordination
• IT Security Controls Support

6.6.1 IT Security Tools Support and Cybersecurity Incident Response


6.6.1.1 Scope of Services Supported
Provide tier 2 support for security events and incidents as well as support IT security service requests.
Ensure all PBGC networks and systems are properly protected from cyber threats through the
deployment of defense-in-depth technologies and operations. Typical IT Security Tools Support and
Cybersecurity Incident Response administration and support activities include:

• Conduct analysis on suspected phishing attempts and coordinate removal of confirmed phishing
attempts including blocking sender (or sender domain) and any associated URLs
• Conduct analysis on blocked and potentially miscategorized websites to determine if
whitelisting is appropriate and coordinate whitelisting if approved
• Inspect email quarantined as part of DLP or anti-malware measures and release if false-positive
• Review security information and event management (SIEM) events and alerts including potential
data exfiltration, excessive failed logins, potential web intrusion and take additional actions as
necessary
• Define security events and correlations based upon PBGC risk and architectural posture (known
risks, vulnerabilities, deficiencies) and configure SIEM to detect and report and/or alert on these
risks
• Coordinate ingestion of logs and configuration of notifications and/or reports in SIEM for
application events of interest to application stakeholders

148
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Blacklist URLs, domains, email addresses, etc. based on threat intelligence feeds
• Malware analysis, impact assessment, and removal
• Update/refine rule sets of various security tools to minimize false positives
• Monitor endpoint management software, e.g. Symantec End Point Protection, BigFix to ensure it
is current and activated and repair/reinstall as required and ensures all application devices are
having their security logs ingested into the SIEM
• Respond to approved data requests about suspicious activities and potential privacy breaches

Support for IT Security Tools Support and Cybersecurity Incident Response in the PBGC data centers is
typically handled using remote management software and protocols, e.g. https, SSH, Powershell, ILO,
OA, etc., but does occasionally require physical visits to address certain hardware and software
problems. See Appendix F - PBGC Locations for PBGC locations, including planned changes to PBGC’s
facilities and data centers over the life of the contract. See Appendix G - IT Service Support Guidelines
for impact, urgency, and prioritization guidelines associated with IT service and support. See Appendix J
- IT Service and Support 2018 Statistical Summary for tier 2 incidents, requests for information (RFIs),
service/access requests, requests for change (RFCs), and RFC tasks processed in calendar year 2018 for
these services. See Appendix K - IT Infrastructure Tools List for the software utilized to provides these
services.

The tables that following provide additional insight into the components of PBGC’s IT Security Tools
Support and Cybersecurity Incident Response infrastructure as well as the volume of activity expected:

Cybersecurity Infrastructure Equipment Summary

IT infrastructure equipment that supports Cybersecurity will be made available in PBGC’s reading room.

Critical Security Events from SIEM:

The following table details the critical security events tracked in PBGC’s SIEM in Calendar Year 2018:

Month Total Actionable False Trend Summary


Events Events Positives
Processed
Jan-18 405 16 389 Majority of false positives originated from
Symantec alerts.
Feb-18 303 14 289 False positives from VMWare traffic as detected
by FirePower IDPS.
Mar-18 86 12 74 False positives originated from unroutable traffic
from database servers.
Apr-18 34 4 30 False positives from VMWare traffic as detected
by FirePower IDPS.
May-18 53 19 34 False positives originated from unroutable traffic
from database servers.

149
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Jun-18 87 14 73 False positives orginated from Office 365
activities.
Jul-18 44 9 35 Majority of false positives originated from
Symantec alerts.
Aug-18 54 20 34 False positives originated from NetBackup
devices.
Sep-18 57 24 33 False positives originated from NetBackup
devices.
Oct-18 40 18 22 Cryptomining alerts and phishing attacks.
Nov-18 128 77 51 JSP Webshell Backdoor attempts were dropped by
FirePower IDPS.
Dec-18 446 414 32 JSP Webshell Backdoor attempts were dropped by
FirePower IDPS.
Total 1737 641 1096

There were two (2) security events that were ultimately classified as security incidents and reported to
US CERT in calendar year 2018.

SecureIT Service Requests

The following table provides the count for SecureIT service requests processed in Calendar Year 2018:

Service Request Category Total Requests


Block Email Sender 99
SOE ACL/Firewall Change 77
URL to Blacklist 23
URL to Whitelist 31
Total Count 230

6.6.1.2 Requirements
The contractor shall provide the IT Security Tools and Incident Response support services outlined in the
following table:

Reference Requirement
(CS-IR-xx)
CS-IR-01 Contractor shall identify a lead for IT Security Tools Support and Cybersecurity
Incident Response. This lead is required to serve as the primary point of contact for all
IT Security Tools Support and Cybersecurity Incident Response related issues and
coordinate all ITIOD cybersecurity investigations.

150
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Reference Requirement
(CS-IR-xx)
CS-IR-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
enterprise IT Security Infrastructure in all PBGC computing environments that
enhances capacity to respond to business changes and protects PBGC’s network and
data. This shall include, but is not limited to:
• Identifying and addressing performance bottlenecks
• Performing capacity planning and management including allocation and
reconfiguration of IT Security infrastructure and associated services as
needed
• Using monitoring tools to proactively plan and manage IT Security
infrastructure resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
CS-IR-03 Contractor shall fully implement, manage, and support all incident management
activities regarding IT Security infrastructure according to PBGC policies. By using ITIL
best practices, this should include, but shall not be limited to:
• Provide support for IT Security infrastructure hardware and software
• Investigate, troubleshoot, and resolve operational issues due to cyberattacks
such as denial-of-service attacks, DNS poisoning, exfiltration attempts,
unauthorized access, and phishing emails
• Assess, troubleshoot and resolve security issues including coordinating with
vendors to ensure PBGC systems and data are restored to normal operations
• Communicate to end users upon receipt of the security incident ticket and
the resolution of the security investigation
• Brief Federal management of any security tickets that require more than 7
business days to resolve
• Collaborate with other ITIOD platform teams to help restore system
availability due to outages from IT security issues.

See SLA section for details on associated SLA measures.


CS-IR-04 Contractor shall fully implement, manage, and support all change management
activities regarding IT Security infrastructure and the associated services they provide
according to PBGC policies. By using ITIL best practices, this should include, but shall
not be limited to:
• Provide installation and configuration of IT Security infrastructure hardware
and associated operating systems
• Apply applicable security patches at least monthly and install applicable
hardware firmware updates at least quarterly
• Address operating system and software vulnerabilities detected on IT Security
infrastructure during monthly vulnerability scans

151
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Reference Requirement
(CS-IR-xx)
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices
• Submit and process approved SecureIT SharePoint requests for PBGC’s web
proxy blacklists and whitelists
• Update/refine rule sets of various security tools to minimize false positives
• Configure ingestion of security logs and define security events and
correlations based upon PBGC risk and architectural posture (known risks,
vulnerabilities, deficiencies) and configure SIEM to detect and report and/or
alert on these risks
• Develop and implement BigFix fixlets to meet defined requirements
• Create new or refine existing DLP policies to meet defined requirements

See SLA section for details on associated SLA measures.


CS-IR-05 Contractor shall leverage technology and threat intelligence feeds to predict risks and
preempt threats to include review of security information and event management
(SIEM) events and alerts regarding potential data exfiltration, excessive failed logins,
potential web intrusion, etc. and take additional actions as necessary. Contractor
shall perform all phases of security event and incident management including:
• Prepare: The process of providing the necessary tools, knowledge, and
understanding in order to adequately respond to a security incident
• Detect: The process of identifying and prioritizing suspicious events
• Analyze: The process of identifying whether a security event meets the
threshold of a security incident and examining the details of an incident
selected based upon criticality; also includes determining the root cause of an
incident
• Respond: The process of containing the immediate impact of a security
incident
• Recover: The process of returning the affected system(s) to the last known
good state
• Remediate: The process of eliminating the vulnerability from the environment
and resolving the incident
• Follow-up: The process of completing the security incident documentation
and providing communications and feedback to all parties regarding the
incident and lessons learned. Any adjustments to policy and process will be
recommended during this phase and implemented during preparation.
For each security incident, the Contractor shall produce an “SIM After Action Report”
within 14 days following the conclusion of a security incident reported to US-CERT.
See Appendix B – Deliverables for more information on this deliverable.
CS-IR-06 The Contractor shall review security intelligence from government or PBGC
subscribed third-party sources including weekly US-CERT briefings to determine if any

152
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Reference Requirement
(CS-IR-xx)
security threats are applicable to PBGC and formulate a defensive strategy to protect
the agency’s data and assets
CS-IR-07 Contractor shall assist Federal staff to respond to any security-related data call from
external government entities including but not limited to DHS, OMB, and Congress
CS-IR-08 Contractor shall assist Federal staff in investigations of potential insider threat or
fraud including but not limited to timesheet fraud and exfiltration of sensitive PBGC
data
CS-IR-09 Contractor shall provide threat and vulnerability management support, e.g., virus
protection, firewalls, IDS/IPS management, DLP, and the coordination of a wide
variety of information regarding threats and vulnerabilities of all PBGC platforms and
services
CS-IR-10 Contractor shall ensure all local accounts supporting the IT Security infrastructure,
e.g. the built-in admin account, are changed periodically in accordance with PBGC
policy and procedures and the passwords are stored for emergency use
CS-IR-11 Contractor shall participate in periodic exercises to demonstrate and assess readiness
for real security incidents. Contractor shall participate in periodic event and incident
handling quality reviews coordinated by PBGC. Contractor shall take appropriate
steps to mature process based on lessons learned and feedback provided.
CS-IR-12 Contractor shall scan non-PBGC media for malware upon request
CS-IR-13 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain PBGC security tools and appliances.

6.6.2 IT Vulnerability Scanning and Reactive Vulnerability Management coordination


6.6.2.1 Scope of Services Supported
Perform periodic and ad-hoc vulnerability scanning of PBGC’s IT infrastructure and business systems
utilizing PBGC’s vulnerability scanning tools. Coordinate activities including patching, configuration
changes, etc. to remove vulnerabilities in accordance with PBGC policies and procedures. Typical IT
Vulnerability Scanning and Reactive Vulnerability Management coordination activities include:

• Perform monthly, authenticated (when technically feasible) vulnerability scanning of all devices
on PBGC’s network
• Prepare for, attend and actively participate in regular meetings of the Patch and Vulnerability
Management Group (PVMG), the group which tracks vulnerabilities and coordinates their
remediation
• Perform web application, authenticated (when technically feasible) vulnerability scanning of
targeted PBGC systems and produce scanning results which include point-in-time web
application vulnerability details and summary information.

153
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Assess vulnerabilities for scope, research steps required to address, and coordinate needed
actions to address with appropriate Contractor resources. Collaborate and coordinate with
System Owners, Business Owners, and Application Developers as needed.
• Optimize the vulnerability scanning tools to improve the efficiency of the scanning process

Support for IT Vulnerability Scanning and Reactive Vulnerability Management coordination activities in
the PBGC data centers is typically handled using remote management software and protocols, e.g. https,
SSH, Powershell, ILO, OA, etc., but does occasionally require physical visits to address certain hardware
and software problems. See Appendix F - PBGC Locations for PBGC locations, including planned changes
to PBGC’s facilities and data centers over the life of the contract. See Appendix G - IT Service Support
Guidelines for impact, urgency, and prioritization guidelines associated with IT service and support. See
Appendix J - IT Service and Support 2018 Statistical Summary for tier 2 incidents, requests for
information (RFIs), service/access requests, requests for change (RFCs), and RFC tasks processed in
calendar year 2018 for these services. See Appendix K - IT Infrastructure Tools List for the software
utilized to provides these services.

154
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
The following image depicts a high-level summary vulnerabilities detected during a credentialed
vulnerability scan of the PBGC network in December 2018:

6.6.2.2 Requirements
The contractor shall provide the security vulnerability scanning and reactive vulnerability management
services outlined in the following table:

Reference Requirement
(CS-VM-xx)
CS-VM-01 Contractor shall identify a lead for IT Vulnerability Scanning and Reactive Vulnerability
Management. This lead is required to serve as the primary point of contact for all IT
Vulnerability Scanning and Reactive Vulnerability Management related issues.
CS-VM-02 Contractor shall provide and maintain a fully functional, optimally performing IT
Vulnerability Scanning infrastructure covering all PBGC computing environments that
enhances PBGC’s understanding of its cybersecurity risk exposure. Contractor shall
provide reactive vulnerability management services that reduces the cybersecurity
attack surface of PBGC’s network and data as well as its cloud hosted systems. This
shall include, but is not limited to:
• Identifying and addressing performance bottlenecks with regard to
vulnerability scanning
• Performing capacity planning and management including allocation and
reconfiguration of IT Security infrastructure and associated services as
needed
• Using monitoring tools to proactively plan and manage IT Security
infrastructure resources to maximize system and service availability
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
CS-VM-03 Contractor shall perform monthly, authenticated (when technically feasible)
vulnerability scanning of all devices on PBGC’s network and produce scanning results
including point-in-time vulnerability details and summary information including the
155
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
“PBGC Security Posture Summary” report. See Appendix B – Deliverables for more
information on this report. Contractor shall troubleshoot and resolve authentication
issues on devices that support authenticated vulnerability scans.
CS-VM-04 Contractor shall fully implement, manage, and support all change management
activities regarding IT Vulnerability Scanning and Reactive Vulnerability Management
services they provide according to PBGC policies. By using ITIL best practices, this
should include, but shall not be limited to:
• Apply applicable security patches and configuration changes to address
vulnerabilities detected during vulnerability scanning
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices

See SLA section for details on associated SLA measures.


CS-VM-05 Contractor shall perform authenticated (when technically feasible) vulnerability
scanning of new IT infrastructure deployed or updated as part of each major
application or infrastructure release and provide report to project team
CS-VM-06 Contractor shall track all vulnerabilities discovered on the network along with
associated remediation efforts, identified false-positives, vulnerabilities requiring risk
acceptance, and vulnerabilities for which remediation is expected to take longer than
the associated PBGC target. Contractor shall prepare for, attend and actively
participate in regular meetings of the Patch and Vulnerability Management Group
(PVMG), the group which tracks vulnerabilities and coordinates their remediation.
Contractor shall present IT risk posture, risks addressed or mitigated, and issues
needing executive attention as it relates to vulnerabilities within the environment and
ongoing vulnerability remediation activities.
CS-VM-07 Contractor shall ensure all local accounts supporting the IT Vulnerability Scanning and
Reactive Vulnerability Management services, e.g. scanning accounts, are changed
periodically in accordance with PBGC policy and procedures and the passwords are
stored for emergency use
CS-VM-08 Contractor shall assess vulnerabilities for scope, research steps required to address,
and coordinate needed actions to address with appropriate Contractor resources.
Contractor shall collaborate and coordinate with Application/Service owners and
Application Developers as needed.
CS-VM-09 Contractor shall perform ad-hoc application and database, authenticated (when
technically feasible) vulnerability scanning of targeted PBGC systems and produce
scanning results which include point-in-time application and database vulnerability
details and summary information.
CS-VM-10 Contractor shall perform ad-hoc authenticated (when technically feasible)
vulnerability scanning of targeted PBGC cloud-hosted systems and produce scanning
results which include point-in-time vulnerability details and summary information.
CS-VM-11 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain PBGC IT Vulnerability Scanning infrastructure

156
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.6.3 IT Security Controls Support
6.6.3.1 Scope of Services Supported
Provide IT Security Controls Support by providing direct support to the IT Infrastructure Services General
Support System (ITISGSS) Information System Security Officer (ISSO) in managing and documenting the
ongoing security posture of the ITISGSS and other ITIOD managed FISMA systems including applicable
NIST 800-53 security controls. Typical IT Security Controls Support activities include:

• Collect information from subject matter experts within ITIOD and develop control
implementation statements
• Update ITIOD FISMA system security documentation (SSP and other) with approved new,
significant changes requiring updates
• Support control assessments
• Provide expertise and assistance in the development of the security policies and procedures and
assist ensuring compliance with those policies and procedures
• Provide assistance in developing and updating the security artifacts, managing and controlling
changes to the ITIOD FISMA systems and assessing the security impact of those changes
• Assist in maintaining POA&Ms and the remediation of identified weaknesses
• Assist in coordination with other FISMA systems to ensure continual compliance with conditions
of inheritance from the GSS

During calendar year 2018, more than 113 IT security controls were assessed of which 97 were assessed
as satisfied/fully implemented and 9 were assessed as partially implemented. In support of this process,
more than 300 artifacts were provided to the security control assessors and several hundred questions
were answered. The ITISGSS has 288 NIST 800-53 rev. 4 applicable security controls of which 215 have
been assessed as satisfied/fully implemented and of which 25 have been assessed as partially
implemented. Approximately 165 of the ITISGSS security controls are offered to and inherited by other
PBGC FSIMA systems. It is worth noting that, in addition to the ITISGSS, ITIOD manages a FISMA system
for ServiceNow and a FISMA child of the ITISGSS for Office 365 cloud services and is working on
establishment of a second FISMA child of the ITISGSS for Microsoft Azure cloud services.

6.6.3.2 Requirements
The contractor shall provide support for the development and ongoing management and maintenance
of the security controls for the ITIOD managed FISMA systems including the IT Infrastructure Services
General Support System (ITISGSS) as outlined in the following table:

Reference Requirement
(CS-SC-xx)
CS-SC-01 Contractor shall identify a lead for IT Security Controls Support. This lead is required
to serve as the primary point of contact for all IT Security Controls Support activities.
CS-SC-02 Contractor shall develop and maintain IT security controls related to and offered by
the ITISGSS to the standards set forth in the NIST Special Publication 800-53 as
157
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
described in PBGC Security Policy. Contractor will consult with ITIOD subject matter
experts and review approved ITIOD work instructions in development of IT security
controls to ensure they accurately reflect the ITISGSS control implementation.
Contractor shall document and communicate any control deficiencies identified
during control development for POA&M consideration.
CS-SC-03 Contractor shall support PBGC IT Governance, Risk and Compliance Activities
(e.g., management of standards, approvals, waivers)
CS-SC-04 Contractor shall provide Continuous Security Monitoring. The Contractor shall
monitor the ITISGSS including all IT infrastructure and functional areas identified in
performance work statement in accordance with agency- defined parameters, for
compliance with PBGC Security Policy (SP) and all System Security Plans (SSPs) for the
ITISGSS
CS-SC-05 Contractor shall conduct detailed security impact analysis for any change that
introduces new (type of) hardware or software, requires modification to a security
baseline, requires a new connection to an external entity, significantly changes a
publicly facing application or DMZ infrastructure. Contractor shall ensure any
appropriate recommendations or information is provided in writing to
service/application owners and change coordinators.
CS-SC-06 Contractor shall support periodic control assessments including supplying requested
artifacts and responding to inquiries; coordinating with ITIOD subject matter experts
as needed
CS-SC-07 Contractor shall update the ITISGSS system security documentation (SSP and other)
with approved new, significant changes requiring updates including updating
boundary description and technical description to reflect current environment and
include inheritance within 30 days of completed RFC
CS-SC-08 Contractor shall assist with FISMA reporting
CS-SC-09 Contractor shall review outputs from POA&Ms to assess completeness and make
recommendations for additional work needed or POA&M closure
CS-SC-10 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain IT Security Controls

6.7 Test Center Operations


The sections below describe the scope and requirements of the PBGC Test Center Operations services
that the Contractor shall provide under this solicitation. Providing tools, IT resources, and services
necessary to develop, test, and release applications and infrastructure solutions is critical to enable the
accomplishment of the agency mission and as such, are critical to excellent performance under this
contract. Thus, Test Center Operations is a key service area. Much of how the customer views the
success of this contract will be dependent on how well the Contractor administers and supports PBGC’s
Test Center Operations, and how satisfied ITIOD staff members are with the IT services provided.

158
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.7.1 Scope of Services Supported
Provide tier 2 support for incidents relating to Test Center Operations that cannot be resolved by an
End-User services team as well as support service requests. Typical Test Center Operations
administration and support activities include:

• Manage controlled development code repositories including code promotion


• Manage the Test (ITC) Reservation Queue and ITC lab schedules
• Coordinate semi-annual database refreshes for the Development and Test environments
• Provide support to project release teams for application release tools including:
o Requirements management, test planning and functional testing, and defect
management
o Functional testing automation
o Load testing
o Process management and workflow automation
o Code repository and version control system
• Establish and maintain work instructions
• Escalate to and work collaboratively with 3rd party vendors on hardware and software issues

Currently, the Test (ITC) lab is a physical facility located at 1275 K Street, but over the next 12-18 months
this is expected to become a virtual test center. See Appendix G - IT Service Support Guidelines for
impact, urgency, and prioritization guidelines associated with IT service and support. See Appendix J -
IT Service and Support 2018 Statistical Summary for IT Service Desk interactions, tier 2 incidents,
requests for information (RFIs), service/access requests, requests for change (RFCs), and RFC tasks
processed in calendar year 2017 for these services. See Appendix K - IT Infrastructure Tools List for the
software utilized to provides these services.

The tables that follow provide information about the size and transactional volume of PBGC’s Test
Center Operations:

The following table lists each Development, Testing, and Release Management Tool and the number of
associated projects and applications it supports:

Development, Testing, and Release Management Tool # of PBGC Projects # of Applications


Services
HP Application Life Cycle Management (HP ALM, HP Sprinter) / HP QC 12.x 189 12
HP Unified Functional Testing (HP UFT) 12.x 189 6
HP LoadRunner 12.x 189 1
Serena Business Manager (SBM) 10.x 27 21
Serena Version Manager (PVCS VM) 8.x 160 56

The following table lists each Development, Testing, and Release Management Tool and the number of
associated releases it supported in calendar year 2018:

159
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Development, Testing, and Release Management Tool # of Releases in 2018
HP Application Life Cycle Management (HP ALM, HP Sprinter) / HP QC 12.x 1
HP Unified Functional Testing (HP UFT) 12.x 2
HP LoadRunner 12.x 1
Serena Business Manager (SBM) 11.x 4
Serena Version Manager (PVCS VM) 8.x 3

The following table lists each Development, Testing, and Release Management Tool and the estimated
number of associated incidents associated with it in calendar year 2018:

Development, Testing, and Release Management Tool # of PBGC Incident


Tickets
HP Application Life Cycle Management (HP ALM, HP Sprinter) / HP QC 12.x 120
HP Unified Functional Testing (HP UFT) 12.x 20
HP LoadRunner 12.x 4
Serena Business Manager (SBM) 10.x 27
Serena Version Manager (PVCS VM) 8.x 61

The following table lists each Development, Testing, and Release Management Tool and the number of
associated service requests were fulfilled for it in calendar year 2018:

Development, Testing, and Release Management Tool # of Service


Requests
HP Application Life Cycle Management (HP ALM, HP Sprinter) / HP QC 12.x 51
HP Unified Functional Testing (HP UFT) 12.x 29*
HP LoadRunner 12.x 6
Serena Business Manager (SBM) 10.x 9
Serena Version Manager (PVCS VM) 8.x 101
User/Employee Separation 209
*UFT requests are fulfilled automatically/programmatically.

6.7.2 Requirements
The contractor shall provide the Test Center Operations administration and support services outlined in
the following table:

Reference Requirement
(TC-TC-xx)
TC-TC-01 Contractor shall identify a lead for the Test Center Operations administration and
support area. This lead is required to serve as the primary point of contact for all Test
Center Operations administration and support related issues.

160
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
TC-TC-02 Contractor shall provide and maintain a fully functional, optimally performing, secure
Test Center Operations infrastructure that supports application/solution release
management and deployment in all PBGC computing environments as appropriate.
This shall include, but is not limited to:

• Identifying and addressing performance bottlenecks


• Performing capacity planning and management including allocation, resizing,
and reconfiguration of application/solution development, testing, and release
management tools as needed
• Monitoring application/solution development, testing, and release
management tools to ensure they are adequately licensed
• Performing preventative and remedial maintenance of components
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
TC-TC-03 Contractor shall fully implement, manage, and support all incident management
activities regarding Test Center Operations infrastructure that supports
application/solution release management and deployment according to PBGC
policies. By using ITIL best practices, this should include, but shall not be limited to:

• Troubleshoot and resolve issues with application/solution development,


testing, and release management tools as needed

See SLA section for details on associated SLA measures.


TC-TC-04 Contractor shall fully implement, manage, and support all change management
activities regarding Test Center Operations infrastructure that supports
application/solution release management and deployment according to PBGC
policies. By using ITIL best practices, this should include, but shall not be limited to:

• Provide installation and configuration of application/solution development,


testing, and release management tools as needed or decommission when no
longer required
• Apply applicable security patches at least quarterly
• Update controlled development code repositories including
• Promote code
• Coordinate semi-annual database refreshes for the Development and Test
environments
• Address operating system and software vulnerabilities associated with
application/solution development, testing, and release management tools as
needed during monthly vulnerability scans
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices

See SLA section for details on associated SLA measures.

161
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
TC-TC-05 Contractor shall provide support to project release teams for application release tools
including, but not limited to:
• Requirements management, test planning and functional testing, and defect
management
• Functional testing automation
• Load testing
• Process management and workflow automation
TC-TC-06 Contractor shall perform account administration functions in accordance with PBGC
policy and procedures for systems and functions where automation is not already in
place including, but not limited to:
• Non-AD-integrated accounts utilized by developers and testers for use of
application/solution development, testing, and release management tools
o Account unlocks and emergency/temporary disable/re-enable
o Password Resets (including remote user verification when applicable
per PBGC policy)
o Account removal upon request or separation
TC-TC-07 Contractor shall ensure all accounts supporting the Test Center Operations
environment including those utilized by/for application/solution development,
testing, and release management tools, e.g. local tools administrator and service
accounts are changed periodically in accordance with PBGC policy and procedures
and the passwords are stored for emergency use
TC-TC-08 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain the Test Center Operations infrastructure and associated services

162
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.8 Development, Modernization and Enhancements
The sections below describe the scope and requirements of the IT infrastructure Development,
Modernization, and Enhancement (DM&E) services that the Contractor shall provide under this
solicitation. Providing current, vendor-supported, optimally performing, highly available, and compliant
systems and service is critical to enable the accomplishment of the agency mission and as such, are
critical to excellent performance under this contract. Thus, Development, Modernization and
Enhancements is a key service area. Much of how the customer views the success of this contract will
be dependent on how well the Contractor performs DM&E, and how satisfied ITIOD staff members are
with the IT services provided. The Contractor is expected to provide DM&E Services for all IT
infrastructure technology noted in this PWS.

6.8.1 Scope of Services Supported


For the purposes of this Contract, the PBGC defines Development as an activity to design and implement
a new, significant product and/or capability or the upgrade existing IT infrastructure technology
(hardware or software or cloud-based services) requiring major system redesign and requiring moderate
to significant coordination with PBGC business areas or end-users. Example development activities
include the design and implementation of an enterprise Wi-Fi capability, migrating PBGC’s unstructured
content to Microsoft’s Office 365, upgrading Oracle’s 11g environment to Oracle 12c, supporting PBGC’s
transition to the Department of Homeland Security (DHS) Continuous Diagnostics and Mitigation (CDM)
solutions, modernizing PBGC’s PBX-based telephony infrastructure to a VoIP-based system, migrating
SQL services on-premise to Azure hosted SQL services, and upgrading PBGC’s Windows Server from
older versions to 2016. For the purposes of this Contract, the PBGC defines Modernization as an activity
to upgrade existing IT infrastructure technology (hardware or software or cloud-based services)
requiring minimal system redesign and requiring little or no coordination with PBGC business areas or
end-users. Example modernization activities include migrating file systems and storage allocations from
an old storage array to a new one, upgrading VMware vSphere 5.5. to 6.5 and upgrading Symantec
Endpoint Protection from version 12.x to 14.x. For the purposes of this Contract, the PBGC defines
Enhancement as an activity to modify existing or deliver new IT infrastructure technology (hardware or
software) or associated procedures to improve efficiency, improve quality of service delivery, or address
compliance issues with agency or government-wide policy. Example enhancement activities include
deploying Microsoft’s Advanced Threat Protection (ATP) to provide an isolated email detonation
chamber capability, deploying Cisco Prime to more efficiently manage Cisco network infrastructure,
configuring IronPort mail gateways to comply with BOD-18-01, and adjusting settings on IIS instances
enterprise-wide to improve compliance with PBGC’s associated security baseline. All planned
development, modernization and enhancement activities are recorded, authorized, and prioritized in
ITIOD’s IT Infrastructure Program Registry and for items of greater user or business impact, visualized for
the PBGC community in the ITIOD Roadmap. See Appendix L - IT Infrastructure Program Registry and
ITIOD Roadmap for exports and copies of this information.

PBGC maintains software assurance or subscriptions for software outlined in this PWS that will require
periodic modernization and will obtain and furnish new versions of COTS software to the contractor as
Government Furnished Equipment (GFE). PBGC will also provide Contractor with any IT infrastructure
163
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
hardware or cloud services necessary to perform required modernization. The contractor shall plan,
develop, test, and execute upgrades as required, in compliance with the PBGC IT infrastructure
environment policies and standards as described throughout this PWS.

6.8.2 Requirements
The contractor shall provide ongoing development, modernization and enhancements as outlined in the
following table:

Reference Requirement
(DME-ME-xx)
DME-ME-01 Contractor shall identify a lead for the Development, Modernization and
Enhancement (DM&E) services area. This lead is required to serve as the primary
point of contact for all DM&E related issues.
DME-ME-02 Contractor shall support and maintain a fully functional, optimally performing on-
premise infrastructure as part of DM&E service delivery. This shall include, but is not
limited to:

• Baselining performance and ensuring new systems perform at similar or


better levels
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
DME-ME-03 Contractor shall perform all tenant (customer) responsibilities, e.g. configuring;
administering; and monitoring, in support of maintaining fully functional, optimally
performing cloud-hosted infrastructure services as part of DM&E service delivery.
This shall include, but is not limited to:

• Baselining performance and ensuring new cloud-based services perform at


similar or better levels to the services they replace or at a PBGC approved
level of service in line with the CSP’s SLAs
• Coordinating performance of work by vendors as required and in accordance
with PBGC Security policies, vendor warranties and maintenance contracts
DME-ME-04 Contractor shall fully implement, manage, and support all incident management
activities regarding DM&E delivered solutions until transition to operational support
is complete according to PBGC policies. By using ITIL best practices, this should
include, but shall not be limited to:

• Monitor incidents and provide required support for a minimum of 30 days


following delivery of new IT infrastructure solutions or development,
modernization or enhancement of existing IT infrastructure solutions
Serve as point of escalation for problems regarding IT infrastructure covered
under this PWS
See SLA section for details on associated SLA measures.

164
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DME-ME-05 Contractor shall fully implement, manage, and support all change management
activities for DM&E services according to PBGC policies. By using ITIL best practices,
this should include, but shall not be limited to:

• Perform replacement and/or upgrades of IT hardware, operating systems,


software, and cloud services as required
• Modify existing or deliver new IT infrastructure technology (hardware,
software, or IT infrastructure cloud services) or associated procedures to
improve efficiency, improve quality of service delivery, or address compliance
issues with agency or government-wide policy
• Apply necessary configuration changes to ensure compliance with PBGC
security baselines and industry best practices
• Apply necessary configuration changes to ensure compliance with
government-wide requirements or support government-wise initiatives

See SLA section for details on associated SLA measures.


DME-ME-06 Contractor shall monitor and report on the life-cycle of all hardware and software
products covered under this Contract and notify the COR and federal
application/service owner if any products require an update or replacement
DME-ME-07 Contractor shall draft presentations to cover development, modernization of existing
products or introduction of new technology included as part of approved
enhancements. This shall include, but is not limited to presentations required to
update PBGC’s Technical Reference Model (TRM)
DME-ME-08 Contractor shall manage requests for new IT infrastructure enhancements including,
but not limited to:
• Documenting requirements utilizing detailed, unambiguous user stories or
Requirements Traceability Matrix
• Developing design summaries for new enhancements including items like
revised data models, level of complexity and effort and technologies to be
utilized
• Manage, track, and publish progress on development efforts through agile
sprints and present updates no less than monthly
DME-ME-09 Contractor shall research, assess, and recommend enhancements to the COR and
federal DM&E leadership
DME-ME-10 Contractor shall provide New and Removed Hardware/Software Report quarterly and
EoSL Recommendations Report annually. This report shall include new cloud-based
IaaS virtual hardware. See Appendix B – Deliverables and Appendix C - List of
Required Meetings and Reports for more information on these documents.
DME-ME-11 Contractor shall work with the Government to develop detailed Bills of Materials
(BOMs) for DM&E projects which will be used by the Government as part of our IGCE
process.
DME-ME-12 Once a DM&E procurement is approved and a delivery date is scheduled, Contractor
shall develop the detailed project plan and execute the design, implementation,
configuration, testing, and validation of the new products. Contractor shall ensure
165
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
knowledge and documentation is properly transferred to the appropriate operational
teams.
DME-ME-13 Contractor shall create Secure Configuration Baselines (SCBs) and associated physical
images or virtual server templates (if applicable) for operating system, database, SQL
server, web server, and application server technologies for deployment and
compliance purposes ahead of planned updates according to the IT Infrastructure
Program Registry. When technically feasible, the Contractor shall, using PBGC’s
current endpoint management tool (IBM Endpoint Manager/Bigfix), establish baseline
configuration checklists to monitor compliance with newly-created SCBs. Contractor
shall maintain SCBs by ensuring they are reviewed at least annually and make any
needed updates, based on revisions from DISA; NIST; etc., or eliminate if they are no
longer required.
DME-ME-14 Contractor shall establish and maintain a periodic task list for DM&E services

166
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
6.9 Disaster Recovery/Continuity of Operations Planning (COOP) and Testing Coordination
and Execution
The sections below describe the scope and requirements of coordinating the Disaster
Recovery/Continuity of Operations Planning (COOP) and Testing Coordination (FFP) and Execution
(CPAF) services that the Contractor shall provide under this solicitation. Providing stable, reliable,
secure, optimally performing, and highly available systems and service is critical to enable the
accomplishment of the agency mission and as such, are critical to excellent performance under this
contract. Thus, Disaster Recovery/Continuity of Operations Planning (COOP) and Testing is a key service
area.

PBGC maintains a DR data center which hosts IT infrastructure and business applications needed to
support critical and essential PBGC functions in the event of a disaster requiring failover. Data is
replicated to the infrastructure located at PBGC’s DR data center through various replication
technologies including file-level replication, transaction log shipping, database mirroring, storage block
replication, etc. PBGC maintains redundant network connectivity to ensure alternative paths are
available to route network traffic in the event of any single failure. See Appendix D – PBGC Network
Overview Diagram for more information on network connectivity between sites and with the internet.
See Appendix F - PBGC Locations for PBGC locations, including planned changes to PBGC’s facilities and
data centers over the life of the contract.

ITIOD has a SharePoint-based solution called the COOP Tracker which is used to store procedures
needed to failover and failback infrastructure systems and services between PBGC’s primary and DR
data centers. The COOP Tracker has workflow that aids in coordinating the appropriate sequencing of
failover/failback procedures. This system also contains personnel lists, PBGC business area test plans,
and other information needed to perform failover and validate systems in the event failover is required.
During exercises and actual events necessitating failover, the system is also used to record and
communicate status of system/application availability.

PBGC has 42 agency critical functions. ITIOD performs 62 discrete steps as part of its failover
procedures, as of 3/25/2019, broken out as follows:

Team Steps
COOP Crisis Management Team 6
Database Administration and Support 7
UNIX/Linux Administration and Support 7
Site Support 3
Test Center Operations 1
Security Tools Operations 2
Windows Server Administration and Support 23
Storage and Backup Administration and Support 4
Network Infrastructure Support 9
Total 62

167
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
ITIOD performs 56 discrete steps as part of its failback procedures, as of 3/25/2019, broken out as
follows:

Team Steps
COOP Crisis Management Team 7
Database Administration and Support 6
UNIX/Linux Administration and Support 5
Site Support 3
Test Center Operations 0
Security Tools Operations 3
Windows Server Administration and Support 17
Storage and Backup Administration and Support 7
Network Infrastructure Support 8
Total 56

6.9.1 Disaster Recovery/Continuity of Operations Planning (COOP) and Testing Coordination


(FFP)
6.9.1.1 Scope of Services Supported
Coordinate establishment and maintenance of infrastructure and associated procedures to ensure
PBGC’s agency critical functions can be performed at all times including immediately following a major
event adversely impacting PBGC’s primary data center and/or infrastructure. Assess outages or risk to IT
infrastructure and make recommendations to ITIOD leadership regarding activation (or not) of some or
all of PBGC’s Information System Contingency Plan (ISCP) for the Information Technology Infrastructure
Services General Support System (ITISGSS). Coordinate execution of some or all of PBGC’s ISCP for the
ITISGSS in response to actual events or mandatory annual testing. Coordinate reconstitution of systems
and services following execution of some or all of PBGC’s ISCP for the ITISGSS in response to actual
events or mandatory annual testing. Coordinate resolution of any issues discovered during periodic
COOP testing.

Typical activities in support of coordinating disaster recovery and continuity of operations capabilities
include the following:

• Maintain personnel lists and work instructions as well as test results in PBGC’s COOP Tracker
• Provide training for COOP readiness
• Update the ISCP for the ITISGSS based and the IT infrastructure Disaster Recovery Plan (DRP) on
changes to the environment including new business applications and infrastructure solutions
and perform ad-hoc failover/failback testing for new systems supporting critical or essential
function upon introduction to the environment
• Prepare for and coordinate execution of periodic DR failover/failback tests (typically 2 per year)
• Prepare reports regarding DR failover/failback tests
• Coordinate resolution of issues discovered during periodic DR failover/failback testing
• Provide support for the following COOP related exercises:
168
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
o Executive Table Top Exercise (typically one per year)
o Forward Challenge Exercise (typically one per year)
o Interagency Exercise each year (typically one per year)
• Coordinate responses to technical questions regarding the System Contingency process and the
overall IT COOP environment

6.9.1.2 Requirements
The contractor shall provide ongoing support for ITIOD’s disaster recovery and continuity of operations
capabilities including ongoing planning and testing coordination as outlined in the following table:

Reference Requirement
(DR-DRC-xx)
DR-DRC-01 Contractor shall identify a lead for Disaster Recovery/Continuity of Operations
Planning (COOP) and Testing. This lead is required to serve as the primary point of
contact for all activities related to Disaster Recovery/Continuity of Operations
Planning (COOP) and Testing.
DR-DRC-02 Contractor shall participate in coordinating recovery/failover operations and assist in
restoring the capability of the PBGC to conduct business, as required. PBGC will
manage the response to an actual disaster affecting Government locations and
provide direction to the Contractor regarding recovery/failover actions.
DR-DRC-03 The Contractor shall develop, obtain PBGC approval, maintain and coordinate
updates to the ISCP for the ITISGSS, the IT infrastructure DRP, the OIT COOP Plan, and
supporting documentation including work instructions, architecture diagrams,
application lists, location lists, personnel lists and other resources and information
related to the disaster recovery planning. See Appendix B – Deliverables for more
information on these documents.
DR-DRC-04 Contractor shall participate in Business Impact Analysis (BIA) for new PBGC business
applications and infrastructure solutions and based on any new requirements,
coordinate deployment of any needed infrastructure systems to the disaster recovery
data center and update the ISCP for the ITISGSS and the DRP as necessary
DR-DRC-05 Contractor shall prepare for and coordinate execution of periodic COOP
failover/failback tests (typically 2 per year) including preparation of the COOP
exercise plan, conducting COOP training, serving as part of the Crisis Management
Team during the exercise, providing end-to-end support for all failover; system
testing; and failback activities including documentation of all aspects of the event,
and communicating to exercise participants; ITIOD leadership; and agency leadership
prior to; during; and following each exercise
DR-DRC-06 Contractor shall provide a minimum of two conference bridges, with no dependency
on PBGC IT infrastructure, that can be utilized to coordinate recovery activities and to
coordinate application validation in the event that PBGC’s phone systems are
unavailable during an outage requiring failover to the DR data center
DR-DRC-07 Contractor shall establish corrective actions for each issue detected during annual
COOP exercises and identified in the After Action Report within 60 days. All
corrective actions must be stored in the appropriate PBGC repository, currently the
169
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
COOP Tracker, and have an assignee and due date. Contractor shall conduct meetings
to discuss and coordinate the COOP Exercise After Action issues, status and
resolutions and record and publish meeting minutes.
DR-DRC-08 Contractor shall establish and maintain a periodic task list to monitor and proactively
maintain ITIOD’s disaster recovery and continuity of operations capabilities

6.9.2 Disaster Recovery/Continuity of Operations Planning (COOP) and Testing Execution


(CPAF)
6.9.2.1 Scope of Services Supported
Establishment and maintenance of infrastructure and associated procedures to ensure PBGC’s agency
critical functions can be performed at all times including immediately following a major event adversely
impacting PBGC’s primary data center and/or infrastructure. Assess outages or risk to IT infrastructure
and make recommendations to ITIOD leadership regarding activation (or not) of some or all of PBGC’s
Information System Contingency Plan (ISCP) for the Information Technology Infrastructure Services
General Support System (ITISGSS). Execution of some or all of PBGC’s ISCP for the ITISGSS in response to
actual events or mandatory annual testing. Reconstitution of systems and services following execution
of some or all of PBGC’s ISCP for the ITISGSS in response to actual events or mandatory annual testing.
Resolution of any issues discovered during periodic COOP testing.

Typical activities in support of executing disaster recovery and continuity of operations capabilities
include the following:

• Maintain technical work instructions as well as test results in PBGC’s COOP Tracker
• Execute periodic DR failover/failback tests (typically 2 per year)
• Resolution of issues discovered during periodic DR failover/failback testing
• Provide responses to technical questions regarding the System Contingency process and the
overall IT COOP environment

6.9.2.2 Requirements
The contractor shall provide ongoing support for ITIOD’s disaster recovery and continuity of operations
capabilities including ongoing planning and testing execution as outlined in the following table:

Reference Requirement
(DR-DRE-xx)
DR-DRE-01 Contractor shall execute recovery/failover operations and assist in restoring the
capability of the PBGC to conduct business, as required. PBGC will manage the
response to an actual disaster affecting Government locations and provide direction
to the Contractor regarding recovery/failover actions.
DR-DRE-02 Contractor shall fully implement, manage, and support all change management
activities regarding for disaster recovery/continuity of operations planning (COOP)
and testing according to PBGC policies. By using ITIL best practices, this should
include, but shall not be limited to:
170
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Provide installation and configuration of IT infrastructure equipment and
services at PBGC’s DR data center including basic network and SAN
connectivity or decommission when no longer required
• Execute some or all of PBGC’s ISCP for the ITISGSS in response to actual
events or mandatory annual testing including travel to disaster recovery data
center facility as needed

See SLA section for details on associated SLA measures.


DR-DRE-03 The Contractor shall develop, obtain PBGC approval, maintain and update the ISCP for
the ITISGSS, the IT infrastructure DRP, the OIT COOP Plan, and supporting
documentation including work instructions, architecture diagrams, application lists,
location lists, personnel lists and other resources and information related to the
disaster recovery planning. See Appendix B – Deliverables for more information on
these documents.
DR-DRE-04 Contractor shall maintain the disaster recovery data center so that all PBGC
infrastructure components supporting critical and essential agency functions
including associated business applications and infrastructure services can failover to
the disaster site in the event of a major outage at the primary production data center
within 12 hours of failover start
DR-DRE-05 Contractor shall maintain procedures so that any or all PBGC infrastructure
components failed over to the disaster recovery data center can be failed back to
PBGC’s primary data center within 12 hours of failback start to include any data
updated while operating out of the disaster recovery data center or note any
exceptions
DR-DRE-06 Contractor shall participate in Business Impact Analysis (BIA) for new PBGC business
applications and infrastructure solutions and based on any new requirements, deploy
any needed infrastructure systems to the disaster recovery data center and update
the ISCP for the ITISGSS and the DRP as necessary
DR-DRE-07 Contractor shall prepare for and execute periodic COOP failover/failback tests
(typically 2 per year) including, attending COOP training, serving as part of the Crisis
Management Team during the exercise, providing end-to-end support for all failover;
system testing; and failback activities including documentation of all aspects of the
event, and communicating to exercise participants; ITIOD leadership; and agency
leadership prior to; during; and following each exercise

6.10 Cloud Integration and Support


The sections below describe the overview, objectives and scope of the Services that the vendor shall
provide PBGC related to cloud computing.

6.10.1 Overview
Through a combination of research, industry feedback and experience with currently contracted cloud
services, PBGC has developed a set of tenets that will guide the acquisition, implementation and use of
171
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
cloud services. These tenets will evolve as the service offerings mature, PBGC’s needs change and its
familiarity with and understanding of cloud services grows. While the acquisition of cloud services will
not occur through this vehicle, PBGC does expect, over the life of this contract, to transition a significant
portion of its IT infrastructure to cloud services.

A. PBGC does not believe it is feasible to use only one cloud service provider but also does not see
effectiveness and efficiency in adopting a “new cloud for every need” approach. PBGC expects
to engage predominantly with one major cloud service provider (CSP) for infrastructure (Iaas)
and platforms (PaaS) services and has selected Microsoft Azure for this purpose. PBGC also
expects to engage with a limited number of additional CSPs based on particular business
requirements, typically providing software services (SaaS).
B. PBGC does not expect to engage in a simple IT infrastructure “lift and shift” approach to cloud
computing. PBGC expects to engage in cloud computing where the service offered is a fully
mature commodity, as the Corporation has done with electronic mail and office automation and
collaboration applications.
C. From an application development perspective, PBGC expects to employ a
“brownfield/greenfield” approach. As funding permits and needs require, PBGC expects to host
either newly developed applications, or those already fully modernized, in a PaaS “greenfield”
cloud environment. Legacy applications will continue to be hosted in PBGC’s existing IT
infrastructure (or perhaps eventually a simple IaaS “brownfield cloud”) until such time as they
can be replaced or modernized. The idea is to allow PBGC’s business areas to plan for – and
acquire the funding needed to – transition to the cloud based on their business requirements
and timeline.
D. PBGC will work with the Contractor to implement the most effective and cost-efficient
integration framework between PBGC’s brownfield and greenfield environments. Through the
use of an integration framework, PBGC will have the flexibility to transition applications and
infrastructure through the appropriate phases of the IaaS/PaaS/SaaS continuum based on
business requirements, prioritization and funding.
E. Given that PBGC is a financial services agency, providing benefits to millions of American
citizens, cloud IT security – and the ability to verify that security as much as possible – is of
paramount importance to PBGC. All cloud solutions and services must meet and maintain a
FEDRAMP moderate ATO.
F. In general, a cloud service will need to meet the following three requirements: 1) it will cost less
to acquire and operate than its locally-hosted predecessor; 2) it will be as secure or more secure
than its locally-hosted predecessor; and 3) it will meet functional and technical requirements of
the organization including speed and reliability.

6.10.2 Objective
PBGC is not seeking a CSP through this vehicle. Rather, we are seeking a trusted and adaptable partner
who can move through a cloud services life cycle with us. Within the context of this contract vehicle,
"partnership" means an interactive, mutually supportive professional relationship that is open,

172
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
collaborative, agile, and customer-oriented. In addition to meeting the objectives described herein, the
Contractor will be expected to:

• consistently take steps to understand PBGC’s crucial business issues and opportunities.
• identify and propose improvements to frameworks, processes, and services throughout the
performance period of the contract.
• share the risks and responsibilities of joint implementations and initiatives.
• ensure its products and services deliver tangible and meaningful business benefits.
• work collaboratively with other contractors, government agencies, and business partners to
ensure project success.

6.10.3 Scope of Services Supported


The Contractor shall provide services in the following four (4) areas:

a) Strategy, Planning and Acquisition Support (SPAS). The Contractor shall consult with PBGC
regarding what type of cloud services may be most appropriate for a particular business
requirement; engage in collaborative planning regarding project scope, timelines and associated
activities; and provide – outside the realm of inherently governmental functions – support for
the acquisition of cloud services. Specific activities in this phase may include, but are not
necessarily limited to: 1) assist with analysis of appropriate cloud service providers vis a vis
PBGC’s business, funding and security requirements; 2) assist in the creation of presentations,
budget plans and projections, strategic roadmaps and project plans to provide
recommendations; and 3) assist with the creation of communications plans for all affected
parties of the migration(s) to ensure end-user adoption, customer satisfaction, successful
organizational process changes, and alignment with PBGC policies, requirements and goals.
b) Implementation and Integration Support (IIS). The Contractor shall work collaboratively with the
CSP, PBGC Federal staff and other contractors as applicable to implement any newly acquired
cloud service and integrate it with the relevant other aspects of PBGC’s IT infrastructure. Specific
activities in this phase may include, but are not necessarily limited to: 1) providing the technical
support necessary to implement or migrate the PBGC target applications and services to the
cloud; 2) providing migration status reporting including milestones and support or implement
specified migration testing plans and related rollback capabilities; and 3) providing support to
ensure that the implemented cloud services are integrated properly with other cloud services
and/or existing on-premise IT infrastructure services managed through this vehicle.
c) Monitoring, Alerting, Logging and Liaison Support (MALLS). The Contractor shall engage in
activities necessary to ensure that the CSP is providing the service(s) acquired, to the required
applicable quality standard(s). These activities will include monitoring the quality of service,
alerting PBGC when service is below standard or security events occur and ensuring that
appropriate logs of the auditing and monitoring activities are created and maintained. The
Liaison activity will entail providing customer service and technical coordination between the CSP

173
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
and PBGC. PBGC anticipates MALLS will be on-going in each technology and service area. Specific
activities in this phase may include but are not necessarily limited to: 1) providing post-
deployment evaluation of the CSP, to ensure compliance with SLAs, and make recommendations
about competition among CSP where cloud provider performance is less than acceptable; 2)
providing recommendations for specified auditable events related to the applications or services;
and 3) providing recommendations for the creation of the most effective compliant risk
assessments, routine vulnerability scanning, system patching and change management
procedures, and the completion of an acceptable contingency plan for the cloud service.
d) Continuous Cloud Services Security Support (CCSSS) Security requirements pervade all three
areas described above. The Contractor, therefore, shall assist with security tasks such as the
review and documentation necessary to produce an Authority to Operate (ATO) or customize an
existing FedRAMP ATO for PBGC’s use. Specific activities in this phase may include but are not
necessarily limited to: 1) providing recommendations for support and cloud services in
compliance and alignment with Federal statutory requirements (e.g. 38 U.S.C. 5725) governing
the protection of Personally Identifiable Information (PII), Federal Risk and Authorization
Management Program (FedRAMP) standardized security assessment, authorization, and
continuous monitoring policies; 2) providing cloud migration recommendations regarding
security and privacy that are consistent with the NIST Special Publication 800-144 – “Guidelines
on Security and Privacy in Public Cloud Computing” or other applicable standards and guidelines;
3) providing recommendations for security for non-standard data transfers both in transit and at
rest resulting from the migration of the applications or services to the cloud; 4) identify any
additional security and privacy standards to which cloud service providers should conform their
services/solutions; and 5) providing recommendations for contract support to verify that
security requirements are documented in the contract between the cloud service provider and
the PBGC.

PBGC anticipates that the Contractor, with two exceptions, will provide these services either with
existing staff or with staff who transition from a particular on-premise technology billet to one
associated with analogous cloud services. For example, an Exchange administrator who was
responsible for on-premise Exchange tasks would have some role to play in the SPAS phase. He/she
would have a primary role in the IISS area. Whether he/she would then transition to the MALLS area
would depend on factors such as number of staff required, the employee’s professional aspirations
and whether the employee’s skill set was applicable to the on-going activities in the particular service
area. It may be that equivalent or roughly equivalent numbers of staff may be required but new
people may need to inhabit those billets.

174
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix A - QASP
Double click on PDF document below to open:

175
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix B - Deliverables
In accordance with FAR 52.246-5, the results of any and all work performed under this contract are
subject to inspection by the COR or the COR’s designated representative. The following list of deliverables
represents that sub-set of those results that will be provided to the COR in 10 business days if it is not
defined by the CDRL, and for which a formal acceptance and approval must be obtained from the COR.
The Contractor is encouraged to propose improvements in both the content and delivery mechanism of
the deliverables.

Number Deliverable Schedule Descriptions


1 Storage Capacity Monthly Provide a breakdown, by the technology or
Planning Report application consuming the storage, of the current
storage allocation for each separate class of
storage repository
Provide quantity of available (usable) storage
capacity remaining for each separate class of
storage repository
2 Asset Inventory and Monthly Provide assets lost, damaged, or stolen (such as
Management Report iPhones, laptops, etc.)
Provide list of disposed assets
Provide list of decommissioned assets
Provide list of assets received, i.e. added to the
inventory including make and model for hardware
and vendor and major version for software and
associated contract
Provide summary of assets assigned to each PBGC
department
Provide count of available assets, organized by
configuration item including vendor-provided
EoSL data
Provide recommended actions and
updates/replacement items to address EoSL
Provide Capitalized Asset Report consisting of a
list of PBGC assets with initial purchase value
greater than equal to $25,000, date of purchase,
contract, location, asset status, and assigned PoC
Provide list of shared PCs and associated users
Provide list of equipment assigned to a common
space
Provide list of users with no primary PC assigned
Provide list of assigned, deployed assets that have
not checked in with their associated management
system to maintain compliance with PBGC policy

176
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Note: as appropriate, “assets” refers to both
hardware and software

3 System Availability Monthly Report of monthly statistics of availability in terms


Report of percentage and uptime for production IT
infrastructure including servers, databases,
networking, etc. excluding planned outages.
Any unscheduled outages should include detailed
explanation.
4 System Availability Monthly Report of fiscal year-to-date statistics of
Report (Fiscal Year to availability in terms of percentage and uptime for
Date) production IT infrastructure including servers,
databases, networking, etc. excluding planned
outages.
Any unscheduled outages should include detailed
explanation.
5 Network Drawings Quarterly Updated drawings for the PBGC network as a
whole including external connectivity, the PBGC
perimeter network including security devises and
services, and each individual PBGC network
segments.
6 COOP Exercise Report 7 business This deliverable is comprised of the COOP
days after Executive Summary report and the COOP AAR.
exercise for The Summary report is an overview type of
executive report, detailing the general course of the exercise
summary, and highlighting significant events. The AAR is a
30 days detailed, comprehensive description of entire
after event exercise, to include those items that will need to
for full be addressed in the corrective action plan for the
report coming year
7 Program Status Monthly The Program Status Report (PSR) is delivered by
Reports the 10th business day of the month, detailing the
accomplishments, planned activities, and issues of
every team during the reporting period or month

177
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
8 Quarterly Maintenance Quarterly The checklist that documents a complete plan and
checklist schedule for quarterly patching

9 Monthly Ticket/Service Monthly The Contractor shall submit a Monthly


Request Quality Ticket/Service Request Quality Control Report to
Control Report the COR no later than the 10th business day of
each month. The report shall summarize the
Contractor’s quality control inspections for the
previous month, to include a list of the
tickets/requests and sampled with an assessment
of each for the following: 1) timeliness, 2) quality
of documentation, 3) use and note of appropriate
KBA/WI, 4) customer satisfaction (if survey
provided. The Contractor may determine the
number of tickets/requests to be inspected, as
long as the following conditions are met:
tickets/requests from all service domains listed in
the IT Infrastructure Technical Services portion of
the PWS shall be inspected no less frequently than
monthly. Prior to obtaining the COR’s formal
acceptance and approval for this deliverable, the
Contractor shall obtain a formal acceptance and
approval from the Federal staff responsible for
the services inspected that month.
10 Quarterly Contract Quarterly The report shall summarize the Contractor’s
Requirement Quality quality control inspections for the previous
Control Report quarter, to include a list of the contract
requirements sampled and the number
determined to have been met satisfactorily. The
Contractor may determine the number of
requirements to be inspected, as long as the
following conditions are met: requirements from
every technology domain listed in the IT
Infrastructure Technical Services portion of the
PWS shall be inspected no less frequently than
quarterly and all requirements in the PWS shall be
inspected at least once annually. Prior to
obtaining the COR’s formal acceptance and
approval for this deliverable, the Contractor shall

178
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
obtain a formal acceptance and approval from the
Federal staff responsible for the services
inspected that month.

11 Monthly Financial Monthly The Contractor shall submit to the COR a Monthly
Report Financial Report no later than 10th business day
of each month. The report shall include
categories that consist of Estimated Hours,
Estimated Cost, Actual Hours, Actual Cost, Hours
Requires Until End of FY (already in Contract LOE),
Cost Requires Until End of FY (Already Obligated
for the Contract), Additional Hours Needed Until
FY and Additional Funding Needed. The
Contractor shall provide these reports for each
Functional Area of Work Requirements specified
in this RFP or Project (as the result of M&E and
Continuous Improvement Program) the
Contractor provides support for PBGC under this
Contract.
12 OIT COOP Plan Annually Establishes a general approach for OIT
departments to adapt that is coordinated with
other PBGC and Federal entities when faced with
a localized or widespread emergency to
accomplish the following priorities:
• Preserve the lives and safety of all
PBGC personnel, contractors and
visitors
• Continue or recommence serving our
customers and stakeholders at the
earliest and safest opportunity
• Resume normal operations when
practicable
This plan shall include the following:
• Identification of all the department’s
critical and essential functions.
• Orders of succession and delegations
of authority to key departmental
positions and responsibilities
• COOP essential records

179
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
• Recovery time objectives for each
function
• Department standard operating
procedures (SOPs) to assist individuals
in the performance of the critical and
essential functions
• Information Technology (IT) assets
(systems and applications) required to
perform all critical and essential
functions
• Notification of emergency and
implementation of COOP procedures
• Human capital information
• Alternate facility site locations and
directions
• Description of COOP essential records
• PBGC Critical, Situationally Critical,
and Essential Functions
• Checklists and worksheets to activate
the COOP Plan and support PBGC
management and leadership
• External contacts necessary to
perform the critical and essential
functions
• Test, Training and Exercise guidance
13 Information System Thirty (30) The Contractor shall develop and submit a
Contingency Plan calendar Information System Contingency Plan (ISCP) for
(ISCP) for the days after the Information Technology Infrastructure
Information the award Services General Support System (ITISGSS)to the
Technology of the Government. The ISCP for the ITISGSS shall be
Infrastructure Services contract, due thirty (30) calendar days after the award of
General Support and will be the contract, and will be updated on a quarterly
System (ITISGSS) updated on basis. The ISCP for the ITISGSS establishes
a quarterly considerations for and procedures to deal with
basis. various contingencies to ensure PBGC’s IT
infrastructure and IT systems supporting PBGC’s
mission and essential functions remain available
including in the event of a disruption requiring
failover to PBGC’s disaster recovery data center or
failback from it. This document describes the crisis
management process utilized to coordinate
activities during contingencies impacting normal
operations. It references the COOP tracker and
associated discrete steps (work instructions) to
180
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
perform failover and failback. The ISCP for the
ITISGSS shall meet the specifications contained in
NIST 800-34 and also include the following:
1. A description of the Contractor’s emergency
management procedures and policy
2. A description of how the Contractor will
account for their employees during an emergency
3. How the Contractor will communicate with
PBGC during emergencies
4. A list of primary and alternate Contractor
points of contact, each with primary and
alternate:
a. Telephone numbers
b. E-mail addresses
14 Disaster Recovery Plan The Contractor shall develop and submit an IT
infrastructure Disaster Recovery Plan (DRP) to the
Government. The DRP for the ITISGSS shall be
due thirty (30) calendar days after the award of
the contract and will be updated on a quarterly
basis. The DRP establishes procedures to recover
PBGC’s IT infrastructure and IT systems supporting
PBGC’s mission and essential functions following a
disruption requiring failover to PBGC’s disaster
recovery data center or failback from it. This
document describes the crisis management
process utilized to coordinate failover and
failback. It references the COOP tracker and
associated discrete steps (work instructions) to
perform failover and failback. The DRP shall
include the following:
1. A description of the Contractor’s emergency
management procedures and policy
2. A description of how the Contractor will
account for their employees during an emergency
3. How the Contractor will communicate with
PBGC during emergencies
4. A list of primary and alternate Contractor
points of contact, each with primary and
alternate:
a. Telephone numbers
b. E-mail addresses

181
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
15 EOSL Report Annually This report shall include technical
recommendations for development and
modernization activities for at least the next 3
years. This report should ensure that PBGC is
considering the most cost and technically effective
approaches to product procurement, replacement
and upgrade. The report shall contain all EoSL
items, the EoSL date, and recommendations for
technology refresh, replacement or removal of
the item and its function from the future PBGC
environment.
16 Periodic Task List Ninety (90) The Contractor shall, within 90 calendar days after
calendar award of the Contract, prepare and submit the
days after tasks on the Periodic Task List (PTL) for all services
award, and work requirements in all functional
updated as areas. The contractor shall establish in the PTL a
needed list of all repeating tasks and apply them to a
thereafter frequency table. The PTL frequency table shall
include tasks to be completed every hour, two (2)
hours, four (4) hours, at shift change, daily,
weekly, monthly, quarterly, semi-annually and
annually. The PTL shall contain all required
repeatable tasks necessary for the continuous
delivery of all IT Services and products. The
Contractor shall maintain the PTL throughout the
life of the Contract. The Contractor shall submit
to the COR changes in PTL not less than 3 days
prior to the desired date of implementation or as
directed by the COR. The Contractor shall not
implement any changes to the PTL until
authorized by the COR. The Contractor shall
notify the Contracting Officer of any changes that
affect contract cost and not implement these
changes until receiving approval from the
Contracting Officer.
17 Contractor Contact and Monthly Each month, the Contractor shall submit the
Staffing Report Contractor Staffing Report according to PBGC
monthly contractor staffing report process
including, as a minimum, the Contract Number,
Contractor Name, Employee Primary User ID,
Employee Last Name, Employee First Name, Work
Location, Office Number, Telephone Numbers,
Electronic Mail Address, Primary Work
Assignment, Start Date, Separation Date, Risk
182
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Level (moderate for regular users, high for AP
accounts) and Developer Status. The Contractor
shall notify the COR of any additions, deletions, or
changes within one working day after the
change(s). The Contractor shall include within the
Staffing Report a summary of all terminated staff
for the past 30 days and a summary of planned
hires for the next 30 days.
18 Major Incident After- 14 days Following the conclusion of a major incident, the
Action Report (AAR) after the Contractor shall submit an after-action report
conclusion including, at a minimum, a description of the
of a Major incident, impact of the major incident, a detailed
Incident timeline of events leading up to the onset of the
issue; steps taken to troubleshoot and/or resolve
the issue; communications regarding the major
incident, root cause analysis, and lessons learned.
19 SIM After Action 14 days Following the conclusion of a security incident,
Report after the Contractor shall submit an after-action report
reporting a including, at a minimum, a description of the
security incident, technical and business impact of the
incident to security incident, a detailed timeline of events
US-CERT leading up to the onset of the issue; steps taken to
respond; recover; and remediate the issue;
communications regarding the security incident,
and lessons learned.
20 PBGC Security Posture Monthly Provides a high-level executive review of
Summary vulnerabilities detected during a credentialed
vulnerability scan of the PBGC network which shall
include, at a minimum:
• Vulnerability totals for current scan and
trends from previous 6 months
• Authentication Success and Failure Rates
• Top 15 most vulnerable subnets
• Top 15 most vulnerable hosts
• Top 15 Vulnerabilities
• Instances of FTP, Telnet, Open SMTP
relays, Default SNMP Community strings

183
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
21 PBGC Vulnerability Monthly This report shall track the status of vulnerabilities
Status Report and compliance monitoring including, as a
minimum:
• Vulnerability Issue
• Vulnerabilities identified as False -
positives
• Vulnerabilities requiring Risk Acceptance
• Vulnerabilities that remain open
• Instances of FTP, Telnet, Open SMTP
relays, Default SNMP Community Strings
• Severity
• Host Name
• IP Address
• Original Discovery Date
• Number of Days Open
• Plan for Remediation

184
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix C - List of Required Meetings and Reports
It is expected that certain number of impromptu meetings and ad hoc reports will be necessary during
the course of this contract. It is not expected that the time and effort required for those items will be
beyond the normal course of business. Below is the list of meetings and reports that PBGC currently
requires in order to perform its mission. Although these meetings and reports do not require formal
acceptance and approval by the COR or the COR’s designated representative, the Contractor shall
conduct or produce them in a fashion consistent with PBGC’s overall expectations of quality and
timeliness.

Meeting or Report Name Frequency Description


New User Report Weekly Displays the Name and user id for the new hire
User Separation Report Weekly Displays all the accounts that are separated
Separation Processing Daily Displays the status of each line item by separation
Daily Executive Summary Daily The daily report providing complete picture of the
day before, to include systems availability, ticket
statistics for change, incident, request and
problem tickets. Must be delivered by 8:00 a.m.
every business day.
Daily Operations Standup Meeting Daily Daily meeting occurring at 08:00 every business
day, to review operations items from previous
day, as well as prospective issues for the coming
day
Weekly Operations Management Status Weekly Meeting to discuss accomplishments from prior
Meeting week, planned upcoming activities and active
projects for each IT Operations team
Monthly ITIOD Director’s Meeting Monthly Meeting to review overall program status with
the ITIOD Director including any notable
successes or serious operational, programmatic
or budgetary concerns.
Change Advisory Board (CAB) Meeting Weekly Meeting to review and approve or reject changes
promulgated via the change management system
Roadmap Meeting Weekly Meeting to review and approve or reject changes
to the ITIOD Roadmap
Detailed Pending Business Owner Daily Displays tickets that are pending for business
owner approval
IMAC Timeliness Report Daily Provide list of IMAC completed within 2 days
Detailed Pending Business Owner Daily Displays service catalog tickets that are pending
for COR/Manager Approval
Executive Summary Report Weekly Shows weekly performance matrix for executive
summary
Executive Summary Report Monthly Shows monthly performance matrix for executive
summary
Current Disk Space Alerts Daily Provide status and summary of Disk Space Alerts
185
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
AutoStore Weekly Report Weekly Provide AutoStore Job Statistics
Incident Resolution Report for PPS Support Weekly Provide weekly PPS Support Incident statistics
Incident Resolution Report for PPS Support Monthly Provide monthly PPS Support Incident statistics
AD User Export Daily List of active users from Active Directory
AD Group Export Daily List of active groups from Active Directory
SM9 Dynamic Catalog Approvers Export Daily List of request fulfillment approvers
Change Logging Report Daily List of RFCs in Change Logging
Pending Line Item Status Report Daily List of pending Line Items
Privileged Users Report Monthly List of elevated privileged account users from
Active Directory
Asset Inventory Report Daily List of available assets with inventory counts
AD Group Membership Export Daily List of Active Directory group membership
Pending Incidents Report Daily Display Open/Pending Incidents by Assignment
groups
RFC Aging Report Daily List of Aging RFCs
PSIS Report Daily Report for Personnel Security Investigation
System (PSIS) to automatically transfer an active
directory file to an externally hosted system.
Developer Production Access Report Monthly Displays all the developers with Production
Access
Planned/Unplanned Outages Monthly List all Planned & Unplanned Outages
First Call Resolution Monthly Monthly Monthly First Call Resolution report displays the
volume of tickets that were resolved by Tier 1 vs
escalated
First Call Resolution Quarterly Quarterly Quarterly First Call Resolution report displays the
volume of tickets that were resolved by Tier 1 vs
escalated
Survey Count Report Monthly Number of survey count sent for Interactions and
Incidents
Median time for End-to-end request Monthly Provides median time for end-to-end request to
fulfillment fulfillment of service catalog requests
FYTD Median time for End-to-end request Quarterly Provides median time for end-to-end request to
fulfillment fulfillment of service catalog requests for fiscal
year to date
Service Interruptions/Degradations Daily List of unplanned service
(Unplanned) Interruptions/Degradations
Service Interruptions/System Outages Daily List of planned service Interruptions/Degradations
(Planned)
Future Changes and tasks Daily List of RFCs and tasks for Next CAB meeting
Interaction Tickets Opened Daily Shows Opened Interaction Tickets data
Incident Tickets Opened Daily Shows Opened Incident Tickets data
Top Five incidents by Assignment Group & Daily List of Top five incidents by Assignement group
Configuration Item and Configuration item

186
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Daily Problem Tickets Daily List of open and aged problem tickets
Incident Ticket Aging Report Daily List of aging Incident and RFI Tickets by
Assignment group
Service Catalog Request Report Daily List of new Service Catalog Request
Daily Backups Daily Provide status of Daily Backup tasks
Backup Errors Daily Provide status of Daily Backup errors
VCenter Data Store Capacity Report Daily Provide capacity status for VCenter Data Stores
Solaris System Performance Report Monthly Provide System Performance for all solaris servers
2018 Availability Report (Business Hours) Monthly Monthly COR Report
High Level Summary -COR Report
SharePoint System Performance Report Monthly Provides SharePoint System Performance
MyPAA Infrastructure Report Daily MyPAA CPU Disk and Memory Performance
Report
MyPAA_WebURL_Report Daily MyPAA WebURL Transaction Availability and
Performance Reports
DataCenter System Availability Detailed Weekly Provide System Availability Details for PBGC data
Report center
Non-Compliant iPhones Users Monthly Compile report from Intune Server to see which
iPhones have not contacted the Server within 30
days
Master Release Schedule Daily Report produced daily on all open and scheduled
changes
ITISGSS Audit Report Semi- Report provided, with fully information from
annually current ITSM tool, twice a year to the auditors, for
all changes up to a certain date
CAB Voting Member Attendance Report Quarterly Report to provide metrics on voting member
attendance for the previous quarter
Unauthorized Change Report Weekly Report providing the number and description of
any changes that occurred without authorization
via the change management process
Emergency Change Review Report Quarterly Report documenting that all emergency changes
were reviewed at the succeeding weekly CAB
meeting
CAB Agenda Weekly Agenda published prior to that week’s CAB
meeting
CAB Minutes Weekly Provides results of changes approved, action
items and status of forward changes
CAB Forward Schedule Changes Report Weekly Provides 3-week forward view of upcoming
changes and tasks
Business Unit Current and Overdue Changes Daily Report provided for review with specific business
Report units of their current and overdue changes and
tasks

187
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Pending Change Closure Report Weekly Report provided weekly for discussion between
release and change management to assist in
closing changes
RSA Report Daily Report of assigned changes, to manage RSA’s
daily workload
Overdue Changes and Tasks Daily Report for review, with change and task owners,
of their overdue items
New and Removed Hardware/Software Quarterly This report shall include any new or removed
Report hardware asset classes (consisting of vendor,
model/version) or infrastructure software either
added or deleted from the PBGC environment
during the period. This report shall include new
cloud-based IaaS virtual hardware. This report
will include the vendor-provided EoSL data
including dates and recommended
update/replacement items

188
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix D - PBGC Network Overview Diagram

In calendar year 2019, PBGC expects to consolidate the vast majority of its WAN, local phone, and all of
its Internet (MTIPS) circuits under a single GSA EIS contract that includes services for/to the co-located
data centers. Transition to this new vendor will likely run into early calendar year 2020. The 1 GB
backbone currently between HQ, COOP, and Kingestowne will be replaced by a 10 GB wave backbone
connecting the col-located data centers, the HQ, and Knigstowne.

189
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix E - References
Meet all Federal mandates and guidelines regarding IT services and operations. Specifically, PBGC must
meet the requirements mandated in the current rules and regulations listed below.

Reference Documents Source

Clinger Cohen Act http://www.cio.gov/Documents/it_mana


gement_reform_act_Feb_1996.html

Information Technology Management Reform Act http://www.whitehouse.gov/omb/memor


(ITMRA) anda/m96-20.html

Federal Information Security Management Act (P.L. 107- http://thomas.loc.gov/cgi-


347, Title III), December 2002. Paperwork Reduction Act bin/query/F?c107:5:./temp/~c107nixsEC
(P.L. 104-13), May 1995. :e151997:

Guidance for Securing Microsoft Windows Vista http://csrc.nist.gov/itsec/guidance_vista.


html

NIST standard 800 series http://csrc.nist.gov/publications/PubsSPs


.html

NIST Information Security Automation Program (ISAP) http://nvd.nist.gov/scap/docs/ISAP.doc


Automating Vulnerability Management, Security
Measurement, and Compliance, Version 1.0 Beta,
5/22/2007

NIST Security Configuration Checklist: http://checklists.nist.gov/

Federal Information Processing Standards http://csrc.nist.gov/publications/PubsFIP


S.html

Federal Information Processing Standards Publication http://www.csrc.nist.gov/publications/fip


(FIPS Pub) 201 s/fips201-1/FIPS-201-1-chng1.pdf

Homeland Security Presidential Directive 12 (HSPD-12), http://www.dhs.gov/xabout/laws/gc_121


July 6, 2007 7616624097.shtm#1

http://www.whitehouse.gov/news/releas
Homeland Security Presidential Directives HSPD-7
es/2003/12/20031217-5.html
“Critical Infrastructure Identification, Prioritization, and

190
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Reference Documents Source
Protection”

Presidential Decision Directive (PDD) 63, “Critical http://www.fas.org/irp/offdocs/pdd/pdd-


Infrastructure Protection” 63.htm

Federal Desktop Core Configuration (FDCC): OMB M-07- http://www.whitehouse.gov/sites/default


11 Implementation of Commonly Accepted Security /files/omb/assets/omb/memoranda/fy200
Configurations for Windows Operating Systems 7/m07-11.pdf

Federal Desktop Core Configuration (FDCC): OMB M-07- http://www.whitehouse.gov/sites/default


18 Ensuring New Acquisitions Include Common Security /files/omb/assets/omb/memoranda/fy200
Configurations 7/m07-18.pdf

Office of Management and Budget (OMB) Circular A-11 http://www.whitehouse.gov/omb/circula


rs/a11/current_year/a11_toc.html

OMB Circular A-130 http://www.whitehouse.gov/omb/circula


rs/a130/a130trans4.html

OMB TIC Requirements, M-08-05, Implementation of http://www.whitehouse.gov/omb/memor


Trusted Internet Connections (TIC), 20 November 2007 anda/fy2008/m08-05.pdf

OMB TIC Requirements, M-08-16, Guidance for Trusted http://www.whitehouse.gov/omb/memor


Internet Connection Statement of Capability Form (SOC), 4 anda/fy2008/m08-16.pdf
April 2008
OMB memos applicable to IT systems, security, privacy http://www.whitehouse.gov/omb/memor
and contracts anda/index.html

http://www.gao.gov/new.items/d04394g.
GAO IT Investment Management pdf

USA PATRIOT Act (P.L. 107-56), October 2001. http://thomas.loc.gov/cgi-


bin/query/D?c107:4:./temp/~c107AaoSy
F::

Privacy Act of 1974 (P.L. 93-579), December 1974. http://www.defenselink.mil/privacy/doc


uments/pa1974.pdf

Section 508 of the Rehabilitation Act http://www.section508.gov/index.cfm?F


useAction=Content&ID=3

191
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
http://www.federalelectronicschallenge.
Federal Electronics Challenge (FEC) net/

Reference Documents Source

FAR Clauses Applicable to Electronics Stewardship http://www.federalelectronicschallenge.


net/resources/docs/farprov.pdf

Federal Legislation and Executive Orders Relevant to the http://www.federalelectronicschallenge.


FEC net/resources/docs/fec_regs.pdf

NARA Regulations at 36 CFR Chapter XII, Subchapter B, http://www.archives.gov/about/regulatio


Records Management ns/subchapter/b.html

NARA Bulletin 2008-05 http://www.archives.gov/records-


mgmt/bulletins/2008/2008-05.html

NARA Bulletin 2010-05 http://www.archives.gov/records-


mgmt/bulletins/2010/2010-05.html

Government Paperwork Elimination Act http://www.archives.gov/records-


mgmt/policy/electronic-signature-
technology.html

E-Government Act of 2002 http://www.archives.gov/about/laws/ego


v-act-section-207.html

http://www.archives.gov/records-
Federal Records Act 44 USC Chapters 21, 29, 31 & 33 mgmt/laws/

Freedom of Information Act http://www.archives.gov/foia/

Information Technology Infrastructure Library (ITIL) v3 http://www.itil-officialsite.com/

192
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix F - PBGC Locations
The following table details the PBGC locations as of March 2019, including user counts and
requirements for site support staff:
Site Street Address City State Zip Code User Site Support Comment
Code Count Staff Required

BRO 15800 Brookefield WI 53005 5 No


Bluemound Road
Suite 400
BUC 925 Euclid Cleveland OH 44115 23 No Actuarial site. This site currently
Avenue 18th has a dedicated MPLS connection
Floor and GFE, but this will be shifted to
a remote access/VDI solution by
the end of calendar year 2019.
COL 105 Technology Broomfield CO 80021- 12 No Current off-site Service Desk. This
Drive Suite 100 3432 staff connects to PBGC via VPN
(primary) or remote access/VDI
(alternate). A dedicated MPLS
circuit is envisioned for the new
ITIOSS contract.
COR 400 Rouser Road Coraopolis PA 15108- 47 No FBA Site;
Bldg 2 5th Floor 2749 Slated for decommission in
12/2019. Workload is already
being shifted to EUC site.
DOR 3750 NW 87th Doral FL 33178 83 No FBA Site.
Avenue Suite
600 Coordination with on-site
technical support staff may be
required.
EUC 24701 Euclid Ave Euclid OH 44117 99 No Growing to approximately 300
users by the end of 2021 to
include the Call Center and
Document Management Center.
Site Support staff may ultimately
be required, but this is unlikely.

Coordination with on-site


technical support staff may be
required.
HQB 1275 K Street Washington DC 20005- 213 Yes HQ Campus (4th and 9th floors
NW 4026 only)
HQI 1225 I Street NW Washington DC 20005 39 Yes HQ Campus (3rd floor only)

HQL 1615 L Street Washington DC 20036 3 No


NW Suite 510
HQW 1200 K Street Washington DC 20005- 1,291 Yes HQ Campus (Main building and
NW 4026 only occupant; floors LL;1-12)
KIN 5971 Kingstown Alexandria VA 22315- 145 Yes Call and Document Management
Village Parkway 5879 Center. Slated for decommission
Suite 300 by 2021. Services (very) likely
moving to EUC site.

193
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
OFF Off-Site * * * 133 N/A

QUI 200 Newport Quincy MA 02171 9 No State Street Bank


Avenue
WIL 2500 Grubb Wilmington DE 19810 56 No FBA and hosts current disaster
Road Suite 221 recovery data center. Slated for
decommission by 2021.

The following table provides a tentative schedule for adjustments to PBGC locations over the life of
this contract and changes to user counts as well as expected requirements for the O&M contractor:
High-level change Estimated Details
Timeline
Expand EUC site Q3-Q4 FY19 Expand infrastructure and workstation count at EUC site to
support COR and WIL FBA consolidations as well as relocation of
Call Center and Document Management Center services; a total
of 300 staff. Work is already underway and will be complete at
award of this task order.

Minimal O&M contractor responsibilities beyond shipping IT


equipment to the sites being expanded and coordination with on-
site technical staff.
Build out new disaster recovery Q4 FY19- Q1 Location TBD. New O&M contractor will be responsible for
(DR) data center at acquired DR FY20 implementation of a new network core router/switch for
Co-Lo and relocate equipment connectivity as well as relocation of DR equipment and services
and services from WIL to the DR from WIL to the DR Co-Lo.
Co-Lo

Close out Coraopolis site Q4 FY19 Current O&M contractor will be responsible for decommission,
collection and shipping of IT equipment back to HQ. User count
will shift to EUC facility. Work is already planned and will be
largely or entirely complete at award of this task order.

Close out Kingstowne site Q3 FY20-Q1 New O&M contractor will be responsible for decommission,
FY21 collection and shipping of IT equipment back to HQ. User count
will shift to EUC facility. Based on current plans, DMC will be out
of Kingstowne by 6/2020 and CCC will be out by 12/2020.

Close out Wilmington site Q3 FY20 New O&M contractor will be responsible for decommission,
collection and shipping of IT equipment back to HQ. User count
will shift to FBA North facility.

194
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Build out new primary data Q4 FY20- Q1 Location TBD (within 60 miles of DC). New O&M contractor will
center at acquired Co-Lo and FY21 be responsible for implementation of a new network core
relocate equipment and services router/switch for connectivity as well as relocation of primary
from HQW to the primary data data center equipment and services from HQW to the primary
center Co-Lo data center DR Co-Lo.

Consolidate and relocate Q4 FY21 – Q4 New O&M contractor will be responsible for initial infrastructure
Headquarters FY22 build-out at location and then subsequent relocation of users at
HQW, HQB, and HQI to new, single GSA leased space at 445 12th
Street, S.W., Washington, D.C.

195
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix G - IT Service Support Guidelines
Priority Table:

196
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix H - IT Infrastructure Maintenance Schedule
The following table details the maintenance windows for production IT infrastructure which have been
agreed to by the PBGC business units:

Frequency Schedule Infrastructure Event


Weekly Tuesdays and Thursdays Weekly Infrastructure Maintenance including
11:00PM to 5:00AM (of the workstation and server patching.
following day)
Quarterly Last non-holiday weekend of Major Infrastructure Maintenance including Oracle
March server and database patching, network infrastructure
First non-holiday weekend of updates, etc.
June
Last non-holiday weekend of
August
First non-holiday weekend of
December
Semi-Annually First non-holiday weekend of COOP Exercise (February),
February, August COOP Validation Test (August)

Additional outages may also be scheduled as needed, but must be negotiated with the PBGC business
units. ITIOD maintains a list of PoCs for each PBGC department with whom such arrangements can be
coordinated.

IT infrastructure maintenance for the PBGC Development and Test environments can be conducted on
the weekends and after normal operating hours (7:00 AM – 5:00 PM, M-F).

197
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix I - OIG IT Infrastructure Summary
Background
The Office of Inspector General (OIG) for Pension Benefit Guaranty Corporation serves as an
independent entity within PBGC. Its mandate is to detect and prevent fraud, waste, abuse, and
violations of law, and to promote economy, efficiency and effectiveness of the PBGC. The OIG strives, as
an agent of positive change, to continually improve PBGC management and program operations by
independently conducting audits, evaluations, and investigations.

IT Infrastructure Summary
The OIG’s local IT infrastructure is typically logically and physically separate from PBGC’s. It is, however,
connected to and integrated with PBGC’s primary production IT Infrastructure for services such as
Internet access and other enterprise-levels needs, e.g. IPAM, email gateways, etc. The OIG generally
purchases and uses the same models, types and versions of IT infrastructure hardware and software as
PBGC does. The OIG largely administers their infrastructure with their own contract staff for day-to-day
activities. They do, however, rely upon the primary IT infrastructure DM&E/O&M contractor to assist
with installation of new technology, modernization of existing technology, and more complicated
technical issues.

PBGC’s Office of the Inspector General (OIG) IT infrastructure environments consists of many
components and tools detailed in the tables that follow:

Active Directory OIG AD User Summary

The following table details the approximate number of Active Directory user accounts in PBGC’s OIG
user and resource domain (oig.ent.pbgc.gov) broken down by function:

Account Type Count of Users


Regular User 50
AP User 3
Service Account 20
Shared Mailbox 20
The OIG is part of PBGC’s Office 365 implementation and utilizes this primarily email and Skype although
they are starting to explore use of SharePoint.

OIG Windows Server Summary

PBGC’s OIG has approximately 22 Windows servers, with almost all of them being virtual running on
VMware 6.5. The following table breaks them down by OS:

Operating System Count


Windows Server 2008 1
Windows Server 2008 R2 10
Windows Server 2012 R2 15

198
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Total Count 26
An upgrade to Windows Server 2016/2019 is planned for FY19.

The OIG also administers and maintains content for the oig.pbgc.gov web site which is actually hosted in
PBGC’s production environment.

199
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix J - IT Service and Support 2018 Statistical Summary

IT Service Desk Interactions

In 2018, approximately 68% of phone calls to the Service Desk were resolved by the Service Desk. A
further 10% were resolved by Site Support. The remaining 22% were escalated to Tier 2 or Tier 3.

The following table provides the approximate count for tier 1 IT Service Desk interactions processed in
calendar year 2018 broken down by contact type and first call resolution (FCR) for calls made to the
Service Desk. PBGC transitioned to ServiceNow as its operational system for incident management on
January 19th. The process for calculating First Call Resolution was adjusted on March 1st. The following
table is of data between January 19th and December 31st, 2018, except for FCR calculations which are
March 1st through December 31st.

Contact Type First Call Resolved (FCR) Count


Telephone (March 1 – Dec 31) Yes 13,578
Telephone (March 1 – Dec 31) No 6,382
Telephone (not subject to FCR) N/A 3,326* + 1,558**
Telephone (Total) N/A 24,844
Email N/A 21,480
Employee Self Service (on-line submission) N/A 1,197
Site Support Walk-Up N/A 2,428
Total 49,949

* Received between January 19th and February 28th.

** Received between March 1st and December 31st, but not subject to SLA (typically tickets opened by
someone who is not a member of the Service Desk team).

200
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Incident, RFI, Service Requests, and Change Management Requests

The following table provides the approximate count for tier 2 incidents, requests for information (RFIs),
service/access requests, requests for change (RFCs), and RFC tasks processed in calendar year 2018
(January 19th- December 31, 2018) broken down by service area for work in the scope of this contract:

Team Incidents RFIs Service RFCs RFC Tasks


Requests
Asset Management 585 180 1,524 0 0
Configuration Management 15 7 0 3 24
Database Administration 2,981 465 2,754 62 1,099
Enterprise Identity Management 12 8 0 1 7
ITSM Tool Support 339 138 711 17 87
Modernization & Enhancements 797 75 2 332 577
IT Infrastructure and Application 59 9 0 2 8
Availability, Capacity, and
Performance Monitoring
Network Infrastructure support 480 74 12 109 454
Network Operations Center 430 173 0 6 5
Office 365 and Messaging 723 225 7 23 39
Administration and Support
Telephony Infrastructure Support 592 236 2,478 12 13
ITIOD Reporting and 5 3 0 1 2
Dashboarding
Security Tools Operations and 2743 154 695 89 176
Security Incident Response
IT Service Catalog Support 65 51 0 1 0
IT Service Desk 13,729 4,308 5,535 0 0
Site Support 3952 326 3,560 1 28
Storage and Backup 436 102 248 61 206
Administration and Support
Test Center Operations 206 33 922 14 84
UNIX/LINUX Administration and 761 326 143 78 731
Support
Windows Desktop Administration 87 10 58 33 126
and Support, Software Packaging,
and Software Deployment
Network Printer Support 481 16 0 1 1
Windows Server Administration 2119 539 1,379 344 1,079
and Support

201
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
The following table provides the approximate count for tier 2 incidents, requests for information (RFIs),
service/access requests, requests for change (RFCs), and RFC tasks processed in calendar year 2018 for
assignment teams outside the scope of this contract. PBGC migrated to ServiceNow for Incident, RFI,
and Change Requests on January 19th; the data shown is here are for incidents and RFCs opened
between January 19th – December 31st, 2018; For Service Request opened between January 1st –
December 31st, 2018; For RFCs with target environment deployment dates in 2018; and for RFC Tasks
scheduled to be completed in 2018:

Team Incidents RFIs Service RFCs RFC Tasks


Requests

Other ITIOD (non-Operations) 1,116 25 48,033 160 240


including automated fulfillment

Non-ITIOD 7,183 461 21,401 89 235

TOTAL 8,299 486 69,434 249 475

All service requests processed by ITIOD in 2018 in the table above were fulfilled
automatically/programmatically.

202
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix K - IT Infrastructure Tools List
The following table provides a list of the primary software/tools utilized in providing the services under
this contract:

Tool Name Purpose


Service Now (London) IT service management and ticketing tool used for IT interactions
and incident management, changes management, problem
management, IT hardware and software asset tracking and
management, IT configuration item classification, tracking and
management, IT configuration item discovery, and will be
utilized for request management
HP Service Manager 9.3.x Legacy IT service management and ticketing tool which was
replaced by Service Now with the exception of Service
Catalog/Access Request module. This is planned to be replaced
in FY19.
Microsoft SCCM 2012 R2 Used for deploying IT approved and packaged software as well
as workstation images
Microsoft Skype for Business 2016 Remote support/desktop sharing
Microsoft Remote Desktop Connection RDP client
Microsoft Remote Desktop Connection RDP client
Manager 2.x
PuTTY SSH client used for managing devices
Microsoft Active Directory 2012 Serves as the primary directory used by PBGC for accounts and
R2/2008 R2 managing access. The majority of PBGC applications leverage AD
for authentication and access control. Also used to support
group policy application in the Windows environment. Domain
controllers are scheduled for upgrade in FY19 to version 2016.
Dell/Quest Active Roles Server (ARS) Used to manage Active Directory in compliance with PBGC
7.1 standards and to support scripted integration with other PBGC
directories, e.g. procurement, HR
Microsoft ADFS v3 Provides federation services for Cloud hosted applications like
Microsoft Office 365 and Service Now to allow authentication to
cloud services using on-premise Active Directory accounts. A
plan to upgrade to version 4 is planned for FY19.
Microsoft Azure AD Connect 1.1.x Synchronizes users and groups with Microsoft Office 365
CyberArk Password Access Security Privileged Account Management tool being deployed at PBGC to
(PAS) 9.x store privileged credentials and broker and record sessions
requiring privileged access
SailPoint Identity IQ 7.X Enterprise identity management tool being deployed at PBGC
(FY19) to control and automate account and access
management on PBGC systems that are not AD integrated.
SailPoint, in combination with the ServiceNow request module,
will replace existing HP Service Manager/Catalog 9 as workflow

203
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
engine for access request, approval, and fulfillment as well as
replace SharePoint and other non-IT managed solutions utilized
for annual account and access recertification.
Oracle Access Manager 11g and 12c Used to provide single sign-on authentication, leveraging
Windows native authentication and OID, to WebLogic hosted
applications
Oracle Internet Directory 11g and 12c Supports OAM for application access via single sign-on.
Synchronized with AD utilizing Oracle’s Directory Integration
Platform.
Dell/Quest Recovery Manager for Provides backup and granular recovery capabilities for PBGC’s
Active Directory (RMAD) 8.8 Active Directory
Flexera Admin Studio 11.5 Used for conflict resolution and project management for
application deployment packages
Flexara InstallShield 19sp1 Used for creating .MSI application deployment packages and
.MST transforms
IBM BigFix 9.5 Used for patching and occasional software deployments.
Includes Security and Compliance Analytics (SCA) 1.9 for
compliance reporting.
Oracle Enterprise Manager 11g/13c Centralized administration of Oracle databases and middleware
Oracle Recovery Manager (RMAN) 12c Provides for automated Oracle backup and recovery in
conjunction with Veritas Netbackup
Oracle Data Guard 11g/12c Provides for automated data replication to disaster recovery site
Dataguise 5.1.x Masks sensitive data
Microsoft SQL Admin Studio Centralized administration of Microsoft SQL Server databases
Dell/Quest Toad for Oracle SQL Development Tool. PBGC has Base, Xpert, and DBA
modules.
VMware vCenter 6.5 Centralized administration of vSphere environment
VMware Horizon 7.x Centralized administration of Virtual Desktop Infrastructure
environment
VMware PowerCLI 6.5 R1 Automation of virtual environment using Powershell
Office 365 Admin Portal Government cloud management portal administration
Microsoft Azure Admin Portal Manages InTune and Azure (separate tenants for each)
Sharepoint Designer 2016 Develop Sharepoint content including sites and workflows
Infopath 2013 Develop Sharepoint forms
Concept Search 5.4.x Utilized to manage taxonomy terms and automatically assign the
appropriate metadata tags to content based on clues
established by lines of business. This product also supports
detection and quarantine of improperly posted PII and migrating
records to the appropriate folder in PBGC’s records center.
Tectia SSH 6.3.x Provides for SSH support for secured file transfer
IBM Connect Direct 4.x Used to perform secure file transfers between IBM Mainframes
(off-site) and Unix servers
Hummingbird Exceed 13.x SSH and X-Windows client

204
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Splunk Enterprise Security 7.x Splunk is the enterprise log management and security incident
and event management (SIEM) tool for PBGC. Splunk gives the
visibility into log data from all servers and security devices. The
data within Splunk is used for conducting security investigation
and diagnosing the root cause for operational issues.
HP Business Service Management IT Infrastructure and application availability and performance
(BSM) 9.x monitoring, reporting and alerting
HP SiteScope 11.x IT Infrastructure availability, performance, and capacity
monitoring, reporting and alerting
Think Automation 4.x Enable and support email integration with SharePoint on-line
and perform other business process automation, e.g. automate
upload to SharePoint on-line reports distributed via email
Cisco Prime 3.x Manage and monitor (health, performance, alerts, notifications)
Cisco network devices, run baseline checks, automate archive
configurations, push configurations to multiple devices
simultaneously
Cisco AnyConnect 4.x VPN client
Cisco ISE 2.x Provides network access control (NAC) services
HP Network Node Manager (NNMi) Monitor local area (LAN) and wide area network (WAN)
10.x availability via performance monitoring in an easy-to see
graphical format. It shows the devices relative location and
status.
Plixer Scrutinizer 17.x Provides netflow data as well as network traffic analysis
Pscp ver 0.63 Executable binary scp client program (CLI) to securely
copy/transfer router/switch configurations and upload IOS
images to routers and switches
Tftpd64 Executable binary TFTP client program (CLI) to copy/transfer
router/switch configurations and upload IOS images to routers
and switches that don’t support secure protocols
Wireshark 2.x Captures network traffic to analyze packets for troubleshooting
NEC SV9500 PCPro 2.1.x PBX/ACD Management Software (HQ Campus)
NEC Univerge UM8700 Administration Voicemail management software
8.7.x
NEC SV8300 PCPro 9.0.x PBX/ACD Management Software (FBA sites)
NEC Global Navigator (GNAV) 11.x A NEC tool for reporting and monitoring agent activity for the
ACD phone system
PBGC NEC Phone Database A Microsoft Access database used to document relationships
between phone numbers/extensions, users and locations and
phone ports/cables
Liebert Sitescan 5.2 Provides comprehensive monitoring and control of the PBGC
Data Center and Field Benefit Administration (FBA) facility
support systems

205
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Temp Guard Micro Technologies Data Software used to manage, configure and monitor environmental
Capture 4.2.7 (temperature and water) sensors that cannot be managed with
Liebert Sitescan.
Polycom RMX Manager 8.x Used to setup video conferences and to create and manage
personnel conference bridges.
Microsoft Intune 5.x Portal Cloud-Based mobile device management tool used to reset
passwords, factory reset phones, enroll new users, locate lost
devices, manage business required applications as well manage
mobile device compliancy for approximately 1000 mobile
phones.
Hitachi Navigator HUSVM SAN provisioning
Hitachi Storage Navigator Modular HUS150 SAN provisioning
28.x
Hitachi Command Suite 8.5.x HUSVM monitor and SAN provisioning
Hitachi Command Control Interface Shadow Image and COW
HORCM
Netapp Ontap 9.x NFS, CIFS and SAN provisioning
Netapp OnCommand System Manager NFS, CIFS and SAN provisioning
Netapp Snapshot Snapshot
Netapp SnapMirror Remote replication
Netapp FlexClone Local clone
Brocade Web Tools Manage and monitor Brocade Fibre Channel switches
Veritas Netbackup 8.x Enterprise Backup and restore
Veritas Auto Image Replication(AIR) Replication backup data from HQ to WIL
IBM System Storage TS3310 Tape Manage and monitor IBM tape library
Library
Veritas NetBackup appliance software Will be use on new Netbackup appliance on HQ and WIL for
3.x backup, restore and replication using AIR
HP Application Life Cycle Management Requirements management, test planning and functional
(HP ALM) and (and HP Sprinter) / HP testing, and defect management
Quality Center Enterprise 12.x
HP Unified Functional Testing (HP UFT) Automates functional testing
12.x
HP LoadRunner 12.x Used to test applications, measuring system behavior and
performance under load
Serena Business Manager (SBM) 10.x Process management and workflow automation platform for IT
and DevOps designed to orchestrate and automate processes
and provide transparency across an organization
Serena Version Manager (PVCS VM) Code repository and version control system
8.x
Oracle BI Publisher 11.x Enterprise reporting tool
Tableau 10.x Enterprise business intelligence tool

206
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
DOJ Cyber Security Assessment and Used to document FISMA systems, associated controls, control
Management (CSAM) status, and control inheritance, track POA&Ms, etc.
The list of the primary software/tools utilized in providing the security services under this contract will
be made available in PBGC’s reading room.

207
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Appendix L - IT Infrastructure Program Registry and ITIOD Roadmap
The following is an export from the IT Infrastructure Program Registry which is used to record all IT
infrastructure development, modernization, and enhancement projects and activities greater than 40
hours:

Title Brief Description ProjectType Start Date Production End Date


Deployment
Completion
Date

IPv6 Readiness Assessment Procure Cisco Assessment Services for IPv6 Activity 10/20/2015 09/20/2016
and Sequencing Plan
Enterprise Identity Establish new Enterprise Identity Project 01/01/2016 12/15/2019 01/01/2020
Management Management Tool and Services per the results
of the ICAM BNA analysis performed in FY
2015.
Oracle Fusion Middleware Oracle Fusion Middleware Upgrade Project 03/14/2016 09/06/2019 09/30/2019
Upgrade (wls/soa(bpel)
Oracle 12c RDBMS Upgrade Oracle 12c RDBMS Upgrade Project 03/14/2016 09/30/2019 10/31/2019
JRE 8u192 Upgrade Upgrade to JRE version 8 wherever possible, Activity 05/02/2016 07/31/2018 03/29/2019
Servers/Workstations (FY17- and patch legacy versions of JRE if still
FY19) required

Network and Local Printer Standardize and consolidate on a limited Activity 08/25/2016 03/31/2019 03/31/2019
Fleet Standardization number of network and local black and white
and color printer models and establish
associated support model

Headquarters Relocation Headquarters relocation planning and support Project 10/02/2016 12/31/2019 12/31/2019
Planning and Support
Developer Tools Developer Tools Consolidation and Project 10/03/2016 04/19/2019 06/30/2019
Consolidation and Modernization effort to identify a PBGC
Modernization standard for development and release tools
that will support Agile and Continuous
Integration (CI), Data Integration, and
Application Performance Monitoring and
Management (APM). The tools identified
should facilitate use of more modern, cost
effective platforms and technologies including
Cloud and DevOps.
FBA Consolidation Consolidate the existing FBAs into two or Project 06/01/2017 06/15/2019 09/30/2021
three facilities
ITSM uCMDB and Asset Tools Consolidate/Replace ITSM, uCMDB, and asset Project 06/08/2017 02/28/2019 02/28/2019
Replacement (Service Now) tools with new Service Now cloud-based
solution
Telephony Infrastructure This project will replace the legacy telephone Project 06/23/2017 09/30/2019 09/30/2019
Modernization systems at all PBGC locations including the
FBA sites. PBX's and telephone handsets will
be replaced with Omni-channel VoIP capable
systems to support Infrastructure and
Business systems.

208
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Cisco Switch Replacements Replace all EOSL 6500 and 3750 closet Project 06/23/2017 10/31/2018 05/31/2019
(FY18) with Wireless and switches. Includes Cisco ISE, Cisco Prime and
Prime and Cisco ISE Wireless AP Implementation in Scope.

RHEL 7 Upgrade (FY18/19) Upgrade RHEL 6.x (and older) servers to the Project 09/29/2017 09/06/2019 09/30/2019
latest version of RHEL in advance of EoSL.

Data Center Colocation and Acquire and migrate to co-location centers for Project 10/02/2017 08/31/2020 09/30/2020
Consolidated 10GB WAN and primary and disaster recovery IT
TIC implementation infrastructure data center services and
upgrade, consolidate WAN infrastructure to
10 GB Wave, and implement high-speed TIC

SAN Infrastructure Brocade SAN Fabric Modernization and Project 06/15/2018 04/15/2019 05/15/2019
Modernization (FY18/19) Storage Array Consolidation (6 arrays to two),
Storage Expansion and Encryption
Symantec Netbackup Activity 06/20/2018 05/15/2019 05/15/2019
Appliance and Capacity Symantec NetBackup Appliance Capacity
Upgrade (FY18/19) Upgrade including addition of 5 Media Shelves
and 60 FETB licenses as well as enablement of
disk encryption.

HP and Serena TCO Tools HP and Serena TCO Tools Annual Activity 06/30/2018 07/30/2019 08/05/2019
Annual Upgrade (FY18/FY19) Upgrade (FY18/FY19)
Data Loss Prevention (DLP) Establish automated systems/tools at PBGC's Project 08/01/2018 06/15/2019 07/15/2019
(FY-19) perimeter to prevent intentional and
unintentional transfer of PBGC data outside of
PBGC's network.

Mobile Phone Solution Recompete mobile/cellular services contract Activity 08/06/2018 08/30/2019 09/30/2019
Update/Refresh (FY19) and replace/upgrade mobile devices .

Network Perimeter and IPAM Modernize Firewalls, Remote Access Activity 08/15/2018 04/12/2019 04/12/2019
Modernization Appliances, Proxies, and IPAM
devices. Replace external DNS and DNSSEC
Service Provider DataMountain with
Oracle/Dyn DNS.

CDM Task Order 2F (CSM, Implement DHS's CDM Task Order 2F Project 08/29/2018 11/15/2019 11/30/2019
SWAM, VUL, HWAM) (CSM,SWAM,VUL,HWAM).

Windows 2016/2019 Server Upgrade Windows 2012 R2 (and older) Project 09/01/2018 03/16/2020 03/31/2020
Upgrade (FY19) servers to the latest supported MS Windows
Server version (2019 where possible, 2016
where necessary) in advance of EoSL.

Web and DB Vulnerability Establish web vulnerability scanning using Activity 09/01/2018 05/19/2020 05/19/2020
Scanning and Remediation Acunetix and database vulnerability scanning
using AppDetective Pro and adjust patch and
vulnerability management processes, SLAs,
and performance metrics as required to
addressed additional findings
Splunk Business Adoption Support business units to adopt Splunk for Activity 10/01/2018 03/16/2020 03/31/2020
their FISMA systems by generating Splunk
reports, alerts, and/or dashboards to help
address their auditable events.

209
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
Security Tools Annual Perform annual upgrade of security tools to Activity 10/01/2018 09/30/2019 09/30/2019
Upgrade (FY19) the latest major version: Symantec Endpoint
Protection (SEP), Splunk, IBM BigFix, Cisco
FirePower, and Tenable SecurityCenter in
FY19.
Active Directory Upgrade and Upgrade to AD and Cleanup of GPO Activity 10/10/2018 09/30/2019 09/30/2019
Cleanup (FY18)
ITIOSS Contract Re-compete Prepare for and award service contract for the Activity 01/01/2019 11/01/2019 12/31/2019
(FY20) full array of O&M and D,M&E services for
FY20 and beyond.
Annual Asset Inventory (FY19) Conduct Annual Physical Inventory for FY19 Project 03/08/2019 09/01/2019 09/15/2019
for PBGC
Cisco SourceFire IDS/IPS Upgrade Cisco SourceFire IDS/IPS in advance Activity 05/01/2019 05/01/2020 05/15/2020
Upgrade of EoSL
HP and Serena TCO Tools HP and Serena TCO Tools Annual Activity 06/30/2019 11/30/2019 11/30/2019
Annual Upgrade (FY19/FY20) Upgrade (FY19/FY20)
JRE Upgrade Upgrade to current version of JRE on servers Activity 09/18/2019 01/31/2020 01/31/2020
Servers/Workstations (FY20) wherever possible, and patch legacy versions
of JRE if still required.
Security Tools Annual Perform annual upgrade of security tools to Activity 10/01/2019 09/30/2020 09/30/2020
Upgrade (FY20) the latest major version: Symantec Endpoint
Protection (SEP), Splunk, IBM BigFix, Cisco
FirePower, and Tenable SecurityCenter in
FY2020.
Headquarters Relocation Build out GSA leased facility and relocate staff Project 01/01/2020 07/01/2021 08/01/2021
and end-user IT equipment there

Annual Asset Inventory (FY20) Conduct Annual Physical Inventory for FY20 Project 03/08/2020 09/01/2020 09/15/2020
for PBGC
HP and Serena TCO Tools HP and Serena TCO Tools Annual Activity 06/30/2020 11/30/2020 11/30/2020
Annual Upgrade (FY20/FY21) Upgrade (FY20/FY21)
Upgrade Cisco Nexus Core Upgrade Nexus 7000 and 7010 Cisco Nexus Project 07/01/2020 03/30/2021 03/30/2021
Routers Routers in the datacenter.
JRE Upgrade Upgrade to current version of JRE on servers Activity 09/18/2020 01/31/2021 01/31/2021
Servers/Workstations (FY21) wherever possible, and patch legacy versions
of JRE if still required.
Security Tools Annual Perform annual upgrade of security tools to Activity 10/01/2020 09/30/2021 09/30/2021
Upgrade (FY21) the latest major version: Symantec Endpoint
Protection (SEP), Splunk, IBM BigFix, Cisco
FirePower, and Tenable SecurityCenter in
FY2021.
Annual Asset Inventory (FY21) Conduct Annual Physical Inventory for FY21 Project 03/08/2021 09/01/2021 09/15/2021
for PBGC
HP Server Modernization Modernize the HP server infrastructure which Activity 01/01/2022 09/15/2022 11/01/2022
(FY19/20) includes all blade infrastructure and servers,
and standalone servers.

Annual Asset Inventory (FY22) Conduct Annual Physical Inventory for FY22 Project 03/08/2022 09/01/2022 09/15/2022
for PBGC

210
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order
The following is the ITIOD Roadmap, a visual representation of the IT Infrastructure Program Registry for
items of significant end-user or PBGC business impact (as of March 2019):

211
Pension Benefit Guaranty Corporation
IT Infrastructure Operations Support Services (ITIOSS)
O&M and DM&E Task Order

You might also like