Meeting_Body
OPERATIONS, SAFETY, AND CUSTOMER EXPERIENCE COMMITTEE
JUNE 18, 2020
Subject
SUBJECT: METRO RED LINE SCADA REPLACEMENT
Action
ACTION: APPROVE RECOMMENDATIONS
Heading
RECOMMENDATION
Title
AUTHORIZE:
A. The Chief Executive Officer to increase the Contract Modification Authority (CMA) specific to Contract No. OP39603035 with ARINC Control and Information Systems (ARINC) in the amount of $3,357,496 increasing the total authorized contract amount from $15,551,028 to $18,908,524.
B. The Chief Executive Officer to execute contract modifications up to the Board-approved CMA in an amount not to exceed $18,908,524.
C. The purchase of additional coverage on the existing $15,000,000 supplemental project insurance in excess of ARINC’s limited liability in an amount not-to-exceed $700,000. This action increases the total coverage cost from $1,449,000 to $2,149,000;
D. An extension to the period of performance of Contract OP39603035 to December 31, 2021 to allow for SCADA-related work on the CLAX/AMC Station to be completed under this Contract.
Issue
ISSUE
This request addresses the following critical needs related to Metro’s SCADA system:
a) The need to implement and integrate a SCADA system in support of the Crenshaw/LAX Airport Metro Connector (AMC) Station.
b) The need to implement and integrate a SCADA system in support of the Little Tokyo station that is part of the Regional Connector project, which is anticipated to occur in the Fall of 2020;
c) The need for acquisition, installation and integration of new SCADA equipment, configured in a Virtual Machine (VM) and Virtual Desktop Infrastructure (VDI) environment to bring the Metro Red Line to compliance with Metro’s updated SCADA systems standards; This will also bring all other Metro Rail lines to compliance.
Background
BACKGROUND
Contract No. OP39603035 was approved by the Board in November 2013, after a competitive procurement, for equipment and services for the replacement of the obsolete Red Line SCADA System. ARINC was chosen as the most technically qualified firm offering the lowest price. Modification No.1 was issued on July 14, 2014 to include integration of the Foothill and EXPO-II light rail expansion projects to the project. Modification No. 2, issued on November 14, 2014, was an administrative action which did not change the contract value. Modification No. 3 was issued on June 19, 2015 to include modifications to technical specifications and to purchase an additional project specific professional liability insurance associated with the increased contract scope. Modification No. 4 was issued on September 22, 2016 to include integration of the Crenshaw/LAX light rail expansion project to the Contract and to purchase an additional project specific professional liability insurance associated with the increased contract scope.
Recommendations (A) and (B) will allow sufficient contract authority for the implementation and integration of SCADA systems at the CLAX/AMC Connector station and the Little Tokyo station to proceed according to the project’s schedule. It will also enable Metro staff to bring the existing SCADA System into compliance with current Metro Information Technology Hardware and Security standards and allow for compliance with recommendations from the recent MASD Report No. 19-OPS-P03. This will provide the SCADA system the capability to seamlessly and effortlessly support all upcoming Rail expansion projects such as the Purple Line and Foothill Extension.
Recommendation (C), authorization for the purchase of additional project insurance coverage, will provide protection in excess of ARINC’s limited liability specifically for the increased contract scope required for Little Tokyo station and CLAX/AMC station, should it be needed.
Recommendation (D), authorizes an extension in the period of contract performance to December 31, 2021 that will ensure ample time is available for completion of SCADA installation and integration work at Little Tokyo station, and CLAX/AMC station. It will also allow time for resolution of issues such as software defect corrections or software enhancements, if and when required.
Discussion
DISCUSSION
Metro Rail Operations and Wayside Maintenance rely heavily on the SCADA system to provide supervisory and control functions that are essential for the safe, reliable and efficient operation of the Metro rail lines. These functions include centralized control and/or monitoring of train movement, traction and auxiliary power, fire detection and suppression, gas detection, emergency tunnel and ancillary ventilation, elevators and escalators, radio, emergency telephone, Transit Passenger Information System (TPIS) and intrusion.
Enhancements to the existing system are necessary to comply with MASD Audit recommendations and ITS security standards. Some of the enhancements include bringing outdated SCADA systems into compliance; deployment of a local Intrusion Detection System (IDS) and enhancing SCADA system logical security and firewall monitoring capabilities.
Adherence to the latest ITS infrastructure and architecture recommendations will enable a seamless and enhanced integration of SCADA with other enterprise services which include:
• State of the art cyber security
• System backup and disaster recovery
• Nextrip Rail
• BOC/ROC integration
• Emergency backup control center
Next generation intrusion detection/prevention systems and firewalls will be deployed to provide sophisticated cyber security enabling a level of threat detection, threat prevention, and management necessary to respond to increasing cyber threats against our Nation’s critical infrastructure. These systems will also provide the configuration management and forensic data that are essential for effective security oversight and audit.
Adoption of the latest ITS virtual application and desktop infrastructure standards will greatly simplify maintenance, system expansion and in assuring 100% compliance with Metro Information Security Policy. This architecture will also support the integration of bus and rail into a multi-modal operations center with minimal modification that will be required.
Adoption of the latest ITS data storage architecture will enable SCADA system configuration and alarm/event database data to be integrated into the ITS automated backup and disaster recovery system ensuring the ability to restore system operation on any level in a minimum amount of time.
Determination_Of_Safety_Impact
DETERMINATION OF SAFETY IMPACT
Approval of these items would greatly enhance the safety and reliability of the Rail SCADA system by enhancing cyber security, system availability and will address Audit findings.
Financial_Impact
FINANCIAL IMPACT
Funding for this effort is included in the approved Life of Project (LOP) budget of project 205038 - Heavy Rail Subway SCADA System Replacement.
Since this is a multi-year project, the Project Manager and Chief Operating Officer will ensure that all related costs are budgeted in future Fiscal Years.
Impact to Budget
The current source of funds for this action is Measure M 2% State of Good Repair. Using this funding source maximizes the project funding allocations allowed by approved provisions and guidelines.
Implementation_of_Strategic_Plan_Goals
IMPLEMENTATION OF STRATEGIC PLAN GOALS
Approval of this recommendation supports the following Metro Strategic Plan Goal 5; Provide responsive, accountable, and trustworthy governance within the Metro organization.
Alternatives_Considered
ALTERNATIVES CONSIDERED
The Board may choose to not approve the requested increase in contract value with ARINC and direct staff to hold an open procurement. This alternative is not recommended. An open solicitation may result in delivery of a new and different SCADA system that may be incompatible with the existing system. This presents training and maintenance difficulties that will negatively affect the overall safe, effective and efficient operation. Award to a new contractor will also present significant schedule and cost risk to the Regional Connector and AMC projects.
Metro must make every effort to comply with current ITS standards and to address the recent Audit findings and recommendations. Compliance with the ITS standard is mandatory and any deviation will adversely affect Metro’s ability to implement cost effective security, enterprise integration and scalability in a timely fashion.
The Board may also decide not to purchase additional supplemental insurance if the Board determines that additional exposure related to the inability to collect damages for ARINC’s professional negligence for their integration activities is an acceptable risk. This alternative is not recommended by Metro Risk Management.
The Board may also reject the request for a contract extension to December 31, 2021. This is not recommended as sufficient time is important for the Little Tokyo and CLAX/AMC Connector Station SCADA systems to be properly implemented and integrated into the existing SCADA system. These activities are considered part of the critical path and any delays will adversely affect the overall project schedule.
Next_Steps
NEXT STEPS
Upon Board approval, staff will issue contract modifications as needed under Contract No. OP39603035 with ARINC on an as needed basis. Staff will also monitor all expenditures associated with this contract modification to ensure remaining within approved cash flow limits for FY21.
Attachments
ATTACHMENTS
Attachment A - Procurement Summary
Attachment B - Contract Modification/Change Order Log
Attachment C - DEOD Summary
Prepared_by
Prepared by: Errol Taylor, Senior Executive Officer, Maintenance and Engineering, (213) 922-3227
Leticia Solis, Deputy Executive Officer, Wayside Systems Engineering and Maintenance (213) 613-2115
David Chu, Director, Wayside Systems, SCADA, (213) 418-3042
Reviewed_By
Reviewed by: James T. Gallagher, Chief Operations Officer, (213) 418-3108
Debra Avila, Chief Vendor/Contract Management Officer, (213) 418-3051
