Commit to testing the new systems thoroughly, develop an acceptance matrix to document status of testing, and perform verification and validation before introducing them to support agency’s transportation operations.
Chattanooga Area Regional Transportation Authority's experience in deploying transit ITS
- Develop an acceptance matrix to document the status and amendments resulting from testing, verification, and validation of requirements for ITS projects. A feature of CARTA's systems engineering approach was a strong commitment to thoroughly testing the ITS technologies before introducing them to support operations. CARTA's general approach worked as follows. All the requirements for a system were documented in an acceptance matrix. This matrix included: an identification number (ID) for each requirement, a description of and amendments to each requirement, and remarks from verification and validation exercises. For example, the acceptance matrix for the computer aided dispatch / automatic vehicle location (CAD/AVL) deployment was maintained in an Excel® worksheet that included nearly 800 lines of requirements information. The deployments themselves were divided up into a series of incremental demonstrations, with each demonstration exhibiting more of the final functionality required. Before each demonstration, the contractor was required to submit for CARTA approval an acceptance test procedure that identified the requirements to be met and the method used to verify that the requirements were met. CARTA and the contractor would follow the test procedures during the demonstration, and CARTA would update the requirements matrix with the result and status of each requirement tested. This approach provided a mechanism for (a) ensuring that all requirements were satisfactorily met and (b) monitoring progress towards meeting the requirements.
- Commit to thoroughly testing systems before introducing them to operations. Having a mechanism for testing is not sufficient without a commitment to conducting thorough testing, and there were numerous examples that demonstrated CARTA's commitment. First was CARTA's recognition of the importance of testing. During discussions with the evaluators, CARTA identified testing as an important part of their system engineering process. Second was CARTA's willingness, if necessary, to change plans based on testing results. For example, during testing of the new operations software for its flex-route operations, CARTA determined that the planned new software would not provide the expected benefits. So, CARTA stopped the introduction of that software, selected an alternative that would better meet operational needs, and rescheduled the deployment to take place after the CAD/AVL system was in place. An example of the thoroughness of CARTA's testing is evident in the rollout of the next-stop announcement system. In late 2008, CARTA completed the onboard database and was capable of activating onboard audio and dynamic message sign (DMS) announcements. Rather than proceeding with system-wide operations after verifying the general functionality of the system, CARTA elected to conduct a comprehensive test of the system's functionality for each route. CARTA, using a test bus set up with the test database, drove each route variation in the system with an observer onboard to note any errors in the messages displayed and announcements made. During this testing, CARTA discovered and corrected several errors in the system, including incorrect announcements and announcements triggered at incorrect locations.
- During thorough testing, identify enhancements to the system that would improve its operations. The test-bus end-use testing narrated above also allowed CARTA to identify enhancements to the system that would improve its operation. The system was originally configured to display a stop announcement first, then display the date/time, on the interior DMS prior to displaying information about the next stop. During testing, CARTA discovered that the time spent displaying the date/time sometimes delayed the next-stop information display, particularly when stops were closely spaced. Thus, performing end-use testing of the system prevented exposing the public to these errors, which could have reduced public confidence. Another example of CARTA's commitment to testing was the 2009 roll out of the Automated Vehicle Monitoring (AVM) system. The core infrastructure needed to support AVM—the onboard equipment, the Wireless Local Area Networks (WLAN) for daily data uploads, and the AVM software—was in place early in 2008. However, CARTA decided to focus on implementing systems that would deliver the most direct and visible benefits to riders, such as the next-stop announcements and the next-arrival-time predictions. After these systems came online in December 2008, CARTA shifted focus to rolling out AVM. By January 2009, the AVM system elements were integrated enabling the system database to receive daily uploads of data from the buses. CARTA then conducted internal testing of the AVM system to confirm it was operating correctly before releasing it for use by the mechanics in March 2009.
- Beware of contractor’s resistance to extra time required for thorough testing. Initially, some contractors were resistant to the extra time CARTA required for testing. This resistance was typically overcome during the initial subsystem implementations, when CARTA first applied its thorough testing processes with that contractor. CARTA required that test procedures fully demonstrate all the specified requirements for the systems. CARTA reinforced its commitment to testing by refusing to accept systems that did not pass the documented test procedures. After working with CARTA's testing process, most contractors recognized that this approach, while it might delay final acceptance, helped prevent errors from being found after the system was in use when corrections would have been costlier and more difficult to make.
Author: Haas, R.; E. Perry; J. Rephlo
Published By: U.S. DOT Federal Highway Administration
Source Date: November 2009URL: http://ntl.bts.gov/lib/32000/32600/32672/index.htm
Average User Rating
Lesson of the Month for February, 2011 !
Intelligent Transportation Systems > Transit Management > Operations & Fleet Management > Planning
Intelligent Transportation Systems > Transit Management > Information Dissemination > In-Vehicle Systems
Intelligent Transportation Systems > Transit Management > Operations & Fleet Management > Automatic Vehicle Location / Computer-Aided Dispatch
automated vehicle location, computer aided dispatch, automatic vehicle locator, AVL, CAD, AVL/CAD