Lesson
Ensure the accuracy and consistency of an MDSS to build user confidence and trust.
From the experience of Maine Department of Transportation.
September, 2007
United States
Background (Show)
Lesson Learned
The MDSS deployed by MaineDOT in the winter of 2006-2007 offered the DOT and the Scarborough crew a useful winter storm planning tool that supplemented other resources. MaineDOT found their experience overall with the MDSS to be a beneficial one. However, it was apparent from the event reconstruction and interviews with the crews that the influence of this tool on their maintenance and treatment conditions could have been greater had the crews perceived a greater degree of accuracy, reliability, and consistency in the forecasts and treatment recommendations provided. MaineDOT experienced an unusually mild winter, which presented a challenge for weather forecasting tools. Many weather events hovered between snow and rain, thus making it difficult for the MDSS (as well as other weather forecasting tools) to forecast precipitation type, amount, and timing with accuracy. An MDSS must seek to demonstrate accuracy and consistency across all storm types that a DOT might experience if it is to be a trusted tool of the maintenance crew. The following set of lessons learned is provided in support of ensuring a tool that will be perceived as accurate and reliable.
- Configure MDSS treatment recommendations in close consultation with the DOT, and customize them to fit the conditions, needs, and practices of the crews in the district where the MDSS will be used. In providing a treatment protocol for incorporation into an MDSS, MaineDOT decided to select an"average" set of treatment guidelines under a set of weather and pavement temperature ranges that could apply across the state. However, the Scarborough crew tends to apply more chemicals under a given set of weather and pavement conditions than the state average treatments that were configured in the MDSS, so throughout the assessment, the Scarborough crew tended to exceed the initial recommended treatment levels of the MDSS. In the judgment and experience of the Scarborough crew, the MDSS was under-recommending the amount of chemicals that were needed.
The state DOT should work closely with the MDSS vendor to implement a protocol of treatment recommendations that offer the potential for effectiveness and cost savings, consistent with the DOT’s standards for road maintenance. Arriving at an optimal configuration is likely to be an iterative process, with the vendor and the DOT working together to make adjustments, based on the performance of the MDSS after trial periods. The DOT may discover in this process that their existing treatment protocols can be modified to achieve greater efficiencies and savings. - Select the alert topics and alert timing that will be most helpful in making road treatment decisions and to avoid generating too many alerts, which can be distracting The MDSS offered MaineDOT a wide variety of possible alerts covering all possible event types for each forecast point created for the study region. Across the 12 events reviewed in this evaluation, there was an average of 50 alerts per event and about 3.3 alerts per hour based on the criteria specified by MaineDOT. According to the Scarborough crew, this was such a large number of alerts that they found themselves disregarding many of them, and they indicated that at some points the large number of alerts was distracting. In MaineDOT's experience, alerts for each of the forecast points proved not to be necessary, since the multiple alerts often said the same thing. Alerts issued for different forecast points are only helpful when the information at each forecast point is significantly different from others. Other lessons learned regarding the alerts include:
- Configure forecast points that are outside the maintenance area along storm paths to offer improved advance warning of storms
- Create alert wording that is short and clear
- Configure the alerts according to the times of day when the crews are prepared to receive them and make the best use of them
- Provide a mechanism in MDSS treatment recommendations to incorporate the effectiveness of prior treatments, along with observational data, in order to better adjust forward looking recommendations. The MDSS used by MaineDOT was of most value at the front end of a storm. The Scarborough crew evaluated the MDSS before every storm event, along with their other sources of road weather information, and made their decisions whether to pre-treat the road and what type and amount of chemicals to apply. The MDSS was used less often during the course of a storm, in part due to the following reasons:
- The MDSS was not able to factor current conditions into its recommendations,
- Data on crew actions and resulting road conditions was not being integrated into the MDSS
- The crew was too busy fighting the storm to pay attention to the MDSS during the storm
States
None defined
Countries
Systems Engineering
Focus Areas
None defined
Goal Areas
Keywords
None defined
Lesson ID: 2007-00406

Lesson Comments
No comments posted to date