Meteogram Tracking Tool Evaluation
In May of 2014, NWS forecasters from all four CONUS Regions joined product developers, training officials, and OPG staff to conduct a rigorous evaluation of a potential tool for AWIPS-2. The tool is known as the Tracking Meteogram (TM) Tool, an application developed by NASA SPoRT in collaboration with the NWS Meteorological Development Lab.
The aim of this Operational Readiness Evaluation (ORE) was to assess the tool’s practical usability, usefulness for decision making, and potential for workload impacts, from the perspective of NWS forecasters. Forecasters from four NWS regions were placed in a simulated WFO environment, and then led through a diverse array of warning and forecast scenarios, using both archived cases and live data sets. Overall, NWS forecasters unanimously endorsed the TM as a useful tool that could add genuine, unique value to several practical forecaster tasks. A few key effective practices, along with specific findings and recommendations from the evaluation, are identified below.
A. Effective Practices
Effective Practice 1: The TM is very useful as a forecaster decision aid for tracking and displaying trends with meteorological features (e.g., mesocyclone, reflectivity core, cloud-top temperatures, and total lightning data).
Effective Practice 2: The TM is useful at interrogating gridded model output.
Effective Practice 3: The TM is useful for mesoanalysis applications such as monitoring trends in instability, convective inhibition, shear, and rainfall rates.
Effective Practice 4: The TM is useful at providing the forecaster an efficient way to extract and communicate meteorological information that is critical to NWS’ core partners.
Effective Practice 5: The TM would be useful in post event analysis to evaluate how trends in data were related to warning decisions (to improve warning process).
B. Findings and Recommendations
Finding 1: Forecasters found it difficult to assimilate and interpret the TM data when plotting four or more meteograms concurrently. Forecasters also observed that the depiction of numerous meteograms impacted system performance.
Finding 2: To utilize the TM to monitor in-situ changes over a fixed location, it was necessary for the user to manually stack the TM’s trace circles.
Finding 3: Forecasters discovered that it is not possible to store a loaded TM location into an AWIPS-2 procedure.
Finding 4: Functionality does not exist to set and plot a user-defined threshold value superimposed on the TM graph. Forecasters identified two specific applications of value for this feature: (1) as a visual reference on the trace to aid the forecaster’s analysis and forecasting decision process, and (2) as an optional trigger point to activate a notification alert message when exceeded.
C. Lessons Learned about the ORE Process
Many topics and observations emerged from written surveys and oral discussions, but the most significant positive take-aways concerning the ORE process fell into four broad categories:
- Integration of the VLab Development Environment in the ORE Process
- Diversity and Realism of Cases Used for Evaluation Sessions
- Emphasis on Human Factors in Evaluating Usability and Usefulness
- Blend of Forecasters, Trainers, Developers, and Support Staff Who Participated
D. Recommendation for Field Implementation
All participating NWS forecasters unanimously endorsed the TM as a useful tool that will add unique value to several specific diagnostic and predictive tasks without posing adverse impacts on operational workflow or forecaster workload. While NWS forecasters believe it will be received well and readily adopted by forecasters in its current version, they recommend two enhancements prior to widespread field implementation. First, it is recommended that the stationary mode option, AWIPS-2 procedure capability, and tendency information be added to the TM’s functionality. Second, a brief TM training video and one-page (front and back) informational handout should accompany the TM’s implementation. Based on the results of this ORE, the OPG recommends implementation of the Tracking Meteogram into the AWIPS-2 baseline without reservation.