U.S. DEPARTMENT OF COMMERCENational Oceanic and Atmospheric Administration
NATIONAL WEATHER SERVICE
Silver Spring, Md. 20910 W/CM2l
U.S. DEPARTMENT OF COMMERCEFebruary 23, 1990
TO: All Holders of Operations Manual
SUBJECT: Transmittal Memorandum for Operations Manual Issuance 90-1
1. Material Transmitted:
WSOM Chapter A-63, Program Evaluation.
2. Summary:
This chapter has been substantially rewritten and reorganized to reflect changes in policy and procedures. The chapter is being reissued with a new title. It assigns responsibility, provides guidelines, and sets reporting procedures for the Weather Service Evaluations Program.
3. Effect on Other Instructions:
Supersedes WSOM Chapter A-63, transmittal issuance 76-21, September 15, 1976, and all Regional Operations Manual Letters filed with it.
Elbert W. Friday, Jr.
Assistant Administrator
for Weather Services
PROGRAM EVALUATION
Table of Contents:
1. Purpose
2. General
3.1 Weather Service Headquarters (WSH) Responsibility
3.2 Regional Headquarters (RH) Responsibility
3.3 Weather Service Forecast Office (WSFO) Responsibility
5. Evaluations
5.1 Evaluations at WSFO's
5.2 Evaluations at Weather Service Offices (WSO), Weather Service Meteorological Observatories (WSMO), and Center Weather Service Units (CWSU)
5.3 Evaluations at Flight Service Stations (FSS)
6. Reports
6.1 Reports on Evaluations at WSFO's
6.2 Reports on Evaluations at WSO's, WSMO's, and CWSU's
6.3 Reports on Evaluations FSS's
6.4 Summary Report on Evaluation Activities
1. Purpose. This chapter establishes policy on the authority and responsibility of managing the Weather Service Evaluations Program. It provides guidelines and establishes the function of Weather Service Evaluations Officer.
2. General. The success of the National Weather Service (NWS) mission is directly related to the quality and timeliness of products and services produced by each office. To maintain a high level of service, periodic evaluation of programs is necessary.
3. Responsibility. Responsibility and management of the evaluations program will be in accordance with policy and instructions set forth in WSOM Chapters A-61, D-26, D-82, this chapter, and the memorandum of Agreement Between the Federal Aviation Administration (FAA) and the National Oceanic and Atmospheric Administration for the Establishment of Working Arrangements for Providing Aviation Weather Service and Meteorological Communications.
3.1 Weather Service Headquarters (WSH) Responsibility. WSH is responsible for establishing policy and procedures for the evaluations program. This is done through the Services Evaluation Branch (W/OM21) in coordination with the Aviation Services Branch (W/OM13), both under the Office of Meteorology.
3.2 Regional Headquarters (RH) Responsibility. Regional Directors are responsible for the management of the program within their regions. Specific responsibilities include monitoring the overall evaluations program, establishing procedures for evaluation visits, reviewing evaluation reports submitted by area managers (AM), and preparing appropriate reports for WSH. Regions shall notify W/OM21 with the names of designated focal points.
3.3 Weather Service Forecast Office (WSFO) Responsibility. AM's are directly responsible for program management and evaluation in their areas of responsibility. This includes evaluating the services and products of the WSFO and all Weather Service Offices (WSO), Weather Service meteorological Observatories (WSMO), and Center Weather Service Units (WSU) evaluation of FAA offices having pilot weather briefing responsibility: and evaluating the coordination of forecasts and warnings among WSFO's, River Forecast Centers, and WSO's. Specific responsibilities are to:
Evaluate the timeliness, quality, and effectiveness of WSFO guidance and user products and services.
Evaluate the timeliness, quality, and effectiveness of WSO, WSMO, and CWSU products and services. This may include some limited quality control of weather observations.
Evaluate the timeliness, quality, and effectiveness of FAA pilot weather briefings and related activities.
Ensure the evaluation of all offices is done effectively and impartially.
Provide the people, time, and resources needed for a successful evaluations program.
Review all evaluation reports and forward them, or a summary report, to RH.
Correct significant deficiencies under the AM's control or notify the appropriate organization responsible.
4. Weather Service Evaluations Officer (WSEO). In order to effectively conduct an evaluations program, WSFO's shall have a WSEO selected from the forecast staff by the AM with the approval of RH. The WSEO shall serve as a staff assistant to the AM. The AM is responsible for ensuring that there is sufficient time available to fulfill WSEO duties.
Under the AM's supervision, the WSEO is responsible for:
Field office visits to WSO's, WSMO's, CWSU's, and Flight Service Stations (FSS) in the AM's area of responsibility (FSS is meant to include Automated Flight Service Stations throughout this chapter). Since these evaluation visits are the key to the Weather Service Evaluations Program, WSEO's are required to visit every facility at least once each calendar year. Two visits per year are preferred and, at the discretion of the AM, can be made by other personnel.
Assisting the AM in evaluating programs at the WSFO.
Certifying pilot weather briefers and making spot checks of briefing performance. The WSEO will also schedule proficiency examinations based on requests of FAA or NWS facility managers, supervisors, or other evaluations personnel (Chapter D-82).
Reporting problems found during evaluations to the AM.
Providing support to weather-related transportation accident investigations (Chapter D-90).
Providing assistance to FSS's in using NWS products and forecasts.
Providing advice and consultation on forecast techniques and other procedures to WSO, WSMO, and CWSU personnel.
The WSEO shall not evaluate the performance of the AM, deputy meteorologist in charge (DMIC), or meteorologists in charge/officials in charge (MIC/OIC) of WSO's, WSMO's, and CWSU's. THE WSEO SHALL BE CONCERNED WITH THE QUALITY OF THE PRODUCTS AND SERVICES OF THE FACILITIES, NOT THAT OF INDIVIDUAL PERSONNEL (except as noted in section 4c above).
5. Evaluations. Evaluations shall be in compliance with guidelines outlined in pertinent WSOM chapters and other directives. General guidelines are outlined in this chapter. Detailed procedures shall be developed by RH's, AM's, and WSEO's. Evaluations shall be conducted in an impartial manner and not interfere with field office operations. All MIC/OIC's shall cooperate with the person making the evaluation.
Real-time and near real-time evaluations are also important. AM's may want to spread these responsibilities among several staff members in addition to the WSEO (i.e., DMIC, lead forecasters, etc.).
Public weather forecasts and warnings.
Specialized forecasts and programs (aviation and pilot weather briefings, agriculture, air pollution, fire-weather, marine, etc.).
Hydrologic programs including flash flood watches and warnings.
Usefulness and timeliness of products supplied to WSO's, WSMO's, and CWSU's.
Preparedness and spotter program execution.
Communications and equipment maintenance.
Forecast procedures and computer applications.
Station Duty Manual and other instructions to employees.
Routine WSFO performance evaluation is encouraged. However, formal evaluations are best done by RH personnel.
5.2 Evaluations at Weather Service Offices (WSO), Weather Service Meteorological Observatories (WSMO), and Center Weather Service Units (CWSU). Offices shall be notified in advance of evaluation visits. A checklist or narrative evaluation report shall be completed and preliminary findings shall be discussed with the MIC/OIC. The entire operation should be evaluated, including (as applicable):
Warning procedures and their utility and timeliness.
Procedures for local forecasts.
Pilot weather briefings.
Quality and timely dissemination of surface, radar, and upper air observations.
Coordination activities with the WSFO.
Disaster preparedness activities.
Hydrologic operations (use of guidance and the dissemination of forecasts).
Station Duty Manual and other instructions to employees.
Communications and equipment maintenance.
Computer applications.
Specific checklists and/or guidelines for narrative reports should be developed by the regions.
5.3 Evaluations at Flight Service Stations (FSS). FAA FSS's shall be notified in advance of evaluation visits. A checklist or narrative evaluation report shall be completed and preliminary findings discussed with the official in charge. The WSEO shall evaluate the overall effectiveness of the station pilot weather briefing program, including:
Quality of individual pilot weather briefings.
Adequacy of the quality assurance program for briefers at automated FSS's.
Pilot weather briefing display only as it relates to federally generated or provided graphics and products.
Detailed instructions are contained in WSOM Chapters D-26 and D-82. Specific checklists and/or guidelines for narrative reports should be developed by the regions.
6. Reports. Documentation on evaluation activities shall include reports on evaluations at the WSFO: reports on regular and special visits to WSO's, WSMO's, CWSU's, and FSS's; and summary reports on evaluation activities.
6.1 Reports on Evaluations at WSFO's. The AM shall provide RH with reports on evaluation activities at the WSFO.
6.2 Reports on Evaluations at WSO's, WSMO's, and CWSU's. An evaluation report shall be prepared following each evaluation visit. Copies shall be sent to the MIC/OIC of each office visited. The AM shall send the originals, with comments as appropriate, to RH. regions have the option to require summary reports of significant information in lieu of the individual evaluation reports.
6.3 Reports on Evaluations at FSS's. An evaluation report shall be prepared following each evaluation visit. The Pilot Weather Service Evaluation Report, or other report forms as outlined in WSOM Chapter D-26 and associated OML's, shall be used. Copies shall be sent to the manager of each facility visited. The AM shall send the originals, with comments as appropriate, to RH. Regions have the option to require summary reports of significant information in lieu of the individual evaluation reports.
6.4 Summary Report on Evaluation Activities. RH's shall prepare annual summary reports on evaluation activities within the region. The reports should be brief and for the most part should contain a general, overall look at the program. Specifics should be included only if WSH action is needed. Reports should cover activities such as current WSEO's, training, communications, quality control, etc., concerns and problems of field offices; resolution of problems and deficiencies; any procedural changes resulting from service evaluation; and a list of those offices that were not visited at least once. Send the reports to WSH, W/0M21, by February 15.
7. Follow-up Actions. When minor deficiencies are found, on-the-spot follow-ups should be made. The AM is responsible for follow-ups and additional actions needed for more significant deficiencies. In cases of more serious or recurring service problems, it may be necessary for RH's to participate. If the service problem is national in scope, WSH may take part in corrective actions.
WSOM Issuance
90-1 2-23-90