Binary Scaling of 2.5-km CONUS Gridded MOS GRIB2 Files
MDL will soon begin producing Gridded Model Output Statistics (GMOS) over the CONUS at 2.5-km grid
resolution via NCEP's parallel production system. The 2.5-km grid is double the resolution of the
current 5-km CONUS GMOS, resulting in more than 3 times the amount of data. With the increase in data, concerns have been raised
as to if this will create problems when products begin to flow to the
NWS Telecommunications Operations Center (TOC).
In order for MDL products to be properly routed to the National Digital Guidance Database (NDGD) and across
the Satellite Broadcast Network (SBN), the NWS Telecommunication Operations Center (TOC) requires GRIB2 files to
contain WMO super headers and individual GRIB record headers. Super headers allow the TOC to route multiple grids in one
GRIB2 file to NDGD. Individual headers are used to route single grids to the SBN.
The current limitation is that no super header can contain more than 20MB of data. For Gridded MOS products, WMO super headers
are organized by element and valid period. The valid periods contain forecast projections for days 1-3 (short-range) and days 4-7
(extended-range). Several GMOS elements (Temperature, Dewpoint Temperature, Wind Speed, Wind Direction, Wind Gust, Sky Cover, and
Relative Humidity) forecast at a 3-hourly resolution out to 192 hours. Therefore, these elements contain many grids per valid
period and in combination with the grid resolution increase, GRIB2 files are exceeding the 20MB data limit.
In order to reduce the file size of GRIB2 files, MDL will use binary scaling when packing 2.5-km NDGD
data into GRIB2. Previously, MDL used only decimal scaling which did not affect the precision of the forecast guidance.
Decimal scaling simply multiplies the data by powers of 10 (i.e. any real value forecast will be packed as integer).
The following equation describes the scaling used in GRIB2 packing.
where X = packed value; Y = actual value; D = decimal scale factor; B = binary scale factor. The
C function, NINT, allows the value to be rounded to the nearest integer,
instead of truncating. As shown from the equation above, binary scaling is performing division arithmetic on the data which will introduce a loss of
precision.
MDL will use binary scaling when packing GRIB2 for the elements listed in the table below. Click on the element name
to see animated GIF images comparing non binary scaling vs. binary scaling.
* Wind Direction data are shown in the Wind Speed images as wind barbs.
Binary scaling is not performed on Wind Direction forecasts. To decrease the
file size of wind direction GRIB2 files, only decimal scaling will be used. Gridded
MOS wind direction forecasts are output to the nearest whole degree. METAR reports wind direction to the
nearest 10's of degrees. Using a decimal scale factor of -1 brings our forecasts back to the original
precision of observations while reducing the size of the GRIB2 files.
Using the scaling factors listed above, we were able to decrease file sizes between 20% - 25%
of their original size for Temperature, Dewpoint Temperature, Maximum Temperature, Minimum Temperature, Wind Speed,
and Wind Gust. File sizes for Wind Direction were reduced about 35%.
|