Metadata or 'the documentation of data' serves the purpose of making data discoverable, usable and understandable. A variety of metadata standards and formats have been developed over time to support data discovery and data documentation.

NOAA's directives for metadata, as part of its data documentation plan, include NOAA Administrative Order 212-15 and NOAA's Environmental Data Management Committee's (EDMC) Data Documentation Planning Directive. The Data Documentation directive "establishes ISO 19115 Parts 1 and 2 and a recommended representation standard (ISO 19139) for documenting NOAA's environmental data and information."

• ISO Technical Committee 211 - Geographic information/Geomatics "aims to establish a structured set of standards for information concerning objects or phenomena that are directly or indirectly associated with a location relative to the earth."
• ISO 19115:2003 Geographic Information - Metadata Order Information
• ISO 19115:2003 Geographic Information - Metadata Workbook (2.5 MB) - Guide to Implementing ISO 19115:2003(E), the North American Profile (NAP), and ISO 19110 Feature Catalogue.
• ISO 19115:2003 Geographic Information - Metadata - Biological Extensions Workbook (2.75 MB) - Guide to Implementing ISO 19115:2003(E), the North American Profile (NAP), and ISO 19110 Feature Catalogue with Biological Extensions.
• ISO 19115-2:2009 Geographic Information - Metadata - Part 2: Extensions for imagery and gridded data Workbook (2.99 MB) - Guide to Implementing ISO 19115-2:2009(E), the North American Profile (NAP), and ISO 19110 Feature Catalogue.

MDL is supporting the NextGen program, and particularly NCWCP's development of operational OGC-based Web Services (Web Coverage Service (WCS), Web Feature Service, (WFS), Web Map Service (WMS) as well as discovery services (i.e. metadata records). For the metadata portion of NCWCP web services, MDL has the primary responsibility for NCWCP in developing an initial set of metadata records for aviation-focused weather products as identified by the Federal Aviation Administration (FAA). MDL developed a repeatable process for creating these records to ensure consistency, compliance and completeness.

• Step 1:Translate raw data from its native format (GRIB, BUFR, Text) into its Web Services output format (NetCDF, XML)

  • The easiest way to do this is request it from the WCS or WFS server
  • This includes inserting raw data into the server if necessary. Details on this can be found elsewhere
  • Note that this can be accomplished using the $XMLMAKE/testing/mdl.py script included with the xmlmake package, but there are many ways to do this. We'll refer to the directory that xmlmake package resides in as $XMLMAKE

• Step 2: Modify the $XMLMAKE/consolidate.py script to point to the directory in which the NetCDF files reside.

  • Note that the script defaults to looking in the latest directory created by the $XMLMAKE/testing/mdl.py script (e.g. /tmp/201507170000)

• Step 3: Create ncml from the netcdf files by running the 'ncdump -x' command.

  • This functionality is part of the Linux NetCDF libraries
  • These files are stored in the $XMLMAKE/ncml/ directory

• Step 4: Script takes the ncml files and submits them individually to https://ngdc.noaa.gov/docucomp/recordServices/resolve to get the initial iso19115-2 XML files.

  • These files are stored in the $XMLMAKE/xmls/ directory

• Step 5: We need to create XSL files using info from the ncml XMLs, plus additional information from the Prototype response to a describe coverage request

  • This is done by the $XMLMAKE/consolidate.py which calls the $XMLMAKE/xslmake.py script which generates the xsl files
  • The info from the Prototype responses mostly relates to the grid size and levels.
  • These files are stored by the script in the $XMLMAKE/xsls/ directory

• Step 6: Translate the iso19115-2 files using the XSLs we created

  • The $XMLMAKE/consolidate.py does this automatically using xmllint
  • These files are stored by the script in the $XMLMAKE/trans/ directory

• Step 7: Files should be manually sent to the SME, who adds, replaces, or removes content as needed from the file. Note that "content" refers to anything not involving the overall XML structure:

  • the text contained by the XML
  • tag attributes and values
  • addition or removal of multiples of the same XML tags such as keywords or units tags

• Step 8: Send to metadata point of contact for checking

  • Validate using NetBeans, XML Spy, or other appropriate program
  • Verify that score is 100% in the rubric (https://www.ngdc.noaa.gov/docucomp/recordServices using "ISO to Rubric" option)
  • Other checks as needed

• Step 9: Metadata point of contact makes any XML structure changes

• Step 10: Send back to SME for content changes as needed

• Step 11: Repeat last 3 steps until file is complete

• Step 12: Enter these files into the reg/rep

  • This is done through the web interface on the server using that server's particular protocols

ISO metadata standards support different agencies and requirements in specific ways:

• ISO 19115-2 documents are the preferred standard for organizations actively implementing ISO Metadata as it includes all of the elements of ISO 19115 as well as additional elements that are relevant to many geospatial data sets (raster, imagery, GPS, monitor stations, instruments, etc.).
• ISO 19139 is an XML schema that specifies the format of the metadata record and is used by application developers to implement the standard.
• The upcoming, ISO 19115-1, which was developed after ISO 19115-2 and is an updated version of the base standard (ISO 19115). In addition to changes to the structure of some base elements and the content of some domains, ISO 19115-1 expands upon the former standard by providing more fields to describe geospatial data services (ISO 19119), multi-dimensional gridded datasets, modeling results, etc., and enabling entity/attribute descriptions developed using ISO 19110: Feature Catalog to be associated with or integrated into the metadata record.