Changes in METAR data are frequent. ASOS sites are commissioned, station IDs are changed, and military or foreign stations come and go. Sources of information regarding changes include NWS service change notices and examination of decoder logs.
When one becomes aware that a new or updated METAR site is available, several files should be examined and updated as needed. These include metarStationInfo.txt, MTR.goodness, national_category_table.template, afosMasterPIL.txt, and textCCChelp.txt.
Consider the case of an ASOS commissioning notice. In all likelihood, metadata are already in the AWIPS tables. It's quite possible, however, that the information is not up to date. A first step might be to consult one or more of the Web sites listed in the comments section of metarStationInfo.txt, to wit:
# Good sources of station information include # NWS/OOS: http://www.nws.noaa.gov/tg/siteloc.shtml # FAA ASOS: http://www.faa.gov/asos/asos.htm # Unidata: http://www.unidata.ucar.edu/staff/chiz/stations/stations.shtml # AFWA: ftp://ncardata.ucar.edu/datasets/ds900.0/data/1999.01 # AirNav, linked to the Unidata site, is excellent for U.S. sites. # AES Canada: http://www.climate.weatheroffice.ec.gc.ca/prods_servs/metstat_e.html
File metstat1.txt at the AES site is large, but useful.
grep for the station ID in metarStationInfo.txt, national_category_table.template, and textCCChelp.txt (trailing 3 characters). Also, "build" a 9-character PIL from the national_category entry + "MTR" + 3-char ID; you should find this in afosMasterPIL.txt. For an ASOS, you might find an old MTT entry in the latter that should be removed.
If the geographic information needs to be changed, check in MTR.goodness and either update the entry or remove it. Localization will pick up any stations in metarStationInfo.txt that aren't in MTR.goodness and set them up for plotting.
While we don't currently use the station name fields found in metarStationInfo.txt, it's a good idea to bring them up to date, as well.
Keep in mind that all of the files mentioned in this write-up (except for those in the Ingest section) are in the NDM set maintained by Fran Curnow. Updated information should be forwarded to her. See the NDM Web site for more information and a contact link for Fran.
Sometimes, the data you want are simply not getting into the system. For example, Guam may want to see plots of Japanese data, while sites in Alaska are interested in Russian obs.
Assuming that the files of interest are being uplinked to the SBN, a couple of files will need to be modified to get them onto the AWIPS data disks for processing. Let's use a Russian METAR collective, SARA31, as an example.
Disposition of incoming data is governed by ~fxa/data/acq_patterns.txt on ds1. This file is created by -auxFiles localization from /data/fxa/nationalData/acq_patterns.template. Examining that file reveals several entries that direct surface observations files to where the METAR decoder will see them:
TEXT ^S[AP]US[78]0.* /point/metar/Raw TEXT ^S[AP]US4.* /point/metar/Raw TEXT ^S[AP]U[CEMW]6.* /point/metar/Raw TEXT ^SAAK7.* /point/metar/Raw TEXT ^S[AP]CN[345].* /point/metar/Raw TEXT ^SACN8[5679].*KWBC /point/metar/Raw TEXT ^SACN9[0-7].*KWBC /point/metar/Raw TEXT ^S[AP]MX[4].* /point/metar/Raw TEXT ^S[AP]DR3*.* /point/metar/Raw TEXT ^SAPA12.* /point/metar/Raw TEXT ^SAPA[34][01].* /point/metar/Raw TEXT ^S[AP]XX60.* /point/metar/Raw
None of these patterns matches SARA31. Thus, we'll need to add a line
TEXT ^SARA31.* /point/metar/Raw
or perhaps something more generic, depending on what other bulletins may be desired. After modifying acq_patterns.txt, find the acqserver process that's ingesting the data you want (search the log files), and kill it. A new acqserver will fire up and read the updated pattern file.
Now that we have the data where the decoder can see 'em, we still need to tell the decoder to look for the files. This is goverened by ~fxa/data/METARPatterns.txt (on as1, and as2 for failover purposes). Add an appropriate line (or more), and adjust the patterns count on the first line, then restart the MetarDecoder process as described below.
Of course, these files need to be installed somewhere for testing. It's always a good idea to keep the old version around, for a while at least. An effective scenario goes like this, using the METAR decoder as an example. We start logged into as1-xxx as user fxa.
as1-xxx{fxa}10: cd data
as1-xxx{fxa}11: ll metarS*
-rwxr-xr-x 1 fxa fxalpha 225029 Aug 24 16:10 metarStationInfo.txt
as1-xxx{fxa}12: cp -p metarStationInfo.txt metarStationInfo.txt.990824
as1-xxx{fxa}13: ftp... [copy the new file from wherever it is]
as1-xxx{fxa}14: proc Met
fxa 5732 5729 0 Feb 4 ? 202:25 /awips/fxa/bin/MetarDecoder as1-xxx/4171/5729 1074472440
as1-xxx{fxa}15: kill 5732
as1-xxx{fxa}16: proc Met
fxa 23914 5729 25 21:53:58 ? 0:00 /awips/fxa/bin/MetarDecoder as1-xxx/4171/5729 1074472440
As suggested above, periodically, it's a good idea to go to the as1 logs and see what's missing. When the METAR decoder processes data for which no entry is found in metarStationInfo.txt, it logs a message like this:
15:57:19.794 DM_METARStationInfo.C PROBLEM: NCF_FAIL: KJKJ not in dictionary
Here's a log-search example on the same system.
as1-xxx{fxa}21: logs
as1-xxx{fxa}22: ll Met*
-rw-r--r-- 1 fxa fxalpha 1665877 Feb 15 21:53 MetarDecoder5732as1-xxx001546
-rw-r--r-- 1 fxa fxalpha 6360 Feb 15 21:56 MetarDecoder23914as1-xxx215359
as1-xxx{fxa}23: grep dict MetarDecoder5732as1-xxx001546|cut -c50-60|sort -u
AIL: CYLK n
AIL: K3A6 n
AIL: KJKJ n
AIL: KOMH n
AIL: KTNB n
AIL: KW99 n
If you can find metadata on any of these, it should be added to metarStationInfo.txt (and the related files listed above). What we hope to see, of course, is the elimination of such messages, and many of these in an "old" log should be addressed by an updated metarStationInfo.txt.
Note that the decoder goes ahead and stores the data, even if it has no metadata, tagging them with (-1,-1) gographic coordinates. Though most sites won't see this, the display will dutifully plot these west of Africa, if it's on the map.
Usually, we want to plot these observations. A new station will automatically plot, based on the latitude and longitude from metarStationInfo.txt recorded in the netCDF file by the decoder. Of course, you probably want it on the map; a -station localization will take care of that. (Make sure you have the updated station file on the workstation before you run the localization.)
METARs are also processed by the collective text decoder on as2. They use information in national_category_table.dat (found in ~fxa/data/localizationDataSets/<LLL>). This file is made from national_category_table.template by localization, but differs in only one line:
AAA @@@
where the @@@ is replaced by the local AFOS CCC.
For testing, it's reasonable to do this by hand.
Again, the test scenario is to put the new file in place, find the
CollDBDecoder process (proc Coll), and kill it. After a
while, check the log file for missing station messages. Here, the log
message is like this:
16:01:53.553 CollDBDecoder.C PROBLEM: NCF_FAIL to map XXX to CCC: MMMS
and, again, a grep, cut, and sort method is effective:
as2-xxx{fxa}5: logs
as2-xxx{fxa}6: ll Coll*
-rw-r--r-- 1 fxa fxalpha 3704962 Feb 15 22:19 CollDBDecoder4836as2-xxx000526
as2-xxx{fxa}7: grep CCC CollDBDecoder4836as2-xxx000526 |cut -c 60-70|sort -u
o CCC:
o CCC:
o CCC:
o CCC: 75
o CCC: CWXC
o CCC: CXBP
o CCC: CXDW
o CCC: CXEG
o CCC: CXJM
.
.
.
You'll find a lot of these, some of which are bogus like
DUPE or NIL.
In this case, we'd want to add WXC, XBP, XDW, etc. to national_category_table.template, if we know where they are. A list of Canadian CCCs can be found at www-sdd.fsl.noaa.gov/~jwake/AWIPS/StationIDs.html.
Any changes of this sort should be reflected in afosMasterPIL.txt and in textCCChelp.txt. These need to be installed and tested on a workstation. Run the former through localization (-text option), then stop and start the text workstation.