The LDAD Gateway is a set of software components that together make up the core data acquisition and dissemination mechanism of the AWIPS LDAD. The LDAD Gateway is split among the AS, DS, and LS servers; i.e. both the Internal and External sides of the Firewall. LDAD Gateway software links the Internal and External sides of LDAD, controlling the transfer of information across the Firewall. The AWIPS workstations, data servers, and application servers are located on the internal side local area network (LAN), while the External side is connected to the outside world. Local data sets that are to be brought into AWIPS are received on the External side of the Firewall, and through control of the LDAD Gateway are brought across the Firewall to the Internal side of LDAD.
The Gateway's many components include:
The User Interface component provides the forecaster access to the LDAD Gateway. Through this graphical interface the user can access all system files, start and stop the LDAD Gateway automatic processes, and incorporate new data sources into the system.
For more on the LDAD UI, see Chapter 8.
The Scheduler's function is to perform certain data acquisition/dissemination events on an automatic basis at specified time intervals (in the background). Access to the scheduler's functions is via a GUI invoked by selecting "Collection/Dissemination..." from the Surface menu. The Scheduler does such things as invoke scripts to acquire or disseminate data from/to remote hosts (by FTP) or automatically dial selected LARC gauges for data at specific intervals.
Using the UI the user can create/modify session files. Session files are simple text files that the CO server program uses to generate appropriate commands to execute a data transfer acquisition session with a remote source (e.g. a LARC gauge).
The CO component is used by the other components to provide the interface to communication hardware, i.e. the terminal server and modems. It uses information from the database and the information in the session files for configuring the dial-out parameters, data to obtain, etc. The CO application, CO_serv, calls the appropriate script that actually performs the communications. CO_serv also acts as the external end of the Inter-Gateway Communication described below.
The Inter-Gateway Communications (IGC) function provides a communications link across the firewall. This link, or inter-process communication (IPC), is handled using sockets, an IPC interface that allows communication through the Firewall using TIS's TCP proxy. On the DS, a process called listener is the point of contact for all communications between processes on the internal system (DS) and the external system (LS), where CO_serv acts as the point of contact. All communications between the internal and external systems go through the listener/CO_serv pair.
The Firewall (Section 6.1.9) is configured to allow socket communication to a single specified port from the LS system and only the LS system. Any internal system can perform socket communication to that port on the LS system.
When listener on the DS receives a message that a new data file is available on the LS, it immediately rcp's the file over to $LDAD_DATA/tmp. listener then checks the LDADinfo table to see if it needs to call a pre-processor for that data type. If not, it moves the file to $LDAD_DATA/Raw and sends a notification.
Pre-processors can be written in any programming language; the only requirement is that they be executable. Several pre-processors are included with the LDAD software installation. These were generated for some basic data sets and should be treated as site configurable. If data providers download files in appropriate format, the pre-processor can be as simple as moving the file to $LDAD_DATA/Raw and/or $LDAD_DATA/Text and sending notification.
When a pre-processor sends notification (via the LDAD comms router and data controller), routerLdadDecoder proceeds to decode files found in $LDAD_DATA/Raw using information from the metadata files described in Section 9.5. Using the information in the description (*.desc) files, the decoder evaluates each line of data by checking for the meteorological variable, the incoming units, and the storage units. Processed files are placed in /data/fxa/LDAD/decoded and the storage module (routerStoreNetcdf) is notified. The storage process reads the station files (*Station.txt) for station metadata such as latitude, longitude, and elevation, and stores netCDF files.
In a simliar fashion, routerStoreText and routerShefEncoder also can get notified and process files as appropriate.
Additional information on this topic is available in Section 4.3.
The BBS system is one means for AWIPS data dissemination. It also performs some data acquisition functions when a provider uses it to upload a data set to the LDAD server. The BBS is a standalone process called tmain that is automatically invoked when an external user logs into the LDAD BBS through his or her assigned UNIX account. tmain is assigned as the user's shell. Users can view files and upload or download files either by selecting a file from the menu or by using a shortcut command and entering a filename. tmain is configurable: during startup of the application, it reads the tmenu.mb configuration file for the menu structure, command shortcuts, etc., as described in Section 9.3. Each logged-in external user starts up an instance of tmain. Thus, multiple instances of tmain may be running at any given time depending on the number of external users.
The Informix trigger mechanism is used by FX to dump products to a file as they become available in the text database. It then runs an LDAD-specific script (textdbNotify.pl) which invokes the IGC component to transfer the file over to the external side. There the hmIngest process places it in the appropriate dissemination directory. Please refer to Section 9.2 for Trigger installation and configuration information.
LDAD observations are quality controlled by the AWIPS Quality Control and Monitoring System (QCMS) which, in turn, relies on the MAPS Surface Assimilation System (MSAS) to perform automated quality control checks. See Section 4.4 for more information on QCMS.
| |
|
|