Presented at the Seventh Workshop on Meteorological Operational Systems
15 – 19 November 1999, European Centre for Medium-Range Weather Forecasts
Sinfield Park, Reading RG2 9AX, UK and published in the Workshop Proceedings, July 2000

AWIPS/LINUX
(Advanced Weather Interactive Processing System on LINUX)

U. Herb Grote
NOAA Forecast Systems Laboratory, Boulder, CO

1. INTRODUCTION

The Forecast Systems Laboratory has been working closely with the National Weather Service for many years to develop the Advanced Weather Interactive Processing System (AWIPS). All weather forecast offices in the United States are currently using the AWIPS system in their day-to-day operation. The latest software release (Version 4.3) contains over two million lines of code, of which approximately 500,000 lines of code are considerd to be part of the D2D display code.

The initial objective of the AWIPS/LINUX project is to convert the existing AWIPS code to run on a LINUX platform. The longer term objective is to develop a new AWIPS system architecture around the LINUX operating system that incorporates the latest advances in sensor technology and forecast techniques.

2. THE AWIPS SYSTEM

The AWIPS software provides a complete set of capabilities for a NWS forecast office. It performs real-time data acquisition, data management, application processing, and display generation. The forecaster has a diverse set of meteorological data at his/her disposal to help produce a timely and accurate weather forecast. These data include a selection of forecast models, radar and satellite imagery, surface and upper air observations, and observations from hydro, buoy, and profiler systems.

At a high level, the AWIPS system architecture (Figure 1) can be broken down into four major components: communications processing, application processing, data storage and retrieval, and display manipulation. Each of these components has one or more computers that are interconnected by a local area network. National guidance products arrive at the communications processor, which handles the communications protocol and some data filtering, and are forwarded to the applications server for decoding and storage formatting. The products are then stored on the disks attached to the data server and remain there until they are removed by a data purger. When a forecaster at the workstation requests a product for display, the data is retrieved from the data server and processed for display on the workstation.

Figure 1

Figure 1. AWIPS Functional Block Diagram

3. LINUX CONSIDERATIONS

The rapid advances in computer technology have spurred FSL's interest in providing a full function meteorological workstation on a low cost, high performance processing platform. FSL considered several candidates (including WindowsNT) for the operating system, but Linux became the top candidate because of its capability and increasing popularity.

    3.1 Strengths

    Although Linux has many strengths, the five characteristics listed below are of particular interest:

    • Low Price – The costs of similarily equipped Hewlett Packard hardware and a PC was obtained towards the end of 1999 (Figure 2). Although, these costs have changed somewhat since then, we still believed that there is a potential cost advantage in using a PC running LINUX.

Figure 2

Figure 2. Cost Comparison

    • Stability – The LINUX system appears to be very stable. FSL has been running a significant number of LINUX PCs for development and testing for an extended period of time without encountering any problems. Similarily, an informal survey of other users outside FSL seemed to confirm that LINUX is as stable, if not more so, than WindowsNT.

    • Performance – FSL has ported three major software packages to LINUX PCs: the Interactive Forecast Preparation System Graphical Editor, 3-D Display, and portions of the AWIPS 2-D Display. In all cases the performance is very good and the performance per cost ratio is considerably better than on other systems. As expected, there are some capabilities which do not execute as fast on the PC as on some high-end hardware from other vendors.

    • Standard Compliance – The LINUX operating system is POSIX compliant and its components support all relevant ANSI and ISO standards.

    • Diverse HW Support – Although FSL is planning to initially run AWIPS/LINUX on Intel processors, the knowledge that LINUX is suppported on 68-k, SPARC, ALPHA, Power PC, PA-RISC and other computer chip sets favors LINUX as the operating system.

    3.2 Perceived Weaknesses

    As is true with most other evolving technologies, the many strengths of Linux are accompanied by some perceived weaknesses, as follows.

    • User Unfriendly – With many years of experience with various flavors of UNIX over the years FSL staff did not perceive to be a problem. FSL developers are evaluating various graphical user interfaces (GUI) available for LINUX, such as KDE, Gnome, and fwm. Their experience with these interfaces will allow them to recommend the preferred GUI for use in AWIPS.

    • Installation Complexity – The LINUX operating system is available at no cost from sites on the Internet, or it can be obtained at a small cost from a number of different distributors. The LINUX distributors tend to package the operating system with drivers and desktop environments, and make it available on a compact disk. FSL is using the latest Red Hat distribution of LINUX. In most cases it took less than an hour to install the operating system, and the installation was relatively easy. Some difficulties can arise with certain peripherals and graphic cards. However, FSL found that these could usually be resolved readily with a search on the Internet. Laptops can be be a little more challenging since they tend to use nonstandard graphics cards. Again, a Web site provides considerable guidance for installing LINUX on a laptop (University of Texas, 2000). For the most part, we have found the installation to be fairly easy. Moreover, ome PC vendors (such as Dell Corporation) will preinstall LINUX on the system at the request of the purchaser.

    • Poor Documentation – There is a large amount of documentation available for LINUX on the Web. However, it can require some effort to locate the desired documentation, which may not always be uniform in depth and presentation. A LINUX Documentation Project has been formed to help provide a central access point to LINUX documentation. In addition, an activity is under way to try to provide more uniform documentation for LINUX.

    • Security Concerns – The LINUX operating system is a true UNIX operating system and thus provides the same level of security as other UNIX operating system. A frequent concern is that since the source code is readily available, it will be easy for a "hacker" to find ways to penetrate the system. AWIPS has its own secure network which should make it difficult for anyone outside the network to gain access to the systems. Also, LINUX users usually find that when a security problem is discovered, a fix is available in a matter of hours or, at most, days.

    • Lack of Support – Support for LINUX is available from hardware vendors, such as Hewlett Packard, IBM (in progress), and an organization called LinuxCare. Many users have found that support from the LINUX community is quite adequate. Although FSL has not needed to request help from the user community on LINUX problems, we requested help on a g77 compiler bug, which was corrected in a matter of hours.

    • Uncertain Roadmap – The direction of LINUX is determined by industry trends in hardware and software technology, not by corporate decision. For example, as 64-bit processor and multiple processor architectures are being introduced, the LINUX community is quick to ensure that LINUX will be able to take advantage of these technological advances. Thus, the user community determines which direction UNIX development evolves.

    3.3 Groundswell Support

    According to IDC Research (IDC, 1999), LINUX is the fastest growing server-environment operating system with about 17% of all the servers using LINUX. The popularity and rapid growth of LINUX servers and workstations during the last few years are illustrated in Figure 3.

Figure 3

Figure 3. LINUX Usage

    3.4 Future Trends

    LINUX continues to evolve and take advantage of advances in technology. Considerable effort is being invested in supporting 64 bit processors and providing fast performance for multiple processor architectures (Symmetric Multi-Processing). There ais also an increasing number of Beowulf clusters, a cluster of processors running LINUX that challenge current high performance computers. A good discussion of the strengths and weaknesses of LINUX can be found on the Web (Open System Opinion, 1999)

4. LINUX PORTING EXPERIENCES

The issues faced in porting software from another operating system to LINUX varies greatly, depending on the type and complexity of the software. A discussion follows of some of the issues faced by FSL in porting the AWIPS software to a Linux platform.

    4.1 Graphics

    The AWIPS display software uses the X11 libraries and their implementation for specifying the color model for the X window by using the X-visual. A somewhat unique feature of the HP hardware and software is the ability to specify multiple concurrent X-visuals. The AWIPS display contains five image display windows, of which four use the 24-bit visual and the fifth uses an 8-bit visual. In addition to multiple X-visuals, the HP graphic card provides overlay and underlay planes.

    The lack of of these features on the PC graphic cards required that the display be set to the 24-bit color mode. This placed a significant extra memory and processing burden on the machine, since pixmaps (containing the image to be displayed in a window) increased by a factor of four (from 8 to 32 bits/byte). The simple operation of removing a graphic overlay from an image required that the entire image be re-rendered instead of merely turning off a graphic plane. Also, functions such as changing the image color required recomputing each pixel value instead of modifying a color lookup table.

    4.2 Programming Languages

    The AWIPS software is written in C, C++, Fortran, and scripting languages such as Tcl/tk. It is compiled on the HP hardware using HP's CC and Fortran compilers. A requirement in porting the software to a PC was to ensure that the modified code would still run on the HP hardware with the HP compilers. As a result, FSL compiles the code daily on the HP using the GNU and HP compilers, and on the PC using the GNU compilers.

    The compilers for the AWIPS/LINUX system are the GNU gcc and g77. Since the compilers are more ANSII compliant than HP CC (c-front) and Fortran compilers, some of the code needed to be modified to make it ANSII compliant. For example, the GNU g77 compiler does not support records and uses much stronger data typing than the HP compiler. Some of FSL's legacy Fortran code frequently used integers in Boolean expressions. Other differences in Fortran compilers were in the intrinsic functions, which were dealt with by writing a "g77 wrapper" for the Fortran functions.

    4.3 Processor Architecture

    The ordering of bytes within a word for a PC is different than for the HP. As a result, data files that were generated on the HP were read incorrectly on the PC. Since the code has to be executable on the PC and the HP, a run-time test is performed to determine if the bytes need to be reordered. Certain data files (netCDF and shapefile) use a data representation that makes them readable correctly on both types of machines. The software that reads binary cardiographic (map backgrounds) data and certain tables was affected the most by the difference in byte order. So far, FSL has not fully investigated the impact of the new 64-bit processors on the software.

    Another challenge involved the difference in the way processing threads are implemented by different hardware and software. The HP systems used DCE threads versus POSIX threads for LINUX. A cruxial component of the AWIPS system is the IPC (Interprocess Cmmunications). Tens of thousands of messages are exchanged every day between processes. The IPC software was designed to use multiple threads to efficiently and reliably handle large volumes of messages; the IPC software had to be rewritten for LINUX.

    4.4 Development Tools

    FSL generally found the development tools available for LINUX to be adequate, including compilers, debuggers, performance analyzers, and editors. However, FSL was unable to find a good software tool, such as HP's Purify software, for finding program memory leaks.

    Most AWIPS commercial off-the-shelf (COTS), as well as public domain software, is available for LINUX. The COTS Informix relational data base manager is a key component of the AWIPS system. COTS accounts for a measurable portion of AWIPS code.

5. Project Status

The AWIPS software is rapidly being ported to LINUX. The first step of this effort, porting all of the software to the gcc compiler, is complete. The second step, moving the software from the HP to a PC running LINUX, is partially complete. In addition, the software infrastructure and the entire product display software to LINUX has been ported. The system was successfully demonstrated at the 16th IIPS Conference Exhibit of the Annual Meeting of the American Meteorological Society in January 2000. Concurrent with the effort to port the software, FSL is investigating and assessing new system architectures for the AWIPS system that include PCs and LINUX as a key component.

References: