By Xiangbao Jing07/06/01
0. Overview
Some AWIPS team members at APO are discussing the BBB issue of the
WMO header, like I analyzed one years ago, it made problem finally.
As a text
WS developer, I'm glad to see we pay attention to it again.
Since working on the WMO capability development on the text WS, I raised
the
issue many times. Let's call it the "BBB Issue 2001" this time. During
designing the WMO header editor in 1999, I had asked our text DB developers
to provide a bbb version query option to support the text editor, but
they
were very busy, no time to implement it. After waiting a year, I felt
it is
a real issue, wrote a notes, did a design and implemented it both on
the text
WS and DB as a prototype version in end of 2000. Earlier of this year,
Carl,
Richard, Fred and I discussed the issue. Fred mentioned the collective
issue
for the BBB, since then, we consider the BBB is tough. The code can't
be used
until there is a good solution from APO. Month ago, we found the duplicate
problem for the WMO header only products. And now we are dealing with
the DR
8426, textWS may sends same type product in the same minute.
In this document, I will discuss following question from the view of
a textWS
developer.
How does the text WS creates the BBB currently?
What is the BBB solution and prototype in 2000?
How to track the BBB version on text WS and DB?
Why exist multiple messages with same headers ?
What are the possible solution for now and future?
What we don't clear or have no solution?
I proposed some solutions to deal with it.
1. The 5.0 Text Header Editor
Begin from 5.0, forecaster can create the WMO header for a text message
by
using the WMO header editor GUI.
The user can use the TTAAii and CCCC entries and BBB select menu to
create
a WMO header. The possible values in the BBB are AAx, CCx, RRx
and NOR.
The version number "x" is from A to X and is generated automatically
by
the editor. Currently it is fixed on A only. The NOR is a empty bbb,
the
default selection.
But how is the "x" generated? We designed the capability to automaticly
handle the bbb version in the editor GUI. When the AAx, CCx or RRx
is selected,
the editor queries the text DB, then the text DB search latest bbb
version,
the X in local textDB. The textWS generate a new bbb version on the
header
editor GUI by increasing the returned X from the text DB. Because
the text
DB doesn't support the bbb version search, we just fixed the bbX at
bbA simply.
2. BBB Issue 2000 and Prototype Code
After a long time worrying, I raised the BBB issue again and did some
works,
in 2000, although it is not on requirements.
Is the bbb versions in the text DB are correct? Is the latest bbb version
the real latest one? We can save a product without send it out. This
will
increase the version number in text DB. In the other words, the real
latest
bbb version in the transmission may smaller than in the text DB. A
jumped bbb
version will be generated when creating this kind product next time.
For
example, first, a message with bbb=CCB is sent out and saved into DB;
second,
a message with same ID is created, the bbb is CCC, it is saved only;
now we
create third message with same ID, send and save it, the bbb is CCD.
Does it
mater if the transmission systems get uncontinuous version order? I
consider
it doesn't. How does the bbb is tracked and changed in transmission
systems?
We don't know. We need help from APO and NWSTG.
The works includes,
1). Designed the interfaces for the text DB access.
textdb -rw TTAAii -rs
CCCC -rb bb?
textdb -rw TTAAii
-rs CCCC -ri NNNXXX -rb bb?
The bb? is CC?, AA? or RR?.
The format of the returned data from the text DB is,
2000\n
X\n
The X is the latest
bbb version which is from A to X. For example,
the AAD, CCA, RRF...
It is '0' if there is no such product,
reading error or data
incorrect, such as the latest time of the
product earler then
one months in the DB, because we can
not reuse a header
in a month.
If the interface is "-rw TTAAii -rs CCCC -ri NNNXXX -rb bb?",
the DB search the
latest product with
TTAAii, CCCC, NNN, XXX, BBB and create-time.
Why using the create-time?
but the header time? The create-time
is the product store
time, is UNIX time(in second), but the header
time only in minute.
If the interface is
"-rw TTAAii -rs CCCC -rb bb?",
the DB search the
latest product with
TTAAii, CCCC, CCC,NNN,XXX, BBB and create-time.
We fix the CCCNNNXXX
as "NOAFOSPIL".
2). Implemented the interfaces
in the textdb.
The modified files
are, CallDB.h, CallDB_Awips.ec, TextDB_Impl.C
and textdb.C. I tested the
new textdb on the ds1-fslt(LINUX).
3). On the WMO header editor
of the text WS, will generate a bbb
version while the RRX, AAX
or CCX is selected. It sets the bbb
version to "A" if the returned
is "0" or "X", otherwise, increases
the latest bbb version as
a new one.
It is implemented in
file textHeaderBlockAwips.tcl.
This solution at lest provides a interface between the textWS and DB.
It is
a simple implementation, but doesn't consider the collective issue
and queries
the text DB. Because the text DB is slow, it may slow the bbb create
time. It
supposes the latest bbb version is correct or higher than real. It's
not good
to the warning product. I call it the "DB-Query solution".
3. Collective Issue
Fred Branski raised the collective issue on a phone conference. Although
each
product may with a bbb, the transmission systems only can track the
bbb of the
collective message, but the individual products in the message body.
So, a
stored product in the text DB may with an incorrect bbb. We ever unclear
does
the collective product use the BBB? Is the original BBB of each
collected
product keept in the collective message?
We may fix the problem in the decoder if a Collective message uses BBB
and all
BBBs of each individual product is in the message. Otherwise, TG need
find a
solution. Anyway, We need the correct BBB in the local text DB.
It dimmed the DB-Query solution. When and how can the problem
be solved?
How to track bbb of each product during transmitting and store in text
DB
correctly? This is not the area of the textWS developer. We should
wait for
it is fixed and then used the DB-Query solution, because the header
editor
need the correct or higher latest bbb version from local textDB.
However, the collective issue is the most important to the BBB issue.
4. Multiple Messages With Same Header (MMWSH)
We identify a product with its header, "TTAAii CCCC DDHHMM BBB NNNXXX".
I met
the MMWSH (or call duplicate) problem since 1999 when I first time
test the
text WS and DB, but never free from it until now.
There are two types of MMWSH, different products with same header and
same
product with same header repeatly sent and saved. The first type should
be
reidentified by changing bbb, and second type should cut off, keep
one
message only.
More than one product can be sent out in same minute on one text WS,
or
different text WSs of one office. For the products only with WMO ID,
NNNXXX=
"NOAFOSPIL", may created at different offices but with a same header,
because
the WMO ID is not uniqued.
A product can be repeatedly sent many times in same minute.
Actionly,many place may generate MMWSH, such as any product creating
software
in a WFO and NCEP and NWSTG software. There are may many product generators
in
a WFO. Each generator must fix the MMWSH problem. For example, TAF
can handle
the BBBs of it created products. What is responded areas of the AWIPS
WS
developers to fix the MMWSH? It is the text WS, DB(APO) and maybe ingest.
Whatever, any MMWSH product in the text DB, will leads problem in the
text
display, D2D display, and relative applications.
We fixed the duplicate for the products with a NNNXXX in text DB writing,
but
how about the product with WMO ID only? I don't know. Another problem
is that,
the text DB can processing the second type MMWSH, the repeated messages
by
cutting, but how about the first type, the different product with same
header?
No way!
So, we must fix MMWSH from the sources. For example, we can control
the MMWSH
when sending a product on a text WS if it is its only original generation
source
In the other words, the product is created or modified using the text
editor.
The way is to assign a new bbb version automaticly by asking the MHS
process if
a product with same header was sent in same minute. Let us discuss
it in next
section in detail.
5. Control MMWSH When sending Message on Text WS
Text WS queries the MHS to avoid MMWSH when sending to correct the edited
BBB, and avoid MMWSH sending out in same minute.
The MHS keeps all product headers which were sent out in last minute.
Before sending a edited product on the text WS, send a
ipc message including "TTAAii CCCC DDHHMM BBB NNNXXX and size" to ask
the MHS
that if the same header product was sent in same minute. The MHS matches
the
"TTAAii CCCC DDHHMM NNNXXX" and replys a bbb list to the text WS. Base
this
list, a new BBB will be assigned, then the product will be sent out
and saved
into the DB at same time.
There are more than one CCX, RRX or AAX may be matched in MHS buffer,
only
biggest version of CCX, RRX or AAx is return in the bbb list. For example,
CCF
is in the bbb list only if matched CCC, CCE and CCF.
If the bbb is empty in the edited product, set it to RRA, then compare
with
the bbb list. If the bbb list is empty, it means no such product was
sent in
same minute.
How to assign a new bbb to sending product? See the table.
----------------------------------------------------------------------
E-BBB
M-BBB
S-BBB
"-"
"-",[CCx],[AAx]
RRA
"-"
RRx,["-"],[CCx],[AAx] RR+1
"-"
[CCx ],[AAx]
"-"
RRx,CCx or AAx "-"
E-BBB
RRx
RRx,[CCx],[AAx],["-"] M-RRx+1 if M-RRx >= E-RRx
E-RRx if M-RRx < E-RRx
RRx
[CCx], [AAx],["-"]
E-RRx
CCx
CCx,[RRx],[AAx],["-"] M-CCx+1 if M-CCx >= E-CCx
E-CCx if M-CCx < E-CCx
CCx
[RRx], [AAx],["-"]
E-CCx
AAx
AAx,[RRx],[CCx],["-"] M-AAx+1 if M-AAx >=
E-AAx
E-AAx if M-AAx < E-AAx
AAx
[RRx], [CCx],["-"]
E-AAx
-----------------------------------------------------------------------
E-BBB -- BBB in the edited product.
M-BBB -- BBB list matched in the MHS.
S-BBB -- assigned BBB for the sending product.
"-" -- Empty.
The E-BBB = RRx will not happen in the Warning case, user will not
select RRx during create a warning header.
The text WS can not send the repeated product in same minute. The chance
is
very very low to creating two exactly same product on two text WSs
and send to
MHS. For example, When D2D generating a WGN product, it is sent to
one local
text WS only. However, from a abnormal operation, same product may
be sent
out and saved into DB twice. For example, After sending a edited product,
enter the editor again and click the "Send" button, this product will
be sent
again. The text WS can buffer the last sent product including its sending
DDHHMM and compare the headers or message bodies before sending to
MHS. It
may delay the sending a little. Otherwise, we just send a repeated
product out
with different header. It is simple, but what problem it will
make?
More than one text WS at one forecast office may query MHS at same time.
They
will get same Replied bbb list and assign same bbb version. The MHS
may fixes
this problem by updating the header buffer after replying earlier querying,
since the text WS has to send the product.
I call this solution as "MHS Multiple Check"(MHS-MC) solution. It will
keep
uniqued header sending from text WSs of one office and fix the DR8426.
We
suppose there is only one MHS in a office.
6. Track BBB in MHS
For some products which are only generated from text WSs of one office
and
required sending as quickly as possible, such as Warning, the BBB can't
be
tracked in the text DB, it too slow. Instead, the MHS is a good place
to
tracked it. Each out going message from text WSs of an office goes
through
it.
The MHS keeps the latest sent headers including BBB of all configued
products.
It should be set to empty if a latest sent header is longer then month.
The
MHS has to search the latest header for configued products when it
started or
keep it empty if the searching is expensive.
When a RRx, CCx or AAx is selected in the header editor, the text WS
queries
the MHS by sending a "TTAAii CCCC BBx NNNXXX", the return is the latest
BBB
version of this product, the "x". It is A-X or "0" which stands for
empty
BBB or didn't send such product in month. Then a BBB version will be
assigned
by increasing the version number or set it to "A".
Each forecast office can do the configuration by creating a product
list file.
The format likes,
#
#
TTAAii CCCC NNNXXX
...
TTAAii CCCC NNNXXX
The NNNXXX is "FOSPIL" for international product.
I would like to call this solution as "MHS BBB Query"(MHS-Query).
7. Solutions for Different Type Products
We can use the above solutions according different type products on
the text
WS. The following table is the combined solutions.
==============================================================================
Product Type
Editing BBB Sending Check
Solution
------------------------------------------------------------------------------
Warning Proc
MHS-Query MHS-MC
MHS-Q-MC
------------------------------------------------------------------------------
Time sensitive Proc
Same office
MHS-Query MHS-MC
MHS-Q-MC
------------------------------------------------------------------------------
Nontime sensitive Proc
Same office
DB-Query MHS-MC
DB-Q-MHHS-MC
------------------------------------------------------------------------------
Other Proc
Diff. Offices
DB-Query MHS-MC
DB-Q-MHHS-MC
------------------------------------------------------------------------------
MHS-Q-MC, is for time sensitive product which is only created on the
text WSs
of one office. The header editor gets the latest BBB version and check
MMWSH
from the MHS. We implement it for the warning products, this will fix
the
DR8426. Later, we can extend it to the others time sensitive products
by change
configuration table if need. Before the extention, we should know how
many
products are time sensitive. We can't put too many product in the MHS
to track
their latest BBB, this will hurt performance of the MHS.
DB-Q-MHHS-MC, is for non-time sensitive products or product may created
from
different offices. The header editor gets the latest BBB version from
the local
textDB and check MMWSH from the MHS. It requires the latest BBB version
is
correct or higher than real in the text DB.
8. Conclusion
Although we stick on the BBB issue for long time, it still is
a black box for
me for how is the BBB tracked outside the a text WS and DB. We
did not use the
DB-Query solution code because of the collective issue. I believe APO
and TG
will solve this problem finally. We should implement the MHS-Q-MC solution
for
warning products to fix the DR8426 now and then extend it to other
time
sensitive products after an evaluation. After solving the collective
problem,
the DB-Q-MHHS-MC solution should be implemented.