Here is a letter about the weather station by Stephen L. Johnston, Radar Engineer, IEEE Fellow, addressed to Representative Sensenbrenner.
- Editor

E-mail: radarslj@ieee.org
December 14, 2000

The Honorable F. James Sensenbrenner, Jr.
Chairman, House Science Committee
2332 Rayburn House Office Building
Washington, DC 20515-4909

Dear Representative Sensenbrenner:

     This letter is being sent to you as an E-mail attachment to Mr. Tom Vanek because of the exi-gency of the present situation. Please recall my letter of November 15, 2000 to you. There I expressed concern over the fact that Mr. Donald Burgess of the National Weather Service (NWS) in Norman, Oklahoma had shown that the average tornado Critical Success Index (CSI) for all NWS Weather Fore-cast Offices (WFOs) was only 0.20 (20%) over the entire period of 1986-1998. Since this period in-cluded both pre-NEXRAD and NEXRAD operation it meant that NEXRAD had not provided any measurable improvement in tornado warning in the nation. This was the first open disclosure of this problem. While that was indeed most disturbing, Mr. Burgess did not pursue this serious matter further before his recent reassignment to another NWS activity. Unfortunately I was unable to get the database of Mr. Burgess so I could do further analysis of his results.

     Thanks to your letter to the NWS on this, General Kelly suggested that I contact Mr. William Lerner at the NWS headquarters for NWS tornado data. Mr. Lerner very kindly provided me with tor-nado data from 116 NWS WFOs. That data was surprisingly so compact that he was able to rapidly transmit it to me as an E-mail attachment! I greatly appreciate his full cooperation even though he had been in this work for only two months following the retirement of Mr. Polger. In this letter I will sum-marize results of our analysis of the new data and make recommendations for further action by your Committee.

     As regards my methodology, my tornado warning "yardstick" was the CSI. It is the combination of radar Probability of Detection (POD) and False Alarm Rate (FAR) in a well-known simple mathe-matical equation. I found that the mathematical equation has been known in the NWS for over one hun-dred years. The first reference to it was in 1884! I sometimes refer to CSI as being the "effective POD" and FAR as being the "Diluter of POD". In that regard at high FAR values, the corresponding CSI value closely approximates the value of (1-FAR), meaning that POD has practically no effect on the CSI value there. We first reduced the number of data sets from 116 to 47 by eliminating the WFOs that had reported no tornadoes in the 14-year period or that had many annual CSI values of zero. This eliminated the Far Western states, New England states, and mid Atlantic states. The eliminated states showed strong correlation to the ranking of states according to Tornado Annual Coverage Fraction (ACF) of Dr. Frank Tatom of VorTeK LLC, Huntsville, Alabama. His top 22 states very closely encompass all the 47 WFOs used in our work. We then ranked the 47 WFO tornado CSI values according to three similar lists, the average of each WFO for the first seven years, the second seven years, and for all 14 years. The two seven year lists approximated dividing the reduced database into two equal portions. We con-centrated on the second seven-year list since NEXRAD was installed in the field during that period. The attached chart shows the WFO CSI averages values for the last five years in the Great Plain states and east of the Mississippi River. The rest of this letter presents principal findings of our data analysis.

     1. The highest second 7 year period average CSI was only 38% for the Amarillo, Texas WFO. This very low maximum value is much less than the 50% value promised by the NWS during Hearings on the Modernization Plan. Inspection of the database showed that some WFOs reported greater annual values than this BEFORE NEXRAD was installed! Clearly, NEXRAD tornado performance has not lived up to promises.

     2. The region of highest annual CSI values was in the Great Plains states. Inspection of the in-dividual annual CSI histories for those WFOs generally shows little improvement of CSI when NEX-RAD was installed. One possible explanation might be due to the nature or tornadoes in the Plains states. I sometimes say facetiously that a blind person can see those tornadoes and a deaf person may hear them at a distance of 100 miles so one does not need radar to detect them! One would think that CSI values for such tornadoes should be much greater without or with NEXRAD. That matter needs further study.

     3. The region of lowest tornado average CSI is east of the Mississippi River. One contributor is certainly the nature of tornadoes in the region. Nature of terrain differences limit low altitude radar cov-erage. Similarly tornado smaller size restricts required radar distant lateral coverage needed to detect Doppler Tornado Vortex Signatures (TVS). The often-short tornado time on the ground for these tor-nadoes mandates the most rapid tornado detection and warning. Unfortunately NEXRAD is now locked into a volume scan. Clearly these low values are UNACCEPTABLE! Second 7 year CSI averages of 11 % for Birmingham (that includes Huntsville, Alabama), 10% for Chicago, and only 9% for New Or-leans/Baton Rouge are simply pitiful. Strong recommendations will be made on this later in this letter.

     4. Comparison of the second 7-year averages with the first 7-year averages shows that for some 40 %, of them the second seven-year averages were LESS than the first 7 year averages. This means that for them, NEXRAD tornado warning CSI was WORSE than before NEXRAD! This is also unac-ceptable. Practically all of these were east of the Mississippi River! In the case of the Birmingham, Alabama WFO, before getting its NEXRAD, it did not have a radar of its own. Instead it had only a re-mote radar monitor from early NWS radar located some 40 miles away at Centerville, Alabama. The Centerville staff analyzed the radar output and coordinated with the Birmingham NWS office. From the data of Mr. Lerner for Birmingham, the first radar lashup actually had better tornado performance than the later NEXRAD!

     5. Inspection of individual annual tornado CSI histories for the second seven years East of the Mississippi River clearly shows sudden very rapid rise in number of warnings issued and number of un-verified warnings when NEXRAD was installed. This is because of the NEXRAD policy of warning on EVERYTHING and not to be concerned with false alarms. Is there any wonder why many citizens now ignore NWS tornado warnings? This MUST be corrected.

     6. Although NEXRAD did provide some increase of tornado POD, the excessive NEXRAD FAR values prevented any improvement of CSI. Reduction of NEXRAD tornado FAR would permit NEX-RAD CSI improvement. This must be accomplished.

     7. From the above items, the many NEXRAD shortcomings will be recognized. A considerable tornado CSI improvement may be achieved by the following actions. The radar must be a real time ra-dar; i.e., it must not be locked in a volume scan and be capable of continual antenna azimuth scan at any selected elevation angle or in a vertical antenna scan. It must be capable of being rapidly manually di-rected to any desired azimuth and elevation angles. The spotter communicator must be seated BESIDE the radar operator instead of in a separate room so the radar operator can quickly investigate spotter re-ports. Although the NWS will not acknowledge it, the FIRST on line use of Doppler radar techniques in the NWS occurred BEFORE NEXRAD at the Montgomery, Alabama NWS office in 1981, followed by a similar installation in the NWS office in Huntsville, Alabama a decade later. Both Doppler modifica-tions were provided by local government bodies and citizens. Performance of both far exceeded tornado performance of any NEXRAD anywhere in the USA. Unfortunately the Montgomery Doppler was im-properly shut down and the excellent radar and Doppler were disposed of. Fortunately the Huntsville Doppler is still in existence although it is now not being properly utilized and its staff is not as skilled as its original staff. This radar is scheduled for disposal and the office permanently closed soon. Clearly, the office should be retained with its Doppler and its warning responsibility should be restored at once.

RECOMMENDATIONS

     This letter should be referred to the General Accounting Office (GAO) at once for full study and investigation. I will be glad to assist the GAO in their investigation. The NWS should be directed to retain the Huntsville NWS Doppler in full operation with return of their warning responsibilities.

     May I wish you and your Committee a very Merry Christmas and a Happy New Year. I look forward to full success of the Republican Party in both bodies of the Congress and in the White House. I am proud that I voted Republican again.

Sincerely,

Stephen L. Johnston
Radar Engineer
IEEE Fellow