WeatherZine #23

eZine lover (@eZine)

Published in 

WeatherZine

 · 4 months ago

WeatherZine #23
Number 23, August 2000

"What's happening on the Societal Aspects of Weather WWW Site."

WeatherZine is a bi-monthly on-line and e-mail distribution newsletter for the Societal Aspects of Weather Site. It contains a summary of recent changes to the site (along with links to relevant sections), and news, events, and announcements of interest to the community.

WeatherZine, and its parent WWW site, accept and encourage the submission of activities, events, or links of interest to the community. You can use the on-line forms in the Feedback section or can send an email to thunder@ucar.edu with the item you would like to see posted on the site or included in the next issue of the Zine.

Contents

1. Editorial
• Policy research for atmospheric scientists
2. Guest Editorial
• How much "skill" is there in forecasting El Nino?
• By Chris Landsea and John Knaff
3. Correspondence
• Correspondence from our readers
4. Weather-Related News
• New Mailing List - The use of the Internet in Disaster Situations
5. New Additions to the WWW Site:
• El Nino/La Nina
• Emergency Management
• Floods
• Summer/Winter
• General Weather Resources
• Insurance
6. Subscribe Now!

[1] Editorial -- Policy research for atmospheric scientists

In recent months I've received a steadily increasing number of inquiries from students and faculty in the atmospheric sciences asking about atmospheric sciences policy, what it is and how one might approach this area of scholarship. Here are some suggestions.

Most importantly, become an expert. Your ability to contribute valuable knowledge depends upon your particular expertise. If you are the world's expert in a particular area, then you will have a much stronger basis for contributing to society's needs. Too often, however, knowledge of policy winds up being very broad and not very deep, and universities that seek to develop such expertise (e.g., generic "environmental studies" programs come to mind) often face criticism from their disciplinary colleagues. But a word of caution is also due: being an expert in one area does not mean that you are an expert in other areas.

But recognize that "expert" does not have to be a synonym for "narrow." Of course, if your expertise is in a very narrow area, then your contributions to policy can be limited. Then again, in 1970 an expert on the then-obscure topic of the atmospheric properties of chlorofluorocarbons could not have anticipated the policy significance of this work over the next three decades. But if you would like to enhance the chances that your expertise is relevant to policy, then there are other types of expertise that have broad relevance across policy topics. For example, with the growing emphasis on the use of forecasts by decision makers, in coming years there will likely be a great need for experts in forecast verification. And a focus on policy decision processes, either in general or in the context of a particular area of science - like chlorofluorocarbons - will enhance the relevance of your expertise. Once, an expert was defined as someone who broke knowledge into small pieces and knew an awful lot about one small piece. Increasingly, the needs of policy require experts who can integrate diverse areas of knowledge.

If you include a focus on problems and decisions as "units of analysis" in your research then you will enhance the potential relevance of your work. Typically researchers focus on phenomena (e.g., storms, precipitation) and processes (e.g., ENSO, radiative forcing). Usually this sort of specialized knowledge is necessary but not sufficient in the policy making process. Ultimately, decision makers want to know the answer to the question: "So what?" You will be in a much better position to contribute to the answer if you can clearly link your specialized knowledge of phenomena and processes with "Is this a problem? For whom? Of what magnitude? What can be done? What should be done?" Etc. Recognize that the answers to these questions often require the integration of a broad range of expertise.

Practice suspending judgment. Good judgment, as John Dewey has written, requires that "we defer conclusions in order to infer more thoroughly." It is this willingness to entertain alternative explanations and solutions that distinguishes the policy advocate from the policy analyst. Always ask yourself, what knowledge would change my mind about this problem or decision? If the answer is no information, then you are simply pushing a particular answer. In this case you should look for expertise in marketing, not policy analysis.

Identify and question assumptions. One of the great challenges facing a policy analyst is to become aware of the assumptions that guide thinking. For example, United States science policy since World War II has been guided by a linear model that holds that investments in "basic" research are the most effective way to address societal concerns. For many decades few questioned this assumption. But is this linear model actually the best? In the atmospheric sciences community we see a similar untested assumption in the area of forecasting. Considerable resources are often devoted to "improving forecasts," under an assumption that society will benefit from such improvements. But by comparison little effort is devoted to investigating the conditions under which this assumption is actually true. Because assumptions are usually unquestioned and sometimes incorrect, there is great value in being able to identify and question them in a rigorous, scientific manner.

Dejargonify your speaking and writing. You can never speak or write too clearly, and there is always room for improvement. The best way to improve these skills is the tried and true formula of practice, practice, practice. Seek opportunities to communicate with people outside your own discipline or subject area expertise. At times this can be frustrating and humbling, but if you truly want to contribute to policy debate and discussion you will have to translate expertise into the language of the policy process, and not expect the opposite!

Understand disciplinary "geography." As an expert with interests in policy it is essential that you understand how other areas of expertise fit in relation to both your own and the broader issues that you are concerned with. This places a greater burden upon you than on the disciplinary scholar who is satisfied to focus on purely disciplinary concerns. Not surprisingly, once stripped of jargon and disciplinary idiosyncrasies there is a great deal of overlap in methods, theories, and conclusions across disparate disciplines, particularly in the context of policy where problems and decisions ultimately bring everything together.

Seek the right education and training. Traditionally, there have not been strong linkages within universities between atmospheric sciences departments and policy programs. For this to change will require the dedicated efforts of students and faculty to overcome the familiar obstacles that stand in the way, usually at the department level. There are a number of schools - Columbia, Penn State, Oklahoma, and Colorado are a few examples - that have the requisite expertise on campus to develop a strong atmospheric sciences program of research and education. And there are other efforts beginning as well, such as the Climate Affairs Program (www.esig.ucar.edu/climate_affairs/) here at NCAR and the AMS Policy Program (www.ametsoc.org/AMS/atmospolicy/). For those interested in this area of study, begin to take advantage of the resources that are available to you. As demand increases, universities will move to meet it.

Help to build a network of those with like interests. To that end we are announcing the initiation of a new email list-serv and an educational resource guide. The list-serv is called weather-policy and to join send an email to majordomo@ucar.edu.
In the body of the message include:

subscribe weather-policy <your email address>

* do not include the <>'s. Majordomo will ignore the subject line.

Its purpose is to provide a forum for discussion and communication among those interested in the educational and research aspects of atmospheric sciences policy. The educational resource guide will provide information directly from atmospheric sciences programs about their offerings in this area, as well collect this information, we will post it on our site or announce it in the WeatherZine. As always we welcome your feedback.

- Roger A. Pielke, Jr.

[2] Guest Editorial -- How much "skill" is there in forecasting El Nino?

Chris Landsea
NOAA/AOML/Hurricane Research Division
Miami, Florida
landsea@aoml.noaa.gov

John Knaff
NOAA/Cooperative Institute for Research in the Atmosphere
Colorado State University
Fort Collins, Colorado
Knaff@CIRA.colostate.edu

The 1997-98 El Nino had dramatic impacts, leading to drought in Indonesia, extreme rains in Peru and Ecuador, and a quiet Atlantic hurricane season. Conventional wisdom holds that predictions of the event's onset, magnitude, decay, and impacts generally were accurate. But a close look at the forecasts reveals that while the impacts of the event, once it had begun, were accurately anticipated based on the climatology of past El Ninos, none of the available forecast techniques accurately predicted the event's onset, magnitude, and decay.

What is an accurate forecast? One definition of forecast accuracy is based on a concept called "skill." Atmospheric scientists define "skill" as a prediction's improvement upon some naive baseline. For example, absent other information a best guess for the high temperature in Washington, DC on September 1 might be the historical average high temperature for that date. A forecast that is closer than the historical average to the actual temperature on September 1 thus has skill. A forecast methodology that consistently improves upon a naive baseline is a skillful methodology. Consequently, judgments of the appropriate baseline against which to measure skill are crucial to claims of forecast accuracy.

Traditionally, scientists have judged seasonal El Nino forecasts of sea surface temperature (SST) in the equatorial eastern Pacific Ocean "skillful" if they improve upon a baseline based upon "persistence." A forecast of "persistence" simply uses current conditions as a predictor of future conditions. For example, if the SSTs were 0.7 C above average, persistence would simply forecast 0.7 C above average for the following months and seasons. Because SSTs associated with El Nino are part of a cycle (the El Nino-Southern Oscillation or ENSO), it turns out that persistence is a very easy baseline to outforecast.

To provide a more stringent, but still naive baseline of skill in El Nino forecasting, we developed the El Nino-Southern Oscillation CLImatology and PERsistence (www.aoml.noaa.gov/hrd/Landsea/cliper/) model as a simple statistical tool that takes advantage of the climatology of past El Nino events, persistence, and contemporary trends. Thus, we recommend that the output of ENSO-CLIPER replace the use of persistence as a skill threshold. There are, of course, other simple statistical models that could be used to set this threshold. In our proposal, "skill" is defined as the ability of a forecast or forecast methodology to improve upon ENSO-CLIPER - which is a more difficult task.

In next month's Bulletin of the American Meteorological Society (www.aoml.noaa.gov/hrd/Landsea/skill/) we evaluate twelve statistical and dynamical models which were available in real-time for the 1997/98 event. We conclude that some of the models were able to outperform ENSO-CLIPER in predicting either the onset or the decay of the 1997-98 El Nino, but none was successful at predicting both onset and decay for a medium-range (6-11 months) lead time. Also, no predictive approaches, including the ENSO-CLIPER baseline, were able to anticipate even one-half of the actual magnitude of the El Nino at medium-range (6-11 months) lead. In addition, none of the models showed skill at short- to medium-range lead times (0-8 months). No dynamical model and only two of the statistical models outperformed ENSO-CLIPER by more than 5% (of the root mean square error) at 9 to 14 months lead-time.

A lesson to be learned from this evaluation is that since the best performing models for the very strong 1997-98 El Nino event were statistical ones, it appears that the use of more complex, physically realistic dynamical models does not automatically provide more accurate forecasts. Increased complexity can increase by orders of magnitude the sources for error, which can cause degradation in skill. Despite the lack of skill in forecasting ENSO up to 8 months in advance, once the 1997-98 El Nino had begun national meteorological centers were able to anticipate correctly many of the impacts because of the tendency for El Nino events to persist into and peak during the winter. Indeed, the U.S. Climate Prediction Center's most skillful tools for predicting U.S. seasonal precipitation were statistical rather than dynamical models. For seasonal temperature anomalies in the United States, the statistical and dynamical approaches were about equal in skill. This implies that in this case the use of dynamical models was not needed to anticipate a wet and stormy winter for the southern tier of the United States and a warm winter for the northern tier of states.

We have two recommendations based on this work: 1) a distinct need exists for the forecasting community to debate and agree on the naive baseline against which "skill" is to be measured in forecasts of ENSO phenomena. Use of the simple persistence is much too easy a benchmark. If not ENSO-CLIPER, then some other more rigorous but simple test is essential for evaluating ENSO forecasting in a useful manner; and 2) for the 1997-98 event none of the more sophisticated models- both other statistical schemes as well as numerical techniques - outperformed the naive ENSO-CLIPER baseline for short to medium lead times (0 to 8 months). Thus these more complex models may not be doing much more than carrying out pattern recognition and extrapolation. National meteorological centers and research agencies may wish to consider carefully their resource priorities (personnel, computers, and budgets) when the most accurate tools presently appear to be the relatively cheap statistical systems, compared to the expensive (developmentally and computationally) dynamical models.

These results may be surprising given the general perception that seasonal El Nino forecasts from dynamical models have been quite successful and may even be considered a solved problem. A particular report in Science in 1998 - "Models win big in forecasting El Nino" - generated widespread publicity for the success in forecasting the 1997-98 El Nino's onset by the comprehensive dynamical models. This report was based upon a conference paper by NOAA's Tony Barnston, which only considered El Nino's onset at the time of the report in October 1997. No follow-up in Science was forthcoming when Barnston and colleagues published a paper in the Bulletin of the American Meteorological Society last year showing that the comprehensive dynamical models did not "win big" after all. (It is worth mentioning that the results from Barnston and colleagues do indeed agree quite well in general with our paper, though the interpretation is very different.)

Also disturbing is the use of the supposed success in dynamical El Nino forecasting to support other agendas. For example, a 1999 overview paper by Ledley and colleagues in support of the American Geophysical Union's "Position Statement on Climate Change and Greenhouse Gases" said the following:

"Confidence in [comprehensive coupled] models [for anthropogenic global warming scenarios] is also gained from their emerging predictive capability. An example of this capability is the development of a hierarchy of models to study the El Nino-Southern Oscillation (ENSO) phenomena..... These models can predict the lower frequency responses of the climate system, such as anomalies in monthly and season averages of the sea surface temperatures in the tropical Pacific."

To the contrary, under this logic and with the results of our study, one could even have less confidence in anthropogenic global warming predictions because of the lack of skill in predicting El Nino. The inability of dynamical models to outperform a relatively simple statistical scheme for ENSO calls into question the consensus opinion that coupled dynamical models are the best way to accurately predict short-term climate variability. The bottom line is that the successes in ENSO forecasting have been overstated (sometimes drastically) and misapplied in other arenas.

We are now engaged in an assessment of the forecast skill of the strong 1998-2000 La Nina event, which immediately followed the 1997-1998 El Nino. Given the most recent complete ENSO warm and cold cycle, it may be that truly skillful predictions from models are available. But the current answer to the question posed in this article's title is that there was essentially no skill in forecasting the very strong 1997-98 El Nino at lead times ranging from 0 to 8 months using the performance of ENSO-CLIPER as the naive baseline. Moreover, the lack of skill at the short- to medium-range lead times continues to confirm what was also observed in independent tests of real-time ENSO prediction models for the period 1993-96 in our earlier work.

For further reading

• Barnston, A. G., M. H. Glantz, and Y. He, 1999: Predictive skill of statistical and dynamical climate models in SST forecasts during the 1997-98 El Nino episode and the 1998 La Nina onset. Bull. Amer. Meteor. Soc., 80, 217-243. (Available at: www.esig.ucar.edu/BAMS8002.pdf)
• Kerr, R. A., 1998: Models win big in forecasting El Nino. Science, 280, 522-523.
• Knaff, J.A, and C.W. Landsea, 1997: An El Nino-Southern Oscillation CLImatology and PERsistence (CLIPER) forecasting scheme. Wea. Forecasting, 12, 633-652. (Available at: www.aoml.noaa.gov/hrd/Landsea/cliper/)
• Landsea, C.W., and J.A. Knaff, 2000: How much skill was there in forecasting the very strong 1997-98 El Nino? Bull. Amer. Meteor. Soc., (in press, September issue). (Available at: www.aoml.noaa.gov/hrd/Landsea/skill/)
• Ledley, T. S., E. T. Sundquist, S. E. Schwartz, D. K. Hall, J. D. Fellows, and T. L. Killeen, 1999: Climate change and greenhouse gases. Eos, 80, 453-458.

[3] Correspondence -- Correspondence from our readers

Dear WeatherZine:

In response to your editorial in WeatherZine #21
(Six Heretical Notions About Weather Policy, www.esig.ucar.edu/socasp/zine/21/editorial.html):

When weather disasters happened in the past, the National Weather Service frequently benefited from an infusion of funds. After a 1974 tornado outbreak, for example, a research group was set up at the National Severe Storms Forecast Center. Lately, disasters come and go with little benefit to the weather community. The problem is that we have been ineffectual at convincing the public and policy-makers that (a) they are getting a bargain at the current cost, and (b) they could do even better with a larger investment.

If we ignore the role that human forecasters have in the process of generating forecasts, then we will get less from any level of investment than is possible. Although it seems we are on a juggernaut aimed at removing any role for human forecasters in the public weather service by 2050 (my forecast!), properly educated and trained humans can add considerable value to what any model can generate. We have chosen not to invest in meaningful training of our weather forecasters, thereby putting our hopes into improvement of the numerical model forecasts.

Much collected data goes unused because of a lack of access. Operational data has not been collected to be used in research. Once these data are used by the operational agencies, they go into some "black hole" where retrieval is difficult and expensive. Moreover, data quality issues tied to operationally-collected observations can be a serious barrier to their use.

Access to data collected in field programs typically is controlled and often limited to those directly involved in the data collection. This can seriously limit the general value of the data sets. We often are our own worst enemies at getting the full value from the data we collect at taxpayer expense.

As for forecast quality, the NWS continues to put forth only a token effort into a truly meaningful forecast verification program. Virtually nothing is happening with respect to "closing the loop" on any verification effort.

Another critical gap in our understanding is what it takes to be a high-quality human weather forecaster. We have no idea what characterizes the best human forecasters, by any measure of forecast quality. An obvious implication is that there is no role envisioned for them in the far future ...learning how humans do their part of the forecasting job well is apparently seen as a meaningless, dead-end task. The lack of any interest in this also tends to become a self-fulfilling prophecy.

We are very bad at applying what we know in the real world of operational forecasting. Decision-makers in research and operations are ignorant of the other's needs and interests so that whatever happens to address the gap is done by individuals with little or no support from either set of decision-makers. If only a small fraction of what is known from research is being applied in operations, we have only ourselves to blame!

The operational bureaucracy is very reluctant to change procedures to reflect new knowledge. We are still putting out superannuated "zone" forecasts when it is possible to provide gridded model fields of virtually any forecast quantity over the Internet. If obsolete products are allowed to continue indefinitely, there is little value to any research tied to product improvement.

The NWS is simply not equipped to provide good answers about the value of what they do, in spite of my belief that even their current forecast quality still manages to be extremely valuable, in spite of being significantly lower than what it could be if they made forecast quality a priority. I see little indication that anyone is doing much to address these critical issues. Defensive reactions to your "heresies" indicate a lack of understanding of what you are trying to accomplish with this dialog.

Dr. Charles A. Doswell III
NOAA/National Severe Storms Laboratory
doswell@nssl.noaa.gov
www.nssl.noaa.gov/~doswell
Standard disclaimer ... my views are my own, so don't blame my employer.

[4] Weather-Related News

New Mailing List - The use of the Internet in Disaster Situations

A new mailing list has been created to discuss the development of improved ways of collecting and communicating information during and after disasters by use of the Internet. The purpose of this list is to promote the use and development of the Internet by public agencies, and to discuss the development of other Internet-based systems for collecting and distributing the "disaster message."

To subscribe or unsubscribe to this list send an email to: disastercom-request@disastercenter.com with the word subscribe or the word unsubscribe only, in the body of the message.

Christopher Effgen
build@micronet.net

[5] New Additions to the WWW Site

El Nino/La Nina

El Niño and La Niña: Tracing the Dance of Ocean and Atmosphere (www4.nationalacademies.org/opus/home.nsf/web/elnino?OpenDocument) This publication surveys basic scientific studies of the last century that have led to our current understanding of El Nino/Southern Oscillation (ENSO).

Emergency Management

Overview:The National Pre-Disaster Mitigation Plan (www.civil.buffalo.edu/aawe/aawe-text/publication/NIBS/NEWFEMA.htm #Document) The National Pre-Disaster Mitigation Plan articulates the vision, mission, goals and objectives of this nation's effort to reduce the escalating cost of natural disasters.

Congressional Natural Hazards Caucus (www.senate.gov/~edwards/cnhc/index.html) The Caucus hopes to foster an important dialogue on steps governments can take to lessen the severity of natural disasters.

Multi Hazard Identification and Risk Assessment (www.fema.gov/mit/tsd/ft_mhira.htm) This site clarifies and documents previous efforts to identify natural and technological hazards, and to assess associated risks.

World Disaster Report (www.ifrc.org/publicat/wdr2000/) The 2000 World Disaster Report focuses on public health in disasters.

Surviving Disaster with Renewable Energy (www.nrel.gov/surviving_disaster/) This site highlights the National Renewable Energy Lab's role in emergency preparedness and response, disaster mitigation and relief, and sustainable development.

Natural Hazards Review (www.pubs.asce.org/journals/nhnews.html) The Natural Hazards Review is "the first cross-disciplinary journal to address all aspects of natural hazards loss and cost reduction."

Emergency Preparedness Information Project (EPIP) Emergency Gateway (www.epipgateway.com) This site is designed to provide minority communities with information about creating disaster resistant families, communities, institutions, businesses and organizations.

Bandelier National Monument Cerro Grande Prescribed Fire Investigation Report (www.nps.gov/cerrogrande/) This is the interagency Fire Investigation Team's report on the Bandelier National Monument fire. The report of the Independent Review Board can be found at www.doi.gov/secretary/reviewteamfinal.htm.

Floods

Significant Floods in the United States During the 20th Century - USGS Measures a Century of Floods (ks.water.usgs.gov/Kansas/pubs/fact-sheets/fs.024-00.html) This report lists, by type of flood, 32 of the most significant 20th century U.S. floods in terms of number of lives lost and/or property damage.

Summer/Winter

Heat Waves and Hot Nights: Ozone Action (www.ozone.org/heatstress/); Heat Waves and Hot Nights: Physicians for Social Responsibility (www.psr.org/heatsheet.html) This report shows that extreme heat waves and overheated nights are becoming more frequent in cities and regions across the United States.

General Weather Resources

NOAA Photo Collection (www.photolib.noaa.gov) This site includes hundreds of images of natural hazards from the National Severe Storms Laboratory and the National Weather Service Historical Image Collection.

Insurance

Society for Risk Analysis 1999 meeting (www.riskworld.com/Abstract/AB9ME001.HTM) This site includes abstracts of the 373 papers presented at the 1999 meeting of the Society for Risk Analysis.

[6] Subscribe Now!

The WeatherZine is produced as a Web site and an email message. Subscribing to the WeatherZine will add you to our distribution list and you will receive email messages whenever the WeatherZine is released.

To subscribe to the WeatherZine, use the on-line form at: www.esig.ucar.edu/socasp/forms/join.html, send an email to thunder@ucar.edu and include the following information:

Name
Organization
Email Address
Interests & Needs

or send an email to majordomo@ucar.edu and in the body of the message include:

subscribe weatherzine <your email address>

* do not include the <>'s. Majordomo will ignore the subject line.

For additional information, please contact thunder@ucar.edu

--------------------

Weather Impacts Group
Societal Aspects of Weather
Environmental and Societal Impacts Group
National Center for Atmospheric Research
US Weather Research Program

Societal Aspects of Weather Web Site: www.esig.ucar.edu/socasp/
WeatherZine: www.esig.ucar.edu/socasp/zine/