Jeffrey L. Anderson

Research Outline

(References can be found in the Curriculum Vitae) This was written for a purpose that required writing from the third person perspective.

Overview

Anderson's research career has spanned two decades and has been focused by the common theme to improve predictions of the earths atmosphere. He has made research contributions in theoretical geophysical fluid dynamics (4, 6), seasonal prediction (21, 24), predictability (14, 15), ensemble prediction (13, 17) and ensemble data assimilation (32, 44). His accomplishments in software engineering, applied mathematics and statistics, while important on their own, have been directly in support of his goal to improve prediction. His software engineering contributions are described after his fundamental science work.

 

  Stability of atmospheric flows
  Stationary states and blocking
  Predictability on medium and seasonal timescales
  Numerical prediction of tropical storm frequency
  Ensemble prediction and ensemble tools
  Ensemble data assimilation
  Adaptive methods to support a generic research testbed
  The Flexible Modeling System
  GFDL Bgrid Dynamical Core, Coupled Seasonal Prediction, and AM Climate Models
  The Data Assimilation Research Testbed
  Table 1 : Summary of DART applications and users

 

Stability of atmospheric flows

Andersons dissertation research concerned instability of the atmospheric flows. This work hypothesized that low-frequency atmospheric variability could be explained by looking at the linear instability of barotropic dynamics linearized around observed mid-tropospheric time mean flows. The most unstable modes strongly resembled low-frequency variability patterns including some types of blocking and certain atmospheric teleconnections. The thesis work led to several publications (3, 4) that were an important part of an evolving literature on instability of non-zonal flows. Anderson developed novel numerical methods for finding the most unstable modes of very large generalized eigenvalue problems. These methods were essential to find unstable modes of large atmospheric models and were later adopted and generalized by applied math researchers.

Stationary states and blocking

In the early 1990s several researchers had explored the possibility that persistent states of the atmosphere might be related to special solutions of atmospheric models that had very small time tendencies. Anderson developed high-dimensional minimization algorithms, later applied in optimization for variational data assimilation, that could find states of large models with very small time tendencies. These states were characterized by special locally linear relations between vorticity and stream function. When minimizations were begun from observed atmospheric states with persistent blocking patterns, nearly zonal flows with imbedded high-amplitude modon-like structures in the blocked region resulted (69). These modons had a linear relation between vorticity and stream function that was distinct from that found elsewhere at the same latitude. Several papers resulted from this research (5, 6) and argued that the persistence of blocking could be attributed to the existence of modon-like atmospheric states with locally small time tendency (64, 65). This work led to a three month visit to KNMI in the Netherlands where Anderson began collaborations on seasonal prediction and ensemble prediction.

Predictability on medium and seasonal timescales

The theoretical work on stability and stationary states was used to explore predictability and prediction on timescales ranging from a week to several seasons. Anderson used extended integrations from both atmospheric and coupled general circulation models in an attempt to improve prediction capabilities. In the early 1990s, the concept of return of skill was becoming important in the use of GCMs for seasonal prediction. Anderson explored this concept using monthly forecasts from the NCEP operational seasonal prediction model (7, 8, 66). Using a newly developed diagnostic test for detecting blocking events (9), he demonstrated that the climate of the GCM as a function of lead time evolved from being similar to the observed climate to the long-term model climate over a period of nearly a month. Significantly, the evolution was not smooth. Instead, the model climate became relatively zonal during the first 10 days and then regenerated high- and medium-frequency variability consistent with the model climatology during the next 10 days. This research indicated a need to make model climates more consistent with the observed climate before they could be used for medium range prediction (11, 16, 28). This work had a significant impact on the direction of seasonal prediction GCMs at NCEP and NOAA research laboratories.

In the mid-1990s , there was growing use of numerical models for seasonal forecasts with an emphasis on the development of large GCMs, both atmospheric and coupled, for prediction (23). Unfortunately, the actual skill of these models were lower than prior expectations. Anderson obtained output from operational seasonal prediction GCMs and simple statistical models and demonstrated that the GCMs were not competitive with statistical methods (21). In a seemingly counterintuitive result, he also demonstrated that simple linear statistics were able to better predict the behavior of atmospheric GCMs than were the GCMs themselves with available computing resources. These results have held up to the present time and continue to signal caution that large model development efforts are not always the best way to make progress on atmospheric prediction at all time and space scales. Follow-up work led to methods for statistically reducing error in GCM output to improve forecast skill (25).

Numerical prediction of tropical storm frequency

While GCMs have distinct limitations, they can also have some powerful capabilities. While working with seasonal prediction models, Anderson noticed that certain convective parameterizations could produce vortices that were reminiscent of tropical storms. In low-resolution GCMs, these vortices were far larger than real tropical storms, but had tracks and frequencies that were similar to those for real storms (72). With graduate student Frederic Vitart, Anderson explored these capabilities and their implications for seasonal prediction of tropical storms (15, 26, 29). The work also led to conjectures about climate model capabilities to predict changes in tropical storm frequency and location in the presence of climate change (20). This work led to operational GCM-based seasonal tropical storm prediction systems at NCEP and later at ECMWF. Vitart has extended this work and now leads a seasonal prediction effort at ECMWF. Andersons expertise on this area led to invitations to speak at seasonal prediction centers around the world and his participation in workshops at Bermudas Risk Prevention Initiative (91).

Ensemble prediction and ensemble tools

Anderson became one of the earliest researchers using model ensembles for prediction in attacking the seasonal forecast problem (67, 68, 78). Initial research on this problem focused on so-called AMIP ensembles in which an ensemble of AGCM integrations are forced by observed SSTs. The results were interpreted as providing an upper bound on the predictability of the atmosphere since SST forecasts themselves would be needed (12, 22, 71). This research later included ensemble predictions with the fully-coupled atmosphere/ocean/land model developed by Andersons group at GFDL (24, 73, 74). Andersons group produced the first results on predictability in coupled ocean-atmosphere models and he also published some of the earliest work looking for seasonal predictability from land surface intial conditions (30, 80, 87). Anderson led the development of tools for interpreting ensemble simulations and prediction while at GFDL. Tools developed included the ranked histogram that became the most widely applied tool for evaluating the quality of ensembles (13, 71). He also pioneered the use of non-parametric statistics such as the Kolmogorov-Smirnov test for detecting statistically significant differences in ensembles (12, 70). These tools are fundamental attempts to detect predictable patterns in ensembles for all types of applications. Tools for detecting the most predictable patterns from ensembles were also developed in collaboration with postdocs at GFDL (18). Anderson participated in several national and international groups that were exploring the potential skill of seasonal prediction (23) during the late 1990s and has served on a variety of review and advisory panels in this field.

Ensemble data assimilation

Exploration of ensemble prediction and predictability quickly demonstrated that initial conditions for ensemble members were crucial to the results. Anderson began to explore methods for generating appropriate initial conditions in the mid-1990s at the same time as similar research was begun at operational prediction centers. While the prediction centers generated heuristic methods for ensemble generation and pushed them into operations, Anderson was the first to point out that dynamical constraints were essential to producing good ensemble predictions (10). His work was also among the early suggestions that ensemble forecasts should be a random sample from the underlying conditional distribution (14, 17). In the last decade, Anderson has become a leader in ensemble data assimilation. The first correctly derived ensemble filter algorithms were developed in Europe in 1998. Shortly after that time, Anderson produced the first ensemble assimilation algorithms derived directly from Bayes theorem without the need for intermediate reliance on the Kalman filter (19, 90). He also introduced the use of kernel methods for producing non-linear, non-Gaussian ensemble data assimilation (85, 90). To date, these non-linear methods have proved too costly for operational atmospheric prediction models but further enhancements in efficiency may yet allow them to become ensemble assimilation tools of choice.Andersons insight into reducing the statistical calculations to a sequence of scalar updating steps has proven to be an important strategy for improving data assimilation algorithms and implementing them for parallel computation (32,34).

To avoid problems of filter divergence that plagued ensemble filters, Anderson also made early contributions to the inflation method for dealing with model and assimilation system error (27). Inflation has become, in various forms, a central component of all ensemble assimilation systems in large models.

Adaptive methods to support a generic research testbed

By 2002, ensemble filters were being widely applied, but they still required expert knowledge for localization (33). Traditionally, localization in ensemble filters has restricted the impact of observations to some set of physically adjacent state variables. Both expert knowledge about models, observations, and ensemble data assimilation and a large amount of tuning are needed to find appropriate localizations for good assimilation performance. Anderson realized that localization could instead be reformulated as a response to sampling error in ensemble assimilation. He developed a method to automatically determine appropriate localizations using ensembles of ensembles. This method, the hierarchical filter, can help model developers or observationalists to use ensemble data assimilation systems effectively without developing expertise on ensemble filtering theory (45). The hierarchical filter also demonstrates that physical distance is not the appropriate measure for localization. Instead, an information or correlation distance is more appropriate. Observations should be allowed to influence state variables when the ensemble system is able to identify significant sample correlations between the state variable and observation prior ensembles. This leads to much more powerful types of localization and adapts to more complicated relationships among the state variables. Proper localization of this type can greatly increase the quality of ensemble assimilations in prediction models.

In order to support the use of ensemble assimilation facilities across a variety of different models and physical systems it is also necessary to improve on the heuristic inflation algorithm. Recently, Anderson has developed hierarchical Bayesian techniques to detect model and assimilation system error (44). The same observations that are used for the traditional part of the assimilation are also used to detect inaccuracies in the prior ensemble variance. Bayes theorem is then used to adjust the ensemble variance. Initial implementations of this adaptive inflation remove the need for the tuning of the inflation factor. Work in press describes adaptive inflation algorithms that are even more powerful. These algorithms detect variance errors for each model state vector component and can correct for model and assimilation system errors that are a function of space, variable type, time, or all three (50, 51).

Anderson's current research continues to lead the ensemble data assimilation field (55). He is exploring ways to detect observation system errors at the same time as model error. Methods for reducing ensemble size while maintaining assimilation quality are also under development. Tracer assimilation and ensemble smoothing (47), in which future observations are used to generate state estimates are also work in progress.

The Flexible Modeling System

From 1992-2002, Anderson worked on the development of the Flexible Modeling System (FMS) at GFDL (62). FMS is a set of tools and a software infrastructure to support modeling of the coupled ocean/atmosphere/land system. While it proved relatively easy to develop tools, it was extremely difficult to convince researchers with existing models to adopt the tools. Becoming the head of the GFDL Experimental Prediction Group gave Anderson an opportunity to push forward by first developing an atmospheric model based on FMS and then a fully coupled model. Along the way, the concept of the 'exchange grid' for coupling component models was developed (38) and implemented in a highly-scalable fashion. Performance of the FMS-based models was so good that other modeling groups at GFDL were eventually compelled to adopt FMS. The resulting enhancements in efficiency and collaboration greatly accelerated progress at GFDL and were essential in GFDL being able to participate in the last two IPCC reports. Anderson's expertise on object-oriented modeling led to his participation in a sequence of panels exploring coding requirements for atmospheric modeling. The OSTP/USGCRP advisory panel on climate modeling, that generated the so-called 'Rood report', made recommendations about software engineering practices that were required to improve atmospheric modeling capabilities. Anderson was involved in a number of follow-on meetings that led to the initial ESMF proposal to NASA (31). FMS served as one of two prototypes for the initial ESMF effort. In particular, the FMS time_manager and the exchange grid capability was direct ancestors of central ESMF capabilities. FMS continues to support modeling at GFDL and is still providing insight for ESMF developers. The development of sound software infrastructure for atmospheric modeling has greatly accelerated scientific advances.

GFDL Bgrid Dynamical Core, Coupled Seasonal Prediction, and AM Climate Models

When Anderson took over the GFDL seasonal prediction group, a legacy spectral model was being used for research. From 1992 on, Anderson led an effort to develop more modern prediction models using the developing FMS framework. The goal was to have efficient models that could easily use a variety of different modular physical parameterizations. Under Anderson's leadership, the E-grid global model developed by Messinger was converted into a B-grid dynamical core. This is the dynamical core that is in the DART facility and is used as in data assimilation research by a number of research groups (e.g. 37). At the same time, a modular interface between dynamics and physics was developed along with versions of a number of physics packages. By 1998, the resulting GCM was being used for seasonal prediction at GFDL with particular focus on extratropical predictability and predictability of tropical storm frequency (70, 72, 76). As GFDL prepared for the next IPCC cycle, this model was adopted by the climate community at GFDL and became the AM model series (39) used in IPCC and is still in use at GFDL.

The Bgrid core was also coupled to the MOM3 ocean model under Andersons leadership using the prototype FMS exchange grid to form a coupled modular seasonal prediction model (73, 74, 80). This model evolved into the GFDL CM model series after Anderson's move to NCAR, but the underlying software framework has remained the same.

The Data Assimilation Research Testbed

The success of the object-oriented software tools developed for FMS and ESMF, along with the development of ensemble filtering algorithms for data assimilation, raised the possibility of developing a generic facility for ensemble data assimilation. In 2001, Anderson began development of the Data Assimilation Research Testbed (63). The goal was to build a data assimilation facility that could be applied to a wide range of models and observational datasets without requiring users to have any special data assimilation expertise. It was vital that very little coding be required to incorporate models or observations into the facility and that it run efficiently on a wide range of computing platforms (43). The key software engineering aspects of DART have been the design of interfaces between the assimilation facility and models or observations. The modularity of the functions in DART allows a modeler to focus on model improvements in the context of prediction without having to develop a companion data assimilation system. Conversely, DART can support research on data assimilation algorithms or new types of observations without requiring that the researcher have detailed knowledge of calling a numerical model. Although DART is computationally efficent and often gives skillful predictions compared to larger operational systems, it also gives university researchers, students and other small teams access to conduct state-of-the-art research in prediction and data assimilation. A number of models have been incorporated into DART including nearly a dozen large models like GCMs. DART is now being used at more than a dozen UCAR member universities, at government labs and prediction centers, and in private industry (Table 1).

The availablility of DART has facilitated scientific research on a wide range of problems. Although many of the collaborations that Anderson facilitates do not lead to coauthorship on publications, he is central to several activities at NCAR. Research on fundamental questions about the application of ensemble filters to prediction models have improved prediction (33, 35). Seasonal prediction in both low-order models (46) and coupled GCMs (36) has been a focus. Exploring the value of existing observations (37, 41, 40) or using new observation types such as GPS radio occultation (42, 48) has led to better uses of a variety of observations. Anderson has actively participated in the use of DART in several NCAR models including CAM (52, 53), WRF (48, 41), and middle atmosphere models (49).

Table 1 : Summary of DART applications and users

Models compatible with DART:

 

 

Many low-order (L63, L84, L96, L2004....)

 

 

Global 2-level PE model from NOAA/CDC

 

 

CAM 2.0, 3.0, 3.1 (global spectral model)

 

 

CAM 3.1 (global finite volume model)

 

 

GFDL FMS B-grid (global gridpoint dynamical core)

 

 

GFDL AM (global gridpoint model)

 

 

MIT GCM (from Jim Hansen, configured for annulus)

 

 

WRF (regional prediction model)

 

 

WRF column physics model

 

 

NCEP GFS (global spectral model; assisted by NOAA/CDC)

 

 

GFDL MOM3/4 (global gridpoint ocean model)

 

 

ROSE (upper atmosphere with chemistry)

 

 

Cane-Zebiak 5 (low-order tropical ocean/atmosphere)
    CMAQ (EPA disperson model)

Government and private industry users of DART:

    GFDL, CDC, CIRES, NRL, NSSL, SAMSI, Livermore, George Mason, Caltech, COLA,

 

  Argonne, JPL, Bell Labs

Universities using DART

    Wisconsin, MIT, Arizona, Oklahoma, Princeton, Chicago
    Utah, Colorado,Berkeley, Purdue, Istanbul Technical University,
    CU Denver, ETH Zurich, Warsaw University
 
Seoul National University, Kobe University,
 
Aachen University, UNC Charlotte, Virginia Tech
    Illinois, Delft, Courant, Washington, Duke