DART Manhattan Documentation

Welcome to the IMPROVED Data Assimilation Research Testbed - DART Manhattan.

DART/CAM spaghetti plot

00Z 01 Feb 2003. 20 of 80 member ensemble. T85 CAM GPH at 500hPa.

Sensitiviy Analysis on Hurricane Katrina

Ensembles allow exploration of circulation on hurricane position.

GPS RO observations

Rapid support for new observation types.

Chemical Weather Forecasting and Analysis

CAM-Chem, WRF-Chem, MOPITT, MODIS

DART, WRF, and MARS

 

CAM inflation values for U Wind

after 1 month of adaptive inflation.

CAM inflation values for U Wind

after 1 month of DAMPED adaptive inflation.

DART/CMAQ CO assimilation

 

Planetary Boundary Layer

Probabilistic nowcasting with a single column model.

A good rank histogram

Lorenz '96 - the true state is as likely as any of the ensemble members.

A bad rank histogram (filter divergence)

Lorenz '96 - the true state generally falls outside the ensemble members.

Tracer advection

One of the tutorial examples explores sources/sinks.

Impact of PS observations on V winds

DART has novel algorithms to explore localization.

Ensemble Trajectories

DART has routines to explore and diagnose.

Ensemble Trajectories

Explore the impact of the increments.

time series of 'total error'

Quantify the effects of observations or parameter settings.

CAM@T85 observation space performance

DART natively supports direct comparisons with observations.

CAM@T85 assimilation performance

See if all the observations are being used.

Download icon for Manhattan Release

DART has been reformulated to better support the ensemble data assimilation needs of researchers who are interested in native netCDF support, less filesystem I/O, better computational performance, good scaling for large processor counts, and support for the memory requirements of very large models. Manhattan has support for many of our larger models (WRF, POP, CAM, CICE, CLM, ROMS, MPAS_ATM, ...) with many more being added as time permits. The hallmarks of DART; tutorials, examples, extensive documentation, introductory materials, and performance diagnostics are maintained in an easier-to-grasp organization:

  1. assimilation_code ( programs and support modules )
  2. build_templates ( hardware/compiler options )
  3. diagnostics
  4. documentation
  5. developer_tests
  6. models
  7. observations ( observation converters and forward operators )

[top]


Data Assimilation Research Testbed - DART


DART is a community facility for ensemble DA developed and maintained by the Data Assimilation Research Section (DAReS) at the National Center for Atmospheric Research (NCAR). DART provides modelers, observational scientists, and geophysicists with powerful, flexible DA tools that are easy to implement and use and can be customized to support efficient operational DA applications. DART is a software environment that makes it easy to explore a variety of data assimiliation methods and observations with different numerical models and is designed to facilitate the combination of assimilation algorithms, models, and real (as well as synthetic) observations to allow increased understanding of all three. DART includes extensive documentation, a comprehensive tutorial, and a variety of models and observation sets that can be used to introduce new users or graduate students to ensemble DA. DART also provides a framework for developing, testing, and distributing advances in ensemble DA to a broad community of users by removing the implementation-specific peculiarities of one-off DA systems.

6 frame animation demonstrating the assimilation schematic

DART employs a modular programming approach to apply an Ensemble Kalman Filter which nudges the underlying models toward a state that is more consistent with information from a set of observations. Models may be swapped in and out, as can different algorithms in the Ensemble Kalman Filter. The method requires running multiple instances of a model to generate an ensemble of states. A forward operator appropriate for the type of observation being assimilated is applied to each of the states to generate the model's estimate of the observation.

The DART algorithms are designed so that incorporating new models and new observation types requires minimal coding of a small set of interface routines, and does not require modification of the existing model code. Several comprehensive atmosphere and ocean general circulation models (GCMs) have been added to DART by modelers from outside of NCAR, in some cases with less than one person-month of development effort. Forward operators for new observation types can be created in a fashion that is nearly independent of the forecast model, many of the standard operators are available 'out of the box' and will work with no additional coding. DART has been through the crucible of many compilers and platforms. It is ready for friendly use and has been used in several field programs requiring real-time forecasting. The DART programs have been compiled with many Fortran 90 compilers and have run on linux compute-servers, linux clusters, OSX laptops/desktops, SGI Altix clusters, IBM supercomputers based on both Power and Intel CPUs, and Cray supercomputers.


[top]


What is Data Assimilation ... "DA"?


Loosely speaking, data assimilation is any method of making models utilize the information from observations of the system being modeled. Good assimilations make the modeled state more consistent with the observations; particularly future observations. Effective data assimilation systems tend to make forecasts more accurate - within the ability of the model, naturally - and tend to make 'hindcasts' (the model state immediately after the observations have been assimilated) more accurately reflect the state of the system.

The low-order models (Lorenz '63, '96, etc.) are a great place to start learning about data assimilation. These dynamical models were created as simple analogues to chaotic systems. Once you get comfortable running and exploring assimilations with the low-order models, you are well on your way to understanding assimilations with high-order (more realistic) models.



[top]


Schematic of Ensemble Data Assimilation - from the DAReS Perspective


This is the DART view of ensemble data assimilation for models that run as separate executables. Starting at the top and working clockwise: Everything is driven by a Fortran namelist and the presence or absence of observations. A Fortran executable named filter reads a namelist, an initial state for the ensemble, and a file containing observations and goes to work. Given the observations and an initial state, filter assimilates the observations and then determines how far to advance the model (using information from the namelist and the observation file). filter forks a shell script to the system that is responsible for advancing the model. [The script is responsible for the lower portion of the diagram.] The model advances each ensemble member either in turn or all-at-once. The shell script finishes and signals filter to continue. We are now back at the beginning and the cycle continues as long as there are observations to assimilate or until the control information in the Fortran namelist is met. When that happens, a set of restart files is written (suitable to continue an experiment with more observations) and diagnostic files are written. These diagnostic files allow for the exploration of the assimilation before and after each assimilation step and for exploration of the assimilation in 'observation space'; each real observation is paired with the estimates of the observation from all of the ensemble members (if desired). Minimally, the ensemble mean estimate of the observation and the ensemble spread of the estimates is recorded.


[top]



DART code distributions


The DART code is distributed via a Subversion (SVN) repository. Anonymous access is allowed, and the repository code is read-only for everyone except the DART development team. DART is distributed as source code, so you must be prepared to build the parts of the system you need before you can run it.


DART continues to evolve. Using subversion makes it easy to update and compare your checked-out version of the code with the latest repository version of the code. If you are not familiar with the svn command (the client application of subversion), you should take a stroll through Tim's svn primer. Or there are several GUI programs that help you manage, check out, and merge subversion distributions. If you cannot use svn (e.g. because you are behind a firewall that does not permit subversion access), please email the DART team and we may be able to send you a tar file as a last resort.


The DART development team keeps released versions of the code which are stable (don't change) except for bug fixes. Generally we recommend users check out one of these versions. The DART development team makes frequent updates to the trunk version of the code as new features are developed. Those users who want to use recently added features are welcome to check out the trunk, but they should be prepared to work around possible non-backwards compatible changes and more lightly tested code than the released versions.


We request that you register for DART and afterwards you will be redirected to instructions on how to download a version of the DART code. Registration helps us track how many people are using our code, and allows us to contact current users in case of bugs or major updates. The DART mailing list is a very low-traffic list -- perhaps 4 emails per year -- so PLEASE use a real email address when signing up. We solemnly swear to protect your email address like it is our own! Even local NCAR users or users who have registered in the past are encouraged to reregister when downloading new versions. Thank you for your understanding.


version date instructions most notable change(s)
Manhattan 15 Mar 2017 [doc] Native netCDF support, less filesystem I/O, better scaling, better computational performance, supportes huge memory models. Simplified organizational layout. Supports ROMS, CICE, WRF-CHEM ...
change log
lanai 13 Dec 2013 [doc] Support for the CESM climate components under the CESM framework; the MPAS models, the NOAH land model, the GITM ionosphere model, the NOGAPS atmosphere model, the NAAPS aerosol model, and the SQG surface quasi-geostrophic model. Support for many new chemistry and aerosol observation types, support for many new observations sources, many new diagnostic routines, and new utilities.
change log
trunk today included in distribution varies, use svn log --revision ####:HEAD to see log messages about changes since revision ####.
kodiak 30 Jun 2011 [doc] Damped Inflation, Sampling Error Correction, Boxcar Kernel Filter option, support for new models, new observation types, new diagnostics, new utilities.
... and more going all the way back to ...
easter 8 March 2004 [doc] initial release


Please suggest ways for us to improve DART.

[top]