INSTRUCTIONS / NAMELIST / FILES / REFERENCES / BUGS / PLANS /

PROGRAM prepbufr

Contact: Hui Liu
Reviewers:  
Revision: $Revision: 1.1 $
Release Name: $Name: $
Change Date: $Date: 2006/06/09 22:45:39 $
Change history: see CVS log

OVERVIEW

Translating NCEP BUFR files into DART obs_seq.out files (input file to filter) is a 2 stage process. The first stage uses NCEP software to translate the BUFR file into an "intermediate" text file. This is described in this document. The second step is to translate the intermediate files into obs_seq.out files, which is done by create_real_obs, as described in create_real_obs .




INSTRUCTIONS

The prep_bufr package is free-standing and has not been completely assimilated into the DART architecture. It also requires adaptation of the sources codes and scripts to the computing environment where it will be run. It is not so robust that it can be controlled just with input parameters. It may not have the same levels of error detection and warning that the rest of DART has, so the user should very careful about checking the end product for correctness.

Installation of the NCEP BUFR decoding program

This package is currently (as of release J) organized as .../DART/ncep_obs/prep_bufr/...:
src           Source code of the NCEP PREPBUFR decoder
lib           NCEP PREPBUFR library
convert_bufr  Source code (grabbufr) to convert the big endian PREPBUFR files to little
               endian files, and a script to compile the code.
install       A script to install the NCEP PREPBUFR decoder and the NCEP PREPBUFR library.
exe           Executables of the decoder and converter.
data          Where the NCEP PREPBUFR files (prepqm****) are loaded into from the NCAR Mass Store.
work          Where we run the script to do the decoding.
docs          Some background information about NCEP PREPBUFR observations.

Program /convert_bufr/grabbufr.f

The program grabbufr.f is used to convert the big endian format NCEP PREPBUFR data into little endian format, when needed for use on the Intel® processors. The grabbufr.f code is written in Fortran 90, and can be compiled with pgf90. It has been compiled with gfortran® on an Intel® based Mac®.

This program reads the whole BUFR file into memory, and needs to know the size of the file (in bytes). This information is obtained in the section "Use STAT function ..." by uncommenting the statements

c        do i = 1,size(JSTAT)
c           print*,'JSTAT(',i,') = ,'JSTAT(i)
c        enddo
c        EXIT(98)
the first time this program is run. The file size (determined from ls -l) will match one of the elements of JSTAT. That element should be assigned to KBYTES in the subsequent runs, which must have the JSTAT print out and STOP commented out again, and which actually convert the BUFR files.

The decoding program: /src/prepbufr.f

The program prepbufr.f is used to decode the NCEP reanalysis BUFR data into intermediate text files. This program was originally developed by NCEP. It has been modified to output surface pressure, dry temperature, specific humidity, and wind components (U/V) of conventional radiosonde, aircraft reports, and satellite cloud motion derived wind. There are additional observation types on the BUFR files, but using them they would require significant modifications of prepbufr and require detailed knowledge of the NCEP BUFR files. The NCEP quality control indexes for these observations based on NCEP forecasts are also output and used in DART observation sequence files. The NCEP PREPBUFR decoding program is written in Fortran 77 and can be successfully compiled on Linux computers using pgi90, f77 on SGI computers, and xlf on IBM SP systems. an Intel® based Mac® with gfortran®.

If your operating system uses modules you may need to remove the default compiler and add the one desired for this package. For example

To compile the PREPBUFR libraries and the decoding program, go to the install directory .../install. Set the CPLAT variable in the install.sh to reflect the correct platform. CPLAT = linux is the default. Execute the install.sh script to complete the compilations for the main decoding program and the NCEP PREPBUFR library.

For platforms with little endian format, go to /convert_bufr/ and run convert_bufr.csh to compile the converting code (grabbufr).

The executables (i.e., prepbufr.x, prepbufr_03Z.x and grabbufr.x) are placed in .../prep_bufr/exe.

Platforms tested: The code has been tested on NCAR systems:

Getting the NCEP Reanalysis BUFR format data from NCAR MSS.

The NCEP PREPBUFR files (prepqmYYMMDDHH) can be found within the NCEP reanalysis dataset, ds090.0, on NCAR Mass Store System (MSS).

To find the files:

The prepqm file will be grouped with others in a weekly tar file. Each tar file has a unique file number like "A#####". For example, for January of 2003, the 4 MSS TAR files are: A21899, A21900, A21901, A21902. These are unblocked files. The decoding program requires blocked version of the files. The corresponding blocked files are A21899.blk, A21900.blk, A21901.blk, and A21902.blk. These names don't appear in the web page, but are found on the mass store. Each is about 400 Mb

From a machine with access to the MSS use

where ##### is the data set number, to read the NCEP BUFR data tar files. Put the filename.tar in subdirectory prep_bufr/data

Then

will yield individual 6-hourly NCEP PREPBUFR data files for the observations in the +/- 3-hours time window of 00Z, 06Z, 12Z, and 18Z of each day. Note that DART obs_seq files are organized such that a 24 hour file with 4 observation times would contain observations from 3:01Z to 3:00Z of the next day, centered on 6Z, 12Z, 18Z and "24Z". In addition, there are some observations at 3:00Z on the BUFR file labelled with 06Z. Then, in order to make a full day intermediate file incorporating all the required obs from the "next" day, you'll need the BUFR files through 6Z of the day after the last day of interest. For example, to generate the observation sequence for Jan 1, 2003, the decoded NCEP PREPBUFR text files for Jan 1 and 2, 2003 are needed, and hence the BUFR files are needed.

Running the NCEP BUFR decoding program

In prep_bufr/work/prepbufr.csh set the appropriate values of the year, month, first day, and last day of the period you desire, and the variable "convert" to control conversion from big- to little-endian. Confirm that the raw BUFR files are in ../data, or that prepbufr.csh has been changed to find them. Execute prepbufr.csh in the work directory. It has code for running in the LSF batch environment, but not PBS.

Currently, the script generates daily decoded PREPBUFR text data which contains the observations within the time window of 3:01Z of the day to 3:00Z of next day. These daily output text files are named as temp_obs.yyyymmdd. These text PREPBUFR data files can then be read by DART/ncep_obs/ create_real_obs . to generate the DART daily observation sequence files.




OTHER MODULES USED




INCLUDE file

The prepbufr package doesn't use any namelists, but prepbufr.prm contains some parameters, while other parameters are set in .../prep_bufr/work/prepbufr.csh



FILES




REFERENCES

.../DART/ncep_obs/prep_bufr/docs/* (NCEP text files describing the BUFR files)


KNOWN BUGS




FUTURE PLANS

Further development to get observations directly from original
(undecoded) NCEP BUFR files.