



The first document you should read is prep_bufr.html.



======================================================================
The rest of this document is development comments.
======================================================================


How to generate decoded NCEP PREPBUFR inter-mediate text data 
As well as DART observation sequence files (daily)
kdr; only daily?

Introduction
x kdr; add section about what files we're starting with, and what's necessary
     to translate them?  Refer to .../docs

  This document describes how to generate decoded NCEP reanalysis PREPBUFR 
inter-mediate text data as well as use of the text files to generate DART daily 
observation sequence files. 

   The decoding package is available as a tar file, prepbufr.tar. 
   tar -xvf prepbufr.tar and you get the following directories:

/src    Source code of the NCEP PREPBUFR decoder
/lib    NCEP PREPBUFR library
/convert_bufr  source code to convert the big endian PREPBUFR files to little
               endian files and a script to compile the code.
/install  A script to install the NCEP PREPBUFR decoder and the NCEP PREPBUFR library.
/exe    Executables of the decoder and converter.
/data   Where the NCEP PREPBURE files (prepqm****) are  
kdr; "PREPBUFR"?
/work   Where we run the script to do the decoding.
/docs   Some useful background informations about NCEP PREPBUFR observations.


======================================================================
The decoding program: /src/prepbufr.f 
======================================================================

 The program prepbufr.f is used to decode the NCEP reanalysis BUFR data into 
intermediate text files. This program was originally developed by NCEP. It 
has been modified to output dry temperature, specific humidity, and wind 
components (U/V) of conventional radiosonde, aircraft reports, and satellite 
cloud motion derived wind. The NCEP quality control indexes for these 
observations based on NCEP forecasts are also output and used in DART 
observation sequence files.
kdr; are more types of obs available? (PS, radiances, ?)  
     How much work to get them?

Program /convert_bufr/grabbufr.f

kdr; I see CVS directory in there; should it still be under DART control?

 The program grabbufr.f is used to convert the big endian format NCEP PREPBUFR 
data into little endian format for use on the LINUX computers only. For SGI, IBM 
SP, and Sun computers, this convert program is not needed.

kdr; is it used automatically, or does the use need to execute it?


======================================================================
How to install the NCEP BUFR decoding program
======================================================================

Required compilers

  The NCEP PREPBUFR decoding program is written in Fortran 77 and the grabbufr.f 
code is written in Fortran 90. The first code can be successfully compiled on 
LINUX computers using pgi90, f77 on SGI computers, and xlf on IBM SP systems. 
The 2nd code can be compiled with pgf90.  
kdr;   Add "See note below about compiling under modules. "

  To compile the PREPBUFR libraries and the decoding program, go to the
install directory /install.  Set the CPLAT variable in the install.sh to reflect 
the correct platform.  Currently CPLAT = linux.

  Execute the install.sh script to complete the compilations for the main 
decoding program and the NCEP PREPBUFR library.

  For platforms with little endian format, like linux systems, go to 
/convert_bufr/ and run convert_bufr.csh to compile the converting code.

   The executables are placed in /exe/, i.e., prepbufr.x and conv.x).

Platforms tested:

  The code is tested on Coral, Ocotillo.mmm, Tempest, Blusky.

  Special note for Coral: the default compiler is Intel64. To use the pgf90, the 
following module updates are needed:

  module rm intel64 netcdf64 mpich64
  module add pgi32

  type which pgf90 to see if pgf90 is available.


===============================================================
How to get the NCEP Reanalysis BUFR format data from NCAR MSS?
===============================================================

  The NCEP PREPBUFR files (prepqm*) can be found within the NCEP reanalysis
dataset, ds090.0 (until August 2003) in NCAR MSS. 
kdr; meaning files run through 8/03, or they'll be in ds090.0 until 8/03?

    To find the files:
    go to http://dss.ucar.edu/pub/reanalysis/
kdr; announcement there says
     Public (non-restricted) version of 200309-200602 prepqm files are released.
    Click "Data Archive Summary"
    Click "Complete Tar list of Table of Contents of All VSN's"
    Look at the years you want and click on the ds090.0 next to the year mark.

Find the data set you want by searching for prepqmYYMMDDHH, with YY, MM, DD, HH = 
the year, month, day, and hour you want.
The prepqm file will be grouped with others in a weekly tar file. 
Each tar file has a unique file number like "A#####". 

    For example, for January of 2003, The 4 MSS TAR files are: A21899, A21900, 
A21901, A21902. These files are unblocked files. 

   The decoding program requires blocked version of the files. The corresponding 
blocked files are A21899.blk, A21900.blk, A21901.blk, and A21902.blk.
(These names don't appear in the web page, but are found on the mass store.
Each is about 400 Mb)

  Within subdirectory prepbufr/data use 
    msread -fbi filename.tar /DSS/A*****.blk 

to read the NCEP BUFR data tar files from NCAR MSS (on Coral, tempest, Bluesky, etc.)

   Then
   tar -xvf filename.tar

   You will get individual 6-hourly NCEP PREPBUFR data files for the 
observations in the +/- 3-hours time window of 06Z, 12Z, 18Z, and 00Z of each 
day. 

Run the decoding program as follows.

In prepbufr/work/prepbufr.csh set the appropriate values of the year, month, begin_day, and 
last_day of the period you desire.  Execute prepbufr.csh in the work directory.
kdr; instructions for running this as a batch job?  BSUB is there.  QSUB?
     Script does not cd to proper directory.  Other batch problems?
     conv.x (grabbufr) is executed unconditionally; is this script only good for Linux,
        which requires translation from big- to little-endian?

Currently, the script generates daily Decoded PREPBUFR text data which contains the 
observations within the time window of 3:00Z of the day to 3:00Z of next day. 
These daily output text files are named as temp_obs.yyyymmdd. 

kdr; why did temp_obs.19980731 show up in .../work, even though I was just doing the first week?

  These text PREPBUFR data files can then be read by 
DART/ncep_obs/create_real_obs.f90 to generate the DART daily observation 
sequence files.

======================================================= 
How to generate DART daily observation sequence files?
======================================================= 
  Go to DART/ncep_obs/work

   The create_real_obs.f90 is used to convert the decoded PREPBUFR text data to 
DART observation sequence files. Currently, this program produces daily 
observation sequence from 3:01Z of the day to 3:00Z of the next day.

  Use mkmf_create_real_obs to generate the makefile for compile 
create_real_obs.f90. Type "make" to get the executable.

  Rename input.nml.create_real_obs_default to input.nml
  Make appropriate changes to the namelist of input.nml.
 
  The selection of any combinations of the specific observation kinds (T, Q, 
U/V, and surface pressure) and types (radiosonde, aircraft reports, or satellite 
wind, etc.) can be done in the namelist "ncepobs_nml". You may also want to change 
the location of the decoded PREPBUFR text data files in the namelist. Additional 
information about the namelist is available from the "create_real_obs.html".

kdr create_real_obs.html not available in repository, as of Pre-J.

  Sample namelists are as below:
&ncepobs_nml
   year = 2003,
   month = 1,
   day = 1,
   tot_days = 31,
   max_num = 700000,
   ObsBase = '/ptmp/hliu/decoded_bufr_data/temp_obs.'
   select_obs  = 0,
   ADPUPA = .true., 
   AIRCAR = .true., 
   AIRCFT = .true., 
   SATEMP = .false., 
   SFCSHP = .false., 
   ADPSFC = .false., 
   SATWND = .true.,
   obs_U  = .true., 
   obs_V  = .true., 
   obs_T  = .true.,
   obs_PS = .false.,
   obs_QV = .false.,
   daily_file = .true./

&obs_sequence_nml
   write_binary_obs_sequence = .true.  /

   This will produces daily (or 12-hourly for CAM) observation sequence files 
for the period of Jan. 2003 which have the selected observation types and kinds. 
The output files looks like "obs_seqyyyymmdd".

  Please note that the current DART/ncep_obs/creat_real_obs.f90 generates daily 
(or 12-hourly for CAM) observation sequence files which contains observation in 
the time window of 3.01Z of the day to 3.00Z of next day. So, generation of each 
such daily DART observation sequence files may require the decoded NCEP BUFR 
text data files for that day AND next day as well (since the 2nd BUFR file may 
also contain the observations at 3:00Z of the next day). 
 
  For example, to generate the observation sequence for jan 1, 2003, the decoded 
NCEP PREPBUFR text files for Jan 1 and 2, 2003 are needed.

==================================================================
Notes about documenting this and/or making sure all of it is in repository.

   DART/ncep_obs/read_bufr.f  is almost identical to 
Obs/prepbufr/src/prepbufr.f   read_bufr.f has more reads (adiag...) and writes of something 
prep_bufr.f wants an input file 'prepqm.little' (endian) instead of 'prepqm'

