PUBLIC INTERFACE ~ PUBLIC DATA ~ PUBLIC ROUTINES ~ NAMELIST ~ DIAGNOSTIC FIELDS ~ ERROR MESSAGES ~ REFERENCES ~ NOTES

Module diag_manager_mod

Contact:  Matt Harrison ,  Giang Nong ,  Seth Underwood
Reviewers: 
Change History: WebCVS Log

OVERVIEW

diag_manager_mod is a set of simple calls for parallel diagnostics on distributed systems. It is geared toward the writing of data in netCDF format.

diag_manager_mod provides a convenient set of interfaces for writing data to disk. It is built upon the parallel I/O interface of FMS code /shared/mpp/mpp_io.F90.

A single group of calls to the diag_manager_mod interfaces provides data to disk at any number of sampling and/or averaging intervals specified at run-time. Run-time specification of diagnostics are input through the diagnostics table.

Usage of diag_manager includes the following steps:
  1. Create diag_table as described in diag_table_tk documentation.
  2. Call diag_manager_init to initialize diag_manager_mod.
  3. Call register_diag_field to register the field to be output. NOTE: ALL fields in diag_table should be registered BEFORE the first send_data call
  4. Call send_data to send data to output fields
  5. Call diag_manager_end to exit diag_manager
Features of diag_manager_mod:
  1. Ability to output from 0-D array (scalars) to 3-D arrays.
  2. Ability to output time average of fields that have time dependent mask.
  3. Give optional warning if register_diag_field fails due to misspelled module name or field name.
  4. Check if a field is registered twice.
  5. Check for duplicate lines in diag_table.
  6. diag_table can contain fields that are NOT written to any files. The file name in diag_table of these fields is null.
  7. By default, a field is output in its global grid, it is now possible to output a field in a region specified by user, see send_data for more details.
  8. To check if the diag table is set up correctly, user should set debug_diag_manager=.true. in diag_manager namelist, then the the content of diag_table is printed in stdout.
  9. New optional format of file information in diag_table.
    It is possible to have just one file name and reuse it many times. A time string will be appended to the base file name each time a new file is opened. The time string can be any combination from year to second of current model time.

    Here is an example of file information:
    "file2_yr_dy%1yr%3dy",2,"hours",1,"hours","Time", 10, "days", "1 1 7 0 0 0", 6, "hours"
    From left to right we have:
    • file name
    • output frequency
    • output frequency unit
    • Format (should always be 1)
    • time axis unit
    • time axis name
    • frequency for creating new file
    • unit for creating new file
    • start time of the new file
    • file duration
    • file duration unit.
    The 'file duration', if absent, will be equal to frequency for creating a new file.

    Thus, the above means: create a new file every 10 days, each file will last 6 hours from creation time, no files will be created before time "1 1 7 0 0 0".

    In this example the string 10, "days", "1 1 7 0 0 0", 6, "hours" is optional.

    Keyword for the time string suffix is %xyr,%xmo,%xdy,%xhr,%xmi,%xsc where x is mandatory 1 digit number specifying the width of field used in writing the string
  10. New time axis for time averaged fields.
    Users can use a namelist option to handle the time value written to time axis for time averaged fields.

    If mix_snapshot_average_fields=.true. then a time averaged file will have time values corresponding to ending time_bound e.g. January monthly average is labeled Feb01. Users can have both snapshot and averaged fields in one file.

    If mix_snapshot_average_fields=.false. The time value written to time axis for time averaged fields is the middle on the averaging time. For example, January monthly mean will be written at Jan 16 not Feb 01 as before. However, to use this new feature users should separate snapshot fields and time averaged fields in different files or a fatal error will occur.

    The namelist default value is mix_snapshot_average_fields=.false.
  11. Time average, Max and Min
    In addition to time average userscan also get Max or Min value during the same interval of time as time average. For this purpose, in the diag table users must replace .true. or .false. by "max" or "min".

    Currently, Max and Min are not available for regional output.
  12. standard_name is added as optional in register_diag_field.
  13. When namelist variable debug_diag_manager = .true. array bounds are checked in send_data.
  14. Coordinate attributes can be written in the output file if the argument "aux" is given in diag_axis_init. The corresponding fields (geolat/geolon) should also be written to the same file.

OTHER MODULES USED

time_manager_mod
      mpp_io_mod
         fms_mod
         mpp_mod
   diag_axis_mod
   diag_util_mod
   diag_data_mod
 diag_output_mod
   diag_grid_mod
   constants_mod

PUBLIC INTERFACE

send_data:
Send data over to output fields.
register_diag_field:
Register Diagnostic Field.
send_tile_averaged_data:
Send tile-averaged data over to output fields.
register_static_field:
Register Static Field.
average_tiles:
diag_manager_end:
Exit Diagnostics Manager.
closing_file:
Replaces diag_manager_end; close just one file: files(file)
diag_manager_init:
Initialize Diagnostics Manager.
get_base_time:
Return base time for diagnostics.
get_base_date:
Return base date for diagnostics.
need_data:
Determine whether data is needed for the current model time step.
set_diag_filename_appendix:
init_diurnal_axis:
Finds or initializes a diurnal time axis and returns its' ID.

PUBLIC DATA

None.

PUBLIC ROUTINES

  1. send_data

     
send_data (diag_field_id, field, time, is_in, js_in, ks_in, mask, rmask, ie_in, je_in, ke_in, weight)
    DESCRIPTION
    send_data is overloaded for fields having zero dimension (scalars) to 3 dimension. diag_field_id corresponds to the id returned from a previous call to register_diag_field. The field array is restricted to the computational range of the array. Optional argument is_in can be used to update sub-arrays of the entire field. Additionally, an optional logical or real mask can be used to apply missing values to the array.

    If a field is declared to be mask_variant in register_diag_field logical mask should be mandatory.

    For the real mask, the mask is applied if the mask value is less than 0.5.

    By default, a field will be written out entirely in its global grid. Users can also specify regions in which the field will be output. The region is specified in diag-table just before the end of output_field replacing "none".

    For example, by default:

    "ocean_mod","Vorticity","vorticity","file1","all",.false.,"none",2

    for regional output:

    "ocean_mod","Vorticity","vorticity_local","file2","all",.false.,"0.5 53.5 -89.5 -28.5 -1 -1",2

    The format of region is "xbegin xend ybegin yend zbegin zend". If it is a 2D field use (-1 -1) for (zbegin zend) as in the example above. For a 3D field use (-1 -1) for (zbegin zend) when you want to write the whole vertical extent, otherwise specify real coordinates. The units used for region are the actual units used in grid_spec.nc (for example degrees for lat, lon). Fatal error will occur if region's boundaries are not found in grid_spec.nc.

    Special note when using regional output: result files containing regional outputs should be different from files containing global (default) output. It is a FATAL error to have one file containing both regional and global results. For maximum flexibility and independence from PE counts one file should contain just one region.

    Time averaging is supported in regional output.

    Physical fields (written in "physics windows" of atmospheric code) are currently fully supported for regional outputs.

    Note of dimension of field in send_data

    Most fields are defined in data_domain but used in compute domain. In send_data users can pass EITHER field in data domain OR field in compute domain. If data domain is used, users should also pass the starting and ending indices of compute domain (isc, iec ...). If compute domain is used no indices are needed. These indices are for determining halo exclusively. If users want to ouput the field partially they should use regional output as mentioned above.

    Weight in Time averaging is now supported, each time level may have a different weight. The default of weight is 1.


    INPUT
    diag_field_id   
       [INTEGER]
       [INTEGER]
       [INTEGER]
       [INTEGER]
    field   
       [REAL]
       [REAL, DIMENSION(:)]
       [REAL, DIMENSION(:,:)]
       [REAL, DIMENSION(:,:,:)]
    time   
       [TYPE(time_type), OPTIONAL]
       [TYPE(time_type)]
       [TYPE(time_type)]
       [TYPE(time_type)]
    is_in   
       [Not Required]
       [INTEGER, OPTIONAL]
       [INTEGER, OPTIONAL]
       [INTEGER, OPTIONAL]
    js_in   
       [Not Required]
       [Not Required]
       [INTEGER, OPTIONAL]
       [INTEGER, OPTIONAL]
    ks_in   
       [Not Required]
       [Not Required]
       [Not Required]
       [INTEGER, OPTIONAL]
    mask   
       [Not Required]
       [LOGICAL, DIMENSION(:), OPTIONAL]
       [LOGICAL, DIMENSION(:,:), OPTIONAL]
       [LOGICAL, DIMENSION(:,:,:), OPTIONAL]
    rmask   
       [Not Required]
       [REAL, DIMENSION(:), OPTIONAL]
       [REAL, DIMENSION(:,:), OPTIONAL]
       [REAL, DIMENSION(:,:,:), OPTIONAL]
    ie_in   
       [Not Required]
       [INTEGER, OPTIONAL]
       [INTEGER, OPTIONAL]
       [INTEGER, OPTIONAL]
    je_in   
       [Not Required]
       [Not Required]
       [INTEGER, OPTIONAL]
       [INTEGER, OPTIONAL]
    ke_in   
       [Not Required]
       [Not Required]
       [Not Required]
       [INTEGER, OPTIONAL]
    weight   
       [Not Required]
       [REAL, OPTIONAL]
       [REAL, OPTIONAL]
       [REAL, OPTIONAL]

    OUTPUT
    err_msg   
       [CHARACTER(len=*), OPTIONAL]
       [CHARACTER(len=*), OPTIONAL]
       [CHARACTER(len=*), OPTIONAL]
       [CHARACTER(len=*), OPTIONAL]

  2. register_diag_field

    INTEGER FUNCTION register_diag_field (module_name, field_name, axes, init_time, long_name, units, missing_value, range, mask_variant, standard_name, verbose)
    DESCRIPTION
    Return field index for subsequent calls to send_data.

    axes are the axis ID returned from diag_axis_init, axes are required for fields of 1-3 dimension and NOT required for scalars.

    For a static scalar (constant) init_time is not needed.

    Optional mask_variant is for fields that have a time-dependent mask. If mask_variant is true then mask must be present in argument list of send_data.

    The pair (module_name, fieldname) should be registered only once or a FATAL error will occur.


    INPUT
    module_name   
       [CHARACTER(len=*)]
       [CHARACTER(len=*)]
    field_name   
       [CHARACTER(len=*)]
       [CHARACTER(len=*)]
    axes   
       [Not Required]
       [INTEGER, dimension(:)]
    init_time   
       [TYPE(time_type), OPTIONAL]
       [TYPE(time_type)]
    long_name   
       [CHARACTER(len=*), OPTIONAL]
       [CHARACTER(len=*), OPTIONAL]
    units   
       [CHARACTER(len=*), OPTIONAL]
       [CHARACTER(len=*), OPTIONAL]
    missing_value   
       [REAL, OPTIONAL]
       [REAL, OPTIONAL]
    range   
       [REAL, OPTIONAL, dimension(2)]
       [real, dimension(2), OPTIONAL]
    mask_variant   
       [Not Required]
       [logical, OPTIONAL]
    standard_name   
       [CHARACTER(len=*), OPTIONAL]
       [character(len=*), OPTIONAL]

  3. send_tile_averaged_data

    LOGICAL send_tile_averaged_data (diag_field_id, field, area, time, mask)
    DESCRIPTION
    send_tile_averaged_data is overloaded for 3-d and 4-d arrays. diag_field_id corresponds to the id returned by previous call to register_diag_field. Logical masks can be used to mask out undefined and/or unused values. Note that the dimension of output field is smaller by one than the dimension of the data, since averaging over tiles (3d dimension) is performed.


    INPUT
    diag_field_id   
       [INTEGER]
       [INTEGER]
    field   
       [REAL, DIMENSION(:,:,:)]
       [REAL, DIMENSION(:,:,:,:)]
    area   
       [REAL, DIMENSION(:,:,:)]
       [REAL, DIMENSION(:,:,:)]
    time   
       [TYPE(time_type)]
       [TYPE(time_type)]
    mask   
       [LOGICAL, DIMENSION(:,:,:), OPTIONAL]
       [LOGICAL, DIMENSION(:,:,:), OPTIONAL]

  4. register_static_field

    INTEGER FUNCTION register_static_field (module_name, field_name, axes, long_name, units, missing_value, range, mask_variant, standard_name, dynamic, do_not_log, interp_method, tile_count)
    DESCRIPTION
    Return field index for subsequent call to send_data.


    INPUT
    module_name   
       [CHARACTER(len=*)]
    field_name   
       [CHARACTER(len=*)]
    axes   
       [INTEGER, DIMENSION(:)]
    long_name   
       [CHARACTER(len=*), OPTIONAL]
    units   
       [CHARACTER(len=*), OPTIONAL]
    missing_value   
       [REAL, OPTIONAL]
    range   
       [REAL, DIMENSION(2), OPTIONAL]
    mask_variang   
       [LOGICAL, OPTIONAL] [Default: .FALSE.]
    standard_name   
       [CHARACTER(len=*), OPTIONAL]
    dynamic   
       [LOGICAL, OPTIONAL] [Default: .FALSE.]
    do_not_log   
       [LOGICAL, OPTIONAL] [Default: .TRUE.]
    interp_method   
       [CHARACTER(len=*), OPTIOANL]
    tile_count   
       [INTEGER, OPTIONAL]

  5. average_tiles

    SUBROUTINE average_tiles (diag_field_id, x, area, mask, out)
    DESCRIPTION


    INPUT
    diag_field_id   
       [INTEGER]
    x   (lon, lat, tile) field to average
       [REAL, DIMENSION(:,:,:)]
    area   (lon, lat, tile) fractional area
       [REAL, DIMENSION(:,:,:)]
    mask   (lon, lat, tile) land mask
       [LOGICAL, DIMENSION(:,:,:)]

    OUTPUT
    out   (lon, lat) result of averaging
       [REAL, DIMENSION(:,:)]

  6. diag_manager_end

    SUBROUTINE diag_manager_end (time)
    DESCRIPTION
    Flushes diagnostic buffers where necessary. Close diagnostics files.

    A warning will be issued here if a field in diag_table is not registered


    INPUT
    TIME   
       [time_type]

  7. closing_file

    SUBROUTINE closing_file (file, time)
    DESCRIPTION


    INPUT
    file   
       [INTEGER]
    tile   
       [TYPE(time_type)]

  8. diag_manager_init

    SUBROUTINE diag_manager_init (diag_model_subset, err_msg)
    DESCRIPTION
    Open and read diag_table. Select fields and files for diagnostic output.


    INPUT
    diag_model_subset   
       [INTEGER, OPTIONAL]

    OUTPUT
    err_msg   
       [CHARACTER(len=*), OPTIONAL]

  9. get_base_time

    TYPE(time_type) FUNCTION get_base_time ()
    DESCRIPTION
    Return base time for diagnostics (note: base time must be >= model time).


  10. get_base_date

    SUBROUTINE get_base_date (year, month, day, hour, minute, second)
    DESCRIPTION
    Return date information for diagnostic reference time.


    OUTPUT
    year   
       [INTEGER]
    month   
       [INTEGER]
    day   
       [INTEGER]
    hour   
       [INTEGER]
    minute   
       [INTEGER]
    second   
       [INTEGER]

  11. need_data

    LOGICAL need_data (diag_field_id, next_model_time)
    DESCRIPTION
    Determine whether data is needed for the current model time step. Since diagnostic data are buffered, the "next" model time is passed instead of the current model time. This call can be used to minimize overhead for complicated diagnostics.


    INPUT
    inext_model_time    next_model_time = current model time + model time_step
       [TYPE(time_type)]
    diag_field_id   
       [INTEGER]

  12. set_diag_filename_appendix

    SUBROUTINE set_diag_filename_appendix (string_in)
    DESCRIPTION


    INPUT
    string_in   
       [CHARACTER(len=*)]

  13. init_diurnal_axis

    INTEGER FUNCTION init_diurnal_axis (n_samples)
    DESCRIPTION
    Given number of time intervals in the day, finds or initializes a diurnal time axis and returns its' ID. It uses get_base_date, so should be in the file where it's accessible. The units are 'days since BASE_DATE', all diurnal axes belong to the set 'diurnal'


    INPUT
    n_samples   Number of intervals during the day
       [INTEGER]

NAMELIST

&diag_manager_nml

append_pelist_name

[LOGICAL, default: .FALSE.]
mix_snapshot_average_fields

[LOGICAL, default: .FALSE.]
max_files

[INTEGER]
max_output_fields

[INTEGER, default: 300]
max_input_fields

[INTEGER, default: 300]
max_axes

[INTEGER, default: 60]
do_diag_field_log

[LOGICAL, default: .FALSE.]
write_bytes_in_files

[LOGICAL, default: .FALSE.]
debug_diag_manager

[LOGICAL, default: .FALSE.]
max_num_axis_sets

[INTEGER, default: 25]
use_cmor
Let diag_manager know if the missing value (if supplied) should be overridden to be the CMOR standard value of -1.0e20.
[LOGICAL, default: .FALSE.]

DATA SETS

None.

ERROR MESSAGES

WARNING in register_diag_field
module/output_field <module_name>/<field_name> registered AFTER first send_data call, TOO LATE
WARNING in register_diag_field
module/output_field <modul_name>/<field_name> NOT found in diag_table
FATAL in register_static_field
diag_manager has NOT been initialized
FATAL in register_static_field
module/output_field <module_name>/<field_name> is not registered for tile_count = 1, should not register for tile_count > 1
FATAL in register_static_field
module/output_field <module_name>/<field_name> ALREADY Registered, should not register twice
FATAL in register_static_field
when registering module/output_field <module_name>/<field_name> then optional argument interp_method = <interp_method>, but it should be "conserve_order1"
FATAL in register_static_field
module/output_field <module_name>/<field_name> has non-positive axis_id
FATAL in register_static_field
output_field <field_name> has pack >= 4, range is REQUIRED in register_diag_field
FATAL in register_static_field
axes of <field_name> must >= 2 for local output
WARNING in register_static_field
module/field <module_name>/<field_name> is STATIC. Cannot perform time operations average, maximum or minimum on static fields. Setting the time operation to 'NONE' for this field.
WARNING in send_data
Mask will be ignored since missing values were not specified
WARNING in closing_file
<input_fields(input_num)%module_name>/<output_fields(i)%output_name> skip one time level, maybe send_data never called
NOTE in closing_file
<output_fields(i)%output_name) NOT available, check if output interval > runlength. NetCDF fill_values are written
WARNING in diag_manager_init
Problem reading the diag_table (line: <line_number>)
WARNING in diag_manager_init
Incorrect file description format in the diag_table (line: <line_number>). Ignoring file.
WARNING in diag_manager_init
Format value out of range for file description in the diag_table (line: <line_number>). Ignoring file.
WARNING in diag_manager_init
Invalid start time in the file description in the diag_table (line: <line_number>). Ignoring file.
WARNING in diag_manager_init
Field description format is incorrect in the diag_table (line: <line_number>). Ignoring field.
WARNING in diag_manager_init
Packing is out of range for the field description in the diag_table (line: <line_number>). Ignoring field.
WARNING in diag_manager_init
Error in spatial options for the field description in the diag_table (line: <line_number>). Ignoring field.
STATUS UNKNOWN in get_base_time
module has not been initialized
FATAL in get_base_date
module has not been initialized

REFERENCES

None.

COMPILER SPECIFICS

COMPILING AND LINKING SOURCE
Any module or program unit using diag_manager_mod must contain the line 
     use diag_manager_mod
If netCDF output is desired, the cpp flag -Duse_netCDF must be turned on. The loader step requires an explicit link to the NetCDF library (typically something like -L/usr/local/lib -lnetcdf, depending on the path to the netCDF library). netCDF release 3 for fortran is required.

PRECOMPILER OPTIONS

PORTABILITY
diag_manager_mod uses standard f90.

LOADER OPTIONS

GFDL users can checkout diag_manager_mod using the cvs command . 

        ACQUIRING SOURCE

TEST PROGRAM

None.

KNOWN BUGS

None.

NOTES

None.

FUTURE PLANS

None.

top