;Main Program Version: idlcr8ascii.pro v4.0b19, 20180218 ; Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ;Sub-versions: ; v3.01, 20081020 - If the HDF4 file is created using the HDF4.2r3 library, then extra information ; regarding the dimensions of the VAR_DEPEND values may be included in the file ; (which show up as extra datasets in the HDF_SD_FILEINFO call). A check for ; this is carried out and, if found, the information is excluded from the output. ; v3.02, 20090311 - Improve HDF4.2r3 library checks by comparing the number of datasets recorded ; under DATA_VARIABLES, with the number returned by the HDF_SD_FILEINFO call; ; Change 'WARNING' to 'INFORMATION; Add procedure to handle 'INFORMATION' text ; output; Add 'INFORMATION' text if anything irregular found with the datasets; ; Change 'Data dimensions exceed 8' and 'Data type not allowable' ERROR messages ; to 'INFORMATION' messages. Make log file output to the same directory as the ; first HDF file to be read. ; v3.03, 20091208 - Incorporate HDF_SD_ISCOORDVAR to differentiate between datasets and dimension ; variable names. Add check for allowable VAR_DATA_TYPE=STRING. Add all ; INFORMATION text messages to INFOTXT_OUTPUT_A procedure to avoid duplication ; of code that creates the messages; Add INFORMATION messages when reading HDF5 ; files; Rename AVDC Button to FORMAT and replace keyword option /AVDC with ; /FORMAT. Improve checks on parameter and keyword inputs; Improve checks for ; non-standard (i.e. non-groundbased) HDF files. ; v4.0b1, 20101122 - Adopt GEOMS metadata standard; Because FORMAT attribute(s) have been dropped ; can no longer use defined formatting values when outputing data to files; ; Default ASCII formatting adopted - scientific notation for REAL and DOUBLE, ; except for DATETIME and related values, which have formatting dependent on ; the data type; Numeric metadata values remain in their original data type ; when writing to session memory, meaning: variable attribute labels and values ; now written to separate structure variables sds.va_l and sds.va_v (previously ; sds.va), and structure has 2 dimensions (n_sds, n_atts) instead of 1 ; (previously n_sds); The dataset is written to the initial n_atts index (i.e. ; sds[n_sds,0].data); Rename saved SDS datatype according to GEOMS rules ; (affects INT, LONG, and FLOAT); Add JDF_2_DATETIME and JULIAN_DATE routines ; to the program; Include coordinate variables if they and the non-coordinate ; variables add up to the number of saved DATA_VARIABLES. Add return error code ; option so that program returns to the calling program if an error is ; generated. ; v4.0b2, 20110323 - If required reverse the order of VAR_SIZE and VAR_DEPEND values so that they ; match the order of the multi-dimensional dataset. Note that the HDF read programs ; automatically transpose the multi-dimensional datasets to conform to the IDL ; convention of fastest changing index being listed first (opposite for HDF). ; v4.0b3, 20111014 - Add netCDF support - writes variable attributes to GEOMS standard attributes ; where possible, otherwise writes to VAR_NOTES ; v4.0b4, 20111208 - Test for dimension ordering of the netCDF files, and reverse if required (same ; rules as for HDF). Create separate Dimension Ordering Routine ; v4.0b5, 20120328 - Add options to convert HDF/netCDF files to other HDF/netCDF formats e.g. H4 to ; H5 etc. Requires IDLcr8HDF in the search path to do this. ; v4.0b6, 20120426 - Bug fix when a netCDF file only contains datasets with single dimensions - ; VAR_SIZE information was not being extracted during the read process. ; v4.0b7 20130114 - Add check for IDL being run in DEMO mode by using LMGR command. Replace PRINTF,-1 ; statements (i.e. when dux[0] eq -1) with PRINT (o/w causes DEMO mode to stop). Also ; disable /LOG, /FORMAT and /DUMP keywords in DEMO mode. ; v4.0b8 20140325 - Fix bug that meant VAR_SIZE and VAR_DEPEND values were not being automatically ; reversed if the array sizes were the same and VAR_DEPEND did not include a DATETIME ; value. Now defaults to the 'reverse' option in the TEST_DIM_ORDER routine if no ; other conditions are satisfied; Fix bug that caused crash when VAR_SIZE values were ; not of type string. ; v4.0b9 20141110 - Fix bug that caused multi-dimensional array ordering to not be correctly identified ; if the first dataset to be checked in the file had the same number of elements in ; the array (e.g. was a set of Averaging Kernels). ; v4.0b10 20150127 - Allow for VAR_FILL_VALUE values for string datasets to be written and saved as an ; (empty) string of the correct length. Ensure string datasets are all written and ; saved to the correct length (that of the longest string in the dataset). ; v4.0b12 20151109 - Convert DATETIME (and related) max, min and fill values to LONG64 data type before ; determining the format for ASCII output to account for very large values. ; v4.0b15 20160725 - Fix bug causing program to crash when the number of DATA_VARIABLES values is greater ; than the number of datasets in the file. ; v4.0b19 20180218 - Fix bug that caused multi-dimensional array ordering to not be correctly identified ; if the datasets have the same VAR_SIZE i.e. rev_vd_vs stays as 2 through all the ; checks. Now rev_vd_vs changes to 1 (VAR_SIZE and VAR_DEPEND ordering is reversed) if ; rev_vd_vs=2 after all the checks. PRO idlcr8ascii_common ;Procedure to define the data COMMON block WIDGET_WIN_A, containing common variables ;associated with the Graphical User Interface ; ---------- ; Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20061004: Introduced to IDLCR8ASCII - Version 2.0 ; 20080302: 'cwidg' variable removed, and 'dux' array added to identify requested ; output options - Version 3.0 ; 20101122: Add rerr - Version 4.0b1 ; ; ; Input: Nil ; ; Output: Nil ; ; Called by: N/A ; ; Subroutines Called: None COMMON WIDGET_WIN_A,wtxt,lineno,base,o3,b3,dux,rerr END ; Procedure idlcr8ascii_common PRO intro_a_event, ev ;Procedure to define how Events from the Start-up Introduction Window are handled ; ---------- ;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20061004: Introduced to IDLCR8ASCII - Version 2.0 ; 20080302: Code added to define the Pop-up Window and Log Output events added to ; the Introduction Window - Version 3.0 ; ; Input: ev - Selected widget event structure ; ; Output: o3 - Common string array defining the various options for program output ; ; Called by: XMANAGER in INTRO_A ; ; Subroutines Called: None COMMON WIDGET_WIN_A ;The uservalue is retrieved from a widget when an event occurs WIDGET_CONTROL,ev.id,GET_UVALUE=uv IF (uv EQ 'C') OR (uv EQ 'P') OR (uv EQ 'idlcr8ascii.log') OR $ (uv EQ 'H4') OR (uv EQ 'H5') OR (uv EQ 'NC') THEN BEGIN CASE 1 OF uv EQ 'C': BEGIN uvi=1 & eadd=12L END uv EQ 'P': uvi=3 uv EQ 'H4': BEGIN uvi=5 & eadd=6L END uv EQ 'H5': BEGIN uvi=6 & eadd=5L END uv EQ 'NC': BEGIN uvi=7 & eadd=4L END ELSE: uvi=4 ENDCASE IF o3[uvi] EQ uv THEN o3[uvi]='0' ELSE o3[uvi]=uv IF (uv EQ 'C') OR (uv EQ 'H4') OR (uv EQ 'H5') OR (uv EQ 'NC') THEN BEGIN IF (o3[1] EQ '0') AND (o3[5] EQ '0') AND (o3[6] EQ '0') AND (o3[7] EQ '0') THEN $ WIDGET_CONTROL,ev.id+eadd,Sensitive=0 $ ELSE BEGIN WIDGET_CONTROL,ev.id+eadd,Sensitive=1 IF uv EQ 'C' THEN BEGIN WIDGET_CONTROL,ev.id+1,/Set_Button o3[3]='P' ENDIF ENDELSE ENDIF ENDIF ELSE IF (uv EQ 'F') OR (uv EQ 'D') THEN o3[0]=uv $ ;Assign button event to a variable name ELSE IF uv EQ 'Cont' THEN o3[0]='0' ;changes o3[0] from -1 to 0 IF (uv NE 'C') AND (uv NE 'P') AND (uv NE 'idlcr8ascii.log') AND (uv NE 'H4') AND $ (uv NE 'H5') AND (uv NE 'NC') THEN WIDGET_CONTROL,ev.top,/DESTROY END ;Intro_Event handler PRO intro_a, intype ;Procedure which creates an Introduction Window at start-up when IDLCR8ASCII is called without parameters, ;or invalid parameters, or is called by IDL Virtual Machine. The user has a choice of continuing with ;the program after selecting input options, or stopping the program. ; ---------- ;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20061004: Introduced to IDLCR8ASCII - Version 2.0 ; 20080302: Swapped the order of the option windows, so that the options indicated by the check ; boxes appear above the option boxes that close the Introduction Window and continue ; with the program; Added options to allow the user to send input/output log information ; to a Pop-Up Window and/or a Log File - Version 3.0 ; 20091208: Change AVDC button to FORMAT and keyword option from /AVDC to /FORMAT - Version 3.03 ; 20101122: Add vertxt array. Modify Introduction text to indicate that program accepts 'GEOMS ; Compliant' files, and also to show the new structure format - Version 4.0b1 ; ; Input: intype - Integer set to -2 or -3: -2 indicates normal state; -3 indicates that input ; parameters or keywords at the IDLCR8ASCII call are invalid. ; ; Output: Nil ; ; Called by: IDLCR8ASCII ; ; Subroutines Called: INTRO_A_EVENT (via XMANAGER) COMMON WIDGET_WIN_A ;procedure which provides an introduction message before starting the program. nhdr=46 & errtxt=STRARR(nhdr) vertxt=['idlcr8ascii-v4.0_Readme.pdf','v4.0b18 December 2016'] errtxt[1]='Welcome to IDLcr8ASCII. This program reads GEOMS compliant HDF4, HDF5 and netCDF files and' errtxt[2]='saves contents to either session memory, an output window (summary only) or to ASCII or formatted' errtxt[3]='files (also refer to '+vertxt[0]+').' errtxt[5]='Inputs to the program:' errtxt[6]=' HDF/netCDF FILE(s) - GEOMS compliant HDF4, HDF5 or netCDF files. Input can be by DIALOG_BOX' errtxt[7]=' (IDL VM or full version of IDL), or by command line (full version of IDL only).' errtxt[9]='Choice of output is made by selecting options in this DIALOG_BOX (IDL VM or full version' errtxt[10]=' of IDL), or by ''Keywords'' at the command line input (full version only), as follows:' errtxt[11]=' /F or /FORMAT - generates two output files (with .META and .DATA extensions). The resulting' errtxt[12]=' files can be used as inputs to GEOMS compliant write programs (IDLcr8HDF etc), OR' errtxt[13]=' /D or /DUMP - generates two output files (with .META and .DATA extensions). Data values' errtxt[14]=' will be shown as indicated in the array format defined by VAR_SIZE, AND/OR' errtxt[15]=' /C or /CATALOG - sends variable index, variable name, data format, and dimension size' errtxt[16]=' information to an output window and/or log file, AND/OR' errtxt[17]=' /P or /POPUP - sends log input/output information to a Pop-up Dialog Box, AND/OR' errtxt[18]=' /L or /LOG - sends log input/output information to a file named idlcr8ascii.log' errtxt[19]=' /H4 - outputs contents of the input file as an HDF4 file' errtxt[20]=' /H5 - outputs contents of the input file as an HDF5 file' errtxt[21]=' /NC - outputs contents of the input file as a netCDF file' errtxt[23]='Example of command line input: idlcr8ascii,DFSpec[,/F][,/D][,/C][,/P][,/L][,/H4][,/H5][,/NC]' errtxt[24]=' where ''DFSpec'' can either be a string array containing filenames or a scalar string containing a' errtxt[25]=' filespec or a single file name.' errtxt[26]='If running the full version of IDL, and input is a single HDF file, output can also be saved' errtxt[27]=' to session memory. This can be done by calling idlcr8ascii with the following command line:' errtxt[28]=' idlcr8ascii,DFFile,GA,SDS[,/F][,/D][,/C][,/P][,/L][,/H4][,/H5][,/NC].' errtxt[29]='DFFile can either be the name of an input file or '''' (in which case a DIALOG_BOX will open' errtxt[30]=' and prompt for the input filename). ''GA'' is a returned string array containing the Global' errtxt[31]=' Attributes, and ''SDS'' is a returned structure using pointers containing the Variable' errtxt[32]=' Attribute Labels (sds.va_l), Variable Attribute Values (sds.va_v), and the Data (sds.data). errtxt[34]='Contacts -' errtxt[35]=' Ian Boyd, University of Massachusetts (iboyd@astro.umass.edu)' errtxt[36]=' Department of Astronomy, 619 Lederle GRC, University of Massachusetts' errtxt[37]=' 710 North Pleasant St, Amherst, MA 01002, USA' errtxt[39]=' Ghassan Taha, AVDC Project Coordinator (ghassan.taha@nasa.gov)' errtxt[40]=' NASA Goddard Space Flight Center, Code 613.3' errtxt[41]=' Greenbelt, MD 20771, USA' errtxt[43]='AVDC Website: Tools and documentation available from http://avdc.gsfc.nasa.gov/Overview/index.html' errtxt[45]='To continue, please choose from the options below.' errtxt=' '+errtxt ;Set-up text display widget demomode=LMGR(/DEMO) & optsens=1 IF intype EQ -3 THEN xtxt=' - Command Line Input Error' ELSE xtxt='' base=WIDGET_BASE(Title='idlcr8ascii '+vertxt[1]+xtxt,Tlb_Frame_Attr=1,/Column) ;,Tab_Mode=1) wtxt=WIDGET_TEXT(base,xsize=88,ysize=25,/Scroll) tip='Left Mouse Click or Tab to entry and hit ' base2=WIDGET_BASE(base,/Nonexclusive) cattxt='Catalog - Generate a summary of the contents of the input file(s) and send to an output window or file' poptext='Open a Pop-Up Window to display log input/output (default if ''Catalog'' option is chosen)' logtext='Append log input/output to the file ''idlcr8ascii.log'' ' IF demomode THEN BEGIN logtext=logtext+'(No log or netCDF file output permitted in IDL DEMO Mode)' & optsens=0 ENDIF ELSE logtext=logtext+'(will create the file IF it doesn''t exist)' h4text='Convert input file(s) to HDF4' h5text='Convert input file(s) to HDF5' nctext='Convert input file(s) to netCDF' fctext1='For the following options, any existing files will be overwritten, IDLcr8HDF must be in the IDL Search Path,' fctext2='and a valid Table Attribute Values (TAV) file must be in the same directory as the Input file(s).' b1=WIDGET_BUTTON(base2,value=cattxt,uvalue='C',frame=3) b2=WIDGET_BUTTON(base2,value=poptext,uvalue='P',frame=3) b3=WIDGET_BUTTON(base2,value=logtext,uvalue='idlcr8ascii.log',sensitive=optsens,frame=3) base3=WIDGET_LABEL(base) WIDGET_CONTROL,base3,set_value=fctext1 base4=WIDGET_LABEL(base) WIDGET_CONTROL,base4,set_value=fctext2 base5=WIDGET_BASE(base,/Nonexclusive,ROW=1) b4=WIDGET_BUTTON(base5,value=h4text,uvalue='H4',frame=3) b5=WIDGET_BUTTON(base5,value=h5text,uvalue='H5',frame=3) b6=WIDGET_BUTTON(base5,value=nctext,uvalue='NC',sensitive=optsens,frame=3) base6=WIDGET_BASE(base,/Row) b7=WIDGET_BUTTON(base6,value='ASCII Formatted',uvalue='F',sensitive=optsens,frame=3) ;,Tooltip=Tip) b8=WIDGET_BUTTON(base6,value='ASCII Dump',uvalue='D',sensitive=optsens,frame=3) ;,Tooltip=Tip) b9=WIDGET_BUTTON(base6,value='Continue',uvalue='Cont',frame=3,Sensitive=0) ;,Tooltip=Tip) b10=WIDGET_BUTTON(base6,value='Stop',uvalue='S',frame=3) ;,ToolTip=Tip) WIDGET_CONTROL,base,/Realize WIDGET_CONTROL,b1,/Input_Focus FOR i=0,N_ELEMENTS(errtxt)-1 do $ WIDGET_CONTROL,wtxt,set_value=errtxt[i],/Append XMANAGER,'intro_a',base END ;Intro_A PRO idlcr8ascii_event, ev ;Procedure to close the pop-up logging window after user selects 'Finish'. ; ---------- ;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20061004: Introduced to IDLCR8ASCII - Version 2.0 ; ; Input: Selected widget event structure ; ; Output: Nil ; ; Called by: XMANAGER in IDLCR8ASCII and STOP_WITH_ERROR_A ; ; Subroutines Called: None WIDGET_CONTROL,ev.top,/DESTROY RETALL & HEAP_GC END ;Proc IDLcr8ASCII_Event PRO stop_with_error_a, txt1, txt2, lu ;Procedure called when an error in the program is detected. An error message is displayed ;and the program stopped and reset. If necessary, open files are closed. The error message ;is displayed in one or more of the following: a Pop-up dialog window; the Pop-up logging ;window; the Output Logging window in the IDLDE. ; ---------- ;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20050729: Original IDLCR8ASCII Routine - Version 1.0 ; 20061004: Set-up so that the error output is displayed in the output window dependent on the ; method that IDLCR8ASCII is called. If txt1 is preceeded by 'D_' or is null, the ; error output is to a Pop-up Dialog window. Otherwise output will be to a logging ; window. Renamed from STOP_WITH_ERROR to STOP_WITH_ERROR_A to avoid a name conflict ; with the equivalent procedure in IDLCR8HDF. Common variable definition WIDGET_WIN_A ; added - Version 2.0 ; 20080302: Added code which sends the error message to the IDLDE output window and/or an external ; file (as determined by the dux array values) - Version 3.0 ; 20101122: Allow routine to return to the calling program, rather than stop the application, if ; a 'reterr' argument is included in the call to idlcr8ascii - Version 4.0b1 ; ; Inputs: txt1 - the initial text line of the error message. Generally contains the name of the routine ; where the error was detected. ; txt2 - the second text line which generally describes the error state. ; lu - Where applicable, the file unit that needs to be closed at the termination of the program, ; otherwise set to -1. ; ; Output: Nil ; ; Called by: The routine in which the error was detected. The following routines call STOP_WITH_ERROR: ; READ_HDF_SDS; IDLCR8ASCII ; ; Subroutines Called: IDLCR8ASCII_EVENT (via XMANAGER) COMMON WIDGET_WIN_A IF lu NE -1L THEN FREE_LUN,lu IF txt1 EQ '' THEN BEGIN ; chosen on Intro box res=DIALOG_MESSAGE('IDLcr8ASCII Stopped!',/Information,Title='AVDC IDLcr8ASCII') ENDIF ELSE BEGIN IF STRMID(txt1,0,2) EQ 'D_' THEN txtx=STRMID(txt1,2) ELSE txtx=txt1 FOR i=dux[0],dux[1],dux[2] DO BEGIN IF i EQ -1 THEN BEGIN PRINT,' ERROR in '+txtx & PRINT,' '+txt2 PRINT,'' & PRINT,'IDLcr8ASCII stopped - Program Ended on '+SYSTIME(0) ENDIF ELSE BEGIN PRINTF,i,' ERROR in '+txtx & PRINTF,i,' '+txt2 PRINTF,i,'' & PRINTF,i,'IDLcr8ASCII stopped - Program Ended on '+SYSTIME(0) ENDELSE ENDFOR IF dux[1] GT -1 THEN FREE_LUN,dux[1] IF (STRMID(txt1,0,2) EQ 'D_') AND (rerr EQ 'NA') THEN BEGIN ;write error to DIALOG Box instead of Popup window errtxt2=STRARR(4) errtxt2[0]=STRMID(txt1,2) & errtxt2[1]=txt2 & errtxt2[3]='IDLcr8ASCII Stopped!' res=DIALOG_MESSAGE(errtxt2,/Error,Title='AVDC IDLcr8ASCII Error') ENDIF ELSE IF rerr EQ 'NA' THEN BEGIN ;write error to Popup window lineno=lineno+4L WIDGET_CONTROL,wtxt,set_value=' ERROR in '+txt1,/Append WIDGET_CONTROL,wtxt,set_value=' '+txt2,/Append WIDGET_CONTROL,wtxt,set_value='',/Append,Set_text_top_line=lineno WIDGET_CONTROL,wtxt,set_value='HDF file read stopped - hit to close program',/Append WIDGET_CONTROL,b3,Sensitive=1,/Input_Focus XMANAGER,'stop_with_error_a',base,Event_Handler='idlcr8ascii_event' ENDIF ENDELSE HEAP_GC IF rerr EQ 'NA' THEN RETALL ELSE rerr='Unable to read HDF or NC file - '+txtx+txt2 END ;Procedure Stop_With_Error_A PRO infotxt_output_a, in0, in1 ;Procedure called to report information relevant to the reading of the HDF file. ;This information can be reported to a Pop-up logging window, IDLDE output log window, and/or ;the log file. Code for this output was originally written directly in the affected procedures ; ---------- ;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20090311: Introduced to IDLCR8ASCII - Version 3.02 ; 20091208: Added all the INFORMATION text messages to the routine, to avoid duplication ; in the HDF4 and HDF5 read sections of READ_HDF_SDS - Version 3.03 ; 20160213: Update information message when in0[0] EQ 13 - Version 4.0b13 ; 20161130: Added in0[0] messages 18 and 19 to cover non-GEOMS metadata attributes issues ; - Version 4.0b16 ; ; Inputs: in0 - Integer array containing input used to make correct text output ; in1 - String array containing input used to make correct text output ; ; Output: Nil ; ; Called by: The routine in which the request for INFORMATION text was made. ; The following routines call INFOTXT_OUTPUT_A: READ_HDF_SDS; OUTPUT_HDF_DATA; ; IDLCR8ASCII ; ; Subroutines Called: None COMMON WIDGET_WIN_A CASE 1 OF in0[0] EQ 0: BEGIN infotxt=STRARR(3) IF LONG(in1[0]) EQ 0L THEN BEGIN infotxt[0]=' INFORMATION: Global Attribute DATA_VARIABLES not found or has no values.' infotxt[1]=' Dataset listing order matches the order that the dataset is read in the HDF file.' ENDIF ELSE BEGIN infotxt[0]=' INFORMATION: Number of Global Attribute DATA_VARIABLES values ('+in1[0]+')' infotxt[1]=' does not match the number of datasets saved to the HDF file ('+in1[1]+')' ENDELSE infotxt[2]=' Number of datasets determined from '+in1[2]+' call' END in0[0] EQ 1: infotxt=' INFORMATION: '+in1[0]+' has '+in1[1]+' data dimensions' in0[0] EQ 2: infotxt=' INFORMATION: '+in1[0]+' dataset is not one of the allowable Data Types - '+in1[1] in0[0] EQ 3: BEGIN infotxt=STRARR(4) IF (STRPOS(STRUPCASE(in1[1]),STRUPCASE(in1[0])) NE -1) AND (in0[1] EQ 4) AND (in0[2] EQ 0) THEN $ infotxt[0]=' INFORMATION: Dataset Name is truncated (File created with HDF4.2r1 library or earlier):' $ ELSE infotxt[0]=' INFORMATION: Dataset Name does not match VAR_NAME value:' infotxt[1]=' SDS_NAME: '+in1[0] infotxt[2]=' VAR_NAME: '+in1[1] IF in0[2] EQ 0 THEN infotxt[3]=' Output uses VAR_NAME from Metadata' $ ELSE infotxt[3]=' Output uses Dataset Name from the HDF file' END in0[0] EQ 4: BEGIN infotxt=STRARR(2) IF in0[1] EQ 0 THEN BEGIN infotxt[0]=' INFORMATION: Metadata label VAR_NAME not found for dataset:' infotxt[1]=' SDS_NAME: '+in1[0] ENDIF ELSE IF (in0[2] EQ 0) AND (in0[3] NE 0L) THEN BEGIN infotxt[0]=' INFORMATION: VAR_NAME value does not match any DATA_VARIABLES values:' infotxt[1]=' VAR_NAME: '+in1[1] ENDIF END in0[0] EQ 5: BEGIN infotxt=STRARR(2) infotxt[0]=' INFORMATION: No Variable Attributes returned after call to '+in1[1] infotxt[1]=' SDS_NAME: '+in1[0] END in0[0] EQ 6: BEGIN infotxt=' INFORMATION: Metadata Dataset listing order may not match the order of' infotxt=infotxt+' the Global Attribute DATA_VARIABLES Values' END in0[0] EQ 7: BEGIN infotxt=' INFORMATION: Can read one HDF or netCDF file into session memory.' infotxt=infotxt+' Only first file in the array will be read' END in0[0] EQ 8: BEGIN infotxt=' INFORMATION: '+in1[0]+' is a coordinate variable' END in0[0] EQ 9: BEGIN infotxt=STRARR(3) infotxt[0]=' INFORMATION: Calibrated data in dataset '+in1[0] infotxt[1]=' has been converted back to its original form using the formula:' infotxt[2]=' Orig_Data = Scale_Factor * (Cal_Data - Offset)' END in0[0] EQ 10: BEGIN infotxt=STRARR(2) infotxt[0]=' INFORMATION: VAR_DATA_TYPE='+in1[1]+' for Calibrated data in dataset '+in1[0] infotxt[1]=' has been changed to the original datatype of '+in1[2] END in0[0] EQ 11: BEGIN infotxt=STRARR(2) CASE 1 OF in0[1] EQ 20: BEGIN infotxt[0]=' INFORMATION: Order of multi-dimensional datasets changed to match the programming' infotxt[0]=infotxt[0]+' language convention: infotxt[1]=' IDL/Fortran - fastest changing dimension first (datasets transposed when' infotxt[1]=infotxt[1]+' reading/writing to the HDF file)' END in0[1] MOD 10 EQ 0: BEGIN infotxt[0]=' INFORMATION: Listing order of multi-dimensional VAR_DEPEND and VAR_SIZE values' infotxt[1]=' in the HDF file changed to match corresponding dataset dimension ordering' END ELSE: BEGIN infotxt[0]=' INFORMATION: Unable to determine the correct listing order of multi-dimensional' infotxt[1]=' VAR_SIZE and VAR_DEPEND values in the HDF file' END ENDCASE END in0[0] EQ 12: BEGIN infotxt=STRARR(2) infotxt[0]=' INFORMATION: Table Attribute Values file must be in the same directory as the input file' infotxt[1]=' if wanting to convert from one Data Format to another' END in0[0] EQ 13: BEGIN infotxt=STRARR(2) infotxt[0]=' INFORMATION: IDLcr8HDF called but it is not in the IDL Search Path or it encountered an' infotxt[1]=' unexpected error. If the latter please contact Ian Boyd at iboyd@astro.umass.edu' END in0[0] EQ 14: BEGIN infotxt=STRARR(2) infotxt[0]=' INFORMATION: /POPUP keyword cannot be used together with the ''reterr'' argument.' infotxt[1]=' The request for a POPUP window has been ignored' END in0[0] EQ 15: BEGIN infotxt=STRARR(2) infotxt[0]=' INFORMATION: Argument ''reterr'' must be of type string.' ;a returnable variable of type string.' infotxt[1]=' idlcr8ascii will stop normally if an error is encountered' END in0[0] EQ 16: BEGIN infotxt=STRARR(2) infotxt[0]=' INFORMATION: /LOG, /FORMAT, and /DUMP keywords cannot be used in IDL DEMO mode and will be ignored.' infotxt[1]=' To generate ASCII files please use the free IDL Virtual Machine or a licenced version of IDL' END in0[0] EQ 17: BEGIN infotxt=STRARR(2) infotxt[0]=' INFORMATION: NetCDF file create feature is disabled in IDL DEMO mode.' infotxt[1]=' To generate files please use the free IDL Virtual Machine or a licenced version of IDL' END in0[0] EQ 18: BEGIN infotxt=' INFORMATION: Value for Attribute Label '+in1[0]+' is an invalid Data Type' END in0[0] EQ 19: BEGIN infotxt=' INFORMATION: '+in1[0]+' is not a valid GEOMS Metadata Attribute' END ENDCASE dm=SIZE(infotxt,/N_Elements) IF o3[3] EQ '' THEN lineno=lineno+dm FOR n=0,dm-1 DO BEGIN IF o3[3] EQ '' THEN BEGIN IF n EQ dm THEN WIDGET_CONTROL,wtxt,set_value='',/Append $ ELSE WIDGET_CONTROL,wtxt,set_value=infotxt[n],/Append,Set_text_top_line=lineno ENDIF FOR m=dux[0],dux[1],dux[2] DO $ IF m EQ -1 THEN PRINT,infotxt[n] ELSE PRINTF,m,infotxt[n] ;write out to log(s) ENDFOR END ;Procedure InfoTxt_Output_A FUNCTION jdf_2_datetime, jdf, MJD2000=mjd2000, SHORTISO8601=iso, LONGISO8601=isoms ;Computes the UT date/time from JDF/MJD2000. ; ---------- ; Bojan R. Bojkov ; bojan.r.bojkov@nasa.gov ; 03/04/2004 ; 03/11/2004 bug fix ; 05/27/2005 add ShortISO8601 and LongISO8601 switches, ; and change seconds value to decimal seconds (Ian Boyd) ; ; References ---------- ; Explan. Supp. Astron. Almanac, p.604 (1992); through E. Celarier ; ; Caveats ---------- ; Valid for all YYYY >= -4712 (i.e. for all JD .ge. 0) ; The true Gregorian calendar was only adopted on 15 October 1582, ; so any dates before this are "virtual" Gregorian dates. ; ; Input ---------- ; jdf : double precision value ; mjd2000 : flag if input is MJD2000 ; iso : flag if output is in ISO8601 (YYYYMMDDThhmmssZ) ; isoms : flag if output is in ISO8601ms (YYYYMMDDThhmmss.sssZ) ; ; Output --------- ; floating point array [YYYY, MM, DD, hh, mn, ss.sss], ; or ISO8601 string format ; External subroutines --------- ; NONE jdf=DOUBLE(jdf) j0=2451544.5D IF KEYWORD_SET(mjd2000) THEN jdhold=jdf+j0 ELSE jdhold=jdf jdi=LONG(jdhold) df=jdhold-jdi ;Determine hh, mm, ss, ms hh=(df+0.5)*24.D q=WHERE(hh GE 24.D) IF q[0] NE -1 THEN BEGIN hh[q]=hh[q]-24.D jdi[q]=jdi[q]+1 ENDIF t1=hh hh=LONG(t1) t2=(t1-hh)*60.D mn=LONG(t2) t3=(t2-mn)*60.D ss=t3 ;ss=LONG(t3) ;ms=LONG((t3-ss)*1.d3) ;Determine YYYY, MM, DD t1=jdi+68569L t2=(4*t1)/146097L t1=t1-(146097L*t2+3L)/4L t3=(4000L*(t1+1L))/1461001L t1=t1-(1461L*t3)/4L + 31L t4=(80L*t1)/2447L dd=t1-(2447L*t4)/80L t1=t4/11L mm=t4+2L-12L*t1 yyyy=100L*(t2-49L)+t3+t1 dt=TRANSPOSE([[yyyy],[mm],[dd],[hh],[mn],[ss]]) IF (KEYWORD_SET(iso)) OR (KEYWORD_SET(isoms)) THEN BEGIN dts=STRARR(6) IF (KEYWORD_SET(iso)) AND (ss-FIX(ss) GE 0.5) THEN BEGIN ;recalculate to get correct seconds value jdhold=jdf+0.000008D ;add ~0.7 secs IF KEYWORD_SET(mjd2000) THEN jdhold=jdhold+j0 CALDAT,jdhold,mm,dd,yyyy,hh,mn,ss dt=TRANSPOSE([[yyyy],[mm],[dd],[hh],[mn],[ss]]) ENDIF dt=LONG(dt) FOR i=1,5 DO $ IF dt[i] LT 10L THEN dts[i]='0'+STRTRIM(dt[i],2) ELSE dts[i]=STRTRIM(dt[i],2) IF KEYWORD_SET(isoms) THEN BEGIN ssd=STRING(format='(f6.3)',ss) IF FLOAT(ssd) LT 10.0 THEN dts[5]='0'+STRTRIM(ssd,2) ELSE dts[5]=STRTRIM(ssd,2) ENDIF RETURN,STRTRIM(dt[0],2)+dts[1]+dts[2]+'T'+dts[3]+dts[4]+dts[5]+'Z' ENDIF ELSE RETURN, dt END ;Function jdf_2_datetime FUNCTION julian_date, date_ut, ISO8601=iso, MJD2000=mjd2000 ;Computes the Julian date (jd) or date in MJD2000 format in double precision. ; ---------- ; Bojan R. Bojkov ; bojan.r.bojkov@nasa.gov ; 06/03/2001 ; 09/24/2001: Code cleanup - Function Version 1.0 ; 10/16/2001: Mode corrections - Function Version 1.01 ; 03/07/2002: Comment cleanup and variable change - Function Version 2.0 ; 10/08/2002: Added jdf option - Function Version 2.1 ; 07/21/2003: Converted to a function - Function Version 3.0 ; 05/25/2005: Added ISO8601 and MJD2000 keywords. ; Input can either be a string or a numeric array ; (see below for input details). Returns -99999.d ; if the Input is invalid - Ian Boyd ; ; References ---------- ; Hughes, D.W., Yallop, B.D. and Hohenkerk, C.Y., "The Equation of Time", ; Mon. Not. R. astr. Soc., 238, pp 1529-1535. 1989. ; ; Results verified using: ; Meeus, J, "Astronomical Algorithms", William-Bell, Inc., Richmond VA, ; p62, 1991. ; ; Caveats ---------- ; NONE ; ; Input ---------- ; date_ut: Numeric array (UT: YYYY, MM, DD, [hh, mm, ss]), or if ISO8601 ; keyword set, a string containing datetime as YYYYMMDDThhmmssZ ; ; Output --------- ; jd: julian day, or if MJD2000 keyword set, ; a double precision day in MJD2000 format ; ; External subroutines --------- ; NONE ON_IOERROR,TypeConversionError valid=0 ;Type conversion check inputerror=0 ;Check that input is as expected ;Check input format IF KEYWORD_SET(iso) THEN BEGIN ;input is YYYYMMDDThhmmssZ IF STRLEN(date_ut) EQ 16 THEN BEGIN ;test for numeric values (will return type conversion error value if not a number) FOR i=0,7 DO check=FIX(STRMID(date_ut,i,1)) FOR i=9,14 DO check=FIX(STRMID(date_ut,i,1)) dt=INTARR(6) dt[0]=FIX(STRMID(date_ut,0,4)) & dt[1]=FIX(STRMID(date_ut,4,2)) dt[2]=FIX(STRMID(date_ut,6,2)) & dt[3]=FIX(STRMID(date_ut,9,2)) dt[4]=FIX(STRMID(date_ut,11,2)) & dt[5]=FIX(STRMID(date_ut,13,2)) date_ut=dt & achk=[0,6] ;to add hhmmss component ENDIF ELSE inputerror=1 ENDIF ELSE BEGIN ;check that input contains at least YMD (and at most YMDhms) information and is either ;an integer, long, float or double array achk=SIZE(date_ut) IF (achk[1] LT 3) OR (achk[1] GT 6) OR (achk[2] LT 2) OR (achk[2] GT 5) THEN inputerror=1 ENDELSE IF inputerror EQ 0 THEN BEGIN ;can perform JD calculation IF date_ut[1] GT 2 THEN BEGIN y=DOUBLE(date_ut[0]) m=DOUBLE(date_ut[1]-3) d=DOUBLE(date_ut[2]) ENDIF ELSE BEGIN y=DOUBLE(date_ut[0]-1) m=DOUBLE(date_ut[1]+9) d=DOUBLE(date_ut[2]) ENDELSE ;Compute Julian date: j=LONG(365.25D0*(y+4712.D0))+LONG(30.6D0*m+0.5D0)+59.D0+d-0.5D0 ;Check for Julian or Gregorian calendar: IF j LT 2299159.5D0 THEN jd=j $ ;If Julian calendar. ELSE BEGIN ;If Gregorian calendar. gn=38.D0-LONG(3.D0*LONG(49.D0+y/100.D0)/4.D0) jd=j+gn ENDELSE ;add hhmmss values if present CASE 1 OF achk[1] EQ 4: jd=jd+DOUBLE(date_ut[3])/24.D ;hh only achk[1] EQ 5: jd=jd+(DOUBLE(date_ut[3])*3600.D + DOUBLE(date_ut[4])*60.D)/86400.D ;hhmm only achk[1] EQ 6: jd=jd+(DOUBLE(date_ut[3])*3600.D + DOUBLE(date_ut[4])*60.D + $ DOUBLE(date_ut[5]))/86400.D ;hhmmss ELSE: ENDCASE valid=1 ;Type conversion OK ;Set to MJD2000 if required IF KEYWORD_SET(mjd2000) THEN jd=jd-2451544.5D ENDIF TypeConversionError: IF valid EQ 0 THEN RETURN,-99999.D ELSE RETURN, jd END ;Function Julian_Date PRO test_dim_order, va_name, va_value, va_type, sds_dim, rev_vd_vs ;Procedure to test the dimension ordering of any multi-dimensional datasets in the input DF file, and ;return correct ordering code (dimension ordering for idlcr8ascii uses IDL/Fortran convention) ; ---------- ;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20111208: Introduced - Version 4.0b4 ; 20140325: Fix bug that meant VAR_SIZE and VAR_DEPEND values were not being automatically ; reversed if the array sizes were the same and VAR_DEPEND did not include a DATETIME ; value. Now defaults to the 'reverse' option if no other conditions are satisfied; ; Fix bug that caused crash when VAR_SIZE values were not of type string - Version 4.0b8 ; 20141110: Fix bug that caused multi-dimensional array ordering to not be correctly identified ; if the first dataset to be checked in the file had the same number of elements in ; the array (e.g. was a set of Averaging Kernels) - Version 4.0b9 ; ; Inputs: va_name - An abbreviated version of the Variable Name; either 'VD' (VAR_DEPEND) or 'VS' ; (VAR_SIZE) ; va_value - The corresponding Variable Value ; sds_dim - Dimension information for the DF Dataset being tested ; ; Outputs: rev_vd_vs - Scalar to indicate whether VAR_DEPEND and VAR_SIZE values (and data, in the case ; of netCDF measurements) need to be reversed ; ; Called by: Read_HDF_SDS ; IF va_type NE 'STRING' THEN BEGIN ;VAR_SIZE contains numeric values instead of in the form of a string vavhold='' & n_vav=N_ELEMENTS(va_value) FOR k=0,n_vav-1 DO BEGIN IF k EQ n_vav-1 THEN vtxt='' ELSE vtxt=';' vavhold=vavhold+STRTRIM(va_value[k],2)+vtxt ENDFOR ENDIF ELSE vavhold=va_value vs_v=STRCOMPRESS(STRSPLIT(vavhold,';, ',/EXTRACT,COUNT=rcnt),/REMOVE_ALL) IF va_name EQ 'VD' THEN BEGIN ;VAR_DEPEND tests dti=WHERE(STRUPCASE(vs_v) EQ 'DATETIME',dticnt) ;Note: if DATETIME has VAR_SIZE=1 then the ordering is dependent on the order of the VAR_SIZE values only IF (rcnt GT 1) AND (dticnt NE 0) THEN BEGIN ;multi-dimensions including DATETIME test1=(dti[0] EQ rcnt-1) AND (sds_dim[0] GT 1) ;DATETIME is the last value and present only once and has VAR_SIZE GT 1 test2=(dti[0] EQ 0) AND (dticnt EQ 1) AND (sds_dim[0] GT 1) ;DATETIME is the first value and present only once and has VAR_SIZE GT 1 IF (test1) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=10 $ ELSE IF (test2) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=11 $ ELSE IF ((test1) AND (rev_vd_vs MOD 10 EQ 1)) OR ((test2) AND (rev_vd_vs EQ 10)) THEN $ rev_vd_vs=3 $ ;inconsistent rules regarding dimension ordering in file ELSE IF (test2) AND (rev_vd_vs EQ 0) THEN rev_vd_vs=20 ;Dimension ordering is slowest changing first so change dataset ordering as well as ;VAR_DEPEND and VAR_SIZE ordering ENDIF ENDIF ELSE BEGIN ;VAR_SIZE tests IF rcnt GT 1 THEN BEGIN ;multi-dimensions ;check all values are numeric numchk=STRJOIN(vs_v,/SINGLE) & valok=1 FOR k=0,STRLEN(numchk)-1 DO BEGIN bchar=BYTE(STRMID(numchk,k,1)) IF (bchar LT 48) OR (bchar GT 57) THEN valok=0 ;i.e. non-numeric character ENDFOR IF valok EQ 1 THEN BEGIN ;all values are numeric so do checks vs_vl=LONG(vs_v) test1=ARRAY_EQUAL(vs_vl,sds_dim) test2=ARRAY_EQUAL(REVERSE(vs_vl),sds_dim) ;test in case values are the same e.g. 40;40 ;If arrays are the same size then the default will be to reverse the VS and VD values ;unless a different dimension ordering has already been identified IF (test1 EQ 1) AND (test2 EQ 0) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=0 $ ELSE IF (test1 EQ 0) AND (test2 EQ 1) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=1 $ ELSE IF ((test1 EQ 1) AND (test2 EQ 0) AND (rev_vd_vs MOD 10 EQ 1)) OR $ ((test1 EQ 0) AND (test2 EQ 1) AND (rev_vd_vs MOD 10 EQ 0)) THEN $ rev_vd_vs=3 $ ;inconsistent VAR_SIZE and sds_dim agreement from dataset to dataset ELSE IF (test1 EQ 1) AND (test2 EQ 1) AND (rev_vd_vs EQ 2) THEN $ rev_vd_vs=2 $ ;first set of measurements tested have same dimensions so do not do anything ELSE IF rev_vd_vs EQ 2 THEN rev_vd_vs=1 ;default if no other criteria are found as this ;will then automatically reverse the VD and VS values ENDIF ENDIF ENDELSE END ;Procedure Test_Dim_Order PRO read_hdf_sds, ifile, ga, sds, catinfo ;Procedure to read the contents of a GEOMS standard HDF4 or HDF5 compatible file ;into session memory ; ---------- ;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20050729: Original IDLCR8ASCII Routine - Version 1.0 ; 20050912: Removed Common variable definition CATALOGINFO and made the variable a parameter passed ; to the procedure; Checks that the attribute variable names read by HDF_SD_GETINFO, match ; those recorded in DATA_VARIABLES in the Global Attributes. If not use the variable name ; listed in DATA_VARIABLES - Version 1.1 ; 20061004: Common variable definition WIDGET_WIN_A added for Error calls; Add option to read HDF5 ; files (needs IDL5.6 or newer); Add check that the HDF4 file has valid information on the ; number of global and variable attributes; Checks added in the event that the HDF4 or ; HDF5 file has been created by the NCSA utility programs H5toH4 or H4toH5 from an original ; AVDC/Envisat/NDACC HDF4 or HDF5 file; Ensures variable attributes and data are listed in ; the same order as that given under DATA_VARIABLES - Version 2.0 ; 20081020 - If the HDF4 file is created using the HDF4.2r3 library, then extra information ; regarding the dimensions of the VAR_DEPEND values may be included in the file (which ; show up as extra datasets in the HDF_SD_FILEINFO call). A check for this is carried out ; and, if found, the information is excluded from the output - Version 3.01 ; 20090311: Improve HDF4.2r3 library checks by comparing the number of datasets recorded under ; DATA_VARIABLES, with the number returned by the HDF_SD_FILEINFO call; Change 'WARNING' to ; 'INFORMATION and make calls to procedure INFOTXT_OUTPUT_A to avoid duplication. Add ; 'INFORMATION' text if anything irregular found with the datasets; Change 'Data dimensions ; exceed 8' and 'Data type not allowable' ERROR messages to 'INFORMATION' messages ; - Version 3.02 ; 20091208: Incorporate HDF_SD_ISCOORDVAR to differentiate between datasets and dimension variable ; names. Add check for allowable VAR_DATA_TYPE=STRING. Add all INFORMATION text messages to ; INFOTXT_OUTPUT_A procedure to avoid duplication of code that creates the messages; ; Add INFORMATION messages when reading HDF5 files; Improve checks for non-standard (i.e. ; non-groundbased) HDF input file - Version 3.03 ; 20101122: Numeric metadata values remain in their original data type when transferring to session ; memory, meaning: variable attribute labels and values now written to separate structure ; variables sds.va_l and sds.va_v (previously sds.va), and structure has 2 dimensions ; (n_sds, n_atts) instead of 1 (previously n_sds); The dataset is written to the initial ; n_atts index (i.e. sds[n_sds,0].data); Rename saved SDS datatype according to GEOMS rules ; (affects INT, LONG, and FLOAT); Compare HDF dimensions with VAR_SIZE values and transpose ; VAR_SIZE and VAR_DEPEND values if required - Version 4.0b1 ; 20111014: Add netCDF read capability - writes variable attributes to standard GEOMS variable ; attribute labels where possible, otherwise appends information to VAR_NOTES - Version 4.0b3 ; 20120426: Bug fix when a netCDF file only contains datasets with single dimensions, then VAR_SIZE ; information was not being extracted during the read process - Version 4.0b6 ; 20140325: Fix bug that caused crash when VAR_SIZE values were not of type string - Version 4.0b8 ; 20150127: Allow for VAR_FILL_VALUE values for string datasets to be written and saved as an ; (empty) string of the correct length. Ensure string datasets are all written and ; saved to the correct length (that of the longest string in the dataset) - Version 4.0b10 ; 20150217: Do not remove whitespace of any variable attribute values that are written as strings ; - Version 4.0b11 ; 20160614: Some HDF4 string datasets may not have 2 dimensions, so do not need to remove the first ; dimension (= maximum number of characters) - Version 4.0b14 ; 20160725: Fix bug causing program to crash when the number of DATA_VARIABLES values is greater ; than the number of datasets in the file - Version 4.0b15 ; 20161130: Assign SDS_name to catinfo for attributes when the VAR_NAME is not present; rewrite ; netCDF section to align it with HDF5 section and to account for dataset ordering that ; does not match that in DATA_VARIABLES - Version 4.0b16 ; 20161213: Fix bug when reading in netCDF files to stop all dataset variable attribute values to ; be converted to string format (now matches HDF4 and HDF5 checks). Fix bug that caused ; the program to crash when VAR_DEPEND=CONSTANT in netCDF files (sds_ndim=0) - Version 4.0b17 ; 20161218: Fix bug when reading netCDF attribute values that are of type byte rather than string - ; Version 4.0b18 ; 20180218 Fix bug that caused multi-dimensional array ordering to not be correctly identified ; if the datasets have the same VAR_SIZE. Now rev_vd_vs changes to 1 (VAR_SIZE and ; VAR_DEPEND ordering is reversed) if rev_vd_vs=2 after all the checks - Version 4.0b19 ; ; Inputs: ifile - a string containing the name of the input file to be read in. ; catinfo - a string array identifying the type of input file ('H4','H5','NC') ; ; Outputs: ga - a string array containing the global attribute labels and values extracted from the HDF ; file. ; sds - a structure using pointers, of size [n_sds,n_atts], containing the variable attribute ; labels and values (sds[n,m].va_l and sds[n,m].va_v) and the data (sds[n,0].data) ; extracted from the HDF file. ; catinfo - a string array of size [n_sds,4] (where n_sds is the number of datasets in the HDF ; file), containing information on the variable name, data type, data dimension, and ; number of attributes. ; ; Called by: IDLCR8ASCII ; ; Subroutines Called: STOP_WITH_ERROR_A (if error state detected) ; INFOTXT_OUTPUT_A (if program can make a change) ; Possible Conditions for STOP_WITH_ERROR call (plus [line number] where called): ; 1. No global attributes or datasets present in the root group of an HDF5 file. ; 2. The number of global attributes or datasets is zero or not able to be determined ; in an HDF4 file. ; ; Information Conditions (when the program is able to make changes): ; 1. DATA_VARIABLES global attribute not found or has no values. ; 2. Number of datasets given under DATA_VARIABLES is not equal to the number saved to the ; file. ; 3. The number of dimensions in a dataset exceeds 8. ; 4. The data type of a dataset is not BYTE, SHORT, INTEGER, LONG, REAL, DOUBLE, or STRING. ; 5. Dataset name is truncated or does not match VAR_NAME value. ; 6. Metadata label VAR_NAME not found for the dataset. ; 7. VAR_NAME does not match any DATA_VARIABLES values. ; 8. No Variable Attributes returned after HDF call. ; 9. Metadata Dataset listing order may not match the order of the Global Attribute ; DATA_VARIABLES Values. ; 10. Variable is written to the HDF file as a coordinate variable. COMMON WIDGET_WIN_A ;Note any calibrated (scaled) HDF4 dataset is corrected using the formula specified in ;UG_print42r3.pdf pg. 3-107: orig = cal * (cal_val - offset) ncsa_cal=['scale_factor','scale_factor_err','add_offset','add_offset_err','calibrated_nt'] ;Initialize scalar to indicate whether VAR_DEPEND and VAR_SIZE values need to be reversed rev_vd_vs=2 ;1 = reverse attribute values; 0 = no reverse (and will generate message); ;2 = no action required; 3 = rev_vd_vs changed between 0 and 1 during checking, therefore error ;Note if rev_vd_vs is still 2 after all the checks then it will be changed to 1 so that dimensions ;are swapped by default ;List of Standard GEOMS Variable Attributes attr_arr_data=['VAR_NAME','VAR_DESCRIPTION','VAR_NOTES','VAR_SIZE','VAR_DEPEND',$ 'VAR_DATA_TYPE','VAR_UNITS','VAR_SI_CONVERSION','VAR_VALID_MIN',$ 'VAR_VALID_MAX','VAR_FILL_VALUE'] n_aad=N_ELEMENTS(attr_arr_data) ;Possible error messages for this procedure proname='Read_HDF_SDS procedure: ' errtxt=STRARR(2) errtxt[0]=' found in the Root Group of the HDF5 File.' errtxt[1]=' is zero or not able to be determined (check for corrupted file).' lu=-1 ;Allowable VAR_DATA_TYPEs avdt=['BYTE','SHORT','INTEGER','LONG','REAL','DOUBLE','STRING'] slabel=['VAR_UNITS','VAR_SI_CONVERSION','VAR_VALID_MIN','VAR_VALID_MAX','VAR_FILL_VALUE'] ;save values as is if dataset is of type STRING ;If necessary, free up memory by destroying the heap variables pointed at by its pointer arguments ;from previous calls to this program IF N_ELEMENTS(sds) NE 0 THEN PTR_FREE,sds.va_l,sds.va_v,sds.data ;Define the HDF SDS storage structure sds_set={va_l: PTR_NEW(), $ ;Variable Attribute Labels va_v: PTR_NEW(), $ ;Variable Attribute Values data: PTR_NEW()} ;SD data array IF catinfo[0,0] EQ 'H4' THEN BEGIN ;The HDF_SD_START function opens an HDF file and initializes the SD interface. sd_id=HDF_SD_START(ifile,/READ) ;Determine the number of SDS (n_sds) and global attributes (n_ga) found ;in the current file. HDF_SD_FILEINFO,sd_id,n_sds,n_ga IF n_sds EQ 0L THEN ntxt='Number of SDS datasets' $ ELSE IF n_ga EQ 0L THEN ntxt='Number of Global Attributes' $ ELSE ntxt='' IF ntxt NE '' THEN BEGIN STOP_WITH_ERROR_A,o3[3]+proname,ntxt+errtxt[1],lu & RETURN ENDIF ga=STRARR(n_ga) ;set the Global Attribute dimensions vn=[''] ;initialize vn array (to hold Dataset names) n_sds_hold=n_sds & n_vn=0L & do_coordvar=0 ;Read the file's Global Attributes and determine number of DATA_VARIABLES FOR i=0L,n_ga-1L DO BEGIN ;The HDF_SD_ATTRINFO procedure reads or retrieves info about an SD attribute. HDF_SD_ATTRINFO,sd_id,i,NAME=ga_name,DATA=ga_data ga_name=STRTRIM(ga_name,2) & ga_data=STRTRIM(ga_data,2) ;Assign global attributes to ga ga[i]=ga_name+'='+ga_data ;read list of DATA_VARIABLES into array IF STRUPCASE(ga_name) EQ 'DATA_VARIABLES' THEN vn=STRSPLIT(ga_data,' ;',/Extract,COUNT=n_vn) ENDFOR IF n_sds_hold NE n_vn THEN BEGIN ;Do check for sds being a Dimension attribute do_coordvar=1 FOR i=0L,n_sds-1L DO BEGIN ;The HDF_SD_SELECT function returns an SD dataset ID given the current ;SD interface ID, and the zero-based SD dataset index. sds_id=HDF_SD_SELECT(sd_id,i) IF HDF_SD_ISCOORDVAR(sds_id) THEN n_sds_hold=n_sds_hold-1L ;Closes the SDS interface. HDF_SD_ENDACCESS,sds_id ENDFOR IF n_sds_hold EQ 0L THEN BEGIN STOP_WITH_ERROR_A,o3[3]+proname,'Number of SDS datasets'+errtxt[1],lu RETURN ENDIF IF n_sds_hold NE n_vn THEN $ INFOTXT_OUTPUT_A,[0],[STRTRIM(n_vn,2),STRTRIM(n_sds_hold,2),'HDF_SD_FILEINFO'] ;DATA_VARIABLES values not read successfully, or number of datasets given under DATA_VARIABLES ;is not equal to the number saved to the file ENDIF ;Determine maximum number of attributes, and match the dataset order to DATA_VARIABLES list if possible max_atts=0L & oi=LONARR(n_sds_hold) dv_order=1 & c_sds=0L catinfo=STRARR(n_sds_hold,4) ;output info for catalog output FOR i=0L,n_sds-1L DO BEGIN ;The HDF_SD_SELECT function returns an SD dataset ID given the current ;SD interface ID, and the zero-based SD dataset index. sds_id=HDF_SD_SELECT(sd_id,i) IF (do_coordvar EQ 0) OR (NOT HDF_SD_ISCOORDVAR(sds_id)) THEN BEGIN ;The HDF_SD_GETINFO procedure retrieves information about an SD dataset. HDF_SD_GETINFO,sds_id,NATTS=sds_natts, $ ;no. attributes TYPE=sds_type, $ ;data type DIMS=sds_dim, $ ;dimension information (automatically reversed by IDL) NAME=sds_name ;dataset name ;Check for string attribute and, if so, remove the first dimension IF (sds_type EQ 'STRING') AND (N_ELEMENTS(sds_dim) NE 1) THEN sds_dim=sds_dim[1:N_ELEMENTS(sds_dim)-1] ;Check for multi-dimensional dataset n_sds_dim=N_ELEMENTS(sds_dim) IF n_sds_dim GT 1 THEN multi_dim=1 ;Check number of dimensions IF n_sds_dim gt 8 THEN INFOTXT_OUTPUT_A,[1],[sds_name,STRTRIM(n_sds_dim,2)] ;Check for coordinate variable IF HDF_SD_ISCOORDVAR(sds_id) THEN INFOTXT_OUTPUT_A,[8],[sds_name] ;Check that a dataset name has been successfully read sds_name=STRTRIM(sds_name,2) IF sds_name EQ '' THEN sds_name='N/A' ;Rename format to be compatible with the Metadata guidelines sds_type=STRTRIM(STRUPCASE(sds_type),2) IF sds_type EQ 'INT' THEN sds_type='SHORT' $ ELSE IF sds_type EQ 'LONG' THEN sds_type='INTEGER' $ ELSE IF sds_type EQ 'FLOAT' THEN sds_type='REAL' vi=WHERE(sds_type EQ avdt,vcnt) IF vcnt EQ 0 THEN INFOTXT_OUTPUT_A,[2],[sds_name,sds_type] IF sds_natts NE 0L THEN BEGIN IF sds_natts GT max_atts THEN max_atts=sds_natts vnf=0 & lcnt=0 & vnv='' & vcnt=sds_natts ;Extract the variable attributes FOR j=0L,sds_natts-1L DO BEGIN ;The HDF_SD_ATTRINFO procedure reads or retrieves information about an SD attribute. HDF_SD_ATTRINFO,sds_id,j,NAME=va_name,DATA=va_value,TYPE=va_type va_name=STRTRIM(va_name,2) schk=WHERE(STRUPCASE(va_name) EQ slabel,schkcnt) IF ((sds_type EQ 'STRING') AND (schkcnt EQ 0)) OR ((sds_type NE 'STRING') AND (va_type EQ 'STRING')) THEN $ va_value=STRTRIM(va_value,2) ;Check returned dataset attributes (sds_) with saved attributes (va_) IF (STRUPCASE(va_name) EQ 'VAR_NAME') AND (vnf EQ 0) THEN BEGIN ;check that the dataset name matches the VAR_NAME vnf=1 IF STRUPCASE(va_value) NE STRUPCASE(sds_name) THEN BEGIN ;Try and determine which is wrong - Dataset name or VAR_NAME - default is Dataset name li=WHERE((STRUPCASE(sds_name) EQ STRUPCASE(vn)) AND (vn[0] NE ''),lcnt) ;If lcnt NE 0 then Dataset name matches DATA_VARIABLES value so assume that VAR_NAME is wrong IF lcnt NE 0 THEN usesds=1 ELSE usesds=0 INFOTXT_OUTPUT_A,[3,4,usesds],[sds_name,va_value] IF usesds EQ 0 THEN sds_name=va_value ELSE va_value=sds_name ENDIF vnv=va_value ;match the order of the dataset to the DATA_VARIABLES list li=WHERE(STRUPCASE(va_value) EQ STRUPCASE(vn),lcnt) ENDIF IF STRUPCASE(va_name) EQ 'VAR_DATA_TYPE' THEN BEGIN va_value=STRUPCASE(va_value) IF va_value NE sds_type THEN BEGIN ;Dataset is not saved in the same data type as that described by VAR_DATA_TYPE IF (sds_type EQ 'INTEGER') AND (va_value EQ 'LONG') THEN $ itxt=['32-bit INTEGER','64-bit LONG'] $ ELSE IF (sds_type EQ 'SHORT') AND (va_value EQ 'INTEGER') THEN $ itxt=['16-bit SHORT','32-bit INTEGER'] $ ELSE itxt=[sds_type,va_value] ENDIF ENDIF ELSE IF STRUPCASE(va_name) EQ 'VAR_DEPEND' THEN $ ;do dimension ordering checks TEST_DIM_ORDER,'VD',va_value,va_type,sds_dim,rev_vd_vs $ ELSE IF STRUPCASE(va_name) EQ 'VAR_SIZE' THEN $ ;do dimension ordering checks TEST_DIM_ORDER,'VS',va_value,va_type,sds_dim,rev_vd_vs ENDFOR ENDIF ELSE BEGIN ;No Variable Attributes found vcnt=3 ;Can the SDS_NAME be matched with a DATA_VARIABLES value? li=WHERE(STRUPCASE(sds_name) EQ STRUPCASE(vn),lcnt) IF lcnt NE 0 THEN vnf=1 ELSE vnf=0 INFOTXT_OUTPUT_A,[5],[sds_name,'HDF_SD_GETINFO'] ENDELSE ;Check for possible problems with determining array index IF lcnt NE 0 THEN BEGIN IF li[0] GE n_sds_hold THEN dv_order=0 $ ;can occur when number of DATA_VARIABLE values is greater than number of datasets ELSE IF catinfo[li[0],0] NE '' THEN dv_order=0 ;This array has already been written to ENDIF ELSE IF (lcnt EQ 0) OR (vnf EQ 0) THEN dv_order=0 ;Write out information text if sds_name cannot be matched and determine array index IF dv_order EQ 0 THEN BEGIN li=WHERE(catinfo[*,0] EQ '') ;determine lowest available array index to write info to in0=[4,vnf,lcnt,n_vn] IF (vnf EQ 0) OR ((lcnt EQ 0) AND (n_vn NE 0L)) THEN INFOTXT_OUTPUT_A,in0,[sds_name,vnv] ENDIF oi[c_sds]=li[0] ;Attribute order index c_sds=c_sds+1L catinfo[li[0],0]=sds_name & catinfo[li[0],1]=sds_type catinfo[li[0],2]=STRTRIM(sds_dim[0],2) & catinfo[li[0],3]=vcnt IF N_ELEMENTS(sds_dim) GT 1 THEN $ FOR j=1,N_ELEMENTS(sds_dim)-1 DO catinfo[li[0],2]=catinfo[li[0],2]+';'+STRTRIM(sds_dim[j],2) ENDIF ;Closes the SDS interface. HDF_SD_ENDACCESS,sds_id ENDFOR IF max_atts EQ 0L THEN max_atts=3 ;Use information from HDF_SD_GETINFO call only IF rev_vd_vs EQ 2 THEN rev_vd_vs=1 ;change default so that VAR_DEPEND and VAR_SIZE are reversed ;if not o/w changed in TEST_DIM_ORDER ;Dimension the structure to the number of datasets x number of attributes sds=REPLICATE(sds_set, n_sds_hold, max_atts) ;Write attributes to structure c_sds=0L !QUIET=1 ;suppress system error and information messages (used for CALDATA call in HDF_SD_GETINFO) FOR i=0L,n_sds-1L DO BEGIN notranspose=1 ;0/1 will change to 0 if the dataset needs to be transposed ;The HDF_SD_SELECT function returns an SD dataset ID given the current ;SD interface ID, and the zero-based SD dataset index. sds_id=HDF_SD_SELECT(sd_id,i) nocal=1 ;Boolean to identify dataset which has been calibrated IF (do_coordvar EQ 0) OR (NOT HDF_SD_ISCOORDVAR(sds_id)) THEN BEGIN ;The HDF_SD_GETINFO procedure retrieves information about an SD dataset. ;Note - any saved pre-defined attributes will be called with HDF_SD_ATTRINFO call HDF_SD_GETINFO,sds_id,NATTS=sds_natts, $ ;no. attributes TYPE=sds_type, $ ;data type DIMS=sds_dim, $ ;dimension information NAME=sds_name, $ ;dataset name CALDATA=sds_cal ;pre-defined calibration info ;Check to see whether data has had scale factor and offset applied IF (sds_cal.Cal NE 0.D) OR (sds_cal.Offset NE 0.D) THEN BEGIN nocal=0 ;Dataset has been calibrated ;identify datatype of the original dataset CASE 1 OF sds_cal.Num_Type EQ 21L: vdt_val='BYTE' sds_cal.Num_Type EQ 22L: vdt_val='SHORT' sds_cal.Num_Type EQ 24L: vdt_val='INTEGER' sds_cal.Num_Type EQ 5L: vdt_val='REAL' sds_cal.Num_Type EQ 6L: vdt_val='DOUBLE' ELSE: vdt_val='' ;invalid or cannot determine original datatype ENDCASE ENDIF IF sds_natts NE 0L THEN BEGIN ;Extract the variable attributes jh=0 ;will only loop if attribute is not a pre-defined calibration attribute FOR j=0L,sds_natts-1L DO BEGIN ;The HDF_SD_ATTRINFO procedure reads or retrieves information about an SD attribute. HDF_SD_ATTRINFO,sds_id,j,NAME=va_name,DATA=va_value,TYPE=va_type va_name=STRTRIM(va_name,2) schk=WHERE(STRUPCASE(va_name) EQ slabel,schkcnt) IF ((sds_type EQ 'STRING') AND (schkcnt EQ 0)) OR ((sds_type NE 'STRING') AND (va_type EQ 'STRING')) THEN $ va_value=STRTRIM(va_value,2) IF (nocal EQ 0) AND (STRUPCASE(va_name) EQ 'VAR_DATA_TYPE') THEN BEGIN ;If required prepare INFORMATION text (written after DATA has been corrected) IF (vdt_val NE STRUPCASE(va_value)) AND (vdt_val NE '') THEN BEGIN io_arr=[STRTRIM(sds_name,2),va_value,vdt_val] va_value=vdt_val ENDIF ELSE BEGIN io_arr=[''] & vdt_val=va_value ENDELSE ENDIF IF ((STRUPCASE(va_name) EQ 'VAR_DEPEND') OR (STRUPCASE(va_name) EQ 'VAR_SIZE')) AND $ ((rev_vd_vs MOD 10 EQ 1) OR (rev_vd_vs EQ 20)) THEN BEGIN ;need to reverse the va_values IF va_type NE 'STRING' THEN BEGIN ;VAR_SIZE contains numeric values instead of in the form of a string vavhold='' & n_vav=N_ELEMENTS(va_value) FOR k=0,n_vav-1 DO BEGIN IF k EQ n_vav-1 THEN vtxt='' ELSE vtxt=';' vavhold=vavhold+STRTRIM(va_value[k],2)+vtxt ENDFOR ENDIF ELSE vavhold=va_value vs_v=STRCOMPRESS(STRSPLIT(vavhold,';, ',/EXTRACT,COUNT=rcnt),/REMOVE_ALL) IF rcnt GT 1 THEN BEGIN ;multi-dimensions IF (rev_vd_vs EQ 20) THEN notranspose=0 ;need to transpose dataset as well vs_v=REVERSE(vs_v) FOR k=0,rcnt-1 DO $ IF k EQ 0 THEN va_value=vs_v[k] ELSE va_value=va_value+';'+vs_v[k] ENDIF ENDIF ;Test for pre-defined calibration attribute and do not save as the data have ;been coverted back to original values pi=WHERE(STRLOWCASE(va_name) EQ ncsa_cal,pcnt) IF pcnt EQ 0 THEN BEGIN sds[oi[c_sds],jh].va_l=PTR_NEW(va_name) sds[oi[c_sds],jh].va_v=PTR_NEW(va_value) jh=jh+1 ENDIF ENDFOR ENDIF ELSE BEGIN ;No Variable Attributes found va_lab=['VAR_NAME','VAR_SIZE','VAR_DATA_TYPE'] FOR j=0,2 DO sds[oi[c_sds],j].va_l=PTR_NEW(va_lab) sds[oi[c_sds],0].va_v=PTR_NEW(sds_name) sds[oi[c_sds],1].va_v=PTR_NEW(sds_dim) ;Rename format to be compatible with the Metadata guidelines IF sds_type EQ 'INT' THEN sds_type='SHORT' $ ELSE IF sds_type EQ 'LONG' THEN sds_type='INTEGER' $ ELSE IF sds_type EQ 'FLOAT' THEN sds_type='REAL' sds[oi[c_sds],2].va_v=PTR_NEW(sds_type) ENDELSE ;Extract the data HDF_SD_GETDATA,sds_id,datasize ;Test to see if dataset dimension ordering needs to be changed IF notranspose EQ 0 THEN datasize=TRANSPOSE(datasize) IF nocal EQ 0 THEN BEGIN ;apply scale factor and offset corrections to data and write text to log file CASE 1 OF vdt_val EQ 'BYTE': datasize=BYTE(sds_cal.Cal*(datasize-sds_cal.Offset)) vdt_val EQ 'SHORT': datasize=FIX(sds_cal.Cal*(datasize-sds_cal.Offset)) vdt_val EQ 'INTEGER': datasize=LONG(sds_cal.Cal*(datasize-sds_cal.Offset)) vdt_val EQ 'REAL': datasize=FLOAT(sds_cal.Cal*(datasize-sds_cal.Offset)) ELSE: datasize=DOUBLE(sds_cal.Cal*(datasize-sds_cal.Offset)) ENDCASE INFOTXT_OUTPUT_A,[9],[STRTRIM(sds_name,2)] IF io_arr[0] NE '' THEN INFOTXT_OUTPUT_A,[10],io_arr ENDIF sds[oi[c_sds],0].data=PTR_NEW(datasize) c_sds=c_sds+1L ENDIF ;Closes the SDS interface. HDF_SD_ENDACCESS,sds_id ENDFOR !QUIET=0 ;Allow system error and information messages ;The HDF_SD_END function closes the SD interface to an HDF file. HDF_SD_END,sd_id ENDIF ELSE IF catinfo[0,0] EQ 'H5' THEN BEGIN ;HDF5 format file stdfm=1 ;Boolean indicating standard AVDC/EVDC/NDACC H5 format or not dv_order=1 ;Boolean indicating whether Metadata Variable listing order matches DATA_VARIABLES values n_vn=0L ;Initialize the number of Variable Names in the standard format h5 file hdf_file_id=H5F_OPEN(ifile) sd_id=H5G_OPEN(hdf_file_id,'/') ;Get number of attributes within the root group (these should be the Global Attributes) n_ga=H5A_GET_NUM_ATTRS(sd_id) ;Get the number of objects within the root group (including datasets, groups etc) n_obj=H5G_GET_NMEMBERS(sd_id,'/') IF n_obj NE 0L THEN BEGIN d_obj=INTARR(n_obj) & g_obj=INTARR(n_obj) ;Determine number of datasets. Note: if H5 file created using H4toH5 then an extra ;group will have been created which will not be used. Also datasets may be listed ;alphabetically. This section also determines whether the H5 file is a standard ;AVDC/EVDC/NDACC groundbased file or otherwise. FOR i=0L,n_obj-1L DO BEGIN sds_name=H5G_GET_MEMBER_NAME(sd_id,'/',i) h5fstat=H5G_GET_OBJINFO(sd_id,sds_name) IF h5fstat.type EQ 'DATASET' THEN d_obj[i]=1 $ ELSE IF h5fstat.type EQ 'GROUP' THEN g_obj[i]=1 ENDFOR di=WHERE(d_obj EQ 1,n_sds) gi=WHERE(g_obj EQ 1,n_grp) IF (n_ga EQ 0) AND (n_sds EQ 0) AND (n_grp NE 0) THEN stdfm=0 ENDIF ELSE n_sds=0L IF stdfm EQ 0 THEN catinfo[0,0]='HE5' ;non-standard dataset so need to call READ_NONSTD_H5 IF n_ga NE 0 THEN BEGIN ga=STRARR(n_ga) ;set the Global Attribute dimensions vn=[''] ;initialize vn array (to hold Dataset names) ;Read in the Global Attribute labels and values FOR i=0L,n_ga-1L DO BEGIN ga_id=H5A_OPEN_IDX(sd_id,i) ga_name=H5A_GET_NAME(ga_id) & ga_name=STRTRIM(ga_name,2) ga_data=H5A_READ(ga_id) & ga_data=STRTRIM(ga_data,2) ;Check for _GLOSDS suffix and strip (added for compatibility with H4toH5 utility) IF STRMID(STRUPCASE(ga_name),STRLEN(ga_name)-7) EQ '_GLOSDS' THEN $ ga_name=STRMID(ga_name,0,STRPOS(ga_name,'_',/Reverse_Search)) WHILE STRMID(ga_name,STRLEN(ga_name)-1) EQ '_' DO ga_name=STRMID(ga_name,0,STRLEN(ga_name)-1) ga[i]=ga_name+'='+ga_data H5A_CLOSE,ga_id ;Extract the list of the DATA_VARIABLES IF STRUPCASE(ga_name) EQ 'DATA_VARIABLES' THEN vn=STRSPLIT(ga_data,' ;',/Extract,COUNT=n_vn) ENDFOR ENDIF ELSE IF stdfm EQ 1 THEN BEGIN STOP_WITH_ERROR_A,o3[3]+proname,'No Global Attributes'+errtxt[0],lu RETURN ENDIF IF n_sds NE 0L THEN BEGIN IF n_vn NE n_sds THEN $ INFOTXT_OUTPUT_A,[0],[STRTRIM(n_vn,2),STRTRIM(n_sds,2),'H5G_GET_NMEMBERS'] ;DATA_VARIABLES values not read successfully, or number of datasets given under DATA_VARIABLES ;is not equal to the number saved to the file ;Determine maximum number of Attributes, correct dataset order, and dimension order max_atts=0L & oi=LONARR(n_sds) dv_order=1 catinfo=STRARR(n_sds,4) ;output info for catalog output FOR i=0,n_sds-1 DO BEGIN sds_name=H5G_GET_MEMBER_NAME(sd_id,'/',di[i]) ds_id=H5D_OPEN(sd_id,sds_name) IF sds_name EQ '' THEN sds_name='N/A' datasize=H5D_READ(ds_id) sds_dim=SIZE(datasize,/DIMENSIONS) & n_sds_dim=N_ELEMENTS(sds_dim) sds_type=SIZE(datasize,/TYPE) ;Determine the number of attributes associated with the dataset sds_natts=H5A_GET_NUM_ATTRS(ds_id) IF sds_natts GT max_atts THEN max_atts=sds_natts IF sds_natts NE 0L THEN BEGIN vnf=0 & lcnt=0 & vnv='' ci=STRARR(4) ;array containing catalog attributes ;Read in Dataset Attributes FOR j=0L,sds_natts-1L DO BEGIN da_id=H5A_OPEN_IDX(ds_id,j) da_name=H5A_GET_NAME(da_id) & da_name=STRTRIM(da_name,2) va_value=H5A_READ(da_id) IF SIZE(va_value,/TYPE) EQ 7 THEN va_type='STRING' ELSE va_type='NON_STRING' schk=WHERE(STRUPCASE(da_name) EQ slabel,schkcnt) IF ((sds_type EQ 7) AND (schkcnt EQ 0)) OR ((sds_type NE 7) AND (va_type EQ 'STRING')) THEN $ va_value=STRTRIM(va_value,2) CASE 1 OF (STRUPCASE(da_name) EQ 'VAR_NAME') AND (vnf EQ 0):BEGIN vnf=1 ;check that the dataset name matches the VAR_NAME IF STRUPCASE(va_value) NE STRUPCASE(sds_name) THEN BEGIN ;Try and determine which is wrong - Dataset name or VAR_NAME - default is Dataset name li=WHERE((STRUPCASE(sds_name) EQ STRUPCASE(vn)) AND (vn[0] NE ''),lcnt) ;If lcnt NE 0 then Dataset name matches DATA_VARIABLES value so assume that VAR_NAME is wrong IF lcnt NE 0 THEN usesds=1 ELSE usesds=0 INFOTXT_OUTPUT_A,[3,5,usesds],[sds_name,va_value] IF usesds EQ 0 THEN sds_name=va_value ELSE va_value=sds_name ENDIF ;Determine actual list order from DATA_VARIABLE listing (H5 lists datasets alphabetically) li=WHERE(STRUPCASE(va_value[0]) EQ STRUPCASE(vn),lcnt) ci[0]=va_value & vnv=va_value END STRUPCASE(da_name) EQ 'VAR_DATA_TYPE': BEGIN ci[1]=va_value vi=WHERE(STRUPCASE(va_value[0]) EQ avdt,vcnt) IF vcnt EQ 0 THEN INFOTXT_OUTPUT_A,[2],[sds_name,va_value] END STRUPCASE(da_name) EQ 'VAR_DEPEND': BEGIN TEST_DIM_ORDER,'VD',va_value,va_type,sds_dim,rev_vd_vs END STRUPCASE(da_name) EQ 'VAR_SIZE': BEGIN TEST_DIM_ORDER,'VS',va_value,va_type,sds_dim,rev_vd_vs END ELSE: ENDCASE H5A_CLOSE,da_id ENDFOR ci[3]=STRTRIM(sds_natts,2) IF ci[0] EQ '' THEN ci[0]=sds_name ENDIF ELSE BEGIN ;No Attributes, so can only write dataset name to Metadata ci=[STRUPCASE(sds_name),'','','1'] & vnv=sds_name ;Can the SDS_NAME be matched with a DATA_VARIABLES value? li=WHERE(STRUPCASE(sds_name) EQ STRUPCASE(vn),lcnt) IF lcnt NE 0 THEN vnf=1 ELSE vnf=0 INFOTXT_OUTPUT_A,[5],[sds_name,'H5A_GET_NUM_ATTRS'] ENDELSE FOR k=0,N_ELEMENTS(sds_dim)-1 DO BEGIN IF k EQ 0 THEN ci[2]=STRTRIM(sds_dim[k],2) ELSE ci[2]=ci[2]+';'+STRTRIM(sds_dim[k],2) ENDFOR ;Check for possible problems with determining array index IF lcnt NE 0 THEN BEGIN IF li[0] GE n_sds THEN dv_order=0 $ ;can occur when number of DATA_VARIABLE values is greater than number of datasets ELSE IF catinfo[li[0],0] NE '' THEN dv_order=0 ;This array has already been written to ENDIF ELSE IF (lcnt EQ 0) OR (vnf EQ 0) THEN dv_order=0 ;Write out information text if sds_name cannot be matched and determine array index IF dv_order EQ 0 THEN BEGIN li=WHERE(catinfo[*,0] EQ '') ;determine lowest available array index to write info to in0=[4,vnf,lcnt,n_vn] IF (vnf EQ 0) OR ((lcnt EQ 0) AND (n_vn NE 0L)) THEN INFOTXT_OUTPUT_A,in0,[sds_name,vnv] ENDIF H5D_CLOSE,ds_id oi[i]=li[0] ;to put datasets in the correct order catinfo[li[0],*]=ci[*] ENDFOR ENDIF ELSE IF stdfm EQ 1 THEN BEGIN STOP_WITH_ERROR_A,o3[3]+proname,'No Datasets'+errtxt[0],lu & RETURN ENDIF IF rev_vd_vs EQ 2 THEN rev_vd_vs=1 ;change default so that VAR_DEPEND and VAR_SIZE are reversed ;if not o/w changed in TEST_DIM_ORDER ;Dimension the structure to the size of the SDS datasets (with dimension n_sds) sds=REPLICATE(sds_set, n_sds, max_atts) ;Put datasets and attributes into sds structure FOR i=0,n_sds-1 DO BEGIN notranspose=1 ;0/1 will change to 0 if the dataset needs to be transposed sds_name=H5G_GET_MEMBER_NAME(sd_id,'/',di[i]) ds_id=H5D_OPEN(sd_id,sds_name) IF sds_name EQ '' THEN sds_name='N/A' datasize=H5D_READ(ds_id) sds_type=SIZE(datasize,/TYPE) ;Determine the number of attributes associated with the dataset sds_natts=H5A_GET_NUM_ATTRS(ds_id) IF sds_natts NE 0L THEN BEGIN ;Read in Dataset Attributes FOR j=0L,sds_natts-1L DO BEGIN da_id=H5A_OPEN_IDX(ds_id,j) da_name=H5A_GET_NAME(da_id) & da_name=STRTRIM(da_name,2) va_value=H5A_READ(da_id) IF SIZE(va_value,/TYPE) EQ 7 THEN va_type='STRING' ELSE va_type='NON_STRING' schk=WHERE(STRUPCASE(da_name) EQ slabel,schkcnt) IF ((sds_type EQ 7) AND (schkcnt EQ 0)) OR ((sds_type NE 7) AND (va_type EQ 'STRING')) THEN $ va_value=STRTRIM(va_value,2) H5A_CLOSE,da_id IF ((STRUPCASE(da_name) EQ 'VAR_DEPEND') OR (STRUPCASE(da_name) EQ 'VAR_SIZE')) AND $ ((rev_vd_vs MOD 10 EQ 1) OR (rev_vd_vs EQ 20)) THEN BEGIN ;need to reverse the va_values IF va_type NE 'STRING' THEN BEGIN ;VAR_SIZE contains numeric values instead of in the form of a string vavhold='' & n_vav=N_ELEMENTS(va_value) FOR k=0,n_vav-1 DO BEGIN IF k EQ n_vav-1 THEN vtxt='' ELSE vtxt=';' vavhold=vavhold+STRTRIM(va_value[k],2)+vtxt ENDFOR ENDIF ELSE vavhold=va_value vs_v=STRCOMPRESS(STRSPLIT(vavhold,';',/EXTRACT,COUNT=rcnt),/REMOVE_ALL) IF rcnt GT 1 THEN BEGIN ;multi-dimensions IF (rev_vd_vs EQ 20) THEN $ ;AND (STRPOS(STRUPCASE(va_value),'DATETIME') NE -1) THEN $ notranspose=0 ;need to transpose dataset as well vs_v=REVERSE(vs_v) FOR k=0,rcnt-1 DO $ IF k EQ 0 THEN va_value=vs_v[k] ELSE va_value=va_value+';'+vs_v[k] ENDIF ENDIF sds[oi[i],j].va_l=PTR_NEW(da_name) sds[oi[i],j].va_v=PTR_NEW(va_value) ENDFOR ENDIF ELSE BEGIN ;No Attributes, so can only write dataset name to structure val='VAR_NAME' sds[oi[i],0].va_l=PTR_NEW(val) sds[oi[i],0].va_v=PTR_NEW(sds_name) ENDELSE H5D_CLOSE,ds_id ;Test to see if dataset dimension ordering needs to be changed IF notranspose EQ 0 THEN datasize=TRANSPOSE(datasize) sds[oi[i],0].data=PTR_NEW(datasize) ENDFOR H5G_CLOSE,sd_id H5F_CLOSE,hdf_file_id ENDIF ELSE BEGIN ;netCDF file ncdf_desc=['long_name'] ncdf_std=['units','valid_range','_fillvalue'] dv_order=1 ;Boolean indicating whether Metadata Variable listing order matches DATA_VARIABLES values n_vn=0L ;Initialize the number of Variable Names in the standard format h5 file fileid=NCDF_OPEN(ifile) ;Get number of global atts and variables filestruct=NCDF_INQUIRE(fileid) n_dim=filestruct.ndims n_ga=filestruct.ngatts n_sds=filestruct.nvars IF n_sds EQ 0L THEN ntxt='Number of SDS datasets' $ ELSE IF n_ga EQ 0L THEN ntxt='Number of Global Attributes' $ ELSE ntxt='' IF ntxt NE '' THEN BEGIN STOP_WITH_ERROR_A,o3[3]+proname,ntxt+errtxt[1],lu & RETURN ENDIF ;read global attributes ga=STRARR(n_ga) & vn=[''] FOR j=0,n_ga-1 DO BEGIN ga_name=NCDF_ATTNAME(fileid,j,/GLOBAL) NCDF_ATTGET,fileid,ga_name,ga_data,/GLOBAL ga_data=STRTRIM(ga_data,2) ga[j]=STRTRIM(STRUPCASE(ga_name),2)+'='+ga_data ;read list of DATA_VARIABLES into array IF STRUPCASE(ga_name) EQ 'DATA_VARIABLES' THEN vn=STRSPLIT(ga_data,' ;',/Extract,COUNT=n_vn) ENDFOR ;read dimension information IF n_dim NE 0 THEN BEGIN mv_dim=STRARR(n_dim,2) FOR j=0,n_dim-1 DO BEGIN NCDF_DIMINQ,fileid,j,dimname,dimsize mv_dim[j,0]=STRTRIM(dimname,2) mv_dim[j,1]=STRTRIM(dimsize,2) ENDFOR ENDIF ELSE mv_dim=[''] IF n_vn NE n_sds THEN $ INFOTXT_OUTPUT_A,[0],[STRTRIM(n_vn,2),STRTRIM(n_sds,2),'NCDF_INQUIRE'] ;DATA_VARIABLES values not read successfully, or number of datasets given under DATA_VARIABLES ;is not equal to the number saved to the file ;Determine maximum number of Attributes, correct dataset order, and dimension order max_atts=0L & oi=LONARR(n_sds) catinfo=STRARR(n_sds,4) ;output info for catalog output FOR j=0,n_sds-1 DO BEGIN varstruct=NCDF_VARINQ(fileid,j) sds_natts=varstruct.natts IF sds_natts GT max_atts THEN max_atts=sds_natts sds_name=varstruct.name sds_datatype=varstruct.datatype sds_dimid=varstruct.dim sds_ndim=varstruct.ndims ;determine the actual dimension values from the dim IDs given in varstruct.dim IF sds_ndim NE 0 THEN BEGIN sds_dim=LONARR(sds_ndim) FOR k=0,sds_ndim-1 DO sds_dim[k]=mv_dim[sds_dimid[k],1] ENDIF ELSE sds_dim=[1L] ;Read Variable Attributes IF sds_natts NE 0L THEN BEGIN vnf=0 & lcnt=0 & vnv='' ci=STRARR(4) ;array containing catalog attributes std_fnd=INTARR(n_aad) ;Read in Dataset Attributes FOR k=0,sds_natts-1 DO BEGIN varattsname=NCDF_ATTNAME(fileid,j,k) NCDF_ATTGET,fileid,j,varattsname,va_value va_name=STRUPCASE(varattsname) vavtype=SIZE(va_value,/TYPE) IF (vavtype EQ 7) OR (vavtype EQ 1) THEN va_type='STRING' ELSE va_type='NON-STRING' schk=WHERE(va_name EQ slabel,schkcnt) test1=(sds_datatype EQ 'CHAR') AND (schkcnt EQ 0) ;string dataset but not one of the slabel values test2=(sds_datatype NE 'CHAR') AND (va_type EQ 'STRING') ;not a string dataset but string attribute value test3=(sds_datatype EQ 'CHAR') AND (schkcnt NE 0) AND (vavtype EQ 1) ;string dataset, slabel and attribute value is of type byte IF (test1) OR (test2) THEN va_value=STRTRIM(va_value,2) $ ELSE IF test3 THEN va_value=STRING(va_value) CASE 1 OF (STRUPCASE(va_name) EQ 'VAR_NAME') AND (vnf EQ 0):BEGIN vnf=1 ;check that the dataset name matches the VAR_NAME IF STRUPCASE(va_value) NE STRUPCASE(sds_name) THEN BEGIN ;Try and determine which is wrong - Dataset name or VAR_NAME - default is Dataset name li=WHERE((STRUPCASE(sds_name) EQ STRUPCASE(vn)) AND (vn[0] NE ''),lcnt) ;If lcnt NE 0 then Dataset name matches DATA_VARIABLES value so assume that VAR_NAME is wrong IF lcnt NE 0 THEN usesds=1 ELSE usesds=0 INFOTXT_OUTPUT_A,[3,5,usesds],[sds_name,va_value] IF usesds EQ 0 THEN sds_name=va_value ELSE va_value=sds_name ENDIF ;Determine actual list order from DATA_VARIABLE listing li=WHERE(STRUPCASE(va_value[0]) EQ STRUPCASE(vn),lcnt) ci[0]=va_value & vnv=va_value END STRUPCASE(va_name) EQ 'VAR_DATA_TYPE': BEGIN ci[1]=va_value vi=WHERE(STRUPCASE(va_value[0]) EQ avdt,vcnt) IF vcnt EQ 0 THEN INFOTXT_OUTPUT_A,[2],[sds_name,va_value] END STRUPCASE(va_name) EQ 'VAR_DEPEND': BEGIN TEST_DIM_ORDER,'VD',va_value,va_type,sds_dim,rev_vd_vs END STRUPCASE(va_name) EQ 'VAR_SIZE': BEGIN TEST_DIM_ORDER,'VS',va_value,va_type,sds_dim,rev_vd_vs END ELSE: ENDCASE ENDFOR ci[3]=STRTRIM(sds_natts,2) IF ci[0] EQ '' THEN ci[0]=sds_name ENDIF ELSE BEGIN ;No Attributes, so can only write dataset name to Metadata ci=[STRUPCASE(sds_name),'','','1'] & vnv=sds_name ;Can the SDS_NAME be matched with a DATA_VARIABLES value? li=WHERE(STRUPCASE(sds_name) EQ STRUPCASE(vn),lcnt) IF lcnt NE 0 THEN vnf=1 ELSE vnf=0 INFOTXT_OUTPUT_A,[5],[sds_name,'NCDF_VARINQ'] ENDELSE FOR k=0,N_ELEMENTS(sds_dim)-1 DO BEGIN IF k EQ 0 THEN ci[2]=STRTRIM(sds_dim[k],2) ELSE ci[2]=ci[2]+';'+STRTRIM(sds_dim[k],2) ENDFOR ;Check for possible problems with determining array index IF lcnt NE 0 THEN BEGIN IF li[0] GE n_sds THEN dv_order=0 $ ;can occur when number of DATA_VARIABLE values is greater than number of datasets ELSE IF catinfo[li[0],0] NE '' THEN dv_order=0 ;This array has already been written to ENDIF ELSE IF (lcnt EQ 0) OR (vnf EQ 0) THEN dv_order=0 ;Write out information text if sds_name cannot be matched and determine array index IF dv_order EQ 0 THEN BEGIN li=WHERE(catinfo[*,0] EQ '') ;determine lowest available array index to write info to in0=[4,vnf,lcnt,n_vn] IF (vnf EQ 0) OR ((lcnt EQ 0) AND (n_vn NE 0L)) THEN INFOTXT_OUTPUT_A,in0,[sds_name,vnv] ENDIF oi[j]=li[0] ;to put datasets in the correct order catinfo[li[0],*]=ci[*] ENDFOR IF rev_vd_vs EQ 2 THEN rev_vd_vs=1 ;change default so that VAR_DEPEND and VAR_SIZE are reversed ;if not o/w changed in TEST_DIM_ORDER ;Dimension the structure to the size of the SDS datasets (with dimension n_sds) sds=REPLICATE(sds_set, n_sds, max_atts) ;Put datasets and attributes into sds structure FOR j=0,n_sds-1 DO BEGIN notranspose=1 ;0/1 will change to 0 if the dataset needs to be transposed varstruct=NCDF_VARINQ(fileid,j) sds_name=varstruct.name IF sds_name EQ '' THEN sds_name='N/A' sds_natts=varstruct.natts sds_datatype=varstruct.datatype sds_ndim=varstruct.ndims ;Extract data from a netCDF dataset and get size information NCDF_VARGET,fileid,j,sds_data IF sds_natts NE 0L THEN BEGIN ;Read in Dataset Attributes FOR k=0L,sds_natts-1L DO BEGIN varattsname=NCDF_ATTNAME(fileid,j,k) NCDF_ATTGET,fileid,j,varattsname,va_value va_name=STRUPCASE(varattsname) vavtype=SIZE(va_value,/TYPE) IF (vavtype EQ 7) OR (vavtype EQ 1) THEN va_type='STRING' ELSE va_type='NON-STRING' schk=WHERE(va_name EQ slabel,schkcnt) test1=(sds_datatype EQ 'CHAR') AND (schkcnt EQ 0) ;string dataset but not one of the slabel values test2=(sds_datatype NE 'CHAR') AND (va_type EQ 'STRING') ;not a string dataset but string attribute value test3=(sds_datatype EQ 'CHAR') AND (schkcnt NE 0) AND (vavtype EQ 1) ;string dataset, slabel and attribute value is of type byte IF (test1) OR (test2) THEN va_value=STRTRIM(va_value,2) $ ELSE IF test3 THEN va_value=STRING(va_value) IF ((STRUPCASE(va_name) EQ 'VAR_DEPEND') OR (STRUPCASE(va_name) EQ 'VAR_SIZE')) AND $ ((rev_vd_vs MOD 10 EQ 1) OR (rev_vd_vs EQ 20)) THEN BEGIN ;need to reverse the va_values IF va_type NE 'STRING' THEN BEGIN ;VAR_SIZE contains numeric values instead of in the form of a string vavhold='' & n_vav=N_ELEMENTS(va_value) FOR i=0,n_vav-1 DO BEGIN IF i EQ n_vav-1 THEN vtxt='' ELSE vtxt=';' vavhold=vavhold+STRTRIM(va_value[i],2)+vtxt ENDFOR ENDIF ELSE vavhold=va_value vs_v=STRCOMPRESS(STRSPLIT(vavhold,';',/EXTRACT,COUNT=rcnt),/REMOVE_ALL) IF rcnt GT 1 THEN BEGIN ;multi-dimensions IF (rev_vd_vs EQ 20) THEN $ ;AND (STRPOS(STRUPCASE(va_value),'DATETIME') NE -1) THEN $ notranspose=0 ;need to transpose dataset as well vs_v=REVERSE(vs_v) FOR i=0,rcnt-1 DO $ IF i EQ 0 THEN va_value=vs_v[i] ELSE va_value=va_value+';'+vs_v[i] ENDIF ENDIF sds[oi[j],k].va_l=PTR_NEW(va_name) sds[oi[j],k].va_v=PTR_NEW(va_value) ENDFOR ;This section commented out for now - Version 4.0b16 ;Check attributes again in case there are missing GEOMS Attributes that can be filled in ;vnt='' ;FOR k=0,sds_natts-1 DO BEGIN ; varattsname=NCDF_ATTNAME(fileid,j,k) ; NCDF_ATTGET,fileid,j,varattsname,va_value ; va_name=STRUPCASE(varattsname) ; va_type=SIZE(va_value,/TYPE) ; schk=WHERE(va_name EQ slabel,schkcnt) ; IF ((sds_datatype EQ 'CHAR') AND (schkcnt EQ 0)) OR ((sds_datatype NE 'CHAR') AND $ ; ((va_type EQ 1) OR (va_type EQ 7))) THEN va_value=STRTRIM(va_value,2) $ ; ELSE IF (va_type EQ 1) OR (va_type EQ 7) THEN va_value=STRING(va_value) ; ;Check for VAR_NOTES entry ; lcname=STRLOWCASE(varattsname) ; ni=WHERE(lcname EQ ncdf_desc,ncnt) ; IF ncnt EQ 0 THEN ni=WHERE(lcname EQ ncdf_std,ncnt) ; IF (ncnt EQ 0) AND (lcname NE 'units') THEN BEGIN ;append label and value to VAR_NOTES ; mi=WHERE(attr_arr_data EQ 'VAR_NOTES') ; IF vnt NE '' THEN BEGIN ; IF STRMID(vnt,STRLEN(vnt)-1,1) EQ '.' THEN itxt=' ' ELSE itxt='. ' ; ENDIF ELSE itxt='' ; vnt=vnt+itxt+STRTRIM(varattsname,2)+': '+STRTRIM(va_value,2) ; ENDIF ; mcnt=0 ; ;Check for VAR_DESCRIPTION entry (long_name) ; IF lcname EQ 'long_name' THEN mi=WHERE(attr_arr_data EQ 'VAR_DESCRIPTION',mcnt) ; ;Check for VAR_UNITS entry ; IF lcname EQ 'units' THEN mi=WHERE(attr_arr_data EQ 'VAR_UNITS',mcnt) ; IF (mcnt NE 0) AND (std_fnd[mi[0]] EQ 0) THEN sds[oi[j],mi[0]].va_v=PTR_NEW(STRTRIM(va_value,2)) ; ;Check for valid_range and _FillValue attributes ; ni=WHERE(lcname EQ ncdf_std,ncnt) ; IF ncnt NE 0 THEN BEGIN ; vdt=SIZE(varattsval,/TYPE) ; IF lcname EQ 'valid_range' THEN BEGIN ; mi=WHERE(attr_arr_data EQ 'VAR_VALID_MIN') ; IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(varattsval[0]) ; mi=WHERE(attr_arr_data EQ 'VAR_VALID_MAX') ; IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(varattsval[1]) ; ENDIF ELSE BEGIN ; mi=WHERE(attr_arr_data EQ 'VAR_FILL_VALUE') ; IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(varattsval) ; ENDELSE ; ENDIF ;ENDFOR ;IF vnt NE '' THEN BEGIN ; mi=WHERE(attr_arr_data EQ 'VAR_NOTES') ; IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(vnt) ;ENDIF ; ;Write VAR_NAME to metadata ;mi=WHERE(attr_arr_data EQ 'VAR_NAME') ;IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(sds_name) ;catinfo[oi[j],0]=sds_name ;;Write VAR_DATA_TYPE to metadata ;mi=WHERE(attr_arr_data EQ 'VAR_DATA_TYPE') ;CASE 1 OF ; sds_datatype EQ 'BYTE':catinfo[oi[j],1]='BYTE' ; sds_datatype EQ 'CHAR':catinfo[oi[j],1]='STRING' ; (sds_datatype EQ 'INT') OR (sds_datatype EQ 'SHORT'):catinfo[oi[j],1]='SHORT' ; sds_datatype EQ 'LONG':catinfo[oi[j],1]='INTEGER' ; sds_datatype EQ 'FLOAT':catinfo[oi[j],1]='REAL' ; sds_datatype EQ 'DOUBLE':catinfo[oi[j],1]='DOUBLE' ;ENDCASE ;sds[oi[j],mi[0]].va_v=PTR_NEW(catinfo[oi[j],1]) ;;Write VAR_DEPEND to metadata ;mi=WHERE(attr_arr_data EQ 'VAR_DEPEND') ;IF std_fnd[mi[0]] EQ 0 THEN BEGIN ; vd='' ; FOR k=0,sds_ndim-1 DO BEGIN ; IF k EQ 0 THEN vd=vd+STRTRIM(mv_dim[sds_dim[k],0],2) $ ; ELSE vd=vd+';'+STRTRIM(mv_dim[sds_dim[k],0],2) ; ENDFOR ; sds[oi[j],mi[0]].va_v=PTR_NEW(vd) ;ENDIF ;;Determine VAR_SIZE from dataset attributes ;mi=WHERE(attr_arr_data EQ 'VAR_SIZE') ;IF std_fnd[mi[0]] EQ 0 THEN BEGIN ; sds_dim=SIZE(sds_data,/DIMENSIONS) ; IF sds_dim[0] EQ 0 THEN sds_dim[0]=1 ; vs='' ; FOR k=0,N_ELEMENTS(sds_dim)-1 DO $ ; IF k EQ 0 THEN vs=vs+STRTRIM(sds_dim[k],2) ELSE vs=vs+';'+STRTRIM(sds_dim[k],2) ; sds[oi[j],mi[0]].va_v=PTR_NEW(vs) ;ENDIF ENDIF ELSE BEGIN ;No Attributes, so can only write dataset name to structure val='VAR_NAME' sds[oi[j],0].va_l=PTR_NEW(val) sds[oi[j],0].va_v=PTR_NEW(sds_name) ENDELSE ;Test to see if dataset dimension ordering needs to be changed IF notranspose EQ 0 THEN sds_data=TRANSPOSE(sds_data) IF sds_datatype EQ 'CHAR' THEN sds_data=STRING(sds_data) sds[oi[j],0].data=PTR_NEW(sds_data) ENDFOR NCDF_CLOSE,fileid ;Add DATA_VARIABLES to global attributes if not already present gi=WHERE(STRMID(ga,0,14) EQ 'DATA_VARIABLES',gcnt) IF gcnt EQ 0 THEN BEGIN FOR j=0,n_sds-1 DO BEGIN IF j EQ 0 THEN dv=catinfo[j,0] ELSE dv=dv+';'+catinfo[j,0] ENDFOR ga=[ga,'DATA_VARIABLES='+dv] ENDIF dv_order=-1 & n_vn=n_sds ENDELSE ;Array listing order may not be correct IF (dv_order EQ 0) AND (n_vn NE 0L) THEN INFOTXT_OUTPUT_A,[6] ;If necessary write comment regarding VAR_SIZE and VAR_DEPEND dimension ordering IF (rev_vd_vs MOD 10 EQ 0) OR (rev_vd_vs EQ 3) THEN INFOTXT_OUTPUT_A,[11,rev_vd_vs] END ;Procedure Read_HDF_SDS PRO output_hdf_data, ifile, ga, sds, catinfo ;Procedure to output the contents of an HDF file in ASCII form, either in a format compatible for ;conversion back to HDF, or in Row and Column format, or as a summary list ; ---------- ;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20050729: Original IDLCR8ASCII Routine - Version 1.0 ; 20050912: Removed Common variable definition CATALOGINFO and made the variable a parameter passed ; to the procedure; Make output filenames lower case; If data type is double and the ; accompanying VIS_FORMAT value starts with an 'F' (e.g. F9.4), then, change the 'F' to a ; 'D' when specifying the format for output - Version 1.1 ; 20061004: Common variable definition WIDGET_WIN_A added for Error calls; Display input/output ; information in a pop-up display window if the program is opened using IDL VM - Version 2.0 ; 20080302: Added code which sends the Catalog and log output to the IDLDE output window and/or an ; external file (as determined by the dux array values); Ensure all data values are ; separated by a space when the Dump output option is chosen - Version 3.0 ; 20101122: Adopt GEOMS metadata standard; Because FORMAT attribute(s) have been dropped can no longer ; use defined formatting values when outputing data to files; Default ASCII formatting ; adopted - scientific notation for REAL and DOUBLE, except for DATETIME and related values, ; which have formatting dependent on the data type - Version 4.0b1 ; 20150127: Ensure all string datasets are written and saved as left justified strings (previously ; defaulted to right justified) - Version 4.0b10 ; 20151109: Convert DATETIME (and related) max, min and fill values to LONG64 data type before ; determining the format for ASCII output to account for very large values - Version 4.0b12 ; 20161130: Add checks for Metadata values written in an invalid data type (e.g. structure) and for ; invalid attributes - Version 4.0b16 ; ; Inputs: ifile - a string containing the name of the HDF file to be read in. ; ga - a string array containing the global attribute labels and values extracted from the HDF ; file. ; sds - a structure using pointers, of size [n_sds,n_atts], containing the variable attribute ; labels and values (sds[n,m].va_l and sds[n,m].va_v) and the data (sds[n,0].data) ; extracted from the HDF file. ; catinfo - a string array of size [n_sds,4] (where n_sds is the number of datasets in the HDF ; file), containing information on the variable name, data type, data dimension, and ; number of attributes ; ; Outputs: Nil ; ; Called by: IDLCR8ASCII ; ; Subroutines Called: None COMMON WIDGET_WIN_A ;determine number of datasets and attributes (from sds.va) as=SIZE(catinfo) n_sds=as[1] indir=FILE_DIRNAME(ifile,/MARK_DIRECTORY) infile=FILE_BASENAME(ifile) vdtype=[1,2,3,4,5,7,12,13,14,15] ;valid IDL data types badlabel=[''] ;list of attribute labels where the values are not valid (to avoid repeated log messages) IF o3[1] EQ 'C' THEN BEGIN IF o3[3] EQ '' THEN BEGIN lineno=lineno+n_sds+1 wintxt='Listing of Var_Name Index; Var_Name; Var_Data_Type; Var_Size' WIDGET_CONTROL,wtxt,set_value=wintxt,/Append,Set_text_top_line=lineno ENDIF FOR i=dux[0],dux[1],dux[2] DO BEGIN IF i EQ -1 THEN PRINT,'Listing of Var_Name Index; Var_Name; Var_Data_Type; Var_Size' $ ELSE PRINTF,i,'Listing of Var_Name Index; Var_Name; Var_Data_Type; Var_Size' ENDFOR FOR j=0,n_sds-1 DO BEGIN IF o3[3] EQ '' THEN BEGIN wintxt=STRTRIM(j,2)+'; '+catinfo[j,0]+'; '+catinfo[j,1]+'; '+catinfo[j,2] WIDGET_CONTROL,wtxt,set_value=wintxt,/Append ENDIF FOR i=dux[0],dux[1],dux[2] DO $ IF i EQ -1 THEN PRINT,STRTRIM(j,2),'; ',catinfo[j,0],'; ',catinfo[j,1],'; ',catinfo[j,2] $ ELSE PRINTF,i,STRTRIM(j,2),'; ',catinfo[j,0],'; ',catinfo[j,1],'; ',catinfo[j,2] ENDFOR ENDIF IF o3[0] NE '0' THEN BEGIN ;create output filenames outf1=STRMID(infile,0,STRPOS(infile,'.',/Reverse_Search))+'.meta' outf2=STRMID(infile,0,STRPOS(infile,'.',/Reverse_Search))+'.data' OPENW,lu1,indir+outf1,/GET_LUN OPENW,lu2,indir+outf2,/GET_LUN ;HDF4 and NETCDF pre-defined attributes ncsa=['long_name','units','format','coordsys','valid_range','_FillValue','scale_factor', $ 'scale_factor_err','add_offset','add_offset_err','calibrated_nt'] PRINTF,lu1,'! Output from IDLcr8ASCII application v4.0' PRINTF,lu1,'! '+ifile & PRINTF,lu1,'!' PRINTF,lu1,'! Global Attributes' FOR i=0,N_ELEMENTS(ga)-1 DO PRINTF,lu1,ga[i] FOR j=0,n_sds-1 DO BEGIN n_atts=FIX(catinfo[j,3]) IF n_atts NE 0 THEN BEGIN PRINTF,lu1,'!' & PRINTF,lu1,'! Variable Attributes' PRINTF,lu2,catinfo[j,0] FOR i=0,n_atts DO BEGIN ;number of attributes. Last loop is for Data IF i EQ n_atts THEN vavt=SIZE(*sds[j,0].data,/TYPE) $ ELSE vavt=SIZE(*sds[j,i].va_v,/TYPE) gti=WHERE(vavt EQ vdtype,g_vavt) ;test to see if the data type is valid IF g_vavt EQ 1 THEN BEGIN ;it is OK to transfer the attribute value to a variable IF i EQ n_atts THEN vav=*sds[j,0].data ELSE vav=*sds[j,i].va_v ENDIF IF (STRPOS(STRUPCASE(catinfo[j,0]),'DATETIME') NE -1) AND ((vavt EQ 4) OR (vavt EQ 5)) THEN BEGIN ns=MAX(STRLEN(STRTRIM(LONG64(vav),2))) IF vavt EQ 4 THEN dp=6 ELSE dp=9 ;indicates the number of decimal places to use in the output ;If the number is very large then output as an exponent IF ns LT 8 THEN fmt='(D'+STRTRIM(ns+dp+1,2)+'.'+STRTRIM(dp,2)+')' ELSE fmt='(E0)' ENDIF ELSE IF (vavt EQ 4) OR (vavt EQ 5) THEN fmt='(E0)' $ ELSE IF vavt EQ 7 THEN fmt='(A-)' $ ELSE fmt='(I0)' IF i NE n_atts THEN BEGIN ;write out metadata in form LABEL=VALUE nvav=N_ELEMENTS(vav) vat=*sds[j,i].va_l ni=WHERE(STRLOWCASE(vat) EQ STRLOWCASE(ncsa),ncnt) ;check for pre-defined attributes and exclude IF (ncnt EQ 0) AND (g_vavt EQ 1) THEN BEGIN IF nvav EQ 1 THEN vat=vat+'='+STRING(FORMAT=fmt,vav[0]) $ ;'(A-'+STRTRIM(STRLEN(vav[0]),2)+')',vav[0]) $ ELSE BEGIN FOR k=0,N_ELEMENTS(vav)-1 DO BEGIN IF k EQ 0 THEN vat=vat+'='+STRTRIM(STRING(FORMAT=fmt,vav[k]),2) $ ELSE vat=vat+';'+STRTRIM(STRING(FORMAT=fmt,vav[k]),2) ENDFOR ENDELSE PRINTF,lu1,vat ENDIF ELSE IF g_vavt NE 1 THEN BEGIN bli=WHERE(vat EQ badlabel,blcnt) IF blcnt EQ 0 THEN BEGIN badlabel=[badlabel,vat] INFOTXT_OUTPUT_A,[18],[vat] ENDIF ENDIF ENDIF ENDFOR ;Write out data IF o3[0] EQ 'F' THEN BEGIN PRINTF,lu2,format=fmt,vav ENDIF ELSE IF o3[0] EQ 'D' THEN BEGIN ;Ensure there is a single character between all values sdssize=SIZE(vav,/Dimension) sdshold=STRTRIM(STRING(FORMAT=fmt,vav),2) maxstr=MAX(STRLEN(sdshold)) ;use default IDL formatting rules sdshold=STRING(format='(A-'+STRTRIM(maxstr,2)+')',sdshold) ;Uncomment line below if format described by VIS_FORMAT is required ;sdshold=STRING(format=p_f,*sds[j].data) IF sdssize[0] NE 0 THEN sdshold=REFORM(sdshold,sdssize) PRINTF,lu2,sdshold ENDIF ENDIF ELSE BEGIN ;Variable attributes do not have an associated dataset INFOTXT_OUTPUT_A,[19],[catinfo[j,0]] ENDELSE ENDFOR PRINTF,lu1,'!' & PRINTF,lu1,'! End of output file created by IDLcr8ASCII' FREE_LUN,lu1 & FREE_LUN,lu2 IF o3[3] EQ '' THEN BEGIN lineno=lineno+2L WIDGET_CONTROL,wtxt,set_value=outf1+' created!',/Append,Set_text_top_line=lineno WIDGET_CONTROL,wtxt,set_value=outf2+' created!',/Append,Set_text_top_line=lineno ENDIF FOR i=dux[0],dux[1],dux[2] DO BEGIN PRINTF,i,' '+outf1+' created!' & PRINTF,i,' '+outf2+' created!' ENDFOR ENDIF END ;Procedure Output_HDF_Data PRO idlcr8ascii, ifile, ga, sds, reterr, $ FORMAT=o1, DUMP=o2, CATALOG=o4, POPUP=o5, LOG=o6, H4=o7, H5=o8, NC=o9 ;Main IDL procedure to convert GEOMS compliant netCDF, HDF4 and HDF5 files to ASCII, Session Memory ;or another recognized Data format. ; ;Program documentation, idlcr8ascii-v4.0_Readme.pdf, available on http://avdc.gsfc.nasa.gov. ; ;Program sub-version 4.0b19 (20180218) ; ---------- ;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu ; ; History: ; 20050729: Original Release - Version 1.0 ; 20050912: Removed Common variable definition CATALOGINFO; If the format of the ; DATA_START_DATE and FILE_GENERATION_DATE values is MJD2000, then change to ISO8601, ; unless the output option is /Dump - Version 1.1 ; 20061004: Common variable definition WIDGET_WIN_A added for Error calls; Make the code suitable ; for running on a licensed version of IDL (using the .pro and .sav versions of the code) ; and on IDL Virtual Machine (using the .sav version of the code); Change the command ; line keyword options. Remove the /Help option (window now opened if there are no command ; line parameters), and /Catalog can now be called together with /AVDC or /Dump; The ; program can now handle multiple HDF files as input (either as a string array or as a ; file spec); Display input/output information as well as errors and warnings in a pop-up ; display window if the program is called using IDL VM; Help window becomes an ; Introduction window, and the user has the option of continuing to run the program with ; file inputs; Include option to read HDF5 versions of the ground-based files ; - Version 2.0 ; 20080302: Remove HELP,/TRACEBACK call, which identified how the program was called. This was used ; to stop output to the IDLDE output window in the event that IDL VM was used, but it is ; not required as IDL VM ignores these print calls; Add options to send Catalog and logged ; input/output information to a Pop-up Box (/Popup) and/or to an external file (/Log). Add ; DIALOG_BOX to show completion of the program if program inputs are via the INTRO box, and ; logging window option isn't requested - Version 3.0 ; 20091208: Add INFORMATION text message to INFOTXT_OUTPUT_A procedure; Replace /AVDC keyword with ; /FORMAT (note /AVDC also retained for backward compatibility purposes); Improve checks ; on parameter and keyword inputs - Version 3.03 ; 20101122: Changes made to account for new GEOMS conventions, including changing the format of the ; structure so that numeric metadata values can be saved in their actual datatype, rather ; than string; Add return error code option so that program returns to the calling program ; if an error is generated - Version 4.0b1 ; 20120328: Add options to convert HDF/netCDF files to other HDF/netCDF formats e.g. H4 to ; H5 etc. Requires IDLcr8HDF in the search path to do this - Version 4.0b5 ; 20130114: Add check for IDL being run in DEMO mode by using LMGR command. Replace PRINTF,-1 ; statements (i.e. when dux[0] eq -1) with PRINT (o/w causes DEMO mode to stop). Also disable ; /LOG, /FORMAT and /DUMP keywords and stop ASCII file output in DEMO mode - Version 4.0b7 ; 20150127: Add extra condition for the file extension when checking whether an input file is netCDF ; due to the program crashing when incorrectly identifying an HDF5 file as netCDF when ; using IDL8.3 (OK for IDL6.4) - Version 4.0b10 ; 20160213: Add CATCH, /CANCEL after idlcr8hdf call - Version 4.0b13 ; ; Inputs: ifile - a string array or filespec containing the name of the HDF file(s) to be read in ; ; OPTIONS ; FORMAT - Program generates two output files per input file (with .meta and .data ; extensions). The resulting files are compatible with programs that can write HDF ; files using these two files as input. ; DUMP - Program generates two output files per input file (with .meta and .data extensions). ; Data values will be shown as indicated by the array format defined by VAR_SIZE. ; CATALOG - Program sends the variable index, variable name, format, and dimension(s) ; information to a pop-up window or the IDL DE output log window. ; POPUP - to append input/output information as well as warnings and errors to a pop-up ; display window ; LOG - to append input/output/catalog information as well as errors to a log file named ; idlcr8ascii.log ; H4 - outputs contents of the input file as an HDF4 file ; H5 - outputs contents of the input file as an HDF4 file ; NC - outputs contents of the input file as a netCDF file ; ; Outputs: ga - If included as a parameter in the command line, a string array containing the global ; attribute labels and values extracted from a single HDF file. ; sds - If included as a parameter in the command line, a structure using pointers, of size ; [n_sds,n_atts], containing the variable attribute labels and values (sds[n,m].va_l ; and sds[n.m].va_v) and the data (sds[n,0].data) extracted from a single HDF file. ; The data is saved in the form defined by the variable attributes ; reterr - optional string input that, if included, returns an error message to the calling ; program rather than stop the program. Note that the Pop-up option will be ; deselected if present together with this argument. ; Two ASCII files per input file containing the attributes and datasets extracted from the HDF ; file(s), if the FORMAT or DUMP options are chosen. Summary metadata information if the ; CATALOG option is chosen. Other Hierachical data format files if H4, H5, or netCDF are chosen. ; ; Called by: Main Routine ; ; Subroutines Called: INTRO_A (if program called without command line parameters) ; READ_HDF_SDS ; JDF_2_DATETIME ; OUTPUT_HDF_DATA (if output is not just to session memory only) ; STOP_WITH_ERROR_A (if error state detected) ; INFOTXT_OUTPUT_A (if program can make a change) ; Possible Conditions for STOP_WITH_ERROR call (plus [line number] where called): ; 1. No valid HDF4, HDF5 or netCDF file selected. ; ; Information Conditions (when the program is able to make changes): ; 1. If Session Memory option is chosen can only read in one HDF file. ; 2. /POPUP keyword cannot be used together with the 'reterr' argument. ; 3. Argument 'reterr' must be a scalar variable of type string. ; 4. /LOG, /FORMAT, and /DUMP keywords cannot be used in IDL DEMO mode ; 5. NetCDF file create feature is disabled in IDL DEMO mode COMMON WIDGET_WIN_A ;Possible error messages for this procedure proname='IDLCR8ASCII procedure: ' IF FLOAT(!Version.Release) GE 5.6 THEN errtxt='No valid HDF4, HDF5, or netCDF file selected.' $ ELSE BEGIN errtxt='No valid HDF4 or netCDF file selected ' errtxt=errtxt+'(Note: Input file can only be HDF4 or netCDF for IDL'+!Version.Release+').' ENDELSE lu=-1 ON_IOERROR,TypeConversionError ;used when checking Datetime format demomode=LMGR(/DEMO) ;Boolean to check for IDL being run in demo mode (disable /LOG option) IF N_PARAMS() EQ 1 THEN BEGIN ;File input only ;check to see what output keyword options are set - if none then intype=-3 (i.e. invalid) IF (~ KEYWORD_SET(o1)) AND (~ KEYWORD_SET(o2)) AND (~ KEYWORD_SET(o4)) AND $ (~ KEYWORD_SET(o7)) AND (~ KEYWORD_SET(o8)) AND (~ KEYWORD_SET(o9)) THEN $ intype=-3 ELSE intype=SIZE(ifile,/Type) ENDIF ELSE IF N_PARAMS() GE 1 THEN intype=SIZE(ifile,/Type) $ ;Check that infile is a string ELSE IF (KEYWORD_SET(o1)) OR (KEYWORD_SET(o2)) OR (KEYWORD_SET(o4)) OR $ (KEYWORD_SET(o7)) OR (KEYWORD_SET(o8)) OR (KEYWORD_SET(o9)) THEN $ intype=-1 $ ;need to open dialog box for input file ELSE IF (KEYWORD_SET(o5)) OR (KEYWORD_SET(o6)) THEN intype=-3 $ ;cannot have these keyword options only ELSE intype=-2 ;no parameter or keyword inputs in idlcr8ascii call IF (intype NE 7) AND (intype GE 0) THEN intype=-3 ;first command line parameter not a string type dux=[-1,-1,1] ;initialize flags for output log to IDLDE Output Window and/or to file ;dux[0] default allows output to the IDLDE output window (ignored by IDL VM) ;dux[1] is the program assigned file unit for sending log output to file ;dux[2] is the step between dux[0] and dux[1] rerr='NA' ;initialize return error string o3=['-1','0','0','0','0','0','0','0'] IF intype LT -1 THEN BEGIN ;either no or invalid input parameters and/or keywords INTRO_A,intype ;Open Intro Box and determine output options (FORMAT/DUMP, Catalog, etc) IF o3[0] EQ '-1' THEN BEGIN STOP_WITH_ERROR_A,'','',lu & RETURN ENDIF IF o3[3] EQ 'P' THEN o3[3]='' ELSE o3[3]='D_' ENDIF ELSE BEGIN ;Set options (FORMAT/DUMP/CONVERT, Catalog, Session Memory, Popup Window, Log Output) IF (KEYWORD_SET(o1)) THEN o3=['F','0','0','D_','0','0','0','0'] $ ELSE IF KEYWORD_SET(o2) THEN o3=['D','0','0','D_','0','0','0','0'] $ ELSE o3=['0','0','0','D_','0','0','0','0'] IF N_PARAMS() GE 3 THEN o3[2]='M' IF KEYWORD_SET(o4) THEN o3[1]='C' IF KEYWORD_SET(o5) THEN o3[3]='' IF KEYWORD_SET(o6) THEN o3[4]='idlcr8ascii.log' IF KEYWORD_SET(o7) THEN o3[5]='H4' IF KEYWORD_SET(o8) THEN o3[6]='H5' IF KEYWORD_SET(o9) THEN o3[7]='NC' ENDELSE ;Check for reterr parameter infoval=INTARR(4) IF (N_PARAMS() EQ 2) OR (N_PARAMS() GE 4) THEN BEGIN ;If reterr included allows error output to return to a calling program IF N_PARAMS() EQ 2 THEN reterr=ga ;session memory option not wanted ;IF (ARG_PRESENT(reterr)) AND (SIZE(reterr,/Type) EQ 7) THEN BEGIN IF SIZE(reterr,/Type) EQ 7 THEN BEGIN rerr='' IF o3[3] EQ '' THEN BEGIN ;Deselect POPUP option and write INFORMATION message o3[3]='D_' & infoval[0]=14 ENDIF ENDIF ELSE infoval[1]=15 ;reterr input included but is of the incorrect type, or parameter cannot be returned to the calling program ENDIF ;Do check for /LOG, /FORMAT, /DUMP, and/or /NC keywords in IDL DEMO Mode and disable if necessary IF (demomode) AND ((o3[4] EQ 'idlcr8ascii.log') OR (o3[0] NE '0') OR (o3[7] EQ 'NC')) THEN BEGIN IF (o3[4] EQ 'idlcr8ascii.log') OR (o3[0] NE '0') THEN infoval[2]=16 IF o3[7] EQ 'NC' THEN infoval[3]=17 o3[0]='0' & o3[4]='0' & o3[7]='0' ENDIF IF N_PARAMS() GE 1 THEN BEGIN ;Check that input HDF/netCDF files exist. If not then prompt the user for the missing files arorfs=SIZE(ifile,/Dimensions) ;Is file input Filespec or String Array? IF (arorfs[0] EQ 0) THEN BEGIN ;input is filespec IF ifile NE '' THEN ifile=FILE_SEARCH(ifile) isize=SIZE(ifile,/Dimensions) IF isize[0] EQ 0 THEN ifile=[''] ;no files found matching filespec ENDIF IF N_ELEMENTS(arorfs) GT 1 THEN ifile=REFORM(ifile,N_ELEMENTS(ifile),/Overwrite) ;Convert to 1-D array file_exist=FILE_TEST(ifile,/Read) gi=WHERE(file_exist EQ 1,gcnt) IF gcnt NE 0 THEN ifile=ifile[gi] ;contains all valid filenames ENDIF ELSE ifile=[''] gi=WHERE(ifile NE '',nfile) IF nfile EQ 0L THEN BEGIN IF o3[2] EQ 'M' THEN $ ;pick one file only ifile=DIALOG_PICKFILE(Filter=['*.hdf','*.h5'],/Must_Exist, $ Title='Select HDF4, HDF5, or netCDF GEOMS Compatible Format File') $ ELSE $ ;multiple selection permissable ifile=DIALOG_PICKFILE(Filter=['*.hdf','*.h5','*.nc'],/Must_Exist, $ /Multiple_Files,Title='Select HDF4, HDF5, or netCDF GEOMS Compatible Format File(s)') ENDIF ;now check to see that files were actually selected, if not STOP! gi=WHERE(ifile NE '',nfile) IF nfile EQ 0L THEN BEGIN STOP_WITH_ERROR_A,'D_'+proname,errtxt,lu IF STRLEN(rerr) GT 2 THEN BEGIN IF N_PARAMS() EQ 2 THEN ga=rerr ELSE reterr=rerr ENDIF RETURN ENDIF ELSE BEGIN ifile=ifile[gi] & dsort=SORT(ifile) & ifile=ifile[dsort] ifile=FILE_SEARCH(ifile,/FULLY_QUALIFY_PATH) ENDELSE ;Set dux values according to output options IF o3[4] NE '0' THEN BEGIN ;open idlcr8ascii.log o3[4]=FILE_DIRNAME(ifile[0],/Mark_Directory)+o3[4] res=FILE_TEST(o3[4],/Write) IF res EQ 0 THEN openw,du,o3[4],/GET_LUN $ ELSE BEGIN OPENW,du,o3[4],/Append,/GET_LUN & FOR i=0,1 DO PRINTF,du,'' ENDELSE dux[1]=du & dux[2]=dux[1]-dux[0] ENDIF FOR i=dux[0],dux[1],dux[2] DO $ IF i EQ -1 THEN PRINT,'HDF/netCDF File Input/Output Log - Program Started on '+SYSTIME(0) $ ELSE PRINTF,i,'HDF/netCDF File Input/Output Log - Program Started on '+SYSTIME(0) IF o3[3] EQ '' THEN BEGIN ;Set-up output window display widget base=WIDGET_BASE(Title='HDF/netCDF File Input/Output Log',Units=2,yoffset=3,xoffset=3,Tlb_Frame_Attr=1,/Column) ;,Tab_Mode=1) wtxt=WIDGET_TEXT(base,/Scroll,xsize=100,ysize=12) base2=WIDGET_BASE(base) b3=WIDGET_BUTTON(base2,value='Finish',uvalue='DONE',frame=3,Sensitive=0) ;,Accelerator='Return') WIDGET_CONTROL,wtxt,/Realize WIDGET_CONTROL,base,/Realize lineno=0L ENDIF IF (o3[2] EQ 'M') AND (N_ELEMENTS(ifile) GT 1) THEN BEGIN ifile=[ifile[0]] & nfile=1L INFOTXT_OUTPUT_A,[7] ENDIF FOR i=0,N_ELEMENTS(infoval)-1 DO IF infoval[i] NE 0 THEN INFOTXT_OUTPUT_A,[infoval[i]] FOR ndf=0L,nfile-1L DO BEGIN catinfo=STRARR(1,4) ;initialize catinfo array ifile[ndf]=FILE_SEARCH(ifile[ndf],/Fully_Qualify_Path) IF o3[3] EQ '' THEN BEGIN lineno=lineno+2L IF ndf NE 0 THEN WIDGET_CONTROL,wtxt,set_value='',/Append WIDGET_CONTROL,wtxt,set_value='Reading '+ifile[ndf],/Append,Set_text_top_line=lineno ENDIF FOR i=dux[0],dux[1],dux[2] DO BEGIN IF i EQ -1 THEN BEGIN IF ndf NE 0 THEN PRINT,'' PRINT,'Reading '+ifile[ndf] & PRINT,'' ENDIF ELSE BEGIN IF ndf NE 0 THEN PRINTF,i,'' PRINTF,i,'Reading '+ifile[ndf] & PRINTF,i,'' ENDELSE ENDFOR IF FILE_TEST(ifile[ndf]) EQ 1 THEN BEGIN ;Test for netCDF validname=1 CATCH, ncdferror ;To catch the error when attempting to open the file as a netCDF ;Error Handler IF ncdferror NE 0 THEN BEGIN ;not a netCDF file validname=0 CATCH, /CANCEL ENDIF IF (validname EQ 1) AND (STRMID(STRUPCASE(ifile[ndf]),STRLEN(ifile[ndf])-3,3) EQ '.NC') THEN BEGIN ;Need to add extra condition because H5 files were being accepted then crashing the program. ;Try to open the file. If successful, file is valid. fileid=NCDF_OPEN(ifile[ndf]) ;If get to here then file is netCDF CATCH, /CANCEL NCDF_CLOSE,fileid catinfo[0,0]='NC' ENDIF ELSE BEGIN ;Not netCDF so test for HDF4 or HDF5 IF HDF_ISHDF(ifile[ndf]) EQ 1 THEN catinfo[0,0]='H4' $ HDF4 ELSE IF FLOAT(!Version.Release) GE 5.6 THEN BEGIN ;Test for HDF5 IF H5F_IS_HDF5(ifile[ndf]) EQ 1 THEN catinfo[0,0]='H5' $ ELSE BEGIN STOP_WITH_ERROR_A,o3[3]+proname,errtxt,lu IF STRLEN(rerr) GT 2 THEN BEGIN IF N_PARAMS() EQ 2 THEN ga=rerr ELSE reterr=rerr ENDIF RETURN ENDELSE ENDIF ELSE BEGIN STOP_WITH_ERROR_A,o3[3]+proname,errtxt,lu IF STRLEN(rerr) GT 2 THEN BEGIN IF N_PARAMS() EQ 2 THEN ga=rerr ELSE reterr=rerr ENDIF RETURN ENDELSE ENDELSE CATCH, /CANCEL ENDIF ELSE BEGIN STOP_WITH_ERROR_A,o3[3]+proname,errtxt,lu IF STRLEN(rerr) GT 2 THEN BEGIN IF N_PARAMS() EQ 2 THEN ga=rerr ELSE reterr=rerr ENDIF RETURN ENDELSE ;Open and read the file READ_HDF_SDS,ifile[ndf],ga,sds,catinfo IF STRLEN(rerr) GT 2 THEN BEGIN reterr=rerr & RETURN ENDIF IF catinfo[0,0] EQ 'HE5' THEN BEGIN STOP_WITH_ERROR_A,o3[3]+proname,errtxt,lu IF STRLEN(rerr) GT 2 THEN BEGIN IF N_PARAMS() EQ 2 THEN ga=rerr ELSE reterr=rerr ENDIF RETURN ENDIF ELSE BEGIN ;Do checks and output standard format file ;Convert format of DATA_START/STOP_DATE and FILE_GENERATION_DATE if required ;Note: for /Dump option output will still be original format isodates=['DATA_START_DATE','DATA_STOP_DATE','FILE_GENERATION_DATE'] niso=N_ELEMENTS(isodates) gaiso=STRARR(niso) & gih=INTARR(niso) FOR i=0,niso-1 DO BEGIN gi=WHERE(STRPOS(STRUPCASE(ga),isodates[i]) NE -1,gcnt) IF gcnt EQ 1 THEN BEGIN res=STRSPLIT(ga[gi[0]],' =',/EXTRACT) IF N_ELEMENTS(res) EQ 2 THEN BEGIN IF STRPOS(STRUPCASE(res[1]),'Z') EQ -1 THEN BEGIN ;i.e. not ISO8601 valid=0 ;used for Type Conversion Check mjd=DOUBLE(res[1]) ;will jump to TypeConversionError if Res[1] is not a number valid=1 ;got to here so valid conversion (presumably it is MJD2000) TypeConversionError: IF valid EQ 1 THEN BEGIN iso='' & iso=JDF_2_DATETIME(mjd,/MJD2000,/S) gaiso[i]=res[0]+'='+iso IF o3[0] EQ 'F' THEN ga[gi[0]]=gaiso[i] gih[i]=gi[0] ENDIF ENDIF ENDIF ENDIF ENDFOR ;Create ASCII Output IF (o3[0] NE '0') OR (o3[1] NE '0') THEN OUTPUT_HDF_DATA,ifile[ndf],ga,sds,catinfo IF (o3[5] NE '0') OR (o3[6] NE '0') OR (o3[7] NE '0') THEN BEGIN ;Check for existence of TAV file in the input file directory or working directory indir=FILE_DIRNAME(ifile[ndf],/MARK_DIRECTORY) tav=FILE_SEARCH(indir+'tableattrvalue*.dat',/FULLY_QUALIFY_PATH) IF tav[0] EQ '' THEN INFOTXT_OUTPUT_A,[12] $ ;No valid TAV file so format conversion cannot be done ELSE BEGIN ;Convert data to a different format validpath=1 & ftest=5 & dferr='' IF demomode THEN ftestmax=6 ELSE ftestmax=7 WHILE (validpath EQ 1) AND (ftest LE ftestmax) DO BEGIN IF o3[ftest] NE '0' THEN BEGIN CATCH, path_error ;Return program generated error if IDLcr8HDF is not in the search path ;Error Handler IF path_error NE 0 THEN BEGIN ;IDLcr8HDF is not in the IDL Search Path validpath=0 INFOTXT_OUTPUT_A,[13] ;PRINT, 'Error index: ', path_error ;PRINT, 'Error message: ', !ERROR_STATE.MSG CATCH, /CANCEL ENDIF IF validpath EQ 1 THEN BEGIN ;Try to call IDLcr8HDF IF demomode THEN BEGIN CASE 1 OF ftest EQ 5: IDLCR8HDF,ga,sds,tav[0],indir,dferr ftest EQ 6: IDLCR8HDF,ga,sds,tav[0],indir,dferr,/H5 ENDCASE ENDIF ELSE BEGIN CASE 1 OF ftest EQ 5: IDLCR8HDF,ga,sds,tav[0],indir,dferr,/L ftest EQ 6: IDLCR8HDF,ga,sds,tav[0],indir,dferr,/H5,/L ftest EQ 7: IDLCR8HDF,ga,sds,tav[0],indir,dferr,/NC,/L ENDCASE ENDELSE CATCH, /CANCEL IF o3[4] NE '0' THEN BEGIN ;transfer contents of idlcr8hdf.log to idlcr8ascii.log OPENR,fu,indir+'idlcr8hdf.log',/DELETE,/GET_LUN dum='' READF,fu,dum & READF,fu,dum WHILE NOT EOF(fu) DO BEGIN READF,fu,dum & PRINTF,du,dum ENDWHILE FREE_LUN,fu ENDIF ELSE FILE_DELETE,indir+'idlcr8hdf.log',/QUIET ENDIF ENDIF ftest++ ENDWHILE ENDELSE ENDIF FOR i=0,niso-1 DO IF gaiso[i] NE '' THEN ga[gih[i]]=gaiso[i] IF o3[2] EQ 'M' THEN BEGIN IF o3[3] EQ '' THEN BEGIN lineno=lineno+1L WIDGET_CONTROL,wtxt,set_value=ifile+' array and heap structure created!',/Append,Set_text_top_line=lineno ENDIF FOR i=dux[0],dux[1],dux[2] DO $ IF i EQ -1 THEN PRINT,' '+ifile[ndf]+' array and heap structure created!' $ ELSE PRINTF,i,' '+ifile[ndf]+' array and heap structure created!' ENDIF ENDELSE ENDFOR IF rerr EQ '' THEN $ IF nfile EQ 1 THEN reterr=ifile[0]+' successfully read' $ ELSE reterr='Files successfully read' IF N_PARAMS() EQ 2 THEN ga=reterr FOR i=dux[0],dux[1],dux[2] DO BEGIN IF i EQ -1 THEN BEGIN PRINT,'' & PRINT,'File read completed - Program Ended on '+SYSTIME(0) ENDIF ELSE BEGIN PRINTF,i,'' & PRINTF,i,'File read completed - Program Ended on '+SYSTIME(0) ENDELSE ENDFOR IF dux[1] GT -1 THEN FREE_LUN,dux[1] IF o3[3] EQ '' THEN BEGIN WIDGET_CONTROL,b3,Sensitive=1,/Input_Focus WIDGET_CONTROL,wtxt,set_value='',/Append,Set_text_top_line=lineno+2L WIDGET_CONTROL,wtxt,set_value='HDF file read completed - hit to close program',/Append XMANAGER,'idlcr8ascii',base ENDIF ELSE IF intype LT 0 THEN BEGIN ;Create Finish Dialog Box res=DIALOG_MESSAGE('HDF file read completed!',/Information,Title='AVDC IDLcr8ASCII') ENDIF END ;procedure IDLcr8ASCII