VIIRS Aerosol Science

Processing Algorithm
(AEROSOL_SPA)
User's Guide
Version 1.5.08.04

April 2015

GODDARD SPACE FLIGHT CENTER

GREENBELT, MARYLAND
 VIIRS Aerosol Science Processing Algorithm
AEROSOL_SPA
General
The NASA Goddard Space Flight Center's (GSFC) Direct Readout Laboratory (DRL), Code
606.3 developed this software for the International Polar Orbiter Processing Package
(IPOPP). IPOPP maximizes the utility of Earth science data for making real-time decisions by
giving fast access to instrument data and derivative products from the Suomi National Polar-
orbiting Partnership (SNPP), Aqua, and Terra missions and, in the future, the Joint Polar
Satellite System (JPSS) mission.

Users must agree to all terms and conditions in the Software Usage Agreement on the DRL
Web Portal before downloading this software.

Software and documentation published on the DRL Web Portal may occasionally be updated
or modified. The most current versions of DRL software are available at the DRL Web Portal:

http://directreadout.sci.gsfc.nasa.gov/?id=software

Questions relating to the contents or status of this software and its documentation should be
addressed to the DRL via the Contact DRL mechanism at the DRL Web Portal:

http://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=66

Algorithm Wrapper Concept

The DRL has developed an algorithm wrapper to provide a common command and execution
interface to encapsulate multi-discipline, multi-mission science processing algorithms. The
wrapper also provides a structured, standardized technique for packaging new or updated
algorithms with minimal effort.

A Science Processing Algorithm (SPA) is defined as a wrapper and its contained algorithm.
SPAs will function in a standalone, cross-platform environment to serve the needs of the broad
Direct Readout community. Detailed information about SPAs and other DRL technologies is
available at the DRL Web Portal.

Software Description
This software package contains the Visible Infrared Imaging Radiometer Suite (VIIRS)
Aerosol Science Processing Algorithm (AEROSOL_SPA). The VIIRS Aerosol algorithm takes
as input VIIRS Moderate Resolution Bands Sensor Data Record (SDR) and Terrain-Corrected
Geolocation products, the VIIRS Cloud Mask Intermediate Product (IP), and meteorological
ancillary, and produces the mission-compliant Aerosol Optical Thickness (AOT) IP, Aerosol
Model Index (AMI) IP, Aerosol Environmental Data Record (EDR), Suspended Matter (SM)
EDR, and Aerosol Geolocation HDF5 product. The SPA functions in two modes: Standalone,
or as an IPOPP plug-in.

AEROSOL_SPA Page 1 April 2015

Software Version
Version 1.5 of the DRL algorithm wrapper was used to package the SPA described in this
document. The VIIRS Aerosol algorithm has been ported from the IDPS OPS Version
1.5.08.04.
Enhancements to this SPA include advanced capabilities to meet low latency requirements,
available in both Standalone and IPOPP Mode (IPOPP v2.4 or later).
This software will execute on a 64-bit computer and has been tested on computers with 32GB
of RAM, with the following operating systems:
a) Fedora 20 X86_64;
b) CentOS Linux 7 X86_64;
c) OpenSUSE Linux 12.1 X86_64;
d) Kubuntu 14.04 X86_64.

Copyright 1999-2007, United States Government as represented by the Administrator for the
National Aeronautics and Space Administration. All Rights Reserved.

Credits
The VIIRS Aerosol algorithm was provided to the DRL by the JPSS Mission. This algorithm
was ported to run outside of the IDPS by the DRL in collaboration with the Land Product
Evaluation and Algorithm Test Element (LPEATE).

Prerequisites
To run this package, you must have the Java Development Kit (JDK) or Java Runtime
Engine (JRE) (Java 1.6.0_25 or higher) installed on your computer, and have the Java
installation bin/ subdirectory in your PATH environment variable. This package contains 64-
bit binaries statically pre-compiled on an x86-compatible 64-bit computer running under
Fedora 14, using gcc 4.5.1.

Program Inputs and Outputs

The VIIRS Aerosol algorithm takes as input VIIRS Moderate Resolution Bands M1, M2, M3,
M4, M5, M6, M7, M8, M9, M10, M11, M12, M15, and M16 Sensor Data Record (SDR)
products, the VIIRS Moderate Resolution Band Terrain-Corrected Geolocation product, the
VIIRS Cloud Mask Intermediate Product (IP), and meteorological ancillary to produce the
mission-compliant Aerosol Optical Thickness (AOT) IP, Aerosol Model Index (AMI) IP, Aerosol
Environmental Data Record (EDR), Suspended Matter (SM) EDR, and Aerosol Geolocation
HDF5 products.

AEROSOL_SPA Page 2 April 2015

Installation and Configuration

Installing as a Standalone Application:

Download the AEROSOL_1.5.08.04_SPA_1.5.tar.gz and

AEROSOL_1.5.08.04_SPA_1.5_testdata.tar.gz (optional) files into the same directory.

Decompress and un-archive the AEROSOL_1.5.08.04_SPA_1.5.tar.gz and

AEROSOL_1.5.08.04_SPA_1.5_testdata.tar.gz (optional) files:

$ tar –xzf AEROSOL_1.5.08.04_SPA_1.5.tar.gz

$ tar –xzf AEROSOL_1.5.08.04_SPA_1.5_testdata.tar.gz

This will create the following subdirectories:

SPA
Aerosol
algorithm
ancillary
mode
station
testdata
testscripts
wrapper

Installing into an IPOPP Framework: This SPA can also be installed dynamically into an
IPOPP framework to automate production of VIIRS Aerosol Optical Thickness IP, Aerosol
Model Index IP, Aerosol Environmental Data Record, Suspended Matter EDR, and Aerosol
Geolocation data products. The SPA installation process will install SPA service(s) into
IPOPP. An SPA service is an IPOPP agent that provides the mechanism necessary for
running an SPA automatically within the IPOPP framework. Once this SPA is installed, users
must enable the service(s) corresponding to this SPA along with any other Prerequisite
service(s). Instructions for installing an SPA and enabling its services are contained in the
IPOPP User's Guide (available on the DRL Web Portal). The SPA services associated with
this SPA are listed in Appendix A.

Software Package Testing and Validation

The testscripts subdirectory contains test scripts that can be used to verify that your current
installation of the SPA is working properly, as described below. Note that the optional
AEROSOL_1.5.08.04_SPA_1.5_testdata.tar.gz file is required to execute these testing
procedures.

Step 1: cd into the testscripts directory.

Step 2: There is a script named run-vaerosol inside the testscripts directory.
To run the VIIRS Aerosol algorithm, use

$ ./run-vaerosol

AEROSOL_SPA Page 3 April 2015

A successful execution usually requires fifteen minutes or more, depending on the speed of
your computer and the size of the input. If everything is working properly, the scripts will
terminate with a message such as:

Output viirs.aot is /home/ipopp/drl/SPA/Aerosol/testdata/output/VAOOO.h5

Output viirs.aotip is /home/ipopp/drl/SPA/Aerosol/testdata/output/IVAOT.h5
Output viirs.amiip is /home/ipopp/drl/SPA/Aerosol/testdata/output/IVAMI.h5
Output viirs.gaero is /home/ipopp/drl/SPA/Aerosol/testdata/output/GAERO.h5
Output viirs.sum is /home/ipopp/drl/SPA/Aerosol/testdata/output/VSUMO.h5

You can cd to the output directory to verify that the science products exist. Test output
product(s) are available for comparison in the testdata/output directory. These test output
product(s) were generated on a 64-bit PC architecture computer running Fedora 14. The
output products serve as an indicator of expected program output. Use a comparison utility
(such as diff, h5diff, etc.) to compare your output product(s) to those provided in the
testdata/output directory. Locally generated files may differ slightly from the provided output
files because of differences in machine architecture or operating systems.

If there is a problem and the code terminates abnormally, the problem can be identified using
the log files. Log files are automatically generated within the directory used for execution.
They start with stdfile* and errfile*. Other log and intermediate files may be generated
automatically within the directory used for execution. They are useful for traceability and
debugging purposes. However it is strongly recommended that users clean up log files and
intermediate files left behind in the run directory before initiating a fresh execution of the SPA.
Intermediate files from a previous run may affect a successive run and produce ambiguous
results. Please report any errors that cannot be fixed to the DRL.

Program Operation
In order to run the package using your own input data, you can either use the run scripts within
the wrapper subdirectories, or modify the test scripts within the testscripts subdirectory.

To Use the Run Scripts

Identify the 'run' scripts: The wrapper directory within this package contains one
subdirectory named Aerosol. The subdirectory contains an executable called 'run'. Execute
'run' within the correct wrapper subdirectory to generate the corresponding product. For
instance, the 'run' within wrapper/Aerosol is used for creating VIIRS Aerosol outputs. Note
that to execute 'run', you need to have java on your path.

Specify input parameters using <label value> pairs: To execute the 'run' scripts, you must
supply the required input and output parameters. Input and output parameters are usually file
paths or other values (e.g., an automatic search flag). Each parameter is specified on the
command line by a <label value> pair. Labels are simply predefined names for parameters.
Each label must be followed by its actual value. Each process has its own set of <label value>
pairs that must be specified in order for it to execute. Some of these
pairs are optional, meaning the process would still be able to execute even if that parameter
is not supplied. The two types of <label value> pairs that the AEROSOL_SPA uses are:

AEROSOL_SPA Page 4 April 2015

 a) Input file label/values. These are input file paths. Values are absolute or relative paths
to the corresponding input file.
b) Output file label/values. These are output files that are produced by the SPA. Values
are absolute or relative paths of the files you want to generate.

The following tables contain labels, and their descriptions, required by the AEROSOL_SPA.

Input File Labels Description Source

viirs.gmtco VIIRS Moderate Resolution Band

1. The C-SDR_SPA and VIIRS-SDR_SPA can
Terrain-Corrected Geolocation
be used to create these products.
input HDF5 file path
2. Real time VIIRS SDR and Geolocation
viirs.svmxx VIIRS Moderate Resolution Band products over the eastern US region are
Mx {x = 1 to 12, 15, 16} SDR input available from the DRL ftp site at:
{xx = 01 to 12, 15, ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/viirs/lev
HDF5 file path
16} el1/<SVMxx|GMTCO>_npp_dyyyymmdd_thh
mmssS_ehhmmssS*.h5
Where yyyy, mm, dd represents the year,
month, and day of month for the start of the
swath; the first hh, mm, ss, S represents the
hour, minutes, seconds, and 10th of a second
for the start of the swath and the second hh,
mm, ss, S represents the end time of the
swath.
3. VIIRS SDR and Geolocation products for other
locations and times are available for download
at
www.class.noaa.gov

viirs.cmip VIIRS Cloud Mask IP input HDF5

1. The CLOUDMASK_SPA can be used to create
file path
this product.
2. Real time Cloud Mask IP products over the
eastern US region are available from the DRL
ftp site at:
ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/viirs/lev
el2/IICMO_npp_dyyyymmdd_thhmmssS_ehh
mmssS*.h5
Where yyyy, mm, dd represents the year,
month, and day of month for the start of the
swath; the first hh, mm, ss, S represents the
hour, minutes, seconds, and 10th of a second
for the start of the swath and the second hh,
mm, ss, S represents the end time of the
swath.
3. Cloud Mask IP products for other locations and
times are available for download at
www.class.noaa.gov

AEROSOL_SPA Page 5 April 2015

ncep_met NCEP Numerical Weather For recent files go to:
Prediction Gridded Binary (GRIB) ftp://is.sci.gsfc.nasa.gov/ancillary/temporal/global/gda
File. This can be either a Global s/
Data Assimilation System
(GDAS1, 6 hourly, 1 degree For archived files go to:
global) analysis field file or a ftp://is.sci.gsfc.nasa.gov/ArchivedAncillary/temporal/gl
Global Model Forecast Fields obal/gdas/
(GFS) file (The GDAS and GFS
files must be in grib1 format).
Refer to the "NOTES" section
below for more details regarding
these meteorological ancillary file
inputs.

Output File Description

Labels

viirs.amiip VIIRS Aerosol Model Index IP output HDF5 file path (Daytime only)

viirs.aot VIIRS Aerosol EDR output HDF5 file path (Daytime only)

viirs.aotip VIIRS Aerosol Optical Thickness IP output HDF5 file path (Daytime only)

viirs.gaero VIIRS Aerosol Geolocation output HDF5 file path (Daytime only)

viirs.sum VIIRS Suspended Matter EDR output HDF5 file path (Daytime only)

Execute the 'run': The following script shows an example of a command line to run the VIIRS
Aerosol algorithm from the testscripts directory:
$ ../wrapper/Aerosol/run \
ncep_met ../testdata/input/gdas1.PGrbF00.140901.18z \
viirs.cmip ../testdata/input/IICMO_npp_d20140901_t1738560_e1740201_b14746_c20140908200708003499_noaa_ops.h5 \
viirs.gmtco ../testdata/input/GMTCO_npp_d20140901_t1738560_e1740201_b14746_c20140908200702642321_noaa_ops.h5 \
viirs.svm01 ../testdata/input/SVM01_npp_d20140901_t1738560_e1740201_b14746_c20140908200638045758_noaa_ops.h5 \
viirs.svm02 ../testdata/input/SVM02_npp_d20140901_t1738560_e1740201_b14746_c20140908200623073487_noaa_ops.h5 \
viirs.svm03 ../testdata/input/SVM03_npp_d20140901_t1738560_e1740201_b14746_c20140908200714455393_noaa_ops.h5 \
viirs.svm04 ../testdata/input/SVM04_npp_d20140901_t1738560_e1740201_b14746_c20140908200714715029_noaa_ops.h5 \
viirs.svm05 ../testdata/input/SVM05_npp_d20140901_t1738560_e1740201_b14746_c20140908200735223402_noaa_ops.h5 \
viirs.svm06 ../testdata/input/SVM06_npp_d20140901_t1738560_e1740201_b14746_c20140908200831329343_noaa_ops.h5 \
viirs.svm07 ../testdata/input/SVM07_npp_d20140901_t1738560_e1740201_b14746_c20140908200729256422_noaa_ops.h5 \
viirs.svm08 ../testdata/input/SVM08_npp_d20140901_t1738560_e1740201_b14746_c20140908200803979409_noaa_ops.h5 \
viirs.svm09 ../testdata/input/SVM09_npp_d20140901_t1738560_e1740201_b14746_c20140908200743076006_noaa_ops.h5 \
viirs.svm10 ../testdata/input/SVM10_npp_d20140901_t1738560_e1740201_b14746_c20140908200925075349_noaa_ops.h5 \
viirs.svm11 ../testdata/input/SVM11_npp_d20140901_t1738560_e1740201_b14746_c20140908200858348915_noaa_ops.h5 \
viirs.svm12 ../testdata/input/SVM12_npp_d20140901_t1738560_e1740201_b14746_c20140908200844775167_noaa_ops.h5 \
viirs.svm15 ../testdata/input/SVM15_npp_d20140901_t1738560_e1740201_b14746_c20140908200911898965_noaa_ops.h5 \
viirs.svm16 ../testdata/input/SVM16_npp_d20140901_t1738560_e1740201_b14746_c20140908200729274370_noaa_ops.h5 \
viirs.amiip ../testdata/output/IVAMI.h5 \
viirs.aot ../testdata/output/VAOOO.h5 \
viirs.aotip ../testdata/output/IVAOT.h5 \
viirs.gaero ../testdata/output/GAERO.h5 \
viirs.sum ../testdata/output/VSUMO.h5

A successful execution usually requires fifteen minutes or more, depending on the speed of
your computer and the size of the input. If execution fails, you will see an error message

AEROSOL_SPA Page 6 April 2015

indicating the cause of failure (e.g., a file cannot be found, or a label cannot be recognized).
Correct it and run again. If the problem has some other cause, it can be identified using the
log files. Log files are automatically generated within the directory used for execution. They
start with stdfile* and errfile* and can be deleted after execution. Other log and intermediate
files may be generated automatically within the directory used for execution. They are useful
for traceability and debugging purposes. However it is strongly recommended that users clean
up log files and intermediate files left behind in the run directory before initiating a fresh
execution of the SPA. Intermediate files from a previous run may affect a successive run and
produce ambiguous results. The 'run' can be executed from any directory the user chooses.
This can be done by prefixing it with the file path for the 'run' script.

NOTES:

1. Either GDAS1 or GFS files may be used for the ncep_met label. Try to use a GDAS1
file that is within ±3 hours of the SDR observation time. If that file is not available (as is
often the case for real-time processing), use a GFS file instead. The naming convention
for grib1 GFS files is gfs.thh.yymmdd.pgrbfxx (here yymmddd and hh represent
analysis time, and xx represents forecast time step). Thus a file named
gfs.t12.100201.pgrbf03 corresponds to 1500 hours (12+3) UTC on February 1, 2010.
If you have to choose GFS files as input, you should attempt to use a file that is within
±1.5 hours of the SDR file. If there is more than one such GFS file, use the one with
the smaller forecast time step. For example, if your data time is 1500 UTC, you should
try to use the 3 hour forecast field from the 1200 UTC model run, instead of the 9 hour
forecast field from the 0600 UTC run. If no GDAS1 or GFS file is available using the
above logic, use a GDAS1 file that is closest in time but within ±7 days of the granule
time.
2. The AEROSOL_SPA will not produce the VIIRS Aerosol products if there are an
insufficient number of daytime granules in the input swath.
3. The data products generated by this SPA may be visualized with the DRL's H2G_SPA
(Hierarchical Data Format [HDF] to Georeferenced Tagged Image File Format
[GeoTIFF] Converter Science Processing Algorithm). H2G is designed specifically for
Direct Readout applications to create geolocated GeoTIFF images, jpeg browse
images, and png browse images for parameter datasets in SNPP products and EOS
products. H2G_SPA and its User Guide are available for download from the DRL Web
Portal. Please refer to Appendix A for information on enabling image production for this
SPA in IPOPP.

To Use the Scripts in the testscripts Directory

One simple way to run the algorithms from the directory of your choice using your own data
is to copy the run-vaerosol script from the testscripts directory to the selected directory.
Change the values of the variables like WRAPPERHOME, INPUTHOME and OUTPUTHOME
to reflect the file paths of the wrapper directories and the input/output file paths. Then modify
the input/output file name variables. Run the script to process your data.

AEROSOL_SPA Page 7 April 2015

 Appendix A
SPA Services
Installation of this SPA in IPOPP mode will make the SPA services listed in Table A-1 available
to IPOPP. These services along with any other Prerequisite services (listed in Table A-2) will
need to be enabled to allow IPOPP to automate production of the AEROSOL_SPA data
products. Furthermore, users who wish to generate image products from the data products
generated by this SPA will need to enable the image-generating services listed in Table A-3.
The SPAs containing the Prerequisite and the image-generating services listed in Tables A-2
and A-3 can be downloaded from the DRL Web Portal, in case they are not already available
in your IPOPP installation. Details about these other SPAs are available in the respective SPA
User's Guide. Please refer to the IPOPP User's Guide for instructions on how to install an
SPA in IPOPP and enable the corresponding services.

Table A-1. SPA Services

Services for this Data Products produced

SPA

Aerosol Product Name Destination

(when installed in IPOPP)
VIIRS Aerosol Model Index IP /raid/pub/gsfcdata/npp/viirs/level2/IVA
MI_npp_dyyyymmdd_thhmmssS_ehh
mmssS*.h5*
VIIRS Aerosol EDR
/raid/pub/gsfcdata/npp/viirs/level2/VAO
OO_npp_dyyyymmdd_thhmmssS_ehh
mmssS*.h5*
/raid/pub/gsfcdata/npp/viirs/level2/IVA
VIIRS Aerosol Optical Thickness IP OT_npp_dyyyymmdd_thhmmssS_ehh
mmssS*.h5*
VIIRS Aerosol Geolocation /raid/pub/gsfcdata/npp/viirs/level2/GAE
RO_npp_dyyyymmdd_thhmmssS_ehh
mmssS*.h5*
VIIRS Suspended Matter EDR /raid/pub/gsfcdata/npp/viirs/level2/VSU
MO_npp_dyyyymmdd_thhmmssS_ehh
mmssS*.h5*

* Where yyyy, mm, dd, hh represents the year, month and day of month for start of swath; the
first hh, mm, ss, S represents the hour, minutes, seconds and 10th of a second for the start
of swath and the second hh, mm, ss, S represents the end time of the swath.

AEROSOL_SPA Page A-1 April 2015

 Table A-2. Prerequisite Services

Prerequisite SPA services SPA in which they are available

VIIRS_C-SDR C-SDR_SPA
OR
VIIRS-SDR VIRS-SDR_SPA
CloudMask CLOUDMASK_SPA
NOTE: The services VIIRS-SDR and VIIRS_C-SDR must never be run simultaneously.

Table A-3. Image-generating Services

Image-generating services SPA in which they are available

vaoth5d-geotiff H2G_SPA
vapsh5d-geotiff
vsumh5d-geotiff
NOTE: Please refer to the H2G_SPA User's Guide for more details about the image products,
including their locations and filename patterns when they are generated in IPOPP.

AEROSOL_SPA Page A-2 April 2015