1.24 2024-11-11
~~~~~~~~~~~~~~~

* Fixed appending non-float variables with different sub-dimensions when
  using the Python harp.concatenate() function.

* Updates to product ingestions:
  - ESACCI_GHG: fixed incorrect unit for CH4/CO variables
  - GOME-2 L2: added support for O3MARS aerosol layer height and added
    support for O3MARP V5
  - S5P L2: added snow_ice_type/sea_ice_fraction ingestion for AAI/CO/CH4
  - S5P L2 CH4: added 'corrected' option to ingest destriped columns and
    fixed issue in NIR/SWIR selection of surface albedo
  - S5P PAL L2: added support for SRON/SICOR HDO (L2__HDO__S) product

1.23 2024-07-30
~~~~~~~~~~~~~~~

* Fixed numpy 2.x compatibility issues in python interface.

* Added derivations to retrieve dry air mass/volume mixing ratio from H2O
  mass/volume mixing ratio with regard to dry air.

* Fixed some issues with building the HARP R interface extension.

* Updates to product ingestions:
  - S5P PAL L2 AOT: added support for cloud_fraction vs.
    effective_cloud_fraction split

1.22 2024-06-12
~~~~~~~~~~~~~~~

* Updates to product ingestions:
  - EARLINET: fixed memory issue when reading extinction data.
  - EarthCARE: updates for new V11.2 format definition baseline
  - GEOMS UVVIS.DOBSON: fixed issue with reading of int16 typed data.
  - S5P PAL L2 SO2 COBRA: fixed cloud_fraction option behaviour and added
    alternative qa value calculation for processor V02.00.00.

1.21 2024-03-15
~~~~~~~~~~~~~~~

* Fixed incorrect weight division for random uncertainties in spatial binning.

* Added check for unit equality when appending variables.

* Fixed CMake build for locating Python 3.12.

* Updates to product ingestions:
  - EarthCARE: updates for new V11 format definition baseline
  - ECMWF GRIB: added general support for new masterTablesVersion numbers
  - S5P L2 CH4: added support for nir aerosol/cloud/surface property ingestion.
  - S5P L2 CLOUD: added support for nir-band spatial grid data
  - S5P PAL L2: added support for OClO
  - S5P PAL L2 AOT: added support for V02.00.00 output format
  - S5P PAL L2 BRO: fixed memory issue with reading total column and
    uncertainty variables
  - S5P PAL L2 TCWV: fixed wrong quantity for density variables

1.20.2 2023-11-13
~~~~~~~~~~~~~~~~~

* Fill values of floating point types are now explicitly set to NaN when
  exporting to a netcdf or hdf5 file.

* Updates to product ingestions:
  - EARLINET: fixed issue where no backscatter/extinction data was ingested.

1.20.1 2023-10-12
~~~~~~~~~~~~~~~~~

* Updates to product ingestions:
  - IASI L1C: added missing angle variables
  - S5P PAL L2 AER_OT: added missing wavelength variable
  - S5P PAL L2 CHOCHO: fixed missing units for number density variables

1.20 2023-09-19
~~~~~~~~~~~~~~~

* Updates to product ingestions:
  - GEOMS FTIR: added support for C2H3NO5/PAN and C2H4, added support for
    template V3.
  - GEOMS TCCON: added support for template V6.
  - S5P L2 ALH: added support for revised surface_albedo variable and added
    surface_albedo_uncertainty.
  - S5P L2 O3: fixed custom V1 QA filter for NRTI products
  - S5P PAL L2: fixed memory issue when reading aerosol_optical_depth_validity
    of AER_OT product.

* Fixed missing documentation on weight factor handling in binning algorithm.

* Updated embedded expat library to version 2.5.0.

* Fixed issue where jpeg dll was missing from windows binary installer.

1.19 2023-07-24
~~~~~~~~~~~~~~~

* The python harp.import_product() function now allows HARP .pth files to be
  provided as input.

* Reintroduced propagation of *_uncertainty_random variables when performing
  spatial binning with area weighted averaging.

* Allowed regridding using a multi-dimensional source axis-variable as input.
  e.g. convert from an 'altitude {latitude,longitude,vertical}' grid to
  a fixed 'altitude {vertical}` grid.

* Added documentation for regridding/binning algorithms.

* Updates to product ingestions:
  - ACE-FTS: fix ingestion of ASCII formatted products
  - GOME-2 L2: added support for offline absorbing aerosol index (O3MARP)
  - EARLINET: added support for new format (removed support for old format)
  - EarthCARE: added support for AM__ACD_2B, AM__CTH_2B, BM__RAD_2B,
    BMA_FLX_2B, MSI_AOT_2A, MSI_CM__2A, and MSI_COP_2A
  - ECMWF GRIB: added support for new masterTablesVersion=30 data
  - S5P PAL L2: added support for AER_OT, BRO, CHOCHO, SIF, SO2 COBRA, and
    TCWV

1.18 2023-06-08
~~~~~~~~~~~~~~~

* Renamed all uses of 'water vapour' to 'water vapor'.

* Added =| bitfield operater to allow filtering for the case where only some
  bits within the mask are set.

* The uncertainty propagation of *_uncertainty_random variables for the
  temporal binning operation now always assumes no correlation. For spatial
  binning the random component is removed for the area weighted average
  approach and it is propagated using no correlation for the simple binning
  based on center points.

* Added derivation for 'area {latitude,longitude}' and fixed area calculation
  where South Pole was an edge point.

* Fixed reading of string arrays in HARP Python interface.

* Updates to product ingestions:
  - ECMWF GRIB: added support for new masterTablesVersion=29 data
  - S5P L2 CLOUD: fixed qa_value ingestion for CRB (now use qa_value_crb)
  - S5P L2 SO2: added support for SO2 layer height
  - OMI L3: added support for OMHCHOd, OMNO2d, and OMSO2e

1.17 2022-12-12
~~~~~~~~~~~~~~~

* Added additional conventions for variable names related to areosol/cloud
  properties.

* Fixed issue with geographical area overlap calculation and improvided
  performance of performing operations on geographical polygon data.

* Windows .msi installer now allows HARP component selection for core, Python,
  IDL, MATLAB, and R.

* Updates to product ingestions:
  - AEOLUS L2B: properly split datetime, datetime_start, and datetime_stop
  - CCI L3 O3 LP: fixed wrong variable name
  - EarthCARE: added additional product types and updated ACM_CAP_2B
  - ECMWF GRIB: changed RWC/SWC/LWC/IWC variable names
  - OSIRIS L2: fixed wrong variable names
  - TES TL2ATMTN/TL2ATMTL: fixed wrong variable names

1.16 2022-10-31
~~~~~~~~~~~~~~~

* Improved algorithm to calculate center point from latitude/longitude bounds
  polygon.

* Added derivation of time dependent from time indepent for datetime variables.

* Updates to product ingestions:
  - GEOMS PANDORA: added systematic/random uncertainty for effective
    temperature, renamed H2CO in harp variable names to HCHO.
  - OCO: added support for ACOS/OCO-2/OCO-3 L2/L3 products.
  - Ozone CCI L3 LP: fixed memory issue when reading
    merged_ozone_concentration.
  - S5P L2 HCHO: added amf=clear_sky ingestion option

1.15.1 2022-05-18
~~~~~~~~~~~~~~~~~

* Updates to product ingestions:
  - CLOUDNET L2 classification: fixed fillvalue mapping of values to NaN.
  - GEOMS PANDORA: fixes for template V003 support
  - GEOMS FTIR: allow instrument lat/lon/alt variables to be one-dimensional
    with replicated values.
  - GEOMS UVVIS.BREWER: added random+systematic uncertainty variables.
  - S5P L2 CO: fixed vertical ordering of new apriori profile.

1.15 2022-03-14
~~~~~~~~~~~~~~~

* Fixed issue where bin() operation was removing scalar enumeration variables.

* Updates to product ingestions:
  - ECMWF GRIB: fixed parameter mapping for CAMS model data that uses
    masterTablesVersion=27 data and added support for new
    masterTablesVersion=28 data
  - GEOMS UVVIS.BREWER: added support for template V001
  - GEOMS PANDORA: added support for template V003
  - S5P L2 AAI: added new 335/367 wavelength pair
  - S5P L2 CO: added new apriori profile and support for new AVK unit
  - S5P L2 O3 Profile: added surface_albedo and cloud_albedo
  - S5P L2 HCHO/O3/SO2: added support for updated NRTI indication attribute.

1.14 2021-10-20
~~~~~~~~~~~~~~~

* Updated embedded udunits2 library to version 2.2.28.

* Improved CMake build system.

* Fixed issue when building against HDF5 1.12.

* Updates to product ingestions:
  - AEOLUS L1B: fixed ingestion of Mie/measurement data; added sensor angles
  - AEOLUS L2B: fixed ingestion of Mie data; added sensor angles
  - CLOUDNET L2 classification: added support for new (netcdf4) format
  - EARTHCARE: added very early support (limited version of ECA_ACM_CAP_2B)
  - ECMWF GRIB: fixed unit of backscatter coefficient 1/(milli sr) -> 1/(m sr),
    added support for new masterTablesVersion=27 data
  - GEOMS UVVIS.DOAS: allow template V007 files to be stored in hdf5 format
  - GHG CCI: added support for TROPOMI WFMD L2
  - OMI L2: added sensor alt/lat/lon and surface altitude/pressure
  - OMI L2 OMAERUV: fix ingestion issues
  - QA4ECV HCHO: added amf=clear_sky ingestion option
  - S5P L2 HCHO: renamed cloud_altitude variable to cloud_height
  - S5P L2 NO2: added retrieval of O2-O2 cloud (O22CLD) data
  - S5P L2 NO2+FRESCO: added scene_albedo and scene_pressure
  - S5P L2 O3 Profile: fixed wrong concentration variables (vmr -> nd)
  - S5P L2 SO2: updated SO2_type enumeration list

1.13 2021-03-12
~~~~~~~~~~~~~~~

* Added rebin() operation.

* Added harp_product_rebin_with_axis_bounds_variable() function.

* Added support for datetime_bounds variable.

* Fixed issue in =& operation (it returned true if any of the bits were 1
  instead of all bits).

* Updates to product ingestions:
  - GEOMS: renamed all site_name variables to location_name
  - GEOMS FTIR H2CO: fixed issue in determining unit for H2O data (that would
    fail the ingestion)
  - S5P L2 CO: added co=corrected ingestion option
  - SCIAMACHY L2: fixed averaging of cloud fraction for co-added pixels

1.12 2020-10-29
~~~~~~~~~~~~~~~

* The csv output of 'harpdump --dataset' can now be used as content of a .pth
  file. This improves the performance of harpcollocate/harpmerge/etc. as they
  no longer need to open and extract metadata from files themselves anymore.

* harpcollocate now stores differences as signed values in collocation result
  file (instead of absolute differences).

* Added harp_dataset_prefilter(). This function is now also called within
  harpmerge and harpcollocate to pre-filter the input dataset(s) for datetime
  and collocate left/right filters that are at the start of the operations
  string.

* Added format and history fields to harp_product_metadata.

* Added harp.import_product_metadata() to Python interface.

* harp_dataset_add_product() now ignores files/directories starting with '.'.

* Added uncertainty propagation to bin() and bin_spatial() operations
  (using uncorrelated assumption for bin() and correlated for bin_spatial()).

* Added global option to determine whether uncertainty propagation should
  use a correlated or uncorrelated approach. Currently only used for the bin()
  operation.

* Added global option to create a collocation_datetime variable when performing
  a collocate_left() or collocate_right() operation.

* Added wildcard pattern matching ('*'/'?') to keep() and exclude() operations.

* Added derivation to retrieve angstrom exponent from aerosol optical depth.

* Added derivation to retrieve tropospheric/stratospheric column avk from
  total column avk and tropopause altitude.

* Fixed potential memory issues and improved overall code stability.

* Updates to product ingestions:
  - S5P L2 CLOUD: added cloud_type and cloud_top_temperature
  - S5P L2 O3: added qa_filter=custom ingestion option to set validity based
    on official PRF recommendation for processor V1.x generated products
  - MLS L2: fixed memory issue in reading of validity variable and updated
    validity flag mapping to version 4.x-4.0 of the Data Quality document

1.11 2020-05-19
~~~~~~~~~~~~~~~

* Added derivations to retrieve column number density from column vmr.

* Fixed issue in bin_spatial() where the weightfactor could be too high when
  oversampling the input data. Grid cells that intersected with the data
  polygon could end up with a weightfactor >1.0 (but <2.0).

* Fixed issue where molar mass was incorrectly provided for the species with
  the shortest matching name. e.g. the molar mass of CO was returned for CO2
  and of NO for NO2. This impacts vmr <-> mmr conversions for BrO2, COF2, COS,
  CO2, C2H2O2, H2O2, NOCl, NO2, N2O, N2O5, and all isotopologues (e.g. H2O_161).

* Removed irrelevant harp_str64() and harp_str64u() functions (HARP does not
  support 64bit integers).

* Fixed issue where molar mass linked to variables with a 'surface',
  'tropospheric', or 'stratospheric' prefix returned 0.

* Fixed error/crash in deriving dry air column vmr from dry air column mmr.

* Updates to product ingestions:
  - GOME-2 L2: added support for SO2 columns at all plume heights
  - OMI L2 CLDO2/CLDRR: added validity variable

1.10 2020-03-31
~~~~~~~~~~~~~~~

* Added support for -ar/--reduce-operations parameter in harpmerge and
  for reduce_operations parameter in harp.import_product() Python function.

* Fixed incorrect "invalid variable" error on weight variables for angular
  quantities when performing a harpmerge.

* Fixed issue where the udunits2 library could not be initialized if the
  environment had a LC_NUMERIC locale that did not have '.' as decimal
  separator.

* Fixed issue in name mangling of embedded expat library.

* Updates to product ingestions:
  - GOMOS L1 Limb: fixed unit of wavelength_photon_radiance (and uncertainty)
    and documented conversion algorithm
  - S5P L2 O3 PR: support for IODD 12.0.0
  - S5P L2 O3 TPR: removed support (future availability of product uncertain)
  - GEOMS PANDORA: allow for trailing \0 in global attribute values

1.9.2 2019-12-01
~~~~~~~~~~~~~~~~

* Fixed issue where using the 'bin()' operation did not correctly create a
  '<variable>_weight' variable from a 'weight' variable when it was missing.

* Added default units to documention on variable name conventions.

* Updates to product ingestions:
  - Calipso L2: fixed unit of backscatter coefficient
  - ERS GOME L2: fixed unit of cloud top height
  - S5P L2 AER_LH: fixed unit of aerosol optical depth
  - S5P L2 SO2: add scaled avk for 1km/7km/15km box profiles

1.9.1 2019-11-01
~~~~~~~~~~~~~~~~

* Fixed issue where results from deriving partial column densities from vmr and
  pressure were a factor 10 too low.

* Fixed issue where operation following an index filter operation did not
  get executed.

1.9 2019-10-21
~~~~~~~~~~~~~~

* Added operations to filter on dimension indices (e.g. 'index(vertical) < 5').

* Changed bin() and bin_spatial() operations to use/produce separate 'count'
  and 'weight' variables. The counts/weights are used to allow the following
  commands to now be equivalent:
    harpmerge -a 'bin()' -ap 'bin()' a.nc b.nc merged.nc
    harpmerge -ap 'bin()' a.nc b.nc merged.nc
  As well as the following:
    harpmerge -a 'bin_spatial(...)'
      -ap 'squash(time, (latitude_bounds, longitude_bounds));bin()'
      a.nc b.nc merged.nc
    harpmerge -ap 'bin_spatial(...)' a.nc b.nc merged.nc

* Fixed issue with invalid bounding box optimization for spherical area
  filter operations.

* Fixed missing error handling when using invalid operations in
  harp.export_product() in Python.

* Fixed Python3 compatibility issue when exporting string data.

* Updates to ECMWF GRIB, IASI L1/L2, GOME-2 L1/L2, MIPAS L1, and S5P L2
  product ingestions.

1.8 2019-07-31
~~~~~~~~~~~~~~

* Added R interface.

* Improved derivation for tropopause altitude/pressure (to not return NaN).

* Added derivations for gravity.

* Added derivations to convert between total air and dry air mixing ratios.

* Fixed valid() operation to properly use valid_min/valid_max attribute values.

* Fixed issue in adding *_count variable in append/merge when count variable
  had more than one dimension.

* Updates to ECMWF GRIB, GEOMS MWR, and S5P L2 product ingestions.

1.7 2019-06-06
~~~~~~~~~~~~~~

* Added --no-history option to harpconvert, hardump, and harpmerge.

* Added harp.execute_operations() and harp.convert_unit() to Python interface.

* Added optional min_collocation_index/max_collocation_index arguments to
  collocate_left()/collocate_right() operations.

* harp_product_set_history() and harp_product_set_source_product() now accept
  NULL as argument to clear the current value.

* Appending/merging products now automatically adds missing *_count variables.

* Fixed potential crash during initialization of HARP ingestion module.

* Fixed invalid results when using bin_spatial with latitude_bounds {time,2}
  and longitude_bounds {time,2} variables (i.e. range instead of polygon).

* Fixed array ordering in MATLAB interface. Multi-dimensional arrays are now
  represented from fastest running to slowest (=fortran ordering) as intended.

* Update to S5P L2 product ingestion.

1.6 2019-04-29
~~~~~~~~~~~~~~

* Added clamp() and squash() operations.

* Allow bin() and sort() operations to be performed using a set of variables.

* harp_product_sort() now takes a list of variables.

* The rename() operation no longer fails if the source variable does not exist
  but a variable with the target name already exists.

* The python harp.import_product() function now uses the C library to perform
  the contactenation of products.

* Added post_operations argument to python harp.import_product() function.

* Besides using a polygon of >=3 vertices, a geospatial area can now also be
  defined via a bounding rect. Use e.g. 'latitude_bounds {time,2}' and
  'longitude_bounds {time,2}' to define the min/max of the bounding rect.

* Added derivation for tropospheric/stratospheric column from partial column
  profile and tropopause.

* Corrected derivation for calculating tropopause altitude/pressure.

* Deriving a variable will now use the conversion that requires the minimum
  amount of intermediate variables to be calculated.

* Added support for scene albedo/pressure/type variables.

* Fixed issue where regridding in the vertical dimension using pressure as axis
  could end up with wrong pressure_bounds values.

* Fixed problem in matlab mex extension determination on linux when using the
  configure script.

* Fixed setting of default CODA definition path and udunits2 xml path for
  Python on Windows.

* Added format, variable count, and dimension sizes to harpcheck output.

* Added support for TCCON products in GEOMS format.

* Updates to AEOLUS L2A, ESACCI_OZONE_L2_TC, GEOMS FTIR, GOME2 L2, OMI L2 NO2,
  QA4ECV, and S5P L2 product ingestions.

* Updated embedded expat library to version 2.2.6.

1.5 2018-12-17
~~~~~~~~~~~~~~

* Fixed issue where vertical smoothing/regridding provided invalid results if
  input variables were available as float instead of double data.

* harp_product_get_derived_variable() no longer converts data to double if
  no data type was provided and data already had the right unit.

* Fixed issue where listing derivations for a specific variable would not show
  all possible conversions.

* Add derivation for cloud height from cloud base/top height.

* Added support for Dobson, Pandora and Sonde products in GEOMS format.

* Updates to ACE-FTS, GOME/GOME-2 L2, and S5P L2 product ingestions.

1.4 2018-09-28
~~~~~~~~~~~~~~

* Added -t/--target option to harpdump --list-derivations.

* Added variable_name parameter to harp_doc_list_conversions().

* Added support for meridional/zonal wind speed, and planetary boundary layer
  height variables.

* Added support for sic (Shannon information content) variables for vertical
  profiles.

* Aerosol optical depth and extinction coefficient variables will now
  automatically use a log/log regridding in the spectral dimension.
  This is equivalent to using the Angstrom Exponent approach.

* Fixed interpolation issue where points beyond the source grid could end up
  having an arbitrary wrong value (instead of NaN). This is a regression of
  harp 1.3.

* Add derivations between wind speed/direction and zonal/meridional velocity.

* Flattening a dimension now special cases a dimension length 1 (with less
  modifications to variables/product).

* Fixed issue where a point distance of 0 could sometimes result in NaN.

* Updates to S5P L2, GOME/GOME-2 L2, GEOMS UVVIS-DOAS-OFFAXIS, ECMWF GRIB, and
  AERONET product ingestions.

1.3 2018-07-05
~~~~~~~~~~~~~~

* Added bin_spatial() operation.

* Improved binning operation to automatically filter for NaN values and add
  variable-specific count variables to specify the number of non-NaN values.

* Performing a binning operation on top of a binning operation will propagate
  averages based on the available count variables (i.e. as if the final bin was
  performed on the original data).

* Entries added to the global 'history' attribute now also include the current
  utc time and harp version before the execution command.

* Allow NULL as argument for harp_variable_set_description() and
  harp_variable_set_unit() to clear the current description/unit.

* Added harp_product_bin_spatial() and harp_variable_copy_attributes()
  functions.

* Added optional parameter to regrid() operation to specify the axis boundaries.

* Python harp.import_product() now ignores empty products when ingesting a list
  of products.

* Added derivations:
  - column vmr/mmr from column density ratios
  - total column mass density from surface pressure and pressure profile
  - mass mixing ratio from mass density
  - between dry air and total air mass (column) density
  - between dry air, total air, and H2O number densities

* Fixed derivation of altitude from gph/latitude.

* Updates to S5P L2 and MIPAS L2 product ingestions.

1.2 2018-04-09
~~~~~~~~~~~~~~

* Allow data compression for HDF5 output:
  - Added harp_set_option_hdf5_compression() and
    harp_get_option_hdf5_compression() functions to C library.
  - Added --hdf5-compression option to harpconvert and harpmerge.
  - Added hdf5_compression parameter to harp.export_product() Python function.

* Allow regrid() operation to interpolate in the time dimension.

* Improved binning of angle values (average of 1 and 359 [deg] is now 0).

* Added bin() operation variant without parameters.

* The bin() operation using a collocation result file now retains the
  collocation_index variable (with the first value for each bin).

* Changed smooth() operation such that each vertical level where the source
  grid point was NaN the target grid point also remains NaN.

* Added -ap option to harpmerge.

* Added --dataset option to harpdump and changed output format of
  harp_dataset_print() and harp_product_metadata_print() to csv format.

* Added 'operations' parameter to Python harp.export_product() function.

* Importing/exporting a product with operations in Python now automatically
  updates the history attribute.

* Fixed issue where harpmerge did not introduce a time dimension for all
  variables.

* Fixed issue where datetime_start and datetime_stop global attributes were not
  created when exporting in HDF4 or HDF5 format.

* String attributes in HDF5 files can now be empty strings (using H5S_NULL).

* Allow HARP to import HDF5 files that are generated by the netCDF4 library
  (allow numeric attributes to be 1-element arrays instead of scalars).

* harpcheck now returns with exit code 1 if any of the imports fail.

* harpcollocate --update now only allows one dataset as argument.

* Fixed performance issue with spatial filtering on systems with slow math lib.

* Fixed mapping description of GOME2 L1 product ingestion.

* Fixed wrong results of area overlap calculation.

* Replaced '_' by '-' in long versions of argument names of harpcollocate.

* Added support for dfs (degrees of freedom for signal) profiles.

* Updates to S5P L2 product ingestions.

1.1 2017-11-06
~~~~~~~~~~~~~~

* Parameters that required a file or a directory as dataset reference can now
  also take a file with a .pth extension as input. This file should contain a
  line-separated list of paths to files and/or directories of the dataset.

* Specifying a directory for a dataset will now recursively add all products
  (including sub-directories and contents of .pth files).

* The variable that is binned on with a 'bin(variable)' operation is now
  always kept in the resulting product.

* The bin() operation can now handle 'count' variables (summing it if present
  and creating it otherwise).

* Fixed issue where collocate filter operation on a product with a single
  sample could end in an infinite loop.

* Fixed issue in harpcollocate when providing a -nx option without a -ny.

* Fixed issue with wrap() operation for values < specified minimum.

* Fixed issue with collocate_left/collocate_right operations not including
  last sample when applying it on products in an external format.

* harpcollocate now ignores empty products instead of raising errors for them.

* Added -oa/-ob and -aa/-ab options to harpcollocate matchup.

* Added solar_declination_angle and solar_hour_angle variables and derivations.

* Added orbit_index variable and added this to various product ingestions.

* Fixed derivations of solar azimuth/zenith angles from datetime and lat/lon.

* Added derivations for averaging kernel conversions.

* Added derivations for tropopause altitude/pressure.

* Added NH3 (amonia) as supported species.

* Updates to ECMWF-GRIB, GEOMS-TE-UVVIS-DOAS-OFFAXIS, GEOMS-TE-FTIR, and
  S5P L2 product ingestions (and several others for the new orbit_index).

1.0 2017-09-25
~~~~~~~~~~~~~~

* The import and ingest functions have been combined into a single import
  function that can both read HARP compliant products as well as ingest from
  the supported external formats. This means:
  - harp_import() and harp_ingest() are combined into a single harp_import()
  - harp_ingest_test() has been renamed to harp_import_test()
  - harpfilter has been removed (use harpconvert instead)
  - '--test' option of harpconvert has been removed (use harpcheck instead)

* harpcollocate has been completely rewritten.
  - any 1D time-dependent variable can now be a collocation criterium
  - nearest neighbour filtering with weighted average is no longer supported
  - collocation result files are not backward compatible due to a change in
    the CSV header
  - arguments to harp_collocation_result_new() and
    harp_collocation_result_add_pair() have changed

* Added support for enumeration variables. Enumeration variables have a int8,
  int16, or int32 type and provide an index into a fixed list of enum names.
  Enumeration variables can be treated as string values in operations.

* Added MATLAB and IDL interfaces for HARP.

* Added -o option to harpdump and harpmerge.

* Added -a and -o options for 'harpdump --list-derivations'.

* '--list-derivations' option of harpconvert has been removed (use harpdump
  instead).

* Added -l option to harpmerge.

* Renamed area/point operations:
  - area_mask_covers_area -> area_covers_area
  - area_mask_covers_point -> point_in_area
  - area_mask_intersects_area -> area_intersects_area
  - point_in_area -> area_covers_point
  The first part of the operation name now reflects the item in the
  product and the second part reflects the parameter that is provided.

* Improved performance of point/area vs. area overlap operations by first
  comparing against the bounding box of a polygon.

* Added sort() and area_inside_area() operations.

* The derive() operation can now be used without dimension specification
  to perform a unit and/or data type conversion on an existing variable.

* The derive_smoothed_column(), regrid() and smooth() operations can now
  use a single merged collocated product (instead of a combination of a csv
  collocation result file and a dataset directory of collocated products).

* An area parameter for operations can now either be a reference to an
  area mask file or a single polygon provided as lat/lon array parameters.

* Added derivation to derive the area (size) of latitude/longitude bounds.

* Added derivation to derive degree of freedom for signal (DFS) from 2D
  averaging kernels (AVK).

* Derivations of bound variables will now clamp to the edge midpoints when
  the 'regrid_out_of_bounds' option is set to 'edge'.

* The 'data_type' parameter for harp_product_get_derived_variable() and
  harp_product_add_derived_variable() is now a pointer to allow it to be
  optional (i.e. pass NULL to use the default data type).

* Added harp_geometry_get_area(),
  harp_product_get_smoothed_column_using_collocated_product(),
  harp_product_smooth_vertical_with_collocated_product(),
  harp_product_sort(), harp_product_regrid_with_collocated_product(),
  harp_set_udunits2_xml_path(), harp_set_udunits2_xml_path_conditional(),
  and harp_variable_set_enumeration_values() functions.

* harp_import_product_metadata() and harp_dataset_import() now have an
  'options' parameter.

* Changed 'percentage' parameter of harp_geometry_has_area_overlap() to
  'fraction'.

* harp_product_set_source_product() now has a product path as parameter
  and stores the base name of this as source_product value.

* Added HARP_UNIT_AREA default unit.

* The C interface for collocation results was modified to support a
  dynamic set of collocation criteria differences (based on variable names
  for these differences).

* The Python interface now sets the udunits2 xml path dynamically,
  relative to the location of the HARP C library.

* wrap() operation now properly updates valid_min/valid_max properties.

* codadef files are now installed in <prefix>/share/coda/definitions.
  This allows a present CODA installation to use the same codadef files.

* Windows installation now installs data and documentation files in the
  subdirectory <prefix>/share/harp.

* Fix issue in deriving total column from partial column profile if partial
  column profile did not have {time,vertical} dimensions.

* Fix invalid results from binning operation/function.

* Improve speed of --list-derivations on a product.

* Split data format documentation into separate sections under a new
  heading 'conventions'.

* Added product ingestions for Cloudnet L2a classification products.

* Updates to ECMWF-GRIB, GEOMS-TE-UVVIS-DOAS-OFFAXIS, MIPAS, QA4ECV, and
  S5P product ingestions.

0.6 2017-04-23
~~~~~~~~~~~~~~

* Added ingestion support for ACE-FTS, CALIPSO, Cloud CCI, and NPP-Suomi.

* Added bin() and set() operations.

* Added optional data_type parameter to derive() operation.

* Major refactoring of internal implementation of operations (greatly
  reducing and simplifying the code).

* Fix application of apriori for smoothed column derivation (for vertical
  range where partial columns are NaN, don't subtract apriori*AVK, but just
  set to apriori).

* Fix memory issue in partial column from vmr dry air derivation.

* Remove double definition of radiance conversions.

* Added global HARP option 'regrid_out_of_bounds'.

* Added harp_product_bin() function.

* Add data_type parameter to harp_product_add_derived_variable() and
  harp_product_get_derived_variable() functions.

* Added data_type parameter to harp_product_get_derived_variable() and
  harp_product_add_derived_variable() functions.

* Allow 'Conventions' global attribute to contain more than just a reference
  to HARP.

* Renamed O3MSAF codadef to ACSAF. When updating HARP, you should remove
  any existing O3SMAF-*.codadef files from your installation directory.
  Otherwise you will get 'detection rule for ... is shadowed by ... ' errors.

* Fixed issue with TES TL2CH4L ingestion.

* Updates to Aeolus, GEOMS, S5P, QA4ECV, and Ozone CCI product ingestions.

0.5 2017-03-26
~~~~~~~~~~~~~~

* Changed Open Source licence of HARP from GPLv2 to BSD (3-Clause).

* Changed all '-' to '_' in operation function names to make them identifiers.

* Added harpmerge command line tool.

* Added regrid(), rename(), smooth(), derive_smoothed_column(),
  point_in_area(), and wrap() operations.

* Removed the harpprofile command line tool. Its functionality is replaced by
  the regrid() and smooth() operations.

* Renamed harp_product_regrid_vertical_with_collocated_dataset() to
  harp_product_regrid_with_collocated_dataset() and
  harp_product_smooth_vertical() to
  harp_product_smooth_vertical_with_collocated_dataset()

* Added vertical_unit parameter to harp_product_smooth_vertical() and
  harp_product_regrid_with_collocated_dataset() functions.

* Replaced harp_product_regrid_vertical_with_axis_variable() with
  harp_product_regrid_with_axis_variable(). It can now be used for other
  dimensions too (except the time dimension).

* Added harp_collocation_result_filter_for_collocation_indices(),
  harp_product_get_smoothed_column(),
  harp_product_get_smoothed_column_using_collocated_dataset(),
  harp_product_remove_variable_by_name(), harp_variable_append(),
  harp_product_append(), and harp_variable_smooth_vertical() functions.

* Renamed harp_product_get_variable_id_by_name() to
  harp_product_get_variable_index_by_name().

* Python harp.import_product() and harp.ingest_product() functions now accept
  a list of filenames or a globbing pattern as parameter (which will return
  the result of harp.concatenate() on the imported/ingested products).

* Automatically set CODA definition path for HARP Python interface
  (path can be overruled with CODA_DEFINITION environment variable).

* The parsing of the operations string has been rewritten using flex/yacc.
  This has fixed several issues related to the parsing and execution of
  operations.

* Fixed issue with flatten() operation providing wrong results.

* Fixed issue where vertical regridding to a common grid provided wrong
  results.

* Fixed issue where interval interpolation for a pressure grid was not
  using a log-linear interpolation.

* Aligned order of latitude/longitude arguments in all functions to have
  latitude first and longitude second.

* Added unit 'ppv' to the included udunits2-common.xml file.

* Added variables for surface temperature, total water vapour content, and
  total cloud cover to ECMWF GRIB ingestion.

* The default internal units have been changes to SI (except for molar mass).
  All algorithms have been adapted accordingly.

* Added variable conversions for: apriori profiles, column density from
  mass density (including pm1/pm2.5/pm10).

* Added support for AEOLUS L1b/L2a/L2b products.

* Added support for Aerosol and GHG CCI L2/L3 products.

* Added support for GEOMS FTIR C2H2 ingestion.

* Added support for OMI OMCHOCHO ingestion.

* Added support for additional TES L2 products.

* Added support for QA4ECV NO2 and HCHO products.

* Added support for GOME, GOME-2, GOMOS, MIPAS, IASI, SCIAMACHY L1 products.

* Added support for ODIN OSIRIS and SMR L2 products.

* Changed --list-conversions command line option to --list-derivations.
  And added --list-derivations option to harpdump.

* Use NC_64BIT_OFFSET when storing data to netcdf larger than 1GB.

0.4 2016-10-16
~~~~~~~~~~~~~~

* Fixed wrong scaling factor in conversions related to the ideal gas law
  (10^2 vs. 10^-2) and barometric formula (10^-9 vs. 10^-6).

* Fixed error in algorithm to derive gravity from latitude and height.

* Fixed invalid sorting of collocation result file (which resulted in wrong
  results when performing a collocation resampling).

* Fixed wrong independent dimension length for altitude/pressure bounds for
  SCIAMACHY Level2 ingestion.

* Added apriori number density variables to SCIAMACHY Level2 ingestion.

* Added variable conversion from latitude/longitude per vertical profile point
  to single (mid-point) latitude/longitude for profile.

* Variable conversions for altitude and pressure bounds now also work when
  they are dependend on latitude and longitude dimensions.

* Fixed crash in derived variable conversion for volume mixing ratio from
  mass mixing ratio (for total air).

* Changed dimension argument of 'flatten()' operation from string to
  identifier.

* Fixed issue where -inf/+inf valid_min/valid_max attributes could not be
  exported as float values in the Python harp.export_product() interface.

* When deriving latitude/longitude bounds from mid points, latitude is now
  clamped to [-90,90] and longitude is wrapped to [-180,180].

* Fixed volume mixing ratio unit for GEOMS-TE-UVVIS-DOAS-OFFAXIS ingestion.

0.3 2016-09-30
~~~~~~~~~~~~~~

* The changes between 0.2 and 0.3 are extensive. The list below provides the
  most important aspects, but there are many more smaller changes that are not
  listed.

* HARP 'actions' are now called 'operations' and are now executed in a strict
  sequence. This means that you can now do things like:
    'derive(var); var < 10; exclude(var)'

* Added harpprofile command line tool that allows regridding and smoothing
  of vertical gridded data. 
  Also added harp_product_regrid_vertical_with_axis_variable(),
  harp_product_regrid_vertical_with_collocated_dataset(), and
  harp_product_smooth_vertical() to C interface.

* Added algorithm descriptions for the derived variable conversions to the
  HARP documentation.

* Introduced a more strict definition of variable names for the HARP data
  format. As part of this the list of supported species and associated naming
  has also been revised.

* Introduced strict separation between dry air and total/wet air. The default
  air density in HARP is now the total/wet air. Anything related to dry air
  should now be annotated as such in the variable name. As a result, 'dry_air'
  is now introduced as a species, and for mixing ratios the '_wet' postfix is
  removed and the '_dry_air' postfix is introduced.
  Derived variable conversions have been added for dry/wet conversion and for
  retrieving the needed 'molar mass' of total air.

* Added functionality to deal with HARP Datasets and Product Metadata to the
  public C library interface. The collocation result interface in the C
  libraryinterface has also been reworked to now work with HARP Datasets.

* HARP_ERROR_ACTION and HARP_ERROR_ACTION_SYNTAX have been renamed to
  HARP_ERROR_OPERATION and HARP_ERROR_OPERATION_SYNTAX.
  harp_product_execute_actions() has been renamed to
  harp_product_execute_operations().

* Added operation to flatten data in a certain dimension and added
  harp_product_flatten_dimension().

* Automatic detection of a product type when ingesting products now happens
  using CODA (using included codadef files with detection rules) for all
  products.

* harp_import_global_attributes() has been removed (its functionality is
  now captured by the new Product Metadata interface).

* Added public functions to allow raising/capturing of warnings in HARP:
  harp_report_warning(), harp_get_warning_handler(), harp_set_warning_handler()

* Added public functions to print product/variable structure and content:
  harp_product_print(), harp_variable_print(), harp_variable_print_data()

* Fixed issue in the HARP Python interface where using the value 0 for
  valid_min or valid_max did not work.

* Added concatenate() function to HARP Python interface.

* The 'include()' action/operation has been renamed to 'keep()'.

* The derived variable conversions are now properly split according to the
  dimension types of a variable. This also solves a problem where certain
  valid conversions where not included in a '--list-conversions' print.

* The derived variable conversions that produce uncertainty and covariance
  variables filled with NaNs from scratch have been removed.

* Added derived variable conversions for gridded data (i.e. variables that
  depend on the latitude and longitude dimensions).

* Removed variables flag_am_pm and flag_day_twilight_night and the conversions
  for flag_am_pm, flag_day_twilight_night, and flag_daytime. These conditions
  can all be determined by directly comparing against the solar angles.

* Added support for ingestion of GOMOS L2, SCIAMACHY L2, ECMF GRIB and TEMIS
  assimilated ozone data.

* Added 'species' ingestion option to the MIPAS L2 ingestion.

* Added support for MLS CH3Cl, CH3CN, and CH3OH products.

* Fixed issue in GEOMS ingestion where long variable names were truncated.

* Introduced convention that dimensionless quantities should have an empty
  'units' attribute string value. Only non-quantities should omit the 'units'
  attribute.

* Resolved incorrect correction for leap seconds in product ingestions.

* Covariance variables have been renamed from '*_cov' to '*_covariance'.

* The split of random/systematic for covariances has been removed.
  For ingested products that had such covariances, the random covariances are
  now retrieved as plain covariances and from systematic covariances only the
  trace is retrieved and provided as a systematic uncertainty.

* Added 'geopotential' as a supported quantity with conversions from/to
  'geopotential height'. Added HARP_UNIT_GEOPOTENTIAL.

* Renamed cloud optical thickness to cloud optical depth.

* Added '-a/--operations' option to harpdump.

* Support for HDF4 and HDF5 (for import/export) is now optional when building
  HARP, but still enabled by default.

0.2 2016-06-30
~~~~~~~~~~~~~~

* Added Python interface.

* CODA is no longer included as part of HARP, but is treated as a separate
  package, just like HDF4 and HDF5.

* The CMake build system can now be used to build HARP on Windows and create
  a binary installer (using WIX).

* Revised and renumbered error codes in C interface.
  Renamed HARP_ERROR_SCRIPT and HARP_ERROR_SCRIPT_SYNTAX error codes to
  HARP_ERROR_ACTION and HARP_ERROR_ACTION_SYNTAX. Changed HARP_ERROR_PRODUCT
  to either HARP_ERROR_INGESTION or HARP_ERROR_IMPORT depending on context.
  Added HARP_ERROR_EXPORT, HARP_ERROR_INVALID_PRODUCT and
  HARP_ERROR_INVALID_VARIABLE.

* The HARP command line tools now link against the dynamic HARP library
  instead of the static version.

* Updated embedded expat library to version 2.1.1.

* Updated embedded udunits2 library to version 2.2.20.
  Also removed the HARP specific udunits XML configuration file since all
  HARP units are now included in udunits2.

* Renamed all 'stdev' variables to 'uncertainty'.

* Fixed parsing of dimension names; string dimension could be interpreted as
  independent dimension.

* Improved the efficiency of the conversion from volume mixing ratio to number
  density and vice versa.

* Update of S5P L2 ingestion module to the lastest version of the product
  format specification (S5P-KNMI-L2-0009-SD and relevant DLR PUMs).

* Update of S5P L1B ingestion module to the lastest version of the product
  format specification (S5P-KNMI-L01B-0012-SD).

* Added ingestion modules for the following product types:
    - Ozone CCI L2 limb profile
    - Ozone CCI L3 limb profile
    - Ozone CCI L2 nadir profile
    - Ozone CCI L3 nadir profile
    - Ozone CCI L4 nadir profile
    - Ozone CCI L2 total column
    - Ozone CCI L3 total column

* Added public functions harp_get_fill_value_for_type(),
  harp_get_valid_min_for_type(), harp_get_valid_max_for_type(),
  harp_is_fill_value_for_type(), harp_is_valid_min_for_type(),
  harp_is_valid_max_for_type(), harp_ingest_test(),
  harp_doc_list_conversions(), harp_doc_export_ingestion_definitions(),
  harp_geometry_get_point_distance(), harp_geometry_has_point_in_area(),
  harp_geometry_has_area_overlap(), harp_basename(),
  harp_product_execute_actions(), harp_variable_set_description(),
  harp_variable_set_unit(), harp_variable_set_string_data_element(),
  harp_variable_verify(), harp_product_set_source_product(),
  harp_product_set_history(), harp_product_verify()

* Added public functions for dealing with HARP collocations.

* Moved defines for default harp units to harp.h.

* Added operators =& and !& to the product actions to test integer variables
  against a bit mask, where a =& mask is equivalent to (a & mask) != 0 and
  a !& mask is equivalent to (a & mask) == 0.

* Added functions to verify the internal consistency as well as compliance to
  HARP conventions of harp_product and harp_variable instances. Any instance
  of harp_product returned by harp_import() or harp_ingest() has been verified
  using these functions.

* Added derived variable conversion from 'latitude {latitude}' (mid-points) to
  'latitude_bounds {latitude, independent}' (bounds); similar for longitude.

* Updated HDF5 data format such that HDF5 products written by HARP are
  compatible with netCDF-4. HDF5 dimension scales are used to represent shared
  dimensions. The "dims" dataset attribute has been deprecated.

* Changed netCDF-3/HDF4/HDF5 backends to be more explicit about implicitly
  created dimensions (which are used to store scalars and strings for formats
  that do not support this natively).

* Fixed handling of scalars and empty strings in netCDF-3/HDF4/HDF5 backends.

* Fixed unit attribute name in HARP products ('unit' -> 'units').

* Changed all constants to be in line with CODATA 2014 as published by NIST.

0.1 2015-11-09
~~~~~~~~~~~~~~

* First beta release (using CODA 2.15).
