DEA Surface Reflectance Median Absolute Deviation (Landsat)

Surface Reflectance Euclidean, Spectral and Bray-Curtis Median Absolute Deviation 2.1.0

Version
2.1.0 flat-square
Product ID
ls5_nbart_tmad_annual, ls7_nbart_tmad_annual, ls8_nbart_tmad_annual
Program
Digital Earth Australia
Collection
Geoscience Australia Landsat Collection 2
Resource type
Derivative
Published Date
30/11/2018

Background

It is important to see how dynamic or variable the Australian landscape is over any given year, especially to help with understanding how the landscape changes, or to discriminate parts of the landscape that stay the same throughout the year (like forests) from those areas that go through big changes in cover (such as cropping areas).

By understanding different patterns of variation, we can characterise various types of land cover and land use, and detect changes of significance in the landscape.

What this product offers

This product provides tools to exploit the time series of Earth Observation data available in Digital Earth Australia.

It uses a Triple Median Absolute Deviation (TMAD), which provides three 'second order' high dimensional statistic composites per time period, which measure how much an area varies from the average in terms of 'distance' based on factors such as brightness and spectra:

  • Euclidean distance (EMAD)
  • Cosine (spectral) distance (SMAD)
  • Bray Curtis dissimilarity (BCMAD)

Together, they provide information on variance in the source data and are useful for change detection applications.

Applications

This product gives information about:

  • variance and change, so it can be used in machine learning for change detection

  • land cover mapping

  • environmental monitoring

Publications

Roberts, D., Dunn, B., & Mueller, N. (2018). Open data cube products using high-dimensional statistics of time series. IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, 8647–8650. https://doi.org/10.1109/IGARSS.2018.8518312

Data access

Link to data THREDDS
eCat record 130482
Product ID ls5_nbart_tmad_annual, ls7_nbart_tmad_annual, ls8_nbart_tmad_annual
CMI RESTful node ID 346
NCI project code fk4
Security classification Unclassified
Update frequency annually

Access notes

Technical information

The 'first order' statistics include the mean and the median.

The 'second order' statistics produced here are the equivalent statistic to the standard deviation for medians, being the Median Absolute Deviation (MAD).

This product is designed to provide measures of variance of the time series from which it is calculated. As such, it provides measures of change in the landscape over the time period of the input data. The various distance measures used in the TMAD provide information on variance in the following ways:

  • Euclidean distance (EMAD) is more sensitive to changes in target brightness.
  • Cosine (spectral) distance (SMAD) is more sensitive to change in target spectral response.
  • Bray Curtis dissimilarity (BCMAD) is more sensitive to the distribution of the observation values through time.

Each MAD provides information on different land cover change features and is useful in classification.

The mathematical derivation of the three MADs can be found in Roberts et al (2018).

Accuracy and limitations

The accuracy of the TMAD is dependent on the accuracy of the underpinning Earth Observation data. As the values of the MADs are synthetic (not observed directly by satellite), they cannot be verified by field work.

To calculate a MAD representative of clear observations, the majority of the input data must be clear of clouds, shadows or other issues that can adversely affect an observation. Where a location experiences more than 50% of time under the impact of clouds and shadows (such as areas of Tasmania), the MAD can produce an output that is essentially cloud / shadow noise. As such, it is necessary to use a large enough collection of data to enable a clear median to be calculated. However, this also provides a basis for cloud and shadow detection through time series analysis.

To mitigate the impact of poor quality observations and other artefacts, the input data is first screened using the Pixel Quality product (PQ_25_2.0.0); however, some poor pixels are likely to still be included in the data after screening. Increasing the period from which observations are included in the input dataset also helps to mitigate the issue. Where no-data pixels occur in locations that clear observations should occur, the cause is a lack of clear pixels to populate the MADs.

References

Roberts, D., Dunn, B., & Mueller, N. (2018). Open data cube products using high-dimensional statistics of time series. IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, 8647–8650. https://doi.org/10.1109/IGARSS.2018.8518312

Lineage

The three layers of the TMAD are calculated by computing the multidimensional distance between each observation in a time series of multispectral (or higher dimensionality such as hyperspectral) satellite imagery with the multidimensional median of the time series. The median used for this calculation is the geometric median corresponding to the time series. 

The TMAD is calculated over annual time periods on Earth observations from a single sensor by default (such as the annual time series of Landsat 8 observations); however, it is applicable to multi-sensor time series of any length that computing resources can support.

For the purposes of the default Digital Earth Australia product, TMADs are computed per calendar year, per sensor (Landsat 5, Landsat 7 and Landsat 8) from terrain-illumination-corrected surface reflectance data (Analysis Ready Data), compared to the annual geometric median of that data.

Data sources

Processing steps

  1. Surface Reflectance Calculation (NBAR + Terrain Illumination Correction)
  2. Geometric Median
  3. Median Absolute Deviation

Major algorithms

Schema / spatial extent

DEA Albers Raster Schema

Update frequency asNeeded
Temporal extent
Min. longitude -1943830.00
Max. longitude 2170690.00
Min. latitude -1119030.00
Max. latitude -4856630.00
Coordinate reference system Australian Albers / GDA94 (EPSG: 3577)
Cell size X 25.00
Cell size Y 25.00

Example images

Owner

Commonwealth of Australia (Geoscience Australia)

Principal contributors

Dale Roberts

Subject matter experts

Norman Mueller

License

CC BY Attribution 4.0 International License

Rights statement

© Commonwealth of Australia (Geoscience Australia) 2018. Creative Commons Attribution 4.0 International License.

Acknowledgments

The high-dimensional statistics algorithms incorporated in this product is the work of Dr Dale Roberts, Australian National University.