xarray-sentinel

Easily access and explore the SAR data products of the Copernicus Sentinel-1 satellite mission in Python.

This Open Source project is sponsored by B-Open - https://www.bopen.eu.

Features

xarray-sentinel is a Python library and Xarray backend with the following functionalities:

• supports the following data products as distributed by ESA:
• Sentinel-1 Ground Range Detected (GRD):
  • Stripmap (SM)
  • Interferometric Wide Swath (IW)
  • Extra Wide Swath (EW)
• Sentinel-1 Single Look Complex (SLC) SM/IW/EW
• creates ready-to-use Xarray Datasets that map the data lazily and efficiently in terms of both memory usage and disk / network access
• reads all SAR imagery data: GRD images, SLC swaths and SLC bursts
• reads several metadata elements: satellite orbit and attitude, ground control points, radiometric calibration look up tables, Doppler centroid estimation and more
• (partially broken, see #127) reads uncompressed and compressed SAFE data products on the local computer or on a network via fsspec
• supports larger-than-memory and distributed data access via Dask and rioxarray / rasterio / GDAL
• provides a few helpers for simple operations involving metadata like cropping individual bursts out of IW SLC swaths, applying radiometric calibration polynomials, converting slant to ground range for GRD products and computing geospatial metadata.

Overall, the software is in the beta phase and the usual caveats apply.

Install

The easiest way to install xarray-sentinel is via pip:

    pip install xarray-sentinel

If you have uv installed the simplest way to test the library is to start a ipython session with:

    uvx --with xarray-sentinel ipython

Usage

The SAR data products of the Copernicus Sentinel-1 satellite mission are distributed in the SAFE format, composed of a few raster data files in TIFF and several metadata files in XML. The aim of xarray-sentinel is to provide a developer-friendly Python interface to all data and several metadata elements as Xarray Datasets to enable easy processing of SAR data into value-added products.

Due to the inherent complexity and redundancy of the SAFE format xarray-sentinel maps it to a tree of groups where every group may be opened as a Dataset, but it may also contain subgroups, that are listed in the subgroups attribute.

The following sections show some example of xarray-sentinel usage. In the notebooks folder you can also find notebooks, one for each supported product, that allow you to explore the data in more detail using the xarray-sentinel functions.

The root dataset

For example let's explore the Sentinel-1 SLC Stripmap product in the local folder ./S1A_S3_SLC__1SDV_20210401T152855_20210401T152914_037258_04638E_6001.SAFE. First, we can open the SAR data product by passing the engine="sentinel-1" option to xr.open_dataset and access the root group of the product, also known as /:

>>> import xarray as xr
>>> slc_sm_path = "tests/data/S1A_S3_SLC__1SDV_20210401T152855_20210401T152914_037258_04638E_6001.SAFE"
>>> xr.open_dataset(slc_sm_path, engine="sentinel-1")
<xarray.Dataset> Size: 0B
Dimensions:  ()
Data variables:
    *empty*
Attributes: ...
    family_name:                         SENTINEL-1
    number:                              A
    mode:                                SM
    swaths:                              ['S3']
    orbit_number:                        37258
    relative_orbit_number:               86
    ...
    start_time:                          2021-04-01T15:28:55.111501
    stop_time:                           2021-04-01T15:29:14.277650
    group:                               /
    subgroups:                           ['S3', 'S3/VH', 'S3/VH/orbit', 'S3/V...
    Conventions:                         CF-1.11
    history:                             created by xarray_sentinel-...

The root Dataset does not contain any data variable, but only attributes that provide general information on the product and a description of the tree structure of the data. The group attribute contains the name of the current group and the subgroups attribute shows the names of all available groups below this one.

Measurements datasets

To open the other groups we need to add the keyword group to xr.open_dataset. The measurement can then be read by selecting the desired beam mode and polarization. In this example, the data contains the S3 beam mode and the VH polarization with group="S3/VH" is selected:

>>> slc_s3_vh = xr.open_dataset(
...     slc_sm_path, group="S3/VH", engine="sentinel-1", chunks=2048
... )
>>> slc_s3_vh
<xarray.Dataset> Size: 6GB
Dimensions:           (slant_range_time: 18998, azimuth_time: 36895)
Coordinates:
  * slant_range_time  (slant_range_time) float64 ...
    pixel             (slant_range_time) int64 ...
  * azimuth_time      (azimuth_time) datetime64[ns] ...
    line              (azimuth_time) int64 ...
Data variables:
    measurement       (azimuth_time, slant_range_time) complex64 ...
Attributes: ...
    family_name:                         SENTINEL-1
    number:                              A
    mode:                                SM
    swaths:                              ['S3']
    orbit_number:                        37258
    relative_orbit_number:               86
    ...
    geospatial_lon_min:                  42.772483374347
    geospatial_lon_max:                  43.75770573943618
    group:                               /S3/VH
    subgroups:                           ['orbit', 'attitude', 'azimuth_fm_ra...
    Conventions:                         CF-1.11
    history:                             created by xarray_sentinel-...

The measurement variable contains the Single Look Complex measurements as a complex64 and has dimensions slant_range_time and azimuth_time. The azimuth_time is an np.datetime64 coordinate that contains the UTC zero-Doppler time associated with the image line and slant_range_time is an np.float64 coordinate that contains the two-way range time interval in seconds associated with the image pixel.

Since Sentinel-1 IPF version 3.40, a unique identifier for bursts has been added to the SLC product metadata. For these products, the list of the burst ids is stored the burst_ids dataset attribute.

Metadata datasets

The measurement group contains several subgroups with metadata associated with the image. Currently, xarray-sentinel supports the following metadata datasets:

• product XML file
• orbit from the <orbit> tags
• attitude from the <attitude> tags
• azimuth_fm_rate from the <azimuthFmRate> tags
• dc_estimate from the <dcEstimate> tags
• gcp from the <geolocationGridPoint> tags
• coordinate_conversion from the <coordinateConversion> tags
• calibration XML file
• calibration from the <calibrationVector> tags
• noise XML file
• noise_range from the <noiseRangeVector> tags
• noise_azimuth from the <noiseAzimuthVector> tags

For example, the image calibration metadata associated with the S3/VH image can be read using group="S3/VH/calibration":

>>> slc_s3_vh_calibration = xr.open_dataset(
...     slc_sm_path, group="S3/VH/calibration", engine="sentinel-1"
... )
>>> slc_s3_vh_calibration
<xarray.Dataset> Size: 172kB
Dimensions:       (line: 22, pixel: 476)
Coordinates:
  * line          (line) int64 ...
  * pixel         (pixel) int64 ...
Data variables:
    azimuth_time  (line) datetime64[ns] ...
    sigmaNought   (line, pixel) float32 ...
    betaNought    (line, pixel) float32 ...
    gamma         (line, pixel) float32 ...
    dn            (line, pixel) float32 ...
Attributes: ...
    family_name:                         SENTINEL-1
    number:                              A
    mode:                                SM
    swaths:                              ['S3']
    orbit_number:                        37258
    relative_orbit_number:               86
    ...
    absoluteCalibrationConstant:         1.0
    group:                               /S3/VH/calibration
    Conventions:                         CF-1.11
    title:                               Calibration coefficients
    comment:                             The dataset contains calibration inf...
    history:                             created by xarray_sentinel-...

Note that in this case, the dimensions are line and pixel with coordinates corresponding to the sub-grid of the original image where the calibration Look Up Table is defined.

The groups present in a typical Sentinel-1 Stripmap product are:

/
└─ S3
   ├─ VH
   │  ├─ orbit
   │  ├─ attitude
   │  ├─ azimuth_fm_rate
   │  ├─ dc_estimate
   │  ├─ gcp
   │  ├─ coordinate_conversion
   │  ├─ calibration
   │  ├─ noise_range
   │  └─ noise_azimuth
   └─ VV
      ├─ orbit
      ├─ attitude
      ├─ azimuth_fm_rate
      ├─ dc_estimate
      ├─ gcp
      ├─ coordinate_conversion
      ├─ calibration
      ├─ noise_range
      └─ noise_azimuth

Advanced usage

TOPS burst datasets

The IW and EW products, that use the Terrain Observation with Progressive Scan (TOPS) acquisition mode, are more complex because they contain several beam modes in the same SAFE package, but also because the measurement array is a collage of sub-images called bursts.

xarray-sentinel provides a helper function that crops a burst out of a measurement dataset for you.

You need to first open the desired measurement dataset, for example, the HH polarisation of the first IW swath of the S1A_IW_SLC__1SDH_20220414T102209_20220414T102236_042768_051AA4_E677.SAFE product, in the current folder:

>>> slc_iw_v340_path = "tests/data/S1A_IW_SLC__1SDH_20220414T102209_20220414T102236_042768_051AA4_E677.SAFE"
>>> slc_iw1_v340_hh = xr.open_dataset(
...     slc_iw_v340_path, group="IW1/HH", engine="sentinel-1"
... )
>>> slc_iw1_v340_hh
<xarray.Dataset> ...
Dimensions:           (pixel: 21169, line: 13500)
Coordinates:
  * pixel             (pixel) int64 ...
    slant_range_time  (pixel) float64 ...
  * line              (line) int64 ...
    azimuth_time      (line) datetime64[ns] ...
Data variables:
    measurement       (line, pixel) complex64 ...
Attributes: ...
    family_name:                         SENTINEL-1
    number:                              A
    mode:                                IW
    swaths:                              ['IW1', 'IW2', 'IW3']
    orbit_number:                        42768
    relative_orbit_number:               171
    ...
    geospatial_lon_min:                  -61.94949110259839
    geospatial_lon_max:                  -60.24826879672774
    group:                               /IW1/HH
    subgroups:                           ['orbit', 'attitude', 'azimuth_fm_ra...
    Conventions:                         CF-1.11
    history:                             created by xarray_sentinel-...

Note that the measurement data for IW and EW acquisition modes can not be indexed by physical coordinates because of the collage nature of the image.

Now the 9th burst out of 9 can be cropped from the swath data using burst_index=8, via:

>>> import xarray_sentinel
>>> xarray_sentinel.crop_burst_dataset(slc_iw1_v340_hh, burst_index=8)
<xarray.Dataset> ...
Dimensions:           (slant_range_time: 21169, azimuth_time: 1500)
Coordinates:
  * slant_range_time  (slant_range_time) float64 ...
    pixel             (slant_range_time) int64 ...
  * azimuth_time      (azimuth_time) datetime64[ns] ...
    line              (azimuth_time) int64 ...
Data variables:
    measurement       (azimuth_time, slant_range_time) complex64 ...
Attributes: ...
    family_name:                         SENTINEL-1
    number:                              A
    mode:                                IW
    swaths:                              ['IW1', 'IW2', 'IW3']
    orbit_number:                        42768
    relative_orbit_number:               171
    ...
    group:                               /IW1/HH
    Conventions:                         CF-1.11
    history:                             created by xarray_sentinel-...
    azimuth_anx_time:                    2136.774327
    burst_index:                         8
    burst_id:                            365923

If IPF processor version is 3.40 or higher, it is also possible to select the burst to be cropped using the burst_id key:

```python-repl

xarray_sentinel.crop_burst_dataset(slc_iw1_v340_hh, burst_id=365923) ... Dimensions: (slant_range_time: 21169, azimuth_time: 1500) Coordinates: * slant_range_time (slant_range_time) float64 ... pixel (slant_range_time) int64 ... * azimuth_time (azimuth_time) datetime64[ns] ... line (azimuth_time) int64 ... Data variables: measurement (azimuth_time, slant_range_time) complex64 ... Attributes: ... family_name: SENTINEL-1 number: A mode: IW swaths: ['IW1', 'IW2', 'IW3'] orbit_number: 42768 relative_orbit_number: 171 ... group: /IW1/HH Conventions: CF-1.11 history: created by xarray_sentinel-... azimuth_anx_time: 2136.774327 burst_index: