Data Sources

Pre/Post Disaster Data for the Caribbean Islands

Note: A lot of studies use open street map for pre-disaster data.

[xBD Dataset for xView2 Challenge] (Paper: https://arxiv.org/abs/1911.09296) (Challenge:https://xview2.org/)

• xBD is a new large scale dataset for the advancement of change detection and building damage assessment for humanitarian assistance and disaster recovery research.
• It includes bounding boxes and labels for environmental factors such as fire, water, and smoke
• Contains 700,000 building annotations across over 5,000km squared of imagery from 15 countries.
• Data has been collected over 8 disaster types
  • Dam Collapse
  • Earthquake/TsunamiFlood
  • Landslide
  • Volcanic Eruption
  • Wildfire
  • Wind
• Includes a Joint Damage Scale that provides guidance and an assessment scale to label building damage in satellite imagery
  • No Damage
  • Minor Damage
  • Major Damage
  • Destroyed
• xBD is used to introduce the xView 2.0 challenge
• All imagery is sourced from DigitalGlobe which is high-resolution at ~0.5m
  • They are also able to obtain pre- and post-disaster imagery in multi-band 3,4,8 formats
• Challenge Statement:
  • xBD provides building polygons, ordinal regression labels for building damage, and multi-class labels for environmental factors that caused the damage. Given training data, the challenge is to create models and methods that can extract building polygons and assess the building damage level of polygons on an ordinal scale. Furthermore, the models and methods must assign an additional multi-class label to each polygon that indicates which natural force caused the damage to the building.
  • Link to full data information: https://xview2.org/dataset
  • Registered Password: GTCExposure2021
  • Link to Data breakdown after registering: https://xview2.org/download
  • Link to download data: https://xview2.org/download-links
  • They are hugeee so I haven't downloaded yet.

[From Satellite Imagery to Disaster Insights] (https://research.fb.com/wp-content/uploads/2018/11/From-Satellite-Imagery-to-Disaster-Insights.pdf?)

Training Data

• Spacenet (30/50cm) - (full list of spacenet datasets and challenges: https://spacenetchallenge.github.io/datasets/datasetHomePage.html)
• DeepGlobe (50cm)
• PlanetLabs (3m)

Validation Data

• Hurricane Harvey flood
  • 143km squared near sugar land, Texas
• Santa Rosa fire
  • 120km squared near Santa Rosa, California
  • found ground truth data from FRAP website from the Californa Department of Forestry and Fire Protection
• They annotated the data themselves following the procedure describe in the DeepGlobe Paper: Ilke Demir, Krzysztof Koperski, David Lindenbaum, Guan Pang, Jing Huang, Saikat Basu, Forest Hughes, Devis Tuia, and Ramesh Raskar. Deepglobe 2018: A challenge to parse the earth through satellite images. ArXiv e-prints, 2018.
  • this was to identify the roads and buildings in a pixel-wise binary mask

IRMA-17: COMPREHENSIVE SATELLITE DETECTED BUILDING DAMAGE ASSESSMENT OVERVIEW AS OF 21 SEPTEMBER 2017

• Pre-event buildings footprint [Baseline Data] found from Humanitarian Open Street Map Data from 13 Sept 2017 (https://www.hotosm.org/tools-and-data) - repositries available but mainly in Java and CSS
• UNITAR-UNOSAT (https://www.unitar.org/maps) - used in a lot of papers, especially ones focusing on damage classification
  • Antigua, Anguilla, Barbuda, Turks and Caicos Islands
• NGA (https://www.nga.mil/index.html) - Commercial
  • Bahamas, British Virgin Islands
• SERTIT (https://sertit.unistra.fr/en/) - Commercial
  • Saint Barthelemy, Saint Martin
• COPERNICUS EMS (https://emergency.copernicus.eu/mapping/ems/how-use-service) - need to request access
  • Saint Maarten, Saint Barthelemy, Saint Martin

Low‑cost UAV surveys of hurricane damage in Dominica: automated processing with co‑registration of pre‑hurricane imagery for change analysis

• No obvious direct link to the data but a useful paper

EVALUATING HURRICANE RECOVERY IN DOMINICA

• UNDP Housing and Building Assessment Database (https://www.undp.org/content/integrateddigitalassessments/en/home/hbda.html) - not open access *ArcMap (https://desktop.arcgis.com/en/arcmap/) - GIS Map
• Full table of data sources and their uses on page 11

Building Damage Detection in Satellite Imagery Using Convolutional Neural Networks

Method: The data generation pipeline: (1) Pre- and post-disaster satellite images are first passed through the building detection model to identify all buildings. (2) Damaged buildings are extracted from manual damage assessments of the region provided by UNOSAT. (3) Negative examples are obtained by removing the buildings tagged as damaged from all detected buildings. (4) Damaged and undamaged examples are normalized, and data augmentation is applied.

• Digital Globe WorldView 2 and 3 from FirstLook Database
• Candid Flyover Images from National Oceanic and Atmosphere Administration
  • Haiti

Open AI Caribbean Challenge: Mapping Disaster Risk from Aerial Imagery

• STAC - many STAC catalogs are live including Sentinel and Landsat, full list here (https://stacspec.org/)

Low‑cost UAV surveys of hurricane damage in Dominica: automated processing with co‑registration of pre‑hurricane imagery for change analysis

*looks like a useful paper, they used UAVs, only thing I could find in the paper was this UAViators Humanitarian map (http://uaviators.org/map)

Browser tools for data download

• https://earthexplorer.usgs.gov/ - archive NASA and USGS data, nothing for pre-,post- that isn't already in Descartes
• https://cgt.disasterscharter.org/en - Lots of imagery for various pre/post disaster events - no download or API available
• https://sertit.unistra.fr/en/rms/ - PDFs of GIS maps of damage
• https://geohazards-tep.eu/geobrowser/#!&context=Community%2FTerrainMotionDemo - All just Sentinel imagery
• https://www.unitar.org/maps/countries/29 - Shapefiles of damage assessments

Previous Exposure Assessments

• METEOR - https://meteor-project.org/data - The data presents the estimated number of buildings, building area, and rebuilding value at a 15-arcsecond grid resolution (approximately 500 meters at the equator)
• GEM Global Exposure Database - https://storage.globalquakemodel.org/what/physical-integrated-risk/exposure-database/ - Focusses on Earthquake exposure, data available through OpenQuake https://www.globalquakemodel.org/tools-products
• Sendai 2015 - https://www.preventionweb.net/files/43291_sendaiframeworkfordrren.pdf - Dry AF

Earth Observation Data

Medium Resolution satellite imagery

• ESA’s Sentinel - https://scihub.copernicus.eu/dhus/#/home
  • Access – ESA’s API, ESA SNAP software (http://step.esa.int/main/download/) or Descartes Labs pre-processed (https://docs.descarteslabs.com/), Google Earth Engine (https://developers.google.com/earth-engine/datasets/catalog/sentinel)
  • Coverage – Global with repeat time ~10 days since 2014 (Sent 1), 2016 (Sent 2)
  • Resolution – 10m for multi-spectral camera (Sentinel 2); 10-40m for SAR, Synthetic Aperture Radar (Sentinel 1)
  • Pros – repeat coverage and free access to data
  • Cons – Resolution too coarse for building specific evaluation
  • Other Info – Sentinel’s 3 & 6 focus on ocean (maybe useful for island risks?), with thermal data from 3; 5 focusses on atmosphere (useful relating to climate risks?); Other ESA datasets - https://earth.esa.int/eogateway/search?text=&category=Data&
• NASA’s LandSat
  • Access – USGS landlook (https://landlook.usgs.gov/landlook/viewer.html) & Descartes Labs platform, Google Earth Engine (https://developers.google.com/earth-engine/datasets/catalog/landsat)
  • Coverage – Global since 2013 for LandSat 8, since 1999 with worse resolution LandSat 7
  • Resolution - 30m resolution - worse resolution than Sentinel but since 2013
  • Pros – repeat coverage since before the beginning of Sentinel
  • Cons – Low resolution

High-Res Disaster Imagery

• xBD dataset – (https://arxiv.org/pdf/1911.09296.pdf) labelled building damage dataset (no damage -> destroyed) from satellite imagery -https://xview2.org/download
  • Access – Data download from https://www.maxar.com/open-data localised
  • Coverage - Data for 50 events (biased to US), 850,000 labelled buildings
  • Resolution – 0.8m image res

Other

• Free sample data from Planet's SkySat

• METEOR (Modelling Exposure Through Earth Observation Routines) - https://meteor-project.org/data, https://meteor-project.org/documents/AGU_iPosterSessions.pdf page 10

• Other datasets: https://github.com/robmarkcole/satellite-image-deep-learning

• Descartes Labs platform: https://docs.descarteslabs.com/

Google Earth Engine layers

• High-Resolution Imagery
  • NAIP: National Agriculture Imagery Program, 1m resolution - 2003-2019 coverage - US only
  • Planet SkySat Public Ortho Imagery, 0.8m - Very small sample patches over cities worldwide
• Mid-resolution repeated imagery (time-dependent)
  • LandSat - 30m resolution - worldwide every 2 weeks - since 1999
  • Sentinel - 10m resolution - worldwide every 10 days - since 2015

Benchmark Datasets

• SARptical, SEN12MS, DynamicEarthNet
• https://www.bgu.tum.de/en/lmf/research/datasets/tum-dlr-multimodal/

Ground Data

Buildings databases

• https://3d.bk.tudelft.nl/opendata/opencities/
• Open Street Map building footprints - openstreetmap.org
• https://www.bgu.tum.de/lmf/research/building-footprint-extraction-using-deep-learning/

Environmental

• EM-DAT disaster database - https://public.emdat.be/data (You have to register but it's free) - Select disaster type, location and time period and it will download an Excel file
• Global Earthquake Model initiative: https://storage.globalquakemodel.org/what/physical-integrated-risk/exposure-database/

Socio-Economic

• World Bank - https://databank.worldbank.org/source/world-development-indicators/preview/on - Vast array of data since 1960 on country-wide measures (maybe not local enough?)
• Human mobility insights https://www.unacast.com/