Foundation models are flexible deep learning algorithms that are designed for general tasks, rather than being immediately focused on specific tasks. Trained on large amounts of unlabeled data, they can be applied to a variety of downstream tasks with minimal fine-tuning. Foundation models are well known from natural language processing (BERT, GPT-x), and image processing (DALL-E).
In August 2023, NASA and IBM released the Geospatial AI Foundation Model for NASA Earth Observation Data. The model is available open source on Huggingface under the name of Prithvi, the Hindu goddess of Mother Earth. It has been trained on NASA satellite data — according to IBM, more than 250 Petabyte of data are available.
In this blog post, we discuss
- The NASA Harmonized Sentinel-2 Landsat dataset used for training,
- The architecture of the Prithvi-100M Geospatial AI Foundation Model,
- The training process on IBM’s Vela supercomputer,
- Example applications: flooding and crop type identification.
The Geospatial AI Foundation Model has been trained on NASA Harmonized LandSat Sentinel-2 data.
Sentinel-2 is a satellite mission coordinated by the European Space Agency, with two satellites currently in orbit taking high-resolution images of the Earth. It focuses on land, coastal areas, and selected open waters. The Landsat satellites were launched by NASA to record surface reflectance. The harmonized data combine the input from both sensors, resulting in a spatial resolution of about 30 meters and an average revisit time of two to three days. This resolution is sufficient for agricultural monitoring, land use classification, and natural disaster detection.
Standard photographs are made up of three colors: red, green, and blue. The Sentinel-2 data is available in a total of 13 “colors”, so-called bands, spanning the visible, near-infrared, and shortwave infrared range of the electromagnetic spectrum. Selected bands can be used to identify different things, e.g. the infrared…