FarmVibes.AI: Multi-Modal GeoSpatial ML Models for Agriculture and Sustainability

With FarmVibes.AI, you can develop rich geospatial insights for agriculture and sustainability.

Build models that fuse multiple geospatial and spatiotemporal datasets to obtain insights (e.g. estimate carbon footprint, understand growth rate, detect practices followed) that would be hard to obtain when these datasets are used in isolation. You can fuse together satellite imagery (RGB, SAR, multispectral), drone imagery, weather data, and more.

Fusing datasets this way helps generate more robust insights and unlocks new insights that are otherwise not possible without fusion. This repo contains several fusion workflows (published and shown to be key for agriculture related problems) that help you build robust remote sensing, earth observation, and geospatial models with focus on agriculture/farming with ease. Our main focus right now is agriculture and sustainability, which the models are optimized for. However, the framework itself is generic enough to help you build models for other domains.

FarmVibes.AI Primer

There are three main pieces to FarmVibes.AI. The first one consists of data ingestion and pre-processing workflows to help prepare data for fusion models tailored towards agriculture. Additionally, we provide model training notebook examples that not only allow the configuration of pre-processing of data but also allow tuning existing models with ease. Finally, a compute engine that supports data ingestion as well as adjusting existing and creating novel workflows with the tuned model.

FarmVibes.AI Fusion-Ready Dataset Preparation

In this step, you can select the datasets that you would like to fuse for building the insights. FarmVibes.AI comes with many dataset downloaders. These include satellite imagery from Sentinel 1 and 2, US Cropland Data, USGS Elevation maps, NAIP imagery, NOAA weather data, private weather data from Ambient Weather. Additionally, you can also bring in any rasterized datasets that you want to make them fusion-ready for FarmVibes.AI (e.g. drone imagery or other satellite imagery) and, in the future, custom sensor data (such as weather sensors).

The key technique in FarmVibes.AI is to use as input for ML models data that goes much beyond types, space and time from where the labels are located. For example, when detecting grain silos from satellite imagery (labeled only in optical imagery), it is better to rely on optical as well as elevation and radar bands. In this scenario, it is also important to combine multiple data modalities with other known agriculture infrastructure entities. Likewise, it is also important to use as input the images of a given silo across various times of the year to help generate a more robust model. Including information from many data streams, while also incorporating historical data from nearby or similar locations has been shown to improve robustness of geospatial models (especially for yield, growth, and crop classification problems). FarmVibes.AI generates such input data for models with ease based on parameters that can be specified.