TerraFM-B (`terrafm`)¶

TerraFM-B adapter supporting both provider and tensor backends, with two input modalities (s2 12-band Sentinel-2 SR or s1 VV/VH Sentinel-1) and model-native feature-map grids.

Quick Facts¶

Field	Value
Model ID	`terrafm`
Aliases	`terrafm_b`
Family / Backbone	TerraFM-B from Hugging Face (`MBZUAI/TerraFM`)
Adapter type	`on-the-fly`
Typical backend	provider backend (`gee`), also supports `backend="tensor"`
Primary input	S2 SR 12-band or S1 VV/VH (selected by `sensor.modality`)
Default resolution	10m default provider fetch (`sensor.scale_m`)
Temporal mode	provider path requires `TemporalSpec.range(...)` (v0.1 behavior)
Output modes	`pooled`, `grid`
Extra side inputs	modality settings on `sensor` (`modality`, `use_float_linear`, `s1_require_iw`, `s1_relax_iw_on_empty`)
Training alignment (adapter path)	Medium-High when modality-specific preprocessing matches the intended TerraFM path

When To Use This Model¶

Good fit for¶

experiments comparing S2 and S1 representations under one backbone family
workflows needing both provider fetch and direct tensor backend
model-native feature-map outputs instead of token-only grids

Be careful when¶

mixing S1 and S2 runs without logging modality and preprocessing path
passing tensor backend inputs with wrong channel count (C must be 2 or 12)
assuming backend="auto" selects a non-provider path; it resolves through the embedder's provider contract unless you use tensor

Input Contract (Current Adapter Path)¶

Backend modes¶

backend="tensor":
requires input_chw as CHW
batch tensor inputs should use get_embeddings_batch_from_inputs(...)
adapter resizes to 224
provider backend (gee / provider-compatible, including auto via provider resolution):
requires TemporalSpec.range(...) in v0.1
fetches S2 or S1 based on modality (or sensor.modality if passed through SensorSpec)

Modality selection (`modality` or `sensor.modality`)¶

s2 (default):
12-band Sentinel-2 SR input (B1,B2,B3,B4,B5,B6,B7,B8,B8A,B9,B11,B12)
provider fetch returns normalized [0,1]
input_chw override expects raw SR 0..10000, adapter scales to [0,1]
s1:
2-band Sentinel-1 VV/VH input (VV,VH)
provider path fetches raw VV/VH then normalizes via shared S1 normalization helper
input_chw override expects raw VV/VH and applies log1p + percentile scaling to [0,1]

Sensor fields used by adapter (provider path)¶

common: scale_m, cloudy_pct, composite
S1-specific: use_float_linear, s1_require_iw, s1_relax_iw_on_empty

Channel sanity:

TerraFM path is strict: C must be 12 (S2) or 2 (S1)

Preprocessing Pipeline (Current rs-embed Path)¶

What the original TerraFM model assumes for S1¶

TerraFM treats Sentinel-1 as a 2-channel input branch (VV, VH). The official model code routes the S1 path by channel count (C == 2). The TerraFM paper describes S1 pretraining data as Sentinel-1 RTC patches, so the strongest original assumption is dual-pol VV/VH plus an analysis-ready S1 product, not a hard-coded IW rule.

Why rs-embed prefers `IW` on GEE¶

Earth Engine Sentinel-1 collections are heterogeneous: different instrument modes, coverage patterns, and product characteristics can appear in the same collection. rs-embed therefore prefers IW by default as a conservative proxy for a more homogeneous land-observation subset when approximating TerraFM's S1 training distribution from COPERNICUS/S1_GRD_FLOAT / COPERNICUS/S1_GRD. This IW preference is an adapter policy, not a TerraFM paper requirement.

S1 fetch options in rs-embed¶

s1_require_iw=True:
first try instrumentMode == "IW" together with dual-pol VV/VH
s1_relax_iw_on_empty=True:
if the strict IW query returns no imagery, retry without the IW filter
s1_require_iw=False:
query dual-pol VV/VH directly without enforcing IW

Metadata behavior:

when provider-backed S1 fetch succeeds, metadata records:
s1_iw_requested
s1_iw_applied
s1_iw_relaxed_on_empty
s1_relax_iw_on_empty
this makes it explicit whether a sample came from strict IW filtering or from the relaxed fallback path

Provider path¶

Validate TemporalSpec.range(...)
Select modality from modality (s2 / s1)
Fetch provider patch:
S2: 12-band SR -> normalize to [0,1]
S1: dual-pol VV/VH raw -> prefer IW by default, optionally relax IW on empty -> shared S1 normalization helper -> [0,1]
Optional input inspection on normalized provider input
Resize to fixed 224x224
Load TerraFM-B from vendored runtime + HF .pth weights
Forward:
pooled: TerraFM forward returns CLS embedding (D,)
grid: adapter calls extract_feature(...) and uses last-layer feature map (D,H,W)

Tensor backend path¶

Read input_chw (CHW)
Resize to 224x224
Validate channel count (2 or 12)
Load TerraFM-B and run same forward/grid extraction path

Notes:

Tensor backend path does not apply provider-specific fetch normalization automatically; you are responsible for matching expected input scale/semantics.

Environment Variables / Tuning Knobs¶

Env var	Default	Effect
`RS_EMBED_TERRAFM_FETCH_WORKERS`	`8`	Provider prefetch workers for batch APIs
`RS_EMBED_TERRAFM_BATCH_SIZE`	CPU:`8`, CUDA:`64`	Inference batch size for batch APIs

Related cache envs (used by HF asset download path):

HUGGINGFACE_HUB_CACHE, HF_HOME, HUGGINGFACE_HOME

Adapter behavior notes:

image size is fixed to 224 in current implementation
runtime code is vendored inside rs-embed
weights are fetched from MBZUAI/TerraFM (TerraFM-B.pth)
although the vendored runtime also exposes a large factory, the current adapter only wires up the TerraFM-B weight path, so model_config variant switching is not exposed yet

Output Semantics¶

`OutputSpec.pooled()`¶

Returns TerraFM forward output (CLS embedding) (D,)
This is not token pooling; it is the model’s pooled embedding path

`OutputSpec.grid()`¶

Returns last-layer TerraFM feature map via extract_feature(...)
xarray.DataArray shape (D,H,W)
Metadata includes grid_type="feature_map"
Grid is model feature-map layout, not georeferenced raster pixels

Examples¶

Minimal provider-backed S2 example¶

from rs_embed import get_embedding, PointBuffer, TemporalSpec, OutputSpec

emb = get_embedding(
    "terrafm",
    spatial=PointBuffer(lon=121.5, lat=31.2, buffer_m=2048),
    temporal=TemporalSpec.range("2022-06-01", "2022-09-01"),
    modality="s2",
    output=OutputSpec.pooled(),
    backend="gee",
)

Minimal provider-backed S1 example¶

from rs_embed import get_embedding, PointBuffer, TemporalSpec, OutputSpec, SensorSpec

sensor = SensorSpec(
    collection="COPERNICUS/S1_GRD_FLOAT",
    bands=("VV", "VH"),
    scale_m=10,
    composite="median",
    use_float_linear=True,
    s1_require_iw=True,
    s1_relax_iw_on_empty=True,
)

emb = get_embedding(
    "terrafm",
    spatial=PointBuffer(lon=121.5, lat=31.2, buffer_m=2048),
    temporal=TemporalSpec.range("2022-06-01", "2022-09-01"),
    sensor=sensor,
    modality="s1",
    output=OutputSpec.pooled(),
    backend="gee",
)