In this page
- Overview
- Installation
- Quick Start
- Module Variables
- Types
- WeatherStation
- WeatherFamily (abstract)
- WeatherSignal
- Ingest (abstract)
is_finite(f: float): bool- WeatherPoint (volatile)
- VarMapping (volatile)
f_to_c(f: float): float- CsvImport (abstract)
- JsonImport (abstract)
- WeatherExport (abstract)
- Provider API
- OpenMeteoService
- OpenMeteoGeocodingResult (volatile)
- OpenMeteoGeocodingResponse (volatile)
- OpenMeteoResponse (volatile)
- NasaPowerService
- PvgisService
- MeteostatService
- VisualCrossingService
- NoaaService
- MeteoblueService
- SolcastService
- OpenWeatherMapService
- WeatherDotComService
- Station discovery
- CamsAdapter
- NetcdfImport
- Derived & analytics layer
- Enums
- Common Use Cases
- Best Practices
Weather Integration
Weather data ingestion, time-series storage, calendar rollups, and forecast tracking for GreyCat.
Overview
The weather library provides a typed, graph-persistent foundation for weather-aware GreyCat applications:
- Persistent weather stations located by id or by geo coordinates
- Typed signal families: temperature, wind, radiation, precipitation, pressure, humidity, cloud, snow, soil, air-quality, and marine — each a first-class GCL type with named signals
- Observed truth + forecast issue-time matrix per signal: the forecast is a
nodeTime<nodeTime<float>>(outer = target time, inner = issued-at) so you can compare forecasts issued on different days for the same target hour - Opt-in calendar rollups: call
materialize(units, tz)once on a signal to get incrementally maintained hourly/daily/monthly/yearlyGaussian<float>buckets - Persistent rollup backfill:
WeatherSignal::rebuild_rollups(units, tz, from, to)folds the signal’s existingobservedseries into its own persistenthourly/daily/monthly/yearlyrollup fields over a window (resetting the targeted buckets first, so it’s idempotent) — the disk-backed complement tomaterialize’s forward maintenance - Provider-agnostic ingest:
Ingest::feedaccepts a declarativeArray<VarMapping>table — adding a new data provider is a one-file change - Data sources: connectors for Open-Meteo, NASA POWER, PVGIS, Meteostat, Visual Crossing, NOAA/GHCN, meteoblue, Solcast, and OpenWeatherMap; an offline adapter for Copernicus CAMS/ERA5 (
CamsAdapter); a native NetCDF-4/HDF5 importer (NetcdfImport); plus the generic CSV (CsvImport) and JSON (JsonImport) importers — every connector funnels through the sameIngest::feedseam, and most handle wind u/v decomposition automatically
Typical use cases: energy forecasting, digital twins of outdoor assets, agriculture, smart-building HVAC optimization, anomaly detection against seasonal baselines.
Installation
@library("weather", "0.0.0");
If you want the tests too:
@library("weather");
@include("weather/test");
Quick Start
// 1. Geocode a city via Open-Meteo
var hits = OpenMeteoService::geocode("Esch-sur-Alzette", 1, "en", "LU", null, null);
if (hits.size() == 0) { throw "no geocoding result"; }
var hit = hits[0];
// 2. Get-or-create a station at that location
var station = WeatherStation::get_or_create(geo{hit.latitude, hit.longitude}, "esch");
// 3. Enable the families you care about (idempotent)
station->enable(Array<type>{ TemperatureFamily, WindFamily, RadiationFamily });
// 4. Feed OBSERVED history + the native forecast horizon from Open-Meteo
var now = time::now();
var provider = OpenMeteoProvider {
variables: Array<OpenMeteoVariable>{
OpenMeteoVariable::temperature_2m,
OpenMeteoVariable::apparent_temperature,
OpenMeteoVariable::wind_speed_10m,
OpenMeteoVariable::wind_direction_10m,
OpenMeteoVariable::shortwave_radiation,
OpenMeteoVariable::direct_normal_irradiance,
OpenMeteoVariable::diffuse_radiation,
}
};
provider.feed_history(station, now - 60_day, now); // archive (ERA5) over [from, to]
provider.feed_forecast(station); // native 16-day forecast horizon
// 5. Read the latest observed temperature
var last_temp = station->signal_for(TemperatureFamily, "air_2m")->observed.last();
println("latest observed air_2m: ${last_temp} °C");
// 6. Read the latest forecast for a future target hour
var target = now + 3_day;
var fc = station->signal_for(TemperatureFamily, "air_2m")->latest_forecast(target);
println("latest forecast for now+3d: ${fc} °C");
Module Variables
weather_station_by_id: nodeIndex<String, node<WeatherStation>>
Primary index keyed by the application-defined id. Only populated for stations created with a non-null id.
weather_station_by_geo: nodeGeo<node<WeatherStation>>
Geographic index. Every station is registered here; use it for spatial lookups (nearest station, stations inside a bounding box, etc.).
Types
WeatherStation
Persistent weather station.
Fields:
| field | type | description |
|---|---|---|
location |
geo |
Station position |
id |
String? |
Optional human-readable id (key in weather_station_by_id) |
name |
String? |
Display name |
timezone |
TimeZone? |
Caller-managed station metadata — not read by any ingest/bucketing path (rollup/profile bucketing is fixed per-signal via the explicit tz argument of materialize/rebuild_rollups, set-once rollup_tz). It is read by the export day-grouping paths — WeatherExport::to_csv(daily=true) and WeatherDotComProvider::export_archive split files by the station-local calendar day (UTC when unset) — and by the weather.com history feed for station-local chunk dates; see also the CSV-importer timezone note below |
temperature |
node<TemperatureFamily>? |
null = family not yet activated |
wind |
node<WindFamily>? |
null = family not yet activated |
radiation |
node<RadiationFamily>? |
null = family not yet activated |
precipitation |
node<PrecipitationFamily>? |
null = family not yet activated |
pressure |
node<PressureFamily>? |
null = family not yet activated |
humidity |
node<HumidityFamily>? |
null = family not yet activated |
cloud |
node<CloudFamily>? |
null = family not yet activated |
snow |
node<SnowFamily>? |
null = family not yet activated |
soil |
node<SoilFamily>? |
null = family not yet activated |
air_quality |
node<AirQualityFamily>? |
null = family not yet activated |
marine |
node<MarineFamily>? |
null = family not yet activated |
convective |
node<ConvectiveFamily>? |
null = family not yet activated |
condition |
node<ConditionFamily>? |
Categorical condition series (special family — reached via conditions(), not the signal dispatch) |
Methods:
static get_or_create(location: geo, id: String?): node<WeatherStation>— Return the existing station with that id (or at thatgeo) if one already exists; otherwise create, index, and return a new one. Safe to call repeatedly. When a station already occupies that exact coordinate but has no id yet and a non-nullidis supplied, the id is adopted onto that node and registered inweather_station_by_id(previously the requested id was silently dropped, so later lookups by it failed).static index(nws: node<WeatherStation>)— Register a station in both indexes. Called byget_or_create; use directly when constructing a station without the helper. The geo write is guarded like the id write: the first node indexed at a coordinate wins, so re-indexing or a distinct node sharing the coordinate cannot silently orphan the existing entry (callers replacing an entry mustun_indexthe old node first).static un_index(nws: node<WeatherStation>)— Remove a station from both indexes.enable(families: Array<type>)— Activate the given families (idempotent). Pass type references, e.g.Array<type>{ TemperatureFamily, WindFamily }.activate(fam: type)— Activate a single family (idempotent). Throws on an unknown family type.is_enabled(fam: type): bool— Returntrueif the family sub-node is non-null (i.e. has been activated).signal_for(fam: type, name: String): node<WeatherSignal>— Get-or-create the signal node fornameinsidefam. Auto-activates the family if needed. This is the single entry point used byIngest::feed. Throws on an unknown family or signal name — and the signal name is validated before the family is activated, so an unknown name throws without persisting (or enabling) the family: a typo’d mapping whose throw is caught by the caller cannot leave a spuriously activated, empty family behind.peek_signal(fam: type, name: String): node<WeatherSignal>?— Read-only counterpart ofsignal_for: the existing signal node for(fam, name), ornullwhen the family is inactive or the signal was never created. Never creates or activates anything — safe for exports, diagnostics and other read paths (delegates to the family’speek). An unknown family or signal name returnsnullinstead of throwing.conditions(): node<ConditionFamily>— Get-or-create this station’s categoricalConditionFamily(the special, non-signal family for coded weather conditions over time).sun_position(t: time): SunPosition— Solar elevation/azimuth at this station’s location for instantt(computed on demand; see Astronomical).sunrise(day: time): time?/sunset(day: time): time?— Sunrise/sunset instant for the UTC calendar day containingday, ornullwhen no such event occurs in that day. Null is not only a polar (day/night) phenomenon: at longitudes where the event falls near 00:00 UTC, the rare day on which it drifts across midnight contains no event at all — handlenullat every latitude. For |longitude| > ~90° the day’s sunset can precede its sunrise in UTC order — see Astronomical.
Example:
var station = WeatherStation::get_or_create(geo{49.49, 5.98}, "esch");
station->enable(Array<type>{ TemperatureFamily, PrecipitationFamily });
// write directly
station->signal_for(TemperatureFamily, "air_2m")->set_observed(time::now() - 1_hour, 14.7);
// read back
var temp = station->signal_for(TemperatureFamily, "air_2m")->observed.last();
println("last observed: ${temp} °C");
WeatherFamily (abstract)
Base type for all weather families. Each concrete family stores its canonical signals as explicit node<WeatherSignal>? fields and implements:
signal(name: String): node<WeatherSignal>— Get-or-create the signal by canonical name. Throws on an unknown name.peek(name: String): node<WeatherSignal>?— Read-only lookup: the existing signal node, ornullwhen it was never created. Never creates anything; an unknown name returnsnullinstead of throwing.signal_descriptors(): Array<Tuple<String, String>>— Every canonical(signal name, unit)pair the family can hold, insignal’s dispatch order. The single registryWeatherExportderives its column set from, so the export followsfamily.gclautomatically.
Concrete families:
TemperatureFamily
Temperature signals, all in °C.
| signal name | description |
|---|---|
"air_2m" |
Air temperature at 2 m |
"apparent" |
Apparent / feels-like temperature |
"dew_point" |
Dew-point temperature |
"wet_bulb" |
Wet-bulb temperature |
"air_80m" |
Upper-air temperature at 80 m |
"air_120m" |
Upper-air temperature at 120 m |
"air_180m" |
Upper-air temperature at 180 m |
static new(): TemperatureFamily
WindFamily
Wind component signals per measurement height, all in m/s. The stored truth is u/v (meteorological components); speed and direction are derived by the connector.
| signal name | description |
|---|---|
"u_10m" / "v_10m" |
East/north wind component at 10 m |
"gust" |
Wind gust |
"u_80m" / "v_80m" |
East/north wind component at 80 m |
"u_120m" / "v_120m" |
East/north wind component at 120 m |
"u_180m" / "v_180m" |
East/north wind component at 180 m |
static new(): WindFamily
RadiationFamily
Solar irradiance signals (W/m² unless noted).
| signal name | unit | description |
|---|---|---|
"ghi" |
W/m² | Global horizontal irradiance |
"dni" |
W/m² | Direct normal irradiance |
"dif" |
W/m² | Diffuse horizontal irradiance |
"global_tilted" |
W/m² | Global tilted irradiance |
"terrestrial" |
W/m² | Terrestrial (extraterrestrial-normal) irradiance |
"ghi_clear_sky" |
W/m² | Clear-sky GHI |
"dni_clear_sky" |
W/m² | Clear-sky DNI |
"dif_clear_sky" |
W/m² | Clear-sky diffuse |
"uv_index" |
index | UV index |
"is_day" |
— | Day/night flag (1 = day, 0 = night) |
static new(): RadiationFamily
PrecipitationFamily
Precipitation signals.
| signal name | unit | description |
|---|---|---|
"total" |
mm | Total precipitation accumulation |
"rain" |
mm | Rain accumulation |
"showers" |
mm | Convective-shower accumulation |
"probability" |
% | Precipitation probability |
static new(): PrecipitationFamily
PressureFamily
Atmospheric pressure signals.
| signal name | unit | description |
|---|---|---|
"msl" |
hPa | Mean-sea-level pressure |
"surface" |
hPa | Surface (station-level) pressure |
static new(): PressureFamily
HumidityFamily
Humidity signals.
| signal name | unit | description |
|---|---|---|
"relative" |
% | Relative humidity |
"specific" |
g/kg | Specific humidity |
"vpd" |
kPa | Vapour-pressure deficit |
static new(): HumidityFamily
CloudFamily
Cloud cover fractions by layer (%) plus visibility and sunshine.
| signal name | unit | description |
|---|---|---|
"total" |
% | Total cloud cover |
"low" |
% | Low-level cloud cover |
"mid" |
% | Mid-level cloud cover |
"high" |
% | High-level cloud cover |
"visibility" |
m | Horizontal visibility |
"sunshine_duration" |
s | Sunshine duration |
static new(): CloudFamily
SnowFamily
Snow signals.
| signal name | unit | description |
|---|---|---|
"snowfall" |
cm | Snowfall accumulation |
"depth" |
m | Snow depth on the ground |
"swe" |
mm | Snow-water equivalent |
static new(): SnowFamily
SoilFamily
Soil / agriculture signals.
| signal name | unit | description |
|---|---|---|
"temperature" |
°C | Soil temperature |
"moisture" |
m³/m³ | Volumetric soil moisture |
"et0" |
mm | Reference evapotranspiration (FAO) |
static new(): SoilFamily
AirQualityFamily
Air-quality signals; concentrations in µg/m³ plus an aggregate index.
| signal name | unit | description |
|---|---|---|
"pm2_5" |
µg/m³ | Particulate matter < 2.5 µm |
"pm10" |
µg/m³ | Particulate matter < 10 µm |
"o3" |
µg/m³ | Ozone concentration |
"no2" |
µg/m³ | Nitrogen-dioxide concentration |
"so2" |
µg/m³ | Sulfur-dioxide concentration |
"co" |
µg/m³ | Carbon-monoxide concentration |
"aqi" |
index | Air-quality index |
"aod" |
index | Aerosol optical depth |
"pollen" |
grains/m³ | Pollen concentration |
static new(): AirQualityFamily
MarineFamily
Marine / sea-state signals.
| signal name | unit | description |
|---|---|---|
"wave_height" |
m | Significant (wind-wave) height |
"wave_period" |
s | Wave period |
"wave_direction" |
° | Wave direction (meteorological degrees) |
"swell_height" |
m | Swell height |
"swell_period" |
s | Swell period |
"swell_direction" |
° | Swell direction |
"sst" |
°C | Sea-surface temperature |
"sea_level" |
m | Sea level / tide height |
static new(): MarineFamily
ConvectiveFamily
Convective / atmospheric-instability signals.
| signal name | unit | description |
|---|---|---|
"cape" |
J/kg | Convective available potential energy |
"cin" |
J/kg | Convective inhibition |
"lifted_index" |
°C | Lifted index |
"freezing_level" |
m | Freezing-level height |
"boundary_layer_height" |
m | Planetary boundary-layer height |
static new(): ConvectiveFamily
WeatherSignal
One measurement stream on a family: observed truth, forecast issue-time matrix, optional calendar rollups, and a seasonal profile.
Fields:
| field | type | description |
|---|---|---|
unit |
String? |
Physical unit label (e.g. "°C", "W/m²") |
observed |
nodeTime<float> |
Target-time → measured value |
forecast |
nodeTime<nodeTime<float>> |
Target-time → (issued-at → value) |
hourly |
nodeTime<Gaussian<float>>? |
Hourly rollup; null until materialize enables it |
daily |
nodeTime<Gaussian<float>>? |
Daily rollup; null until materialize enables it |
weekly |
nodeTime<Gaussian<float>>? |
Dormant placeholder (CalendarUnit has no week) |
monthly |
nodeTime<Gaussian<float>>? |
Monthly rollup; null until materialize enables it |
yearly |
nodeTime<Gaussian<float>>? |
Yearly rollup; null until materialize enables it |
rollup_tz |
TimeZone? |
Calendar timezone set by materialize |
max_forecast_issues |
int? |
Forecast-retention bound: null = keep all issues per target (default); N = keep the newest N. Set via limit_forecast_issues |
slots |
nodeList<Gaussian<float>> |
Maintained seasonal profile: one Gaussian<float> per (day-of-year × hour-of-day) slot, slot index dayOfYear*24 + hourOfDay (dayOfYear is 0-based 0…365, so the index is 0…8783), bucketed in rollup_tz when set else UTC |
gaussian |
Gaussian<float> |
Overall distribution across all observed points |
Methods:
static new(unit: String?): WeatherSignal— Allocate a fresh signal. Called by familysignal()methods; you normally don’t call it yourself.materialize(units: Array<CalendarUnit>, tz: TimeZone)— Allocate the requested rollup granularities. Subsequentset_observedcalls maintain them incrementally (O(1) per point). The wholeunitsset is validated first: supported values areCalendarUnit::hour,CalendarUnit::day,CalendarUnit::month,CalendarUnit::year, and passing any other unit (minute/second/microsecond, and the absentweek) throws beforerollup_tzis armed or any rollup is allocated — so a bad unit leaves the signal completely untouched rather than half-materialized. Idempotent:rollup_tzis set-once, so the first call’s timezone wins — calling again with a differenttzdoes not re-bucket already-materialized rollups. Note that callingmaterialize— even with an emptyunitsarray — fixesrollup_tz, which is also the timezone used to bucket the always-onslotsseasonal profile. Call it before the firstset_observedso the profile is bucketed in your chosen timezone from the start; calling it later mixes UTC-bucketed early slots with timezone-bucketed later ones — remediable withrebuild_profile, which re-buckets every slot in the armedrollup_tz.rebuild_rollups(units: Array<CalendarUnit>, tz: TimeZone, from: time?, to: time?)— Backfill the requested rollup granularities (hour/day/month/year) from the existingobservedseries over[from?, to?](a null bound ⇒ the first/last observed point) into the signal’s own persistenthourly/daily/monthly/yearlynodeTime<Gaussian<float>>fields. Allocates them likematerializeif absent (and armsrollup_tzset-once, so bucketing always uses the fixedrollup_tz, not a differingtzpassed on a later call). The window is snapped out to whole calendar-unit boundaries so the edge buckets cover a complete unit. Resets the targeted buckets in the window before folding, so re-running it (or running it after incrementalmaterializemaintenance) never double-counts — Gaussians can’t subtract. Only one bucket Gaussian is live at a time (O(1) memory) and the result lands on disk, reachable from the station index. This is the persistent backfill complement tomaterialize(forward maintenance) — and the replacement for the removed transientcalendar_reduce.rebuild_profile()— Rebuild the overallgaussianand the seasonalslotsprofile from the existingobservedseries: both accumulators are reset, then every observed point is re-folded, re-bucketed in the currentrollup_tz(else UTC) using the same slot encoding asset_observed. Always a full rebuild — there is nofrom/towindow, because these two accumulators span all observed points and Gaussians cannot subtract (the calendar rollups keep their own windowedrebuild_rollups). Use it after corrected re-feeds (the accumulators are first-write-only) or after a latematerialize(tz)left theslotskey space mixing UTC- and tz-bucketed slots — the rebuild unifies the slot keys in the armedrollup_tz.set_observed(t: time, value: float)— Write an observed (truth) point. Maintains the overallgaussianprofile, the per-(day-of-year × hour-of-day)slotsseasonal profile, and any materialized rollups incrementally. The profile, slots, and rollups are updated only on the first write of a given timestamp; re-feeding a timestamp correctsobservedwithout touching the Gaussian accumulators (which cannot subtract), so there is no double-counting. To reflect corrections exactly afterwards, rebuild fromobserved:rebuild_rollupsfor the calendar rollups,rebuild_profileforgaussian/slots. The slot bucket and the rollup buckets are computed inrollup_tzwhen set, else UTC.set_forecast(target: time, issued_at: time, value: float)— Write a forecast point. By default all issues for the same target are kept (keep-all); oncelimit_forecast_issues(N)has been set, the write prunes the target’s issues to the newest N.latest_forecast(target: time): float?— Forecast value at the largest (most recent) issued-at timestamp fortarget. The target lookup is at-or-before (resolveAt): atargetfalling between two stored targets resolves to the closest preceding one. Returnsnullonly when no stored target is at or beforetarget.forecast_as_of(target: time, cutoff: time): float?— Forecast fortargetas issued at or beforecutoff. Returns the value from the most recent issue that is<= cutoff.limit_forecast_issues(max: int?)— Bound the retained forecast issues per target to the newestmax(the last N issues retention policy);nullrestores keep-all.maxmust be >= 1 or null: amax < 1throws before any mutation (it would turn every subsequentset_forecastinto a silent insert-then-delete of all issues). Applies immediately to every existing target and to all subsequentset_forecastwrites. Use it on high-churn signals (forecasts overwrite constantly) to keep the issue-time matrix from growing without bound. See forecast-skill scoring.
Forecast issue-time matrix layout:
forecast: nodeTime<nodeTime<float>>
^ ^
target time issued-at → value
This stores every issue for every target. latest_forecast always reads from the latest run; forecast_as_of lets you replay any past run.
Ingest (abstract)
Provider-agnostic scatter: routes WeatherPoints into station signals via a declarative mapping table.
Methods:
static feed(station: node<WeatherStation>, points: Array<WeatherPoint>, mappings: Array<VarMapping>)— A thin internal wrapper over the new streaming primitiveIngestFeeder(inweather_import.gcl): the library’s own importers stream each value directly into the disk-backed signal node viaIngestFeeder(built once per import; no intermediateArray<WeatherPoint>and, for importers without wind decomposition, no per-rowMap), whilefeed(Array)is retained as a thin wrapper for callers who build their own small batch. For eachWeatherPoint, look up eachvalueskey inmappings, apply anyconvertfunction, and callset_observed(whenhistorical) orset_forecast(when nothistorical) on the target signal. Keys not covered by any mapping are silently skipped. The library-wide hindsight-forecast rule lives inIngestFeeder::feed_value(the single central guard every importer routes through): a forecast value whose targettprecedes its issue time (t < issued) is hindsight, not a forecast — it is dropped (t == issuedis kept; historical points are unaffected), so a “forecast” issued after its target can never winlatest_forecastor inflate skill comparisons, whatever the provider. For historical points the feeder never reads the clock (issuedis an ignored placeholder), so bulk historical imports are free of per-rowtime::now()calls; forecast callers should pre-resolve a singleissued_atfor the whole batch (a nullissued_atfalls back to a freshtime::now()per point, fragmenting the issue-time matrix). The family is auto-activated if needed. A mapping’sunitis never applied — families are the sole source of truth for units (seeVarMappingbelow). A mapping that names a signal the family does not define is skipped with a warning (and dropped for the rest of the batch) rather than aborting the whole feed, so one bad mapping can’t lose an entire import. Non-finite values (NaN / ±Inf) are dropped centrally here — after theconvertfunction runs — so they never reach a signal’s rawobserved/forecastseries. (Gaussian.additself now self-rejects non-finite samples, so the overall/seasonal/rollup stats are protected regardless; the central drop additionally keeps the rawnodeTimeseries clean and stops a non-finite value from consuming a timestamp’s first-write slot — which would otherwise block a later finite value at the same timestamp from updating the accumulators.) (WeatherSignal.set_observed/set_forecastcarry the same guard.) This central drop is what makesNetcdfImport’s documented “fill/missing → NaN → dropped byIngest::feed” contract hold even when the NaN is produced by aconvertfn rather than by the C reader.
Shared helpers — reusable static utilities every connector funnels through (so each provider no longer re-implements them inline):
static coerce_float(raw: any?): float?— Coerce a JSON numeric slot to afloat. JSON numbers arrive as either anintor afloatslot, and a bareas floatdoes not widen anintslot, so the int case is widened explicitly. Returnsnullfor anullor non-numeric slot, so callers skip it instead of throwing.static decompose_wind(values: Map<String, float>, speed_key: String, dir_key: String, speed_scale: float)— Meteorological wind decomposition. Readsspeed_key(multiplied byspeed_scale, e.g.1.0/3.6for km/h → m/s,1.0for m/s) anddir_key(degrees); when both are present it replaces them withu_10m/v_10munder the “wind from” convention (u = -speed*sin(dir),v = -speed*cos(dir)). If exactly one is present the orphan is dropped with a warning. The raw speed/dir keys are always removed.static recompose_wind(u: float?, v: float?): Tuple<float?, float?>— Inverse ofdecompose_wind: reconstructs the meteorological(speed_ms, dir_deg)pair from theu_10m/v_10mcomponents under the same “wind from” convention (speed = sqrt(u²+v²),dir = atan2(-u, -v)in degrees normalized to[0, 360); a calmu = v = 0vector yields direction0). Null handling mirrorsdecompose_wind’s orphan rule — both-null returns(null, null), and a single non-null orphan is dropped with a warning and(null, null)returned. Used byWeatherExport/WeatherDotComProvider::export_archiveto recoverwinddirAvg/windspeed*from the stored components.static check_status(status_code: int, context: String)— Uniform HTTP status guard: throws a consistent error on any non-200 status.contextmust be a non-secret label (provider/endpoint name or a base URL without an api key) — never the full query string, which may embed a key.static require_body(content: any?, context: String, error_msg: String?)— Uniform guard for a 200 response whose body is empty or failed to deserialize (Http<T>then returnscontent == nullwith the detail inerror_msg): throws"<context>: 200 response with an empty or unparseable body[: <error_msg>]"instead of letting a connector skip the chunk silently (an invisible hole of up to a whole chunk). Same non-secretcontextrule ascheck_status.static parse_time(cell: String, format: String?, tz: TimeZone?): time?— Parse a timestamp cell, returningnullinstead of throwing on a malformed value (the timestamp analogue ofCsvImport::parse_cell).formatnull→ ISO-8601. Whentzis supplied the offset-less calendar fields are bound to it (Date::parse(...).to_time(tz)); otherwise the naive value is read in the host’s global timezone viatime::parse.
is_finite(f: float): bool
Free function. Returns true for a finite float, false for NaN or ±Inf. Implemented as !isNaN(f - f) (f - f is 0 for any finite value but NaN for NaN and ±Inf). Used by Ingest::feed to drop non-finite samples centrally before they reach a signal’s raw observed/forecast series. (Gaussian.add itself also self-rejects non-finite samples, so the rollup folds in rebuild_rollups / set_observed are safe regardless.)
WeatherPoint (volatile)
Provider-agnostic input row.
Fields:
| field | type | description |
|---|---|---|
t |
time |
UTC timestamp of the measurement target |
historical |
bool |
true → observed truth; false → forecast |
issued_at |
time? |
Forecast issue time (null for historical points) |
values |
Map<String, float> |
Provider-key → raw value |
VarMapping (volatile)
One declarative mapping row connecting a provider variable to a family/signal.
Fields:
| field | type | description |
|---|---|---|
provider_key |
String |
Key in WeatherPoint.values |
family |
type |
Target family type reference (e.g. TemperatureFamily) |
signal |
String |
Signal name within the family (e.g. "air_2m") |
unit |
String? |
Advisory label, never applied or stored. The family constructor (WeatherSignal::new) sets the canonical unit, so Ingest::feed ignores a mapping’s unit entirely. Families are the sole source of truth for units. |
convert |
function? |
fn(float): float applied before storing; null for identity. The result is coerced via Ingest::coerce_float: an int return widens to float, while a non-numeric return warns once and skips that mapping’s entire stream (like a mapping naming an unknown signal) |
f_to_c(f: float): float
Free function. Converts a Fahrenheit value to Celsius: (f - 32) * 5 / 9. Provided as a ready-to-use convert function for VarMapping.
CsvImport (abstract)
Provider-agnostic CSV importer. Turns already-parsed CSV rows into WeatherPoints and WeatherStations, and can feed a CSV file straight into a station’s signals. The pure row-level converters (rows_to_points / rows_to_stations) are the testable boundary; import_stations is a thin I/O wrapper, while feed streams the file row-by-row into the station’s signals (no raw-rows buffer, no points buffer).
Methods:
static to_var_mappings(cols: Array<CsvColumnMapping>): Array<VarMapping>— Convert eachCsvColumnMappingto aVarMappingwhoseprovider_keyis the column name, so the per-rowvaluesMap keys line up with whatIngest::feedexpects. Reuses the ingest seam.static read_rows(path: String, separator: char?, header_lines: int?): Tuple<Array<String>, Array<Array<String>>>— Read a CSV file into a(header, rows)tuple of raw strings (.x= the column-name header line’s columns,.y= the data rows). It forcesheader_lines: 0on the underlyingCsvReader<Array<String>>so every line — including the header — is returned, then peels off the firstheader_lineslines itself, keeping the last of them as the column-name header (the line immediately before the data), which the reader would otherwise consume.header_linesnulldefaults to1; pass0for a headerless file, or a larger count to discard a multi-line preamble (the real header line after the preamble is the one kept). Used internally byfeedandimport_stations, and reusable directly — e.g. to read a CAMS file whose#-comment preamble precedes the header line (note thatCamsAdapter::feed_cams_radiation_csvnow auto-detects that header itself).static parse_cell(cell: String): float?— Non-throwing numeric parse. A blank, non-numeric sentinel (e.g."M","N/A","--"), or otherwise unparseable cell returnsnullinstead of throwing, so a single bad cell no longer aborts the whole import. Integer cells are coerced to real floats. It does not cover numeric missing-value codes (-9999,-999.9, …) — those parse as real values; declare them per column inCsvColumnMapping.sentinels. Used by bothrows_to_pointsandrows_to_stations.static rows_to_points(header: Array<String>, rows: Array<Array<String>>, mappings: Array<CsvColumnMapping>, ts_column: String, ts_format: String?, historical: bool, issued_at: time?, tz: TimeZone?): Array<WeatherPoint>— Pure converter. Locatests_columninheaderand parses each row’s timestamp. A header name that is duplicated and actually used — thets_columnor a mapped value column — throws ("ambiguous import refused"): a silent last-wins pick could import the wrong column undetectably. Duplicates among columns no mapping references are tolerated. (Shared with the streamingfeed, which resolves columns the same way.) Whentzisnull, the cell is read withtime::parse(cell, ts_format)(host-global behavior;ts_formatnull → ISO8601). Whentzis non-null, a naive (offset-less) timestamp is parsed in that timezone viaDate::parse(cell, ts_format).to_time(tz), so a station-local timestamp lands on the correct UTC epoch. For each mapping whosecolumnis present in the header, the cell is parsed withparse_cell(then checked against the mapping’s numericsentinels); blank/sentinel/unparseable cells are skipped rather than aborting. Rows with a blank timestamp cell are skipped.issued_atis carried only on forecast (!historical) points — and on forecast imports a nullissued_atresolves to a singletime::now()for the whole batch (one consistent forecast-matrix issue key, matchingCsvImport::feedandJsonImport). Ideal for testing without file access.static rows_to_stations(header: Array<String>, rows: Array<Array<String>>, cols: CsvStationColumns): Array<node<WeatherStation>>— Pure converter. For each row reads lat/lng (viaparse_cell) and optional id/name, then callsWeatherStation::get_or_create(geo { lat, lng }, id)(which indexes the node) and sets->namewhen present. The same duplicate-header rule asrows_to_pointsapplies: a duplicated header name that is actually used (lat/lng/id/name) throws instead of silently picking the last occurrence. Rows with a missing, non-numeric, or out-of-range lat/lng (outside the web-mercatorgeo::min/geo::maxbounds; NaN included) are skipped. The first non-empty name wins — a later duplicate-coordinate row will not clobber an already-named station. Timezone is not applied — see the note below.static feed(station: node<WeatherStation>, path: String, mappings: Array<CsvColumnMapping>, ts_column: String, ts_format: String?, separator: char?, header_lines: int?, historical: bool, tz: TimeZone?, issued_at: time?)— Full pipeline: peels the header off the front exactly likeread_rows(keeping the last of theheader_linesleading lines as the column-name header), then streams each data row — converted viarow_to_point— straight into the station’s signals viaIngestFeeder, so only one row (and its smallvaluesMap) is ever resident.tzis forwarded for naive-timestamp interpretation. Forecast points are stamped withissued_at(ortime::now()whenissued_atis null) as their issue time.static import_stations(path: String, cols: CsvStationColumns, separator: char?, header_lines: int?): Array<node<WeatherStation>>— Reads a station-metadata CSV and creates/indexes one station per row viarows_to_stations.
Timezone note:
TimeZoneis a closed enum with no dynamic string-to-value lookup, so a free-texttzcell cannot be coerced into the enum without an exhaustive mapping.CsvStationColumns.tzis therefore accepted as metadata but not applied toWeatherStation.timezone; set the timezone explicitly on the returned node if you need it.
Header handling: the file-reading methods force
header_lines: 0on the underlyingCsvReader<Array<String>>so every line (including the header) is returned, then peel off the firstheader_lineslines themselves — preserving the header text, which the CSV reader would otherwise silently consume. Whenheader_linesisnullit defaults to 1 (the natural single-header-line CSV), so a standard CSV needs noheader_linesargument; passheader_lines: 0explicitly for a headerless file. Withheader_lines> 1 the last of those leading lines (the column-name line immediately before the data) is kept as the header and the preamble lines above it are discarded — so a CAMS-style file whose comment preamble precedes its header line parses correctly whenheader_linesis set to (preamble lines + 1).
CsvColumnMapping (volatile)
Maps one CSV header column to a family/signal.
| field | type | description |
|---|---|---|
column |
String |
CSV header column name; becomes the VarMapping.provider_key |
family |
type |
Target family type reference (e.g. TemperatureFamily) |
signal |
String |
Signal name within the family (e.g. "air_2m") |
unit |
String? |
Advisory unit label, carried into the VarMapping but never applied — the family owns the canonical unit |
convert |
function? |
fn(float): float applied before storing; null for identity |
sentinels |
Array<float>? |
Per-column numeric missing-value codes (e.g. -9999.0, -999.9), matched exactly (no epsilon) after parse_cell; a matching cell is skipped like a blank one. parse_cell itself only rejects blank/non-numeric cells |
CsvStationColumns (volatile)
Header names locating station-metadata fields in a station CSV.
| field | type | description |
|---|---|---|
id |
String? |
Header of the station-id column |
lat |
String |
Header of the latitude column (mandatory) |
lng |
String |
Header of the longitude column (mandatory) |
name |
String? |
Header of the station-name column |
tz |
String? |
Header of a timezone column (metadata only — not applied, see note) |
Usage — import a measurements CSV into a station:
var s = WeatherStation::get_or_create(geo { 49.6, 6.12 }, "esch");
var mappings = Array<CsvColumnMapping> {
CsvColumnMapping { column: "temp", family: TemperatureFamily, signal: "air_2m", unit: "°C", convert: null },
CsvColumnMapping { column: "humidity", family: HumidityFamily, signal: "relative", unit: "%", convert: null }
};
// CSV: timestamp,temp,humidity (ISO8601 timestamps, 1 header line, comma-separated)
// header_lines null defaults to 1; tz null keeps host-global timestamp parsing;
// issued_at null stamps forecast points with time::now() (here historical=true).
CsvImport::feed(s, "files/esch.csv", mappings, "timestamp", null, ',', null, true, null, null);
Usage — import a station-metadata CSV:
// CSV: station_id,latitude,longitude,label
// ST-1,49.6,6.12,Esch
// ST-2,48.85,2.35,Paris
var cols = CsvStationColumns { id: "station_id", lat: "latitude", lng: "longitude", name: "label", tz: null };
var stations = CsvImport::import_stations("files/stations.csv", cols, ',', 1);
// stations are now registered in weather_station_by_id / weather_station_by_geo
JsonImport (abstract)
Generic importer for weather JSON documents shaped as an array of record objects (one record per timestamp) — the common shape of weather REST APIs. The importer streams each value directly into the disk-backed signal node (no intermediate Array<WeatherPoint>); feed_json adds the JSON-string parse + navigation.
Mappings are expressed as JsonFieldMapping rows whose path is a field-path relative to each record: dot-separated keys with optional [i] array indices (e.g. "main.temp", "hourly[0].ghi", "a.b[1][2]").
Methods:
static resolve_path(root: any?, path: String): any?— Walk a dotted field-path within an already-parsed JSON value (Map<String, any?>/Array<any?>tree). Each.-separated segment may carry one or more trailing[i]array indices. Descends through map keys and array indices; returnsnullif any step is missing, an index is out of bounds, or a type mismatches. An empty path ("") returnsroot, but a path with an empty segment — a trailing or double dot such as"main."or"a..b"— is malformed and returnsnull, as is a segment with junk after a]index (e.g."values[0]x").static to_var_mappings(ms: Array<JsonFieldMapping>): Array<VarMapping>— Adapt JSON field mappings to the provider-agnosticVarMappingrows consumed byIngestFeeder. EachVarMapping.provider_keyis set to the mapping’spath, because the importer feeds each value under its mapping path.- Importer streams internally (no exposed pure converter). For each record,
time_fieldis resolved: anintvalue is treated as epoch seconds (time::new(<int>, DurationUnit::seconds)); afloatvalue is treated as float-valued epoch seconds and truncated to integer seconds (time::new((v as float) as int, DurationUnit::seconds)); any other value is read as aStringand parsed withtime_format(ISO 8601 whennull). String timestamps are bound totzwhen supplied; with a nulltz, offset-less strings bind to the host’s global timezone — passTimeZone::"UTC"for offset-less UTC sources (same semantics asCsvImport). Each mapping’spathis resolved to a number (int coerced to float;nullskipped) and fed under that path. Thetime_fieldand every mapping path are tokenized once up front, so a large pull never re-splits a path string per record. The forecast issue-time is hoisted once for the whole batch (issued_at ?? time::now()when!historical, elsenull). Records that arenullor have a missing time field are skipped. static feed_json(station: node<WeatherStation>, json_text: String, records_path: String, time_field: String, time_format: String?, tz: TimeZone?, mappings: Array<JsonFieldMapping>, historical: bool, issued_at: time?)— Full pipeline:Json<any> {}.parse(json_text), navigate to the records array viarecords_path, then stream each value into the station’s family signals viaIngestFeeder.issued_atis forwarded for forecast points (null →time::now()). Arecords_paththat is missing or resolves to a non-array (e.g. a provider error payload) emits a warning and imports nothing; an empty records array stays silent (a valid empty pull).
JsonFieldMapping (volatile)
One declarative mapping row for the generic JSON importer.
| field | type | description |
|---|---|---|
path |
String |
Field-path relative to each record: dot-separated keys with optional [i] array indices |
family |
type |
Target family type reference (e.g. TemperatureFamily) |
signal |
String |
Signal name within the family (e.g. "air_2m") |
unit |
String? |
Advisory unit label, carried into the VarMapping but never applied — the family owns the canonical unit |
convert |
function? |
fn(float): float applied before storing; null for identity |
Import a records-array JSON document
var station = WeatherStation::get_or_create(geo { 49.594, 6.119 }, "my-station");
// JSON shaped as { "list": [ { "ts": "...", "main": { "temp": .. }, "rad": { "ghi": .. } }, ... ] }
var json_text = "{\"list\":[{\"ts\":\"2026-04-21T00:00:00Z\",\"main\":{\"temp\":7.5},\"rad\":{\"ghi\":42}}]}";
var mappings = Array<JsonFieldMapping> {
JsonFieldMapping { path: "main.temp", family: TemperatureFamily, signal: "air_2m", unit: "°C", convert: null },
JsonFieldMapping { path: "rad.ghi", family: RadiationFamily, signal: "ghi", unit: "W/m²", convert: null }
};
// records_path = "list", time_field = "ts", ISO-8601 time (format null),
// tz null (the "Z"-suffixed stamps carry their own offset),
// historical = true, issued_at null (ignored for historical points).
JsonImport::feed_json(station, json_text, "list", "ts", null, null, mappings, true, null);
For epoch-seconds timestamps, point time_field at the numeric field and pass time_format = null — an int time value is interpreted as Unix seconds automatically, and a float value (e.g. 1700000000.0) is truncated to integer seconds. To test without the JSON-string parse, build Array<any?> records by hand and stream them through a feeder: var feeder = IngestFeeder::new(station, JsonImport::to_var_mappings(mappings)); JsonImport::records_scatter(records, time_field, time_format, tz, mappings, feeder, historical, issued_at); then read back from station->signal_for(...)->observed.getAt(t).
WeatherExport (abstract)
Neutral, round-trippable station export — the counterpart to CsvImport / JsonImport. Writes any station’s canonical signals to a wide CSV (one column per present signal) or a record-major JSON document, both re-importable through the same import seams. An abstract type with static methods only (no instances). The weather.com archive-exact 33-column layout (WeatherDotComProvider::export_archive) is a specialization that reuses this scaffolding (observed_union, fmt_cell, read_signal_at, the per-day grouping loop).
Scope and guarantees:
- Observed series only — forecasts are excluded by design. The wide one-row-per-instant layout cannot carry the
(target, issued)two-key forecast matrix, so the export is lossy for forecasts; read the per-signalforecastnodes directly if forecast retention matters. - The column set is derived, not hand-listed:
canonical_columns()walkscanonical_families(), whose descriptors come from each family’s ownsignal_descriptors()— all 69 canonical(family, signal)pairs, in stable family-major order. Adding a signal tofamily.gclextends the export automatically. A signal name declared by more than one family is qualified<family_key>_<signal>(today:total→precipitation_totalandcloud_total), so a full export never carries two identical headers; qualification is decided on the static registry, never on the data. - Export never mutates the graph: signals are reached through the read-only
WeatherStation::peek_signal/ familypeekpaths, so exporting can never activate a family or allocate a signal node. - Empty export ⇒ no file for both formats: a station with nothing to export (no present signal, or no observed point in the window) writes no file at all (
to_jsonno longer writes{"observations": []}). - Timestamps carry microsecond precision: the CSV
timecolumn and the JSONtimefield are ISO-8601 UTC with 6 fractional digits (%Y-%m-%dT%H:%M:%S.%fZ), so sub-second points neither shift nor collide on re-import. Re-import must usets_format/time_formatnull(ISO mode) — an explicit"%Y-%m-%dT%H:%M:%SZ"format no longer parses them.
Methods:
static observed_union(sigs: Array<node<WeatherSignal>>, from: time?, to: time?): Array<time>— the sorted, de-duplicated UNION of observed timestamps acrosssigswithin[from,to](a null bound is open on that side). One entry per distinct instant, chronological — the row index for any export.static fmt_cell(v: float?): String— render a float for a cell, or""when null. Integer-valued floats drop the decimal point (17.0→"17"); other values use the default float rendering when it parses back exactly through the importers’ number parser, and escalate to 17-significant-digit scientific notation when the default rendering would be lossy (e.g.0.1 + 0.2) — keeping the round-trip exact for every value the runtime can re-parse exactly. (Also used byWeatherDotComProvider::export_archivecells.)static canonical_families(): Array<ExportFamily>— the canonical families in stable export order, each as anExportFamily { family: type, key: String, descriptors: Array<Tuple<String, String>> }(thekeymatches theWeatherStationfield name and qualifies ambiguous column names;descriptorsis the family’ssignal_descriptors()). The single registry the export derives from.static canonical_columns(): Array<ExportColumn>— every canonical(family, signal)pair as anExportColumn { family: type, signal: String, column: String }in stable family-major order;columnequalssignalexcept for shared names, which are qualified<family_key>_<signal>.static present_signals(station): Array<Tuple<String, node<WeatherSignal>>>— the canonical signals that actually exist (active family with a non-empty observed series), as(column-name, node)pairs in canonical order —.xis the column name, which may be family-qualified, not always the bare signal name. Strictly read-only (peek_signal).static read_signal_at(sig: node<WeatherSignal>?, t: time): float?— a signal’s observed value att, or null when the signal is null or has no point att.static csv_mappings(): Array<CsvColumnMapping>— the import mapping that exactly invertsto_csv: oneCsvColumnMappingper canonical column, keyed by the same column nameto_csvwrites, no convert (u/v are stored truth, re-imported verbatim). Use withCsvImport::feed/read_rows(ts_column "time",ts_format null= ISO 8601) to round-trip a neutral CSV export.static json_mappings(): Array<JsonFieldMapping>— the JSON twin ofcsv_mappings: oneJsonFieldMappingper canonical column (each column is a flat top-level field on every record object). Use withJsonImport::feed_json(records_path "observations",time_field "time",time_format null) to round-trip a neutral JSON export.static to_csv(station, path: String, from: time?, to: time?, daily: bool)— wide CSV export: header ="time"+ each present canonical column name (canonical order), one row perobserved_uniontimestamp over[from,to]. Timestamps are ISO-8601 UTC with microsecond precision (round-trippable independent of the host tz).daily = falsewrites one file atpath;daily = truetreatspathas a directory and writes<path>/<id-or-export>_YYYYMMDD.csvgrouped by the station-local calendar day (station.timezone ?? UTC). No file on an empty export.static to_json(station, path: String, from: time?, to: time?)— record-major JSON export: a single document{ "observations": [ { "time": "<iso8601 UTC, µs precision>", "<column>": <float>, ... }, ... ] }, one object perobserved_uniontimestamp (absent points omit the field). Re-imports directly viaJsonImport::feed_jsonwithrecords_path = "observations",time_field = "time",time_format nullandjson_mappings()— the exact inverse of import. No file on an empty export.
var station = weather_station_by_id.get("my-station")!!;
// Wide neutral CSV (single file) — re-import with WeatherExport::csv_mappings().
WeatherExport::to_csv(station, "./export/my-station.csv", null, null, false);
var dst = WeatherStation::get_or_create(geo { 49.6, 6.1 }, "round-trip");
// ts_format null = ISO mode — the µs-precision "Z" timestamps carry their own
// offset, so no tz binding is needed.
CsvImport::feed(dst, "./export/my-station.csv", WeatherExport::csv_mappings(),
"time", null, ',', 1, true, null, null);
// Record-major JSON — re-import with JsonImport over records_path "observations".
WeatherExport::to_json(station, "./export/my-station.json", null, null);
Provider API
The uniform provider contract every connector’s public surface implements: one configured WeatherProvider instance per connection, carrying its own credentials/endpoints as fields so the method surface stays uniform. Reference implementations: OpenMeteoProvider, MeteostatProvider, WeatherDotComProvider.
WeatherProvider (abstract)
GreyCat has no method override (a subtype only implements an abstract method or inherits a concrete one), so every feed/capability method is abstract and every provider implements all of them — a provider that does not serve a mode implements it as this.unsupported("<method>"):
abstract id(): String— stable lower-case key (e.g."openmeteo"), for logs and provenance.abstract capabilities(): ProviderCaps— what this provider serves (cheap introspection; no try/catch needed).abstract feed_current(station: node<WeatherStation>)— pull the latest/current observation(s).abstract feed_history(station: node<WeatherStation>, from: time, to: time)— backfill OBSERVED history over[from, to].abstract feed_forecast(station: node<WeatherStation>)— pull the provider’s native FORECAST horizon.abstract station_discovery(): StationDiscovery?— upstream station discovery, ornullwhen the provider has no station network.unsupported(method: String)— the one concrete, inherited helper (never overridden): throws the uniform"<id>: <method> unsupported". The convention for every mode a provider does not serve, so the failure message is centralized and identical across connectors.
ProviderCaps (volatile)
Cheap capability introspection so callers branch without try/catch: current: bool, history: bool, forecast: bool, discovery: bool.
var p = MeteostatProvider::new("RAPIDAPI_KEY");
if (p.capabilities().forecast) {
p.feed_forecast(station); // gated: never hits an "unsupported" throw
}
var disc = p.station_discovery(); // null when the provider has no station network
if (disc != null) {
var nodes = disc.register_bbox(sw, ne, null);
}
Station-discovery contract
station_discovery() returns a StationDiscovery whose max parameter follows one binding contract across every implementation: null = uncapped (provider default); max <= 0 = empty result (an explicit zero/negative cap caps to nothing — it is never an “uncapped” sentinel; implementations short-circuit before any network call). Bounding boxes follow spatial::GeoBox corner semantics (see Spatial): latitudes are order-agnostic, longitudes are taken as given — sw.lng() > ne.lng() means an antimeridian-crossing box.
Writing a custom provider
type MyProvider extends WeatherProvider {
api_key: String?;
fn id(): String {
return "myprovider";
}
fn capabilities(): ProviderCaps {
return ProviderCaps { current: false, history: true, forecast: false, discovery: false };
}
fn feed_current(_station: node<WeatherStation>) {
this.unsupported("feed_current");
}
fn feed_history(station: node<WeatherStation>, from: time, to: time) {
// fetch [from, to] from the upstream API, then stream each point
// through the shared ingest seam, e.g.:
// Ingest::feed(station, points, my_mappings);
}
fn feed_forecast(_station: node<WeatherStation>) {
this.unsupported("feed_forecast");
}
fn station_discovery(): StationDiscovery? {
return null;
}
}
OpenMeteoService
Connector for the Open-Meteo family of APIs.
The public entry point is OpenMeteoProvider (a WeatherProvider): it carries the archive_base_url/forecast_base_url/api_key/variables as fields, so they are off every method signature, and the history-vs-forecast distinction is the method you call. Open-Meteo is a gridded model — it serves OBSERVED history (archive/ERA5) and a native FORECAST horizon, but has no “current” snapshot product (so feed_current is unsupported) and no station network (so station_discovery() is null). Forward geocode(…) stays a provider helper (geocoding is not station discovery). The old single ranged feed_from_openmeteo(station, from, to, …) that straddled now is gone: feed_history is archive-only over the literal [from, to] (a future to never spills into a forecast call — the archive simply returns nulls there), and feed_forecast has no range argument (the native horizon [now, now+15d] is the range).
var p = OpenMeteoProvider::new(); // or OpenMeteoProvider { api_key: "…", variables: … }
p.feed_history(station, from, to); // archive (ERA5) over [from, to]
p.feed_forecast(station); // native 16-day forecast horizon
// p.capabilities() => { current:false, history:true, forecast:true, discovery:false }
// p.station_discovery() => null (gridded: no station network)
var hits = p.geocode("Esch", null, null, "LU", null); // forward geocoding helper
OpenMeteoProvider delegates to the OpenMeteoService statics below (which remain the testable, lower-level surface). When variables is null, OpenMeteoProvider::default_variables() is requested — the canonical family set (temperature, humidity, precipitation, pressure, wind speed+direction+gust, cloud, radiation GHI/DNI/DIF, snow, soil).
Provider methods (OpenMeteoProvider):
static new(): OpenMeteoProvider— all-defaults constructor (public endpoints, no api key,default_variables()).static default_variables(): Array<OpenMeteoVariable>— the default hourly variable set requested whenvariablesis null.id(): String—"openmeteo".capabilities(): ProviderCaps—{ current: false, history: true, forecast: true, discovery: false }.feed_current(station)—unsupported(Open-Meteo has no current-snapshot product; usefeed_forecastfor the near-term horizon).feed_history(station, from, to)— backfill OBSERVED history over the literal[from, to]from the archive (ERA5) endpoint, streaming viaIngestFeeder(delegates tofeed_archive_range). Never spills into the forecast endpoint — a futuretosimply returns archive nulls there. Throws iffrom > to. The archive lags ~5 days behind real time, so a window ending nearnowlogs a warning about the possible recent-past gap.feed_forecast(station)— pull the native FORECAST horizon (issued now): the forecast endpoint over[now, now+15d], clamped to Open-Meteo’s 16-day cap (delegates tofeed_forecast_horizon). No range argument — the horizon is the range.station_discovery(): StationDiscovery?—null(gridded: no station network).geocode(name, count, language, country_code, geocoding_base_url): Array<OpenMeteoGeocodingResult>— forward geocoding helper threading this instance’sapi_keyintoOpenMeteoService::geocode.
Methods (OpenMeteoService):
static geocode(name: String, count: int?, language: String?, country_code: String?, base_url: String?, api_key: String?): Array<OpenMeteoGeocodingResult>— City search. Defaults:count=10,language="en",base_url="https://geocoding-api.open-meteo.com".country_codeis an ISO-3166-1 alpha-2 filter (e.g."LU"). Whenapi_keyis non-null it is appended as&apikey=<key>— needed only for commercial endpoints. Thename(after Unicode normalization/casefolding),country_code, andapi_keyare URL-encoded, so names with spaces or non-ASCII characters (e.g."New York","Düsseldorf") are handled correctly. Throws on non-200 responses or when the server returnserror: true.static mappings(): Array<VarMapping>— Return the declarative mapping table from Open-Meteo variable names to unified family/signal pairs. It maps temperature (temperature_2m→air_2m,dew_point_2m→dew_point,apparent_temperature→apparent), precipitation (precipitation→total,precipitation_probability→probability), radiation (shortwave_radiation→ghi,direct_normal_irradiance→dni,diffuse_radiation→dif), wind (u_10m/v_10m,wind_gusts_10m→gust), humidity (relative_humidity_2m→Humidity/relative,vapour_pressure_deficit→Humidity/vpd), pressure (pressure_msl→Pressure/msl,surface_pressure→Pressure/surface), cloud (cloud_cover→Cloud/total,cloud_cover_low/mid/high→Cloud/low|mid|high), snow (snowfall→Snow/snowfall,snow_depth→Snow/depth), and soil (soil_temperature_0cm→Soil/temperature,soil_moisture_0_to_1cm→Soil/moisture). The humidity, pressure, cloud, snow, and soil variables were previously fetched but silently dropped; they are now mapped. Wind speed/direction are converted to u/v components inline as each record is streamed, emitting"u_10m"/"v_10m"as provider keys — those map to theWindFamilysignals"u_10m"and"v_10m"(reached viasignal("u_10m")/signal("v_10m")). The table still maps only a curated subset ofOpenMeteoVariable(e.g.wet_bulb_temperature_2mis in the enum but not currently mapped); on aprovider_keycollision the lastVarMappingwins.- Importer streams internally (no exposed pure converter); the per-hour mapping and the
wind_speed_10m+wind_direction_10m→u_10m/v_10mdecomposition described above happen inline as each record is fed. Rows outside the requested branch window are skipped: requests are date-granular (whole UTC days), so a response also carries edge-day hours beforefrom/afterto— and, on a forecast pull, today’s hours beforenow— which are no longer ingested. The window clamp is value-precise: the preceding-hour-mean radiation keys are clamped on their re-stamped interval-start instant (t − 1h), all other values on the provider stampt— so the last in-window radiation hour (carried by the row att = to + 1h) survives and no radiation lands beforefrom. A response reportingutc_offset_seconds != 0logs a warning (timestamps are parsed as UTC; requests must not carry&timezone). static feed_archive_range(feeder: IngestFeeder, lat: float, lng: float, from: time, to: time, variables: Array<OpenMeteoVariable>, archive_base_url: String?, api_key: String?)— IO loop (archive endpoint): backfill OBSERVED rows over[from, to]intofeederfrom Open-Meteo’s archive (ERA5) endpoint. Chunks the window into ≤1-year requests (end-inclusive, +1-day step to avoid overlap); each chunk’s rows are clamped to the branch window[from, to]. The request pinswind_speed_unit=ms, sowind_speed_10m-derivedu_10m/v_10mandwind_gusts_10marrive in true m/s (the provider default is km/h, which would store every wind value 3.6× too high). The request date windows (start_date/end_date) are formatted in UTC, and each response’s HTTP status is checked — a non-200 throws (the URL is omitted from the message because it may carry an api key), as does a 200 chunk whose body is empty or unparseable (sharedIngest::require_bodyguard).archive_base_urldefaults to the public archive endpoint. The non-_instantradiation columns (shortwave_radiation,direct_radiation,diffuse_radiation,direct_normal_irradiance,global_tilted_irradiance,terrestrial_radiation) are means over the preceding hour stamped at the interval end by the provider; the converter re-stamps them one hour earlier (interval start) so the same physical hour lands on the same instant as CAMS/Solcast/PVGIS. (OpenMeteoProvider::feed_historyis the public entry point.)static feed_forecast_horizon(feeder: IngestFeeder, lat: float, lng: float, now: time, variables: Array<OpenMeteoVariable>, forecast_base_url: String?, api_key: String?)— IO loop (forecast endpoint): pull the FORECAST horizon (issuednow) intofeederfrom Open-Meteo’s forecast endpoint. Plans the fetchable window withforecast_window(the native horizon[now, now+15d]clamped to Open-Meteo’s 16-day horizon — 16 days including today, the last validend_dateis UTC-today + 15 days), returning early when nothing is fetchable. Samewind_speed_unit=mspin, UTC date windows, status/body guards, and radiation re-stamping asfeed_archive_range.forecast_base_urldefaults to the public forecast endpoint. (OpenMeteoProvider::feed_forecastis the public entry point.)- Pure request/planning helpers (no IO — the unit-testable seams
geocode/feed_archive_range/feed_forecast_horizondelegate to):static build_api_key_param(api_key: String?): String— the&apikey=<url-encoded key>suffix, or""for null.static build_geocode_url(name: String, count: int?, language: String?, country_code: String?, base_url: String?, api_key: String?): String— the geocoding search URL (free-text params URL-encoded).static geocode_decode(body: OpenMeteoGeocodingResponse?): Array<OpenMeteoGeocodingResult>— unwrap a geocoding body: empty array for a null body / missingresults; throws on anerror: truepayload with the provider’sreason.static build_hourly_param(variables: Array<OpenMeteoVariable>): String— comma-join the requested variables for thehourly=query value.static build_archive_url(base_url: String?, lat: float, lng: float, start: time, end: time, variables: Array<OpenMeteoVariable>, api_key: String?): String/static build_forecast_url(…)(same parameters) — one chunk’s archive/forecast URL: whole-UTC-daystart_date/end_date,wind_speed_unit=mspinned.static chunk_windows(from: time, to: time, span: duration): Array<Tuple<time, time>>— split[from, to]into end-inclusive windows of at mostspan, each next window starting one day past the previous end.static forecast_window(from: time, to: time, now: time): Tuple<time, time>?— the forecast portion of[from, to]clamped to the 16-day horizon;null(with a warning when skipping) when no fetchable forecast hours remain.
OpenMeteoGeocodingResult (volatile)
Deserialized geocoding result.
Required fields: id: int, name: String, latitude: float, longitude: float.
Optional fields: elevation: float?, feature_code: String?, country_code: String?, country_id: int?, country: String?, timezone: String?, population: int?, postcodes: Array<String>?, admin1..4: String?, admin1_id..admin4_id: int?.
OpenMeteoGeocodingResponse (volatile)
Envelope returned by the geocoding API. Either results: Array<OpenMeteoGeocodingResult>? on success, or error: bool? / reason: String? on failure.
OpenMeteoResponse (volatile)
Internal deserialization shape for the archive and forecast endpoints. hourly is a Map<String, any?>? (and hourly_units a Map<String, String>?) — nullable so a degraded/empty 200 body deserializes and is skipped by the scatter instead of throwing at parse time. The "time" entry is Array<String>, every other entry is Array<float?> (or Array<int?> for integer-valued variables such as precipitation_probability and cloud_cover) keyed by the Open-Meteo variable name. Integer columns are coerced to float as each record is streamed. You normally don’t handle this yourself; feed_archive_range / feed_forecast_horizon (driven by OpenMeteoProvider) consume it internally.
NasaPowerService
Connector for the NASA POWER hourly point API — no API key, history-only, hourly data from 2001-01-01 onward (the often-quoted 1981 start applies to the daily API).
Methods:
static default_parameters(): Array<String>— The PARAM list requested from NASA POWER:T2M,T2MDEW,ALLSKY_SFC_SW_DWN,ALLSKY_SFC_SW_DNI,ALLSKY_SFC_SW_DIFF,WS10M,WD10M,PRECTOTCORR,RH2M,PS. Wind is requested as raw speed (WS10M) + direction (WD10M); they are decomposed inline into theu_10m/v_10mcomponents theWindFamilystores.static mappings(): Array<VarMapping>— Declarative mapping from NASA POWER parameter names to unified family/signal pairs. Emitsu_10m/v_10mprovider keys (from the WS10M/WD10M decomposition) mapped toWindFamily.PS(kPa) → Pressure/surface (hPa vianasa_kpa_to_hpa).- Importer streams internally (no exposed pure converter). NASA POWER data is PARAM-major (
param → {ts → value}); the importer drives off one parameter’s timestamp keys (the old TS-majorby_tspivot is gone) and feeds each PARAM’s value per"YYYYMMDDHH"timestamp (parsed as UTC) straight into the signals.-999/-999.0fill cells are skipped, integer values coerced to float, andWS10M+WD10Mdecomposed tou_10m/v_10minline. static feed_from_nasa_power(station: node<WeatherStation>, from: time, to: time, base_url: String?)— Full pipeline: pullsGET /api/temporal/hourly/pointover[from, to](chunked into ≤1-year requests, end-inclusive with a +1-day step to avoid overlap), alwayshistorical: true, streaming each response into the station’s signals viaIngestFeeder.base_urldefaults tohttps://power.larc.nasa.gov. Throws iffrom > to, on a non-200 response, or on a 200 with an empty or unparseable body (Ingest::require_body).- Pure request helpers (no IO):
static build_parameters_param(params: Array<String>): String— comma-join the PARAM list for theparameters=query value;static build_hourly_point_url(base_url: String, lat: float, lng: float, start: time, end: time, parameters: String): String— one chunk’s hourly point URL (YYYYMMDDUTC day bounds,time-standard=UTC,community=RE,format=JSON).
NasaPowerResponse (volatile)
Deserialization shape: NasaPowerResponse { properties: NasaPowerProperties }; NasaPowerProperties { parameter: Map<String, Map<String, any?>> }. parameter is keyed by PARAM name; each inner map is keyed by "YYYYMMDDHH" UTC strings → numeric values (NASA’s -999 sentinel marks missing data and is skipped by the converter).
nasa_kpa_to_hpa(v: float): float
Free function. Converts NASA POWER surface pressure (PS, reported in kPa) to hPa (v * 10). Wired as the convert function for the PS mapping.
var station = WeatherStation::get_or_create(geo{ 49.594, 6.119 }, "esch");
station->enable(Array<type>{ TemperatureFamily, WindFamily, RadiationFamily, PressureFamily });
// NASA POWER is history-only — request a past window
NasaPowerService::feed_from_nasa_power(station, time::now() - 365_day, time::now() - 1_day, null);
var last_temp = station->signal_for(TemperatureFamily, "air_2m")->observed.last();
PvgisService
Connector for the PVGIS v5.3 (Photovoltaic Geographical Information System, JRC) hourly radiation API. History-only (no forecast) and key-free (public, rate-limited to roughly 30 requests/second).
Methods:
static mappings(): Array<VarMapping>— Declarative mapping from PVGIS hourly keys to unified family/signal pairs: the syntheticghi_sum→RadiationFamily/"ghi"(see below),Gd(i)→RadiationFamily/"dif",T2m→TemperatureFamily/"air_2m". Wind (WS10m) is intentionally not mapped — see the limitation note below — andGb(i)is not ingested on its own (see the note below). Acomponents=1response (the formfeed_from_pvgisrequests) carries noG(i)— the global irradiance is replaced by its components — so the importer reconstructs ghi per record asGb(i)+Gd(i)+Gr(i)and feeds it under the syntheticghi_sumkey (a record missing one component skips the reconstruction with a warning, once per response).- Importer streams internally (no exposed pure converter; the streaming path is Map-free and clamps to
[from, to]inline). Each hourly record’stimefield is parsed with the"%Y%m%d:%H%M"format (note the colon; PVGIS stamps records at the satellite scan minute, 10 minutes past the hour) explicitly as UTC — PVGISseriescalcstamps are UTC, so a non-UTC host global timezone can no longer shift the epoch — then floored to the hour so the same physical hour lands on the same instant as CAMS/Solcast (interval-START convention) instead of forming a disjoint:10twin series; integer-valued cells are coerced to real floats as the record is fed. static feed_from_pvgis(station: node<WeatherStation>, from: time, to: time, base_url: String?)— Full pipeline: builds theseriescalcURL for the station’s location, requests the JSON time-series, and streams it into the station’s signals viaIngestFeeder(alwayshistorical: true). The endpoint only accepts whole-year bounds and returns every hour of the requested years, so to bound memory this requests one year at a time (startyear == endyear == y, looping over the UTC years offrom/to, likeNasaPowerService) instead of building one giant multi-year response, and clamps each year’s points to the requested[from, to]window inline as it streams.base_urldefaults tohttps://re.jrc.ec.europa.eu. Throws iffrom > to, on a non-200 response, or on a 200 with an empty or unparseable body (Ingest::require_body).
No wind direction: PVGIS reports only wind speed (
WS10m) and no direction.WindFamilystores meteorological u/v components, which cannot be computed without a direction. Wind therefore cannot be ingested from PVGIS, soWS10mis left unmapped andIngest::feedsilently skips it.
Gb(i)is not ingested as dni: PVGISGb(i)is the beam component on the requested plane (here horizontal,angle=0) — not beam-normal irradiance; the two differ by a1/cos(solar zenith)factor, unbounded near sunrise/sunset. CAMS (BNI) and Solcast (dni) feed true beam-normal intoRadiation/dni, so PVGIS contributes nodnito keep the canonical series physically consistent —Gb(i)only enters theGb(i)+Gd(i)+Gr(i)sum behind the syntheticghi_sumkey (withangle=0that sum is the global horizontal irradiance;Gr(i)is ~0 on the horizontal plane but is summed regardless).
PvgisResponse / PvgisOutputs (volatile)
Internal deserialization shape. PvgisResponse { outputs: PvgisOutputs? }; PvgisOutputs { hourly: Array<Map<String, any?>>? } — both nullable so a 200 with a non-seriescalc body is skipped instead of null-derefing the feed. Each hourly record is a Map<String, any?> because PVGIS keys (G(i), Gb(i), …) are not valid GCL field names; numeric cells arrive as int or float and are coerced to float as each record is streamed.
MeteostatService
Connector for Meteostat, offering two ingestion paths: a LIVE JSON path (point/hourly via RapidAPI, requires a key) and a BULK CSV path (free, no auth).
The public entry point is MeteostatProvider (a WeatherProvider): it carries the rapidapi_key/base_url as fields, so they are off every method signature, and the history-vs-current distinction is the method you call (no historical flag). It serves current + observed history and exposes station discovery; Meteostat has no forecast product, so feed_forecast is unsupported.
var p = MeteostatProvider::new("RAPIDAPI_KEY"); // or MeteostatProvider { rapidapi_key: "…", base_url: "…" }
p.feed_history(station, from, to); // observed hourly over [from, to]
p.feed_current(station); // most recent observed 24h
p.feed_bulk_csv(station, "/path/to/<station>.csv"); // gunzip'd bulk dump (observed)
var disc = p.station_discovery()!!; // MeteostatDiscovery
var nodes = disc.register_bbox(sw, ne, null); // stations in a bbox -> WeatherStation nodes
MeteostatProvider delegates to the MeteostatService statics below (which remain the testable, lower-level surface). The feed_bulk_csv(station, path) instance method is provider-specific (bulk file formats differ per provider) and is not on the WeatherProvider contract; the gunzip prerequisite is unchanged (see the bulk recipe below).
Model substitution caveat: the
point/hourlyendpoint fills missing records with model data (&model=true, the provider default, pinned explicitly inbuild_hourly_url), and the date range is day-granular — so a window ending today (everyfeed_current) returns the full current UTC day, including model predictions for hours that have not happened yet. The importer clamps historical records tot <= now(warn-free, expected on everyfeed_current), so those future model hours never land inobservedas truth; past gaps may still be model-filled (Meteostat’s documented gap-filling behavior).
Methods:
static mappings(): Array<VarMapping>— Declarative mapping from the provider keys emitted by the importer to unified family/signal pairs. All unit conversions and the wind u/v decomposition happen inline during ingest (the LIVE streaming path and the BULK converter), so these mappings carry noconvertfunction. Keys:temp→Temperature/air_2m (°C),dwpt→Temperature/dew_point (°C),rhum→Humidity/relative (%),prcp→Precipitation/total (mm),u_10m/v_10m→Wind/u_10m,v_10m (m/s),gust→Wind/gust (m/s),pres→Pressure/msl (hPa),snow_m→Snow/depth (m),tsun_s→Cloud/sunshine_duration (s).coco(condition code) is deserialized but intentionally not ingested — there is noConditions::mapper for Meteostat’s coco vocabulary.- LIVE JSON importer streams internally (no exposed pure converter). Timestamps arrive as
"YYYY-MM-DD HH:MM:SS"(UTC, because the endpoint is queried withtz=UTC) and are parsed with"%Y-%m-%d %H:%M:%S". The unit conversions below are applied and integer JSON fields (rhum,wdir,snow) coerced to float as each record is streamed. static feed_live(station: node<WeatherStation>, from: time, to: time, rapidapi_key: String, historical: bool, base_url: String?)— Full LIVE pipeline. Chunks[from, to]into ≤30-day requests (the Meteostat hourly endpoint serves at most 30 inclusive calendar dates per request; end-inclusive with a +1-day step to avoid overlap), issuingGET <base_url>/point/hourly?lat=<>&lon=<>&start=<YYYY-MM-DD>&end=<YYYY-MM-DD>&tz=UTCper chunk with headersx-rapidapi-keyandx-rapidapi-host: meteostat.p.rapidapi.com, and streams each response into the station’s signals viaIngestFeeder(one feeder reused across chunks).base_urldefaults tohttps://meteostat.p.rapidapi.com. Throws iffrom > to, on a non-200 response, or on a 200 with an empty or unparseable body (Ingest::require_body).- Pure request helpers (no IO —
feed_livedelegates to them):static chunk_windows(from: time, to: time): Array<Tuple<time, time>>— the ≤30-day window tiling (end-inclusive, +1-day step);static build_hourly_url(base_url: String, lat: float, lon: float, start: time, end: time): String— one chunk’spoint/hourlyURL (%Y-%m-%dUTC bounds,tz=UTC, and&model=true— the provider’s default model-substitution pinned explicitly: missing records are filled with model data);static rapidapi_headers(rapidapi_key: String): Map<String, String>— thex-rapidapi-key/x-rapidapi-hostheader pair. static meteostat_bulk_to_points(rows: Array<Array<String>>, historical: bool): Array<WeatherPoint>— Internal pure converter for the BULK hourly CSV, kept as the unit-test boundary over small canned rows (the productionfeed_bulk_csvpath streams row-by-row instead of building this array). The CSV is headerless and positional — column order:date(YYYY-MM-DD), hour(HH), temp, dwpt, rhum, prcp, snow, wdir, wspd, wpgt, pres, tsun, coco. The timestamp is rebuilt fromdate+hour(parsed with"%Y-%m-%dT%H"). Cells parsed with the non-throwingCsvImport::parse_cell(int coerced to float); blank / sentinel / unparseable cells and short rows are skipped rather than aborting the bulk file, andfeed_bulk_csvenablestrimto strip stray\r/spaces. Same unit conversions as the LIVE path.static feed_bulk_csv(station: node<WeatherStation>, path: String, historical: bool)— Reads a decompressed bulk CSV atpathviaCsvReader<Array<String>>(headerless, comma-separated) and streams it row-by-row into the station’s signals viaIngestFeeder— no raw-rows buffer and no points buffer; only the current row’s smallvaluesMap is ever resident.
Bulk download recipe (no native gzip): Meteostat serves bulk files gzip-compressed at https://data.meteostat.net/hourly/<station>.csv.gz (the new host; legacy bulk.meteostat.net stops updating Jan 2026). GreyCat’s std library has no gzip decompressor, so download and gunzip externally first, then pass the resulting .csv to feed_bulk_csv:
curl -sL https://data.meteostat.net/hourly/10637.csv.gz | gunzip > 10637.csv
Unit conversions (both LIVE and BULK paths): wind wspd/gust wpgt km/h → m/s (÷3.6); wind decomposed to u/v from speed (m/s) + direction wdir; snow depth snow mm → m (÷1000, emitted as snow_m); sunshine tsun minutes → seconds (×60, emitted as tsun_s — the BULK path consumes positional column 11; column 12 coco is skipped). Types: MeteostatResponse { data: Array<MeteostatRecord> } (volatile) and MeteostatRecord (volatile) with time: String plus nullable measurement fields (integer fields typed any?, coerced to float).
VisualCrossingService
Connector for the Visual Crossing Timeline API (a single endpoint serving both history and forecast). Handles wind u/v decomposition automatically and normalizes the provider’s native units: wind speed/gust km/h → m/s (÷3.6) and snow depth cm → m.
Methods:
static cm_to_m(v: float): float— Centimetres → metres (v / 100). Wired as theconvertfunction for snow depth inmappings().static km_to_m(v: float): float— Kilometres → metres (v * 1000). Wired as theconvertfunction for visibility inmappings()(unitGroup=metricreports visibility in km; the canonical signal stores metres).static mappings(): Array<VarMapping>— Declarative mapping from Visual Crossing hourly keys to unified family/signal pairs (temp→Temperature/air_2m,feelslike→Temperature/apparent,dew→Temperature/dew_point,humidity→Humidity/relative,precip→Precipitation/total,precipprob→Precipitation/probability,pressure→Pressure/msl,cloudcover→Cloud/total,solarradiation→Radiation/ghi,uvindex→Radiation/uv_index,visibility→Cloud/visibility viakm_to_m,snow→Snow/snowfall,snowdepth→Snow/depth viacm_to_m). Wind speed/direction →"u_10m"/"v_10m"and gust →"gust"(m/s) inline as each record is streamed.- Importer streams internally (no exposed pure converter). It iterates
days[].hours[], timestamps each hour fromdatetimeEpoch(time::new(.., DurationUnit::seconds)), coerces integer JSON columns to float, convertswindspeed/windgustkm/h → m/s and decomposes speed + direction to u/v (removing the raw wind keys) inline. The forecast issue time is resolved once per import byfeed_from_visualcrossing(time::now()when!historical, else null), so every chunk’s forecast targets share one issued-at. static feed_from_visualcrossing(station: node<WeatherStation>, location: String, from: time, to: time, api_key: String, historical: bool, base_url: String?)— Full pipeline: chunks[from, to]into ≤1-year requests, building a Timeline URL per chunk (dates"%Y-%m-%d";locationmay be"lat,lng"or free-text and is URL-encoded), sending each GET, checking status, and streaming each response into the station’s signals viaIngestFeeder.base_urldefaults tohttps://weather.visualcrossing.com. Throws iffrom > toor on a non-200 response — and also on a 200 chunk whose body is empty or unparseable ("Visual Crossing: 200 response with an empty or unparseable body (<d1>..<d2>)", with the stdHttpResponse.error_msgdeserialization detail appended when present, via the helperstatic require_body(body, d1, d2, error_msg)); previously such a chunk was silently skipped, leaving an invisible up-to-one-year hole. Whenhistorical=false, points flow through the forecast issue-time matrix; because the Timeline request is date-granular, today’s already-elapsed hours arrive too — rows whose target precedes the (batch-hoisted) issue time are dropped by the library-wide hindsight guard in the ingest seam (IngestFeeder::feed_value, see Ingest), so a hindsight “forecast” issued after its target can’t winlatest_forecastor inflate skill comparisons.
Forecast-as-of: because Visual Crossing forecasts are written through set_forecast, every issue for a target time is retained. Feed the same forward window every few hours and latest_forecast(target) always returns the most recent run, while forecast_as_of(target, cutoff) replays the forecast as published at any past cutoff.
VisualCrossingResponse / VisualCrossingDay (volatile)
Internal deserialization shapes for the Timeline endpoint. VisualCrossingResponse holds latitude/longitude plus days: Array<VisualCrossingDay>; each VisualCrossingDay carries datetime plus hours: Array<Map<String, any?>> (numeric values may arrive as int or float — coerced to float by the converter).
NoaaService
Connector for NOAA / US National Weather Service data. Two independent paths: a LIVE path against api.weather.gov (US only, no API key, but a User-Agent header is mandatory), and a GHCN-Daily bulk CSV path (global daily summaries, no auth) served from S3 in long format.
Methods:
static mappings(): Array<VarMapping>— Declarative mapping for the LIVE path:temperature→Temperature/air_2m (°C),dewpoint→Temperature/dew_point (°C),relativeHumidity→Humidity/relative (%),u_10m/v_10m→Wind/u_10m,v_10m (m/s),barometricPressure→Pressure/surface (hPa, the station/surface pressure),seaLevelPressure→Pressure/msl (hPa, reduced to mean-sea-level),precipitationLastHour→Precipitation/total (mm). All conversions to canonical units happen inline as each measure is streamed (vianoaa_value, driven by each measure’s WMOunitCode), soconvertisnullhere. TheunitCode==ladder is now aMap<String, function>lookup built once.- Importer streams internally (no exposed pure converter). Per GeoJSON feature it reads each
NoaaMeasurevianoaa_value, skipping any with a nullvalue(NWS reports many quantities asnull). Unit-aware: rather than hard-coding km/h÷3.6 and Pa÷100, each measure is converted to the family’s canonical unit based on the WMOunitCodeit reports — handling temperature (degC/degF/K→ °C), pressure (Pa/hPa/kPa→ hPa), wind speed (km_h-1/m_s-1→ m/s),percent,degree_(angle), and precip length (mm/m→ mm). A measure whoseunitCodeis missing or unrecognized is skipped with a warning (never silently mis-scaled by a wrong factor, and never assumed to already be canonical). Wind speed (already normalized to m/s) + direction are then decomposed intou_10m/v_10m(raw keys removed). Timestamps parsed withIngest::parse_time(ts, "%Y-%m-%dT%H:%M:%S%z", null). The internal streaming converter (noaa_scatter) returns aNoaaScatterStats(count: int— features processed,oldest: time?— oldest parsed timestamp) sofeed_from_noaacan detect a truncated full page. static noaa_value(key: String, measure: NoaaMeasure?, converters: Map<String, function>): float?— Read one optional NWS measure to its canonical-unit float, honoring the per-measure WMOunitCode(see the unit list above).convertersis theunit_converters()table, built once per import and reused for every measure. Returnsnullfor a null measure/value, and for a missing or unrecognizedunitCodewarns and returnsnullrather than guessing.static put_noaa(values: Map<String, float>, key: String, measure: NoaaMeasure?, converters: Map<String, function>)— Resolve one measure vianoaa_valueand store it invaluesunderkeyonly when non-null.static build_observations_url(base_url: String, station_id: String, start: time, end: time): String— Pure request construction (no IO): the station observations URL for one[start, end]window, bounds formatted as UTC%Y-%m-%dT%H:%M:%SZ.static next_page_end(stats: NoaaScatterStats, chunk_start: time, page_end: time, query: String): time?— Pure re-page decision (no IO) for one fetched page: returns the nextendbound to re-request, ornullwhen re-paging must stop (a non-full page, the window start reached, a full page with zero parseable timestamps, or a non-shrinking window — the latter two warn instead of looping).static feed_from_noaa(station: node<WeatherStation>, station_id: String, from: time, to: time, user_agent: String, base_url: String?)— Full LIVE pipeline. Chunks[from, to]into ≤7-day windows, building<base_url>/stations/<station_id>/observations?start=<ISO>&end=<ISO>per chunk, sending a GET with the requiredUser-Agentheader (pass a contact string; api.weather.gov rejects requests without one), checking status, and streaming each response as historical into the station’s signals viaIngestFeeder(one feeder reused across chunks). The NWS endpoint silently truncates an over-full window to its newest 500 features (HTTP 200, no pagination Link), so each window is re-paged:startstays fixed andendwalks down past the oldest feature seen until a page returns fewer than 500 features (or the window start is reached) — sub-hourly (ASOS) stations no longer silently lose the oldest ~5 days of each chunk. On a full page with no parseable timestamps, or a non-shrinking window, it warns and stops rather than looping.base_urldefaults tohttps://api.weather.gov. Throws iffrom > to, on a non-200 response, or on a 200 with an empty or unparseable body (Ingest::require_body— previously such a page silently imported nothing).static ghcn_mappings(): Array<VarMapping>— Mapping for the GHCN-Daily path. Because the GHCN stream already pivots and scales into canonical signal names, keys line up 1:1:air_2m→Temperature/air_2m (°C),total→Precipitation/total (mm),snowfall→Snow/snowfall (cm),depth→Snow/depth (m). The recognized GHCN-Daily LONG format elements (ID,DATE,ELEMENT,DATA_VALUE,…, one row per(DATE,ELEMENT)) are scaled and renamed during ingest:TAVG(tenths °C, ÷10) →air_2m;PRCP(tenths mm, ÷10) →total;SNOW(mm → cm, ÷10) →snowfall;SNWD(mm → m, ÷1000) →depth.TMAX/TMINare skipped — there are no max/min signals in the unified contract. Rows with a non-emptyQ_FLAG(column 5 — the value failed a NOAA quality-assurance check, e.g.D/G/I/L/O/X) are skipped. GHCN dates (YYYYMMDD) are parsed explicitly as UTC, so daily points always land at UTC midnight regardless of the host’s global timezone.static feed_ghcn_csv(station: node<WeatherStation>, path: String, historical: bool)— Streams the CSV atpath(header_lines: 1) one row at a time through aGhcnAccumulator, which coalesces consecutive same-DATErows into one point and flushes on each date change. Real by-station files are not date-sorted — they mirror the.dlylayout (year-month block, thenELEMENT, then day), so same-date rows are rarely contiguous and in practice one single-element point flushes per row. Correctness is order-independent: each value lands independently at its(t, signal)viafeed_value, and only the current date’s smallvaluesMap is ever live (no full rows buffer, noby_dateMap-of-Maps over the whole history). Source files:https://noaa-ghcn-pds.s3.amazonaws.com/csv/by_station/<ID>.csv.
NoaaMeasure / NoaaObsResponse / NoaaScatterStats (volatile)
NoaaMeasure { value: any?; unitCode: String? } — one measured quantity (value is any? because NWS reports some quantities as JSON integers; a null value means the station didn’t report it and is skipped). NoaaObsResponse { features: Array<NoaaFeature>? } (nullable, so a degraded body deserializes instead of throwing); NoaaFeature { properties: NoaaObsProps }; NoaaObsProps carries the ISO-8601 timestamp plus optional temperature, dewpoint, windSpeed, windDirection, relativeHumidity, barometricPressure, seaLevelPressure, precipitationLastHour (each a NoaaMeasure?). NoaaScatterStats { count: int; oldest: time? } — one page’s processed-feature count and oldest parsed timestamp, returned by the streaming converter and consumed by next_page_end.
MeteoblueService
Connector for the meteoblue packages API. meteoblue exposes data as packages (e.g. basic-1h) that map roughly 1:1 to weather families; this connector requests a single <package> and ingests its data_1h block. Handles wind u/v decomposition automatically and supports meteoblue’s opt-in HMAC-SHA256 URL signing.
Methods:
static mappings(): Array<VarMapping>— Declarative mapping from meteobluedata_1hkeys to unified family/signal pairs:temperature→Temperature/air_2m (°C),felttemperature→Temperature/apparent (°C),relativehumidity→Humidity/relative (%),precipitation→Precipitation/total (mm),sealevelpressure→Pressure/msl (hPa),gust→Wind/gust (m/s),ghi_instant→Radiation/ghi (W/m²),dni_instant→Radiation/dni (W/m²),dif_instant→Radiation/dif (W/m²),totalcloudcover→Cloud/total (%). Wind speed/direction (windspeed/winddirection) are converted to"u_10m"/"v_10m"(m/s) inline as each record is streamed. meteoblue’ssnowfractionis a dimensionless 0…1 ratio (fraction of precipitation falling as snow), not a snowfall depth in cm, so it is intentionally not mapped. meteoblue key names vary by package, so the table maps the common ones; on aprovider_keycollision the lastVarMappingwins.- Importer streams internally (no exposed pure converter). It iterates the
data_1hcolumns (same shape as Open-Meteo’shourly), parses each"YYYY-MM-DD HH:MM"timestamp with"%Y-%m-%d %H:%M"as UTC — guaranteed because the request forcestz=UTC; without it the provider auto-detects the location’s local timezone, which would shift every ingested point by the UTC offset — coerces integer columns to real floats, and decomposeswindspeed+winddirectionintou_10m/v_10m(removing the raw keys) inline. The forecast issue time is hoisted once for the whole batch. static build_query_path(lat: float, lng: float, package: String, api_key: String): String— Pure request construction (no IO): the path+query"/packages/<package>?lat=..&lon=..&apikey=..&format=json&tz=UTC"(package/api_keyURL-encoded;tz=UTCsits inside the string that gets HMAC-signed).static build_signed_url(base_url: String?, lat: float, lng: float, package: String, api_key: String, shared_secret: String?): String— Pure URL builder (no IO):base_url ?? "https://my.meteoblue.com"+build_query_path(...), HMAC-SHA256-signing exactly that path+query (includingtz=UTC) and appending&sig=<hex>whenshared_secretis non-null.static feed_from_meteoblue(station: node<WeatherStation>, package: String, api_key: String, shared_secret: String?, base_url: String?, historical: bool)— Full pipeline: builds the (optionally signed) URL viabuild_signed_url, sends a single GET (no date-range chunking is possible — the package URL carries no date range), checks status, and streams the response into the station’s signals viaIngestFeeder. Throws on a non-200 response and on a 200 with an empty or unparseable body (Ingest::require_body— a single-request connector would otherwise silently import nothing).
Optional HMAC signing: when
shared_secretis non-null, the connector signs the path+query string (the exactbuild_query_pathoutput, includingtz=UTC) withCrypto::sha256_hmac_hex(query_path, shared_secret)and appends the result as&sig=<hex>(meteoblue’s signed-URL scheme). Whenshared_secretisnull, the request is sent unsigned (plain apikey auth).
MeteoblueResponse (volatile)
Internal deserialization shape. data_1h is a Map<String, any?> — the "time" entry is Array<String>, every other entry an Array<any?> aligned by index (numeric values arrive as int or float, coerced to float by the converter); metadata/units are optional passthrough maps.
SolcastService
Connector for the Solcast radiation & weather API — a single endpoint serving three time horizons under Bearer-token (Authorization: Bearer <api_key>) auth. The horizon is selected by the SolcastKind enum (live, forecast, historic) rather than a magic string, so a typo’d mode is a compile error, not a late HTTP 404 with a silently-wrong historical flag.
Methods:
static mappings(): Array<VarMapping>— Declarative mapping from Solcast output parameters to unified family/signal pairs:ghi→Radiation/ghi (W/m²),dni→Radiation/dni (W/m²),dhi→Radiation/dif (W/m²),air_temp→Temperature/air_2m (°C),relative_humidity→Humidity/relative (%),surface_pressure→Pressure/surface (hPa),cloud_opacity→Cloud/total (%), plusu_10m/v_10m→Wind/u_10m,v_10m (m/s) from the wind decomposition. Wind speed is already m/s, so no conversion is applied.- Importer streams internally (no exposed pure converter). It picks
forecastswhen present, elseestimated_actuals, decomposeswind_speed_10m(m/s) +wind_direction_10mintou_10m/v_10mand coerces integer JSON columns to floats inline. The forecast issue time is hoisted once for the whole batch. static period_duration(period: String?): duration— Pure parser (no IO) for the record’s ISO-8601periodfield ("PT<minutes>M"); a null/malformed/non-positive value falls back to PT60M (the period the URL requests).static kind_path(kind: SolcastKind): String— Pure helper: the URL path segment for a kind (live→"live",forecast→"forecast",historic→"historic").static build_radiation_url(base_url: String?, kind: SolcastKind, lat: float, lng: float, hours: int?, start: time?, end: time?): String— Pure request construction (no IO): theradiation_and_weatherURL forkind;hoursis the live/forecast window-length selector,start/end(ISO-8601 UTC instants) key the historic endpoint — each appended only when supplied.static historic_windows(from: time, to: time, span: duration): Array<Tuple<time, time>>— Pure chunking helper (no IO): split[from, to]into edge-to-edge instant windows of at mostspan(31 days for Solcast’s historic cap); a boundary-instant re-feed is idempotent.static feed_from_solcast(station: node<WeatherStation>, kind: SolcastKind, hours: int?, from: time?, to: time?, api_key: String, base_url: String?)— Full pipeline: builds the URL(s) viabuild_radiation_urlforkind(SolcastKind::live,::forecastor::historic), sends with anAuthorization: Bearer <api_key>header, checks status, and streams each response into the station’s signals viaIngestFeeder(one feeder shared across requests). Window selection is per kind:live/forecastare bounded by thehourswindow-length selector (passingfrom/tothrows);historicrequiresfrom<to(passinghoursthrows), requests the range as ISO-8601 UTCstart/endquery params, and chunks ranges longer than Solcast’s 31-day historic cap into ≤31-day edge-to-edge windows — one request per window, all streamed through one feeder.historicalis derived askind != SolcastKind::forecast, solive/historicwrite observed truth whileforecastroutes through the forecast issue-time matrix.base_urldefaults tohttps://api.solcast.com.au. Throws on a non-200 response and on a 200 with an empty or unparseable body (Ingest::require_body).
Timestamp parsing: Solcast stamps each record’s
period_endas ISO-8601 with 7 fractional digits + aZsuffix (e.g."2026-04-21T00:30:00.0000000Z").time::parsedoes not accept the fractional+Z form, so the converter slices the leading 19 chars (String.slice(0, 19)) and parses them with"%Y-%m-%dT%H:%M:%S"as UTC.period_endis the end of the averaging interval; the connector re-stamps every point at the interval start (period_end - period, with the record’speriodfield parsed byperiod_duration, PT60M fallback) to match the library’s interval-start convention (same as CAMS) — observed and forecast target times shift accordingly.
GTI is not stored: Solcast can return
gti(global tilted irradiance), but the frozenRadiationFamilyhas onlyghi/dni/dif.gtiis deliberately left unmapped and dropped.
SolcastResponse / SolcastRecord (volatile)
SolcastResponse { estimated_actuals: Array<SolcastRecord>?; forecasts: Array<SolcastRecord>? } — live/historic populate estimated_actuals, forecast populates forecasts. SolcastRecord carries period_end: String plus the measured quantities; fields that may arrive as int OR float (ghi, dni, dhi, gti, relative_humidity, wind_direction_10m, cloud_opacity) are any? and coerced.
OpenWeatherMapService
Connector for the OpenWeatherMap One Call 3.0 API (appid key required). Forecast-only: it consumes the hourly block of GET /data/3.0/onecall, which is always a forecast. Requests use units=metric (temperature °C, wind speed m/s, pressure hPa) and exclude=current,minutely,daily,alerts — every block except the consumed hourly is excluded so the response carries no dead payload.
Methods:
static mappings(): Array<VarMapping>— Declarative mapping from OWM keys to unified family/signal pairs. Wind speed/direction are decomposed to u/v inline as each record is streamed. OWMpopis a 0…1 fraction, scaled to a percentage via thefrac_to_pctconverter before being stored in Precipitation/probability (%). OWMsnow."1h"is mm, converted to cm via themm_to_cmconverter before Snow/snowfall (cm).uvi→ Radiation/uv_index (index) andvisibility→ Cloud/visibility (metres — the stored unit) map directly, no conversion.static from_owm_code(id: int, is_day: bool): WeatherCondition— Map an OWM condition code (weather[0].id) to the canonicalWeatherCondition— the OWM analogue ofConditions::from_wmo/from_vc_icon/from_pictocode: 2xx → thunder; 3xx (drizzle) and 500–504 → rain; 511 (freezing rain) → sleet; 520–531 → showers; 600–602 and 620–622 → snow; 611–613 and 615/616 (“rain and snow” — a rain+snow mix reads as sleet, matching the pictocode-11 convention) → sleet; 771 (squall) and 781 (tornado) → wind; remaining 701–762 (obscuration) → fog; 800 → clear_day/clear_night and 801/802 → partly_cloudy_* viais_day; 803/804 and unknown codes → cloudy. No connector auto-feedsConditionFamily— record conditions yourself viastation->conditions()->set(t, OpenWeatherMapService::from_owm_code(id, is_day)).- Importer streams internally (no exposed pure converter). Each
hourlyentry is streamed:dt(epoch seconds) →time::new(dt, DurationUnit::seconds);wind_speed(m/s, no conversion) +wind_degdecomposed intou_10m/v_10m;wind_gust→gust; integer columns coerced to float; the nestedrain/snowobjects (keyed by"1h") flattened intorain_1h/snow_1h. Issue time hoisted once for the whole batch. static build_onecall_url(base_url: String?, lat: float, lng: float, api_key: String): String— Pure request builder (no IO): the One Call URL withunits=metricandexclude=current,minutely,daily,alertspinned (base_urlnull ⇒https://api.openweathermap.org; the key is URL-encoded).static feed_from_openweathermap(station: node<WeatherStation>, api_key: String, base_url: String?)— Full pipeline: reads lat/lng from the station, pulls thebuild_onecall_urlquery, and streams thehourlyblock into the station’s signals viaIngestFeeder(alwayshistorical: false). Throws on a non-200 response — and the thrown message never echoes the query, so theappidapi key is not leaked. A 200 response with an empty/unparseable body also throws rather than silently doing nothing (the sharedIngest::require_bodyguard). Whole-number JSON fields (pressure/humidity/clouds/wind_deg, etc.) are coerced consistently from int to float. Observed history requires the separate/data/3.0/onecall/timemachineendpoint, out of scope here.
frac_to_pct(v: float): float
Free function. Scales an OWM pop probability fraction (0…1) to a percentage (v * 100). Wired as the convert for the pop mapping.
mm_to_cm(v: float): float
Free function. Converts OWM hourly snow (snow."1h", mm) to centimetres (v / 10). Wired as the convert for the snow_1h mapping.
OwmResponse / OwmHour / OwmWeather (volatile)
OwmResponse { hourly: Array<OwmHour>? } (only hourly is consumed, always treated as a forecast). OwmHour carries dt: int (epoch seconds); every other scalar numeric field (temp, feels_like, dew_point, pressure, humidity, clouds, uvi, visibility, wind_deg, wind_gust, pop) is any? — OWM sends whole numbers as JSON integers, so they are coerced via Ingest::coerce_float — except wind_speed: float?; rain/snow are Map<String, any?>? keyed by "1h"; weather: Array<OwmWeather>? carries the categorical condition codes. OwmWeather { id: int } — one entry of an hour’s weather array; only the OWM condition code is consumed, mapped via from_owm_code.
WeatherDotComService
Connector for the Weather.com / Wunderground PWS (Personal Weather Station) API — the data source behind a Wunderground member’s own stations. Three capabilities behind one mapping table: (1) a live API path (nearby-station discovery, current observations, dated history), (2) a bounding-box discovery + import built on top of it, and (3) archive CSV import/export of the on-disk <ID>/<ID>_YYYYMMDD.csv files the legacy Python downloader produced. Pure GCL — no native code.
Auth is a single apiKey=<key> query param (issued to Wunderground members who run a registered PWS). Every request pins format=json&units=m&numericPrecision=decimal — numericPrecision=decimal is mandatory: the default rounds temperatures/wind to integers and silently degrades the data. Unit-independent fields (solarRadiation W/m², uv, humidity %, winddir°) sit at the top level; unit-dependent fields (temp °C, windSpeed/windGust km/h, pressure hPa, precipTotal mm, dewpt) live under a nested metric object. The API returns two record shapes — instantaneous (observations/current, observations/all/1day) and High/Low/Avg aggregate (history/*, observations/hourly/7day, and the CSV archive) — both normalized into the same canonical keys before mapping.
Field mapping (mappings()):
| canonical key | family / signal | notes |
|---|---|---|
solarRadiation (*High in aggregate) |
Radiation / ghi |
W/m², no conversion |
uv (uvHigh) |
Radiation / uv_index |
index |
temp (tempAvg) |
Temperature / air_2m |
°C |
dewpt (dewptAvg) |
Temperature / dew_point |
°C |
heatIndex (heatindexAvg) |
Temperature / apparent |
°C |
humidity (humidityAvg) |
Humidity / relative |
% |
windSpeed+winddir (windspeedAvg+winddirAvg) |
Wind / u_10m,v_10m |
km/h→m/s (÷3.6), decomposed to u/v |
windGust (windgustHigh) |
Wind / gust |
km/h→m/s |
pressure (pressureMax) |
Pressure / msl |
hPa |
precipTotal |
Precipitation / total |
mm |
Methods:
The public surface is WeatherDotComProvider (a WeatherProvider — see Provider API above); the WeatherDotComService statics below are the connector internals it composes (live HTTP, the bbox grid/quadtree geometry helpers, the CSV archive importers). Discovery now lives on WeatherDotComDiscovery (see Station discovery below); the archive-exact export lives on the provider as WeatherDotComProvider::export_archive (built on the neutral WeatherExport — see WeatherExport below).
WeatherDotComProvider (the instance API):
static new(api_key: String?): WeatherDotComProvider— one configured PWS connection.base_url(null ⇒https://api.weather.com) andhistory_granularity(null ⇒hourly) are plain fields; set them via the record literalWeatherDotComProvider { api_key: ..., base_url: ..., history_granularity: PwsHistoryGranularity::daily }.id(): String→"weatherdotcom";capabilities(): ProviderCaps→{ current: true, history: true, forecast: false, discovery: true }.feed_current(station)— pulls the station’s trailing-24 h rapid observations and streams them as observed. Requiresstation->id. Threads this connection’sapi_key/base_url. An idle station (HTTP 204 ‘no data’ — the documented PWS response for an offline station) is skipped with a warning, not an error.feed_history(station, from, to)— backfills[from,to]from/v2/pws/history/{hourly,daily}(chunked into ≤31-day windows) at this provider’shistory_granularity(default hourly). Requiresstation->id. Chunk dates are formatted in the station’s timezone (station.timezone, UTC fallback) because the history endpoints interpretstartDate/endDateas the station’s local apparent day — formatting in UTC clipped up to ~half a day at the range edges for stations far from UTC. A 204 ‘no data’ chunk is warned and skipped (the remaining chunks continue).feed_forecast(station)— unsupported (PWS has no forecast product); throwsweatherdotcom: feed_forecast unsupported.require_key(): String— the configured api key, or throwsweatherdotcom: api_key required(a loud, non-secret failure used by every call).station_discovery(): StationDiscovery?— the default discoverer for this connection: aWeatherDotComDiscoverywith a 5 km uniform grid. For customstep_km/delay/max_tiles, build aWeatherDotComDiscoverydirectly and pass it toimport_bbox.import_bbox(disc: WeatherDotComDiscovery, sw: geo, ne: geo, mode: PwsImportMode, from: time?, to: time?): Array<node<WeatherStation>>— discover every PWS in[sw,ne]viadisc.register_bbox(register asWeatherStationnodes), then ingest each permode:discover_onlyreturns them untouched,currentpulls trailing-24 h obs,historybackfills[from,to]athistory_granularity. History args and the provider key are validated up front — before the slow, throttled discovery sweep spends its rate budget — except fordiscover_only, which needs no provider key (the discoverer carries its own). The discoverer’sdelayis passed through to the per-station ingest phase too (the most rate-limit-sensitive one). Replaces the old 13-paramimport_bbox/import_bbox_adaptivestatics.export_archive(station: node<WeatherStation>, dir: String, from: time?, to: time?)— serializes the station’s stored signals back to the archive-exact layout<dir>/<ID>/<ID>_YYYYMMDD.csv, one row per observed instant in[from,to](null bound ⇒ unbounded), grouped into files by the station-local calendar day (station.timezone ?? UTC).High=Low=Avgare written from the single stored value;winddirAvg/windspeed*are recomposed from the u/v components viaIngest::recompose_windandwindgust*from the gust signal; columns the model does not store (windchill*,precipRate,pressureTrend) are left blank;qcStatusis a constant1. Requiresstation->id(throws otherwise). Reuses the neutralWeatherExportscaffolding (observed_union/fmt_cell/read_signal_at); the output is byte-for-byte identical to the formerWeatherDotComService::export_csv_archive. (api_key/base_urlare unused — export is offline.)
WeatherDotComService connector internals (statics):
static mappings(): Array<VarMapping>— the canonical-key → family/signal table above, shared by the live API, the bbox importer, and the CSV archive importer. Wind speed/direction are decomposed intou_10m/v_10min the scatter, so only the components appear here.static near(location: geo, api_key: String, base_url: String?): Array<DiscoveredStation>— oneGET /v3/location/near?geocode=LAT,LON&product=pwscall → up to 10 nearest PWS, de-parallelized from the response’s parallel-array shape into clean records.base_urldefaults tohttps://api.weather.com. (Wrapped byWeatherDotComDiscovery.)static feed_current(station, api_key, base_url?): bool/static feed_history(station, from, to, granularity, api_key, base_url?): bool— the HTTP workers the provider’sfeed_current/feed_historythread credentials into; requirestation->id(throw otherwise), andfeed_historythrows iffrom > to. Both returntruewhen data was ingested andfalseon an HTTP 204 no-data skip (feed_history: false when every chunk was empty), so batch callers can count skips without try/catch; any other non-200, or a 200 with an unparseable body, still fails loudly.static history_chunk_date(t: time, tz: TimeZone?): String— one history-chunk bound as aYYYYMMDDcalendar day in the station’s timezone (tznull ⇒ UTC fallback) — the PWS history endpoints readstartDate/endDateas the station-local apparent day.static redact_key(msg: String): String— strip an embeddedapiKey=<value>from a message before logging (transport errors may echo the full request URL; no warn/error in this lib may ever carry an API key).static require_history_args(mode, from, to)/static ingest_discovered(stations, api_key, base_url, delay, mode, granularity, from, to)— the validation + per-station ingest the provider’simport_bboxcomposes after discovery.ingest_discoveredis batch-isolated (the sweep before it was paid — it never aborts mid-way): a station with no data (204) counts as skipped; a station whose feed throws is warned (api key redacted viaredact_key), counted as failed, and the batch continues. Anok/skipped/failedsummary is logged at the end; it throws only when every station failed.static feed_from_csv_archive(path: String)— imports the on-disk archive into the station/signal model. The path kind is auto-dispatched: a single.csvfile is imported directly; a directory is walked recursively for*.csv(so both a single<ID>/station folder and the whole archive root work). The station id is parsed from each filename (<ID>_<YYYYMMDD>.csv), the location from each file’s firstlat/lon, and each row is scattered as an observed aggregate point (reusing the live-historyscatter). Idempotent (per-timestampset_observeddedup).static archive_columns(): Array<String>/static join_semicolons(cells)/static archive_row_to_obs(row, index_of)— archive-layout helpers reused by both the importer and the provider’sexport_archive: the canonical 33-column header, the;-join, and the row → synthetic-aggregate-observation adapter.- The archive export now lives on the provider as
WeatherDotComProvider::export_archive(station, dir, from, to)(see Provider API above), reusing the neutralWeatherExportscaffolding (observed_union/fmt_cell/read_signal_at) andIngest::recompose_wind. The oldWeatherDotComService::export_csv_archivestatic has been removed. - Pure helpers (no IO — exposed for testing):
near_decode(body),grid_points(sw,ne,step_km)withtile_box(pts, box, step_km)/grid_axis(v0, v1, step)(the bbox geometry used byWeatherDotComDiscovery: corners go throughspatial::GeoBox, so an antimeridian-crossing box is tiled as its two non-crossing lobes instead of the 340° complement; each axis is SW-anchored with ceil semantics — the last row/column lands exactly on the north/east boundary, so the NE edge is never left up to a full step uncovered; throws whenstep_km <= 0),in_bbox(g,lat0,lat1,lng0,lng1)(axis-aligned only — for crossing boxes usespatial::GeoBox::contains),build_near_url/build_obs_url/build_history_url,obs_values(obs, aggregate),obs_time(obs),scatter(body, feeder, aggregate)with the per-observationscatter_obs(obs, feeder, aggregate). - Quadtree helpers are reserved:
quad_split/should_subdivide/box_smaller_side_km/box_half_diagonal_km(overPwsBox) have no production caller yet —within_bboxis currently a uniform grid sweep. They are intentionally kept (with their truth-table tests) for the planned adaptive-densification mode that subdivides only saturated/nearcells; do not treat them as dead code. - Archive/export internals:
feed_csv_file_into(path, file_name, feeders)(one archive file into the per-station feeder map),archive_signals(station): PwsArchiveSignals(the 11 mapped archive signal nodes resolved once per export pass —PwsArchiveSignalsis a@volatileholder of 11 nullable signal-node slots; a null slot = family inactive / signal absent, its cells stay blank),archive_signal_list(sigs)(the non-null slots in canonical archive order), andexport_signals(station)(retained as a thin wrapper over the two).
/nearis a point query, not a bbox query. It returns the ≤10 nearest stations to a single geocode, so “every station in a box” requires tiling + union + clip —WeatherDotComDiscovery::within_bboxdoes this over a uniformstep_kmgrid (thecos(lat)longitude correction included). Choosestep_kmsmall enough that no cluster of >10 stations sits between grid nodes.
Rate limits. A free PWS/contributor key allows ~30 calls/min and ~1500 calls/day. A 5 km grid over a country is hundreds of
/nearcalls, andimport_bboxadds one (current) or⌈days/31⌉(history) calls per station — use the discoverer’sdelay/max_tiles, cache discovery, and prefer wide-window history. Read429at runtime.
qcStatus policy. Records are dropped only on an explicit QC failure (
qcStatus == 0).qcStatus == -1(“not checked” — the bulk of historical PWS data and the entire pre-2020 CSV archive) andqcStatus == 1(“passed”) are kept — dropping< 1would silently discard nearly the whole archive.
Export is functional, not byte-identical. Because the lib stores one value per
(signal, time), export writesHigh=Low=Avgfrom that value and re-formats numbers — it round-trips the data (re-importing an export reproduces the signals) but won’t reproduce the rare rows where the originalHigh/Low/Avgdiverged, legacy number formatting, or the unstored columns.
Example — discover + import a Lebanon bbox, then archive round-trip:
// 0. One configured PWS connection (history_granularity defaults to hourly).
var p = WeatherDotComProvider::new(my_api_key);
// 1. Per-station feeds (station->id required).
p.feed_current(station); // trailing-24 h observations
p.feed_history(station, from, to); // backfill at history_granularity
// 2. Discover every PWS in a Lebanon bbox and pull the last 24 h of observations.
// Tiling knobs (step_km / delay / max_tiles) live on the discoverer.
var sw = geo { 33.05, 35.10 };
var ne = geo { 34.69, 36.62 };
var disc = WeatherDotComDiscovery { api_key: my_api_key, step_km: 5.0, delay: 2_s };
var stations = p.import_bbox(disc, sw, ne, PwsImportMode::current, null, null);
println("registered ${stations.size()} PWS stations");
// 3. Bulk-import the historical CSV archive (a station folder OR the whole root).
WeatherDotComService::feed_from_csv_archive("./data/archive/weather");
// 4. Export one station back to the identical on-disk archive format
// (`export_archive` is an instance method on the provider).
var st = weather_station_by_id.get("IBEIRU11");
if (st != null) {
p.export_archive(st!!, "./export/weather", null, null);
}
DiscoveredStation (volatile)
One station from /v3/location/near, de-parallelized: station_id: String, name: String?, location: geo, distance_km: float?, qc_status: int? (-1 not-checked / 0 fail / 1 pass), update_time_utc: time? (last-obs time), probe: Map<String, float>? (reserved for a future data-aware discovery filter; currently unset). WeatherDotComDiscovery::hit_of maps this to a provider-agnostic StationHit.
PwsNearResponse / PwsNearLocation / PwsObsResponse (volatile)
Internal deserialization shapes. PwsNearResponse { location: PwsNearLocation? }; PwsNearLocation holds the parallel arrays stationId[], stationName[], latitude[], longitude[], distanceKm[], qcStatus[], updateTimeUtc[] (index i is the same station across all arrays). PwsObsResponse { observations: Array<Map<String, any?>>? } — each observation is free-shaped (values arrive as int or float, coerced via Ingest::coerce_float; the nested metric object is read with an as cast).
PwsHistoryGranularity / PwsImportMode (enums)
PwsHistoryGranularity { hourly, daily } is the WeatherDotComProvider.history_granularity field — it selects the feed_history endpoint (hourly observations vs daily summaries). PwsImportMode { discover_only, current, history } selects what WeatherDotComProvider::import_bbox does after discovery (register only / pull current obs / backfill history).
Station discovery
A provider-agnostic station-discovery surface that turns a point or a bounding box into StationHits and (optionally) materializes them into the station graph. WeatherDotComDiscovery wraps the Weather.com PWS connector (the point /near endpoint plus the grid geometry helpers); MeteostatDiscovery composes the Meteostat RapidAPI /stations/nearby + /stations/meta endpoints. The pure mappers (hit_of, collect_near_hits, collect_hits, nearby_candidates, merge_candidates, hit_from_meta) and pure builders (build_nearby_url, lobe_center, lobe_radius_m) carry no IO, so they unit-test against canned data with no network. All bbox geometry goes through the shared spatial::GeoBox (antimeridian-aware — see Spatial).
StationHit (volatile)
One discovered station, provider-agnostic: id: String?, name: String? (either may be null when the provider omits it), location: geo (always present), distance_m: float? (great-circle distance in metres from the query point for point queries; null for bbox sweeps).
StationDiscovery (abstract)
The discovery contract. Two provider-specific network paths plus one shared registration convenience.
max contract (binding for every implementation): null = uncapped (provider default); max <= 0 = empty result. An explicit zero/negative cap caps to nothing — it is never an “uncapped” sentinel — and implementations short-circuit on it before any network call.
bbox corner contract: [sw,ne] boxes follow spatial::GeoBox semantics — latitudes are order-agnostic (south = min, north = max); longitudes are taken as given, the box spanning sw.lng() going east to ne.lng(), so sw.lng() > ne.lng() means an antimeridian-crossing box (not a mistake to auto-correct).
abstract near(point: geo, max: int): Array<StationHit>— the stations nearestpoint, capped atmax(max <= 0⇒ empty), nearest-first when the provider reports distances.abstract within_bbox(sw: geo, ne: geo, max: int?): Array<StationHit>— every station the provider knows inside[sw,ne], capped atmaxper the contract above.register_bbox(sw: geo, ne: geo, max: int?): Array<node<WeatherStation>>— runswithin_bbox, then registers each hit into the station graph viaWeatherStation::get_or_create(location, id)(id-keyed, so re-runs are idempotent); a hitnameis copied onto a freshly created station only when the station has no name yet. Returns the registered nodes in discovery order.
WeatherDotComDiscovery
Weather.com / Wunderground PWS discoverer. Fields: api_key: String (PWS key), base_url: String? (defaults to https://api.weather.com), step_km: float (uniform grid spacing for within_bbox), delay: duration? (rate-limit between /near calls), max_tiles: int? (caps grid nodes; loud warn on truncation, no silent cap).
static hit_of(ds: DiscoveredStation): StationHit— pure map: aDiscoveredStation(km distance) to aStationHit(metre distance, km×1000).static collect_near_hits(found: Array<DiscoveredStation>, max: int): Array<StationHit>— pure: map a decoded/nearcandidate list to hits, capped atmax(max <= 0⇒ empty, per theStationDiscoverycontract);near_decodealready orders nearest-first.static collect_hits(sw: geo, ne: geo, tiles: Array<Array<DiscoveredStation>>): Array<StationHit>— pure: union+dedup bystation_idacross the already-fetched per-tile candidate lists, clip each survivor to the[sw,ne]spatial::GeoBox(the nearest-N/nearbleeds past the box edges; the box may cross the antimeridian), emit oneStationHitper survivor withdistance_m = null(bbox sweeps carry no caller-meaningful distance). No network —within_bboxfetches the tiles and passes them here.near(point: geo, max: int): Array<StationHit>— one/v3/location/nearpoint query (≤10 nearest PWS), mapped + capped (max <= 0⇒ empty with no network call).within_bbox(sw: geo, ne: geo, max: int?): Array<StationHit>— a uniformstep_kmgrid sweep (one/nearper node; an antimeridian-crossing box is swept one non-crossingspatial::GeoBoxlobe at a time), deduped+clipped bycollect_hits;max_tilescaps grid nodes,delayrate-limits,maxtruncates the final hit list. Throws whenstep_km <= 0(a zero/negative step would tile forever). A failed tile is warned and skipped — the already-fetched (rate-limited, paid) tiles are kept and the sweep continues, an end-of-sweepok/failedsummary is logged, and the call throws only when every tile failed.
MeteostatDiscovery
Meteostat (RapidAPI) station discoverer over the two real /stations endpoints (/stations/nearby returns id + localized name + distance only — no coordinates; /stations/meta?id= returns the full record with the nested location). There is no server-side /stations/bounds query, so coordinates are always resolved with a /stations/meta follow-up per candidate: near issues 1 + N HTTP requests (one nearby + one meta per candidate), and within_bbox is emulated — one covering /stations/nearby probe per non-crossing spatial::GeoBox lobe (lobe centre + circumradius, so 1–2 probes), candidates deduped by id across lobes, one meta per unique candidate, then clipped to the box.
Fields: rapidapi_key: String, base_url: String? (defaults to https://meteostat.p.rapidapi.com), radius_m: int (bounds the near search, metres), nearby_limit: int? (per-lobe bbox probe limit, default 100; throws if explicitly <= 0; a saturated probe warns about possible truncation — raise nearby_limit or shrink the box). The RapidAPI key travels in the request headers, never in a URL.
static pick_name(names: Map<String, String?>?): String?— pure: a display name from the localized{ "<lang>": name }object —enwhen present, else the first non-null localization.static nearby_candidates(resp: MeteostatNearbyResponse?, max: int?): Array<MeteostatNearbyRecord>— pure: the usable candidates of a nearby response (non-nullidonly — an id-less record cannot be meta-resolved), response order (nearest-first), capped atmaxper theStationDiscoverycontract.static merge_candidates(batches: Array<Array<MeteostatNearbyRecord>>): Array<MeteostatNearbyRecord>— pure: union the per-lobe batches, deduplicating by station id (first occurrence wins).static hit_from_meta(rec: MeteostatNearbyRecord, meta: MeteostatMetaRecord?, keep_distance: bool): StationHit?— pure: combine a nearby candidate with its meta record into aStationHit, or null when the meta record carries no usable coordinates;keep_distancekeeps the nearbydistanceon point queries (bbox sweeps pass false sodistance_mstays null).build_nearby_url(point: geo, limit: int, radius_m: int): String— pure builder for/stations/nearby?lat&lon&limit&radius.static lobe_center(lobe: spatial::GeoBox): geo/static lobe_radius_m(lobe: spatial::GeoBox): int— pure: a non-crossing lobe’s centre and a covering nearby-probe radius (larger centre→corner distance + 5% margin).fetch_meta(id: String): MeteostatMetaRecord?— one/stations/meta?id=lookup (throws on non-200 / unparseable body; the batch loop catches per item).locate_hits(cands, clip, max)— the shared coordinate-resolution loop (batch semantics): one meta call per candidate; a failed or coordinate-less item is warned and skipped, a summary is logged, and it throws only when every attempted lookup failed outright.near(point: geo, max: int): Array<StationHit>— themaxnearest stations: one/stations/nearbycall + one/stations/metaper candidate (max <= 0⇒ empty with no network call);distance_mkept from the nearby response.within_bbox(sw: geo, ne: geo, max: int?): Array<StationHit>— the emulated bbox sweep described above, truncated tomax; bbox hits carrydistance_m = null.
Meteostat discovery response types (volatile)
Pinned to the real RapidAPI shapes: MeteostatNearbyResponse { data: Array<MeteostatNearbyRecord>? } with MeteostatNearbyRecord { id: String?, name: Map<String, String?>?, distance: float? } (metres; no coordinates in nearby responses), and MeteostatMetaResponse { data: MeteostatMetaRecord? } (a single record, not an array) with MeteostatMetaRecord { id, name, location: MeteostatStationLocation?, timezone } and MeteostatStationLocation { latitude: float?, longitude: float?, elevation: int? }. All fields are nullable so a partial record degrades gracefully. (These replace the former flat MeteostatStationRecord/MeteostatStationsResponse, whose flat shape never matched the live nested JSON.)
CamsAdapter
Offline adapter for Copernicus climate data: the CDS reanalyses (ERA5 / ERA5-Land) and the ADS CAMS Radiation Service (McClear / CAMS-rad). There is no live synchronous connector — these datasets are NetCDF/GRIB files served through the Copernicus CDS (cds.climate.copernicus.eu) and ADS (ads.atmosphere.copernicus.eu) async download queue (the cdsapi Python client). This adapter ships only the column→signal mapping tables, a pure CAMS-radiation row converter (the unit-test boundary), and a streaming CSV-file feed that reuses the CsvImport / IngestFeeder seam.
Offline recipe (cdsapi → CSV → feed):
- Request + download the dataset via
cdsapi(the request is queued, then the file is downloaded once ready) — e.g. an ERA5-Land or CAMS-radiation NetCDF/GRIB. - Extract it to CSV with a NetCDF tool (
cdo,xarray, orncdump). - Feed the CSV: for the CAMS Radiation Service use
CamsAdapter::feed_cams_radiation_csv(station, path, header_lines); for ERA5-Land use the genericCsvImport::feed(...)withCamsAdapter::era5_land_mappings().
Native alternative (skip the CSV step): the downloaded granules are NetCDF-4/HDF5, so for MERRA-2 / ERA5 / ERA5-Land you can read them directly with
NetcdfImport::feed(...)instead of extracting to CSV withcdo/xarray/ncdump. Theconvertfunctions and family routing are identical (both funnel through theIngestFeederstreaming seam); useNetcdfImport::era5_mappings()(this table adapted toArray<VarMapping>) for ERA5 / ERA5-Land.
GLOBE vs ESA clarification: “GLOBE” refers to the NASA/NSF GLOBE Program citizen-science observation network (globe.gov) — distinct from ESA. ESA-origin climate data reaches this library via Copernicus CDS (ERA5 reanalysis) and ESA CCI / Sentinel satellite products (NetCDF, offline only); all of these are file-based and ingested through this same offline CSV recipe.
Methods:
static cams_radiation_mappings(): Array<CsvColumnMapping>— Column→signal table for the CAMS Radiation Service CSV (the;-separated McClear/CAMS-rad product):GHI→Radiation/ghi,BNI(beam-normal irradiation = DNI)→Radiation/dni,DHI(diffuse horizontal)→Radiation/dif, all W/m².static era5_land_mappings(): Array<CsvColumnMapping>— Example table for a user-extracted ERA5-Land CSV using the common ERA5 short names (adjust to your export’s column names). See the table below.static cams_normalize_header(header: Array<String>): Array<String>— Normalize a parsed CAMS header row: strips a leading'#'(plus surrounding whitespace) from every cell, so the verbatim CAMS comment-header line ("# Observation period;TOA;...;GHI;...") indexes cleanly by column name.static cams_period_to_interval(period_cell: String): Tuple<time, time>?— Parse a CAMSObservation periodcell ("<start>/<end>") to its(start, end)instants: both sides sliced to 19 chars (dropping fractional seconds) and parsed with"%Y-%m-%dT%H:%M:%S"as UTC. Returnsnullwhen either side is missing, short, or unparseable.static cams_period_to_time(period_cell: String): time?— The interval’s start instant (the interval-start stamping convention). Delegates tocams_period_to_interval, so it now requires both<start>/<end>sides — a lone instant returnsnull.static cams_radiation_rows_to_points(header: Array<String>, rows: Array<Array<String>>, historical: bool): Array<WeatherPoint>— Internal pure converter (network/file-free; the tested boundary, over small canned rows). The header is normalized first (leading'#'stripped viacams_normalize_header), so a verbatim CAMS comment-header works. Each row’sObservation periodinterval is parsed viacams_period_to_interval: the start instant stamps the point, and each GHI/BNI/DHI Wh/m² cell is divided by the period length in hours to store a mean W/m² for every CAMS product step (1-min/15-min/hourly/daily/monthly — identity for hourly).valuesis built via the non-throwingCsvImport::parse_cell(integer cells coerced to real floats, blank/sentinel/unparseable cells skipped); oneWeatherPointper row. Rows with a malformed period side, a zero/negative period length, or ragged rows shorter than the period column are skipped (not fatal). The productionfeed_cams_radiation_csvpath streams instead of building this array.static feed_cams_radiation_csv(station: node<WeatherStation>, path: String, header_lines: int?)— Full pipeline: streams the;-separated CSV atpathrow-by-row into the station’s signals viaIngestFeeder(Map-free, onefeed_valueper GHI/BNI/DHI cell), usingCsvImport::to_var_mappings(cams_radiation_mappings()). The feed always writes observed history (CAMS Radiation is a historical product). Withheader_linesnull the header is auto-detected: the#preamble is scanned until the row whose first cell normalizes to exactlyObservation period(the verbatim"# Observation period;TOA;..."comment-header line), which becomes the header; passheader_lines(preamble lines + the 1 column-name line) only as an override for a non-standard export. Each row’s Wh/m² cells are normalized to a mean W/m² (÷ period hours, per row) exactly likecams_radiation_rows_to_points.
ERA5-Land example mapping (era5_land_mappings()):
| column | family / signal | unit | converter |
|---|---|---|---|
t2m |
Temperature / air_2m | °C | kelvin_to_celsius (K→°C) |
d2m |
Temperature / dew_point | °C | kelvin_to_celsius (K→°C) |
u10 |
Wind / u_10m | m/s | none — ERA5 provides u/v directly (no decomposition) |
v10 |
Wind / v_10m | m/s | none — direct component |
sp |
Pressure / surface | hPa | pa_to_hpa (Pa→hPa) |
tp |
Precipitation / total | mm | m_to_mm (m→mm) |
ssrd |
Radiation / ghi | W/m² | none — see SSRD caveat |
Unit caveats:
- K → °C (ERA5
t2m/d2m): subtract 273.15 (kelvin_to_celsius). - Pa → hPa (ERA5
sp): divide by 100 (pa_to_hpa). - m → mm (ERA5
tp, metres of water equivalent): multiply by 1000 (m_to_mm). - CAMS irradiation is Wh/m² over the period, not instantaneous W/m². Both
cams_radiation_rows_to_pointsandfeed_cams_radiation_csvnormalize it automatically per row: each GHI/BNI/DHI cell is divided by the row’s observation-period length in hours (identity for the hourly product), so the stored series is a mean W/m² for every CAMS step. Rows with a zero/negative or malformed period are skipped. - Precipitation accumulation stamping differs per provider (no automatic re-stamp is applied — unlike radiation, the library declares no accumulation-stamping convention): Visual Crossing’s
precipis the amount falling in the hour starting at the stamp, while Open-Meteo (precipitation), OpenWeatherMap (rain."1h"), Meteostat (prcp) and NOAA (precipitationLastHour) are accumulations for the hour ending at the stamp. Mixing two of these into one station’stotalsignal offsets hourly totals by 1 h depending on the source — prefer a single precipitation provider per station. - ERA5
ssrdis J/m² accumulated over the model step. To recover a mean W/m² divide by the accumulation period in seconds (÷3600 for the hourly product). No converter is wired because the divisor depends on the export’s accumulation step — pre-scale during extraction or supply your ownconvert.
kelvin_to_celsius(v: float): float · pa_to_hpa(v: float): float · m_to_mm(v: float): float
Free functions wired as convert functions in era5_land_mappings(): Kelvin → Celsius (v - 273.15), Pascal → hectopascal (v / 100), and metre → millimetre (v * 1000).
NetcdfImport
Native NetCDF-4 / HDF5 importer for gridded reanalysis granules — MERRA-2 (GES DISC), ERA5, and ERA5-Land. It reads the granule directly, the native alternative to the CamsAdapter offline “download → extract to CSV → feed” recipe. This is the weather library’s only native © component; the .gclib statically links netcdf-c + HDF5.
Scope (v1): local files only (no OPeNDAP / Earthdata streaming), NetCDF-4 / HDF5 only (not classic NetCDF-3 or GRIB), nearest grid cell (no interpolation), and standard gregorian / proleptic-gregorian CF calendars only (noleap, 360_day, … are rejected with an error).
Methods:
static native read_series_columnar_points(path: String, var_names: Array<String>, lat_var: String, lon_var: String, time_var: String, lats: Array<float>, lons: Array<float>): Array<NetcdfColumns>— The batched native core every other entry funnels through: openspathonce, finds the grid cell nearest each(lats[p], lons[p])point, and for each name invar_namesreads that point’s single cell across the whole time axis — never the full cube. Batching matters because all file work (open, HDF5 metadata parse, per-variable attribute resolution, chunk decompression) is serialized process-wide behind the non-thread-safe HDF5 library, and nearby points share the same compressed chunks — N single-point calls repeat that work N times.lats/lonsmust be non-empty, equal-sized, and finite (a NaN target would silently match cell 0 — rejected loudly instead). An extra (non time/lat/lon) dimension is pinned to its first index — with a warn log when its size > 1 — except a legacy-CDSexpverdimension (case-insensitive), whose indices are coalesced per timestep to the first non-fill value (ECMWF’s documented ERA5/ERA5T merge), so granules spanning the final/preliminary boundary keep their recent data. Applies CF unpacking (real = raw · scale_factor + add_offset) and CF time decoding (the time variable’sunits="<seconds|minutes|hours|days> since <base>"; the seconds field may be omitted, and a udunits UTC-offset suffix is honored)._FillValue/missing_value(and a variable absent from the granule) becomeNaN, dropped on the GCL side; a variable declaring neither attribute has the NetCDF default fill for its type (e.g.9.96921e36for floats) treated as missing (resolved vianc_inq_var_fill;NC_NOFILLvariables get no sentinel). A packing/sentinel attribute that is present but not readable as a number (e.g. a textscale_factor) raises an error rather than silently importing unscaled values. Returns oneNetcdfColumnsper point, aligned withlats/lons; all entries share the sameaxisArray object (identical for every point by construction) — treat it as read-only. The native side allocates onlyO(npoints · nvars · ntime)primitivei64/f64— no per-timestepWeatherPoint/Map/key-string churn.static read_series_columnar(path: String, var_names: Array<String>, lat_var: String, lon_var: String, time_var: String, lat: float, lon: float): NetcdfColumns— Single-point convenience overread_series_columnar_points(same contract, one target, oneNetcdfColumns). Prefer the batched form (orfeed_many) when importing several stations from the same granule: each call re-opens the file and repeats the serialized HDF5 work.static feed(station: node<WeatherStation>, path: String, mappings: Array<VarMapping>, lat_var: String, lon_var: String, time_var: String, historical: bool)— Full pipeline (signature unchanged): readsstation’s geo, takes the nearest cell, and streams the per-variable columns into the station’s family signals viaIngestFeeder— one value at a time, no intermediateArray<WeatherPoint>and (since MERRA-2/ERA5 supplyU10M/V10Mdirectly, so no wind decomposition) zero per-row Map. Whenhistoricalis false, one issue time (time::now()) is hoisted for the whole batch, like every other importer.mappingsprovider_keymust be the NetCDF variable name. Usemerra2_mappings()for MERRA-2 orera5_mappings()for ERA5 / ERA5-Land. Delegates tofeed_manywith a single station.static feed_many(stations: Array<node<WeatherStation>>, path: String, mappings: Array<VarMapping>, lat_var: String, lon_var: String, time_var: String, historical: bool)— Batchedfeed: reads all stations’ nearest cells in a single granule open (viaread_series_columnar_points), then streams each station’s columns through its ownIngestFeeder. Use this for fleet backfills over granule sets — per-station semantics are identical tofeed, but the serialized per-open HDF5 work is paid once instead of once per station.static merra2_mappings(): Array<VarMapping>— MERRA-2 single-level (e.g.M2T1NXSLV) variable→family table; see below.static era5_mappings(): Array<VarMapping>— ERA5 / ERA5-Land variable→family table: wrapsCsvImport::to_var_mappings(CamsAdapter::era5_land_mappings()), so eachprovider_keyis the ERA5 short name (t2m,d2m,u10,v10,sp,tp,ssrd). Use this — not the rawCamsAdapter::era5_land_mappings()table, which isArray<CsvColumnMapping>and does not type-check againstfeed(GCL generics are invariant). Thessrdaccumulated-J/m² caveat applies unchanged.
NetcdfColumns (@volatile): a transient native-build result holding axis: Array<time> (the shared timestep axis) and columns: Array<Array<float>> (one float column per requested variable; column v aligns with var_names[v], row i with axis[i]; a cell is NaN for a fill/missing value or an absent variable). Results from read_series_columnar_points share one axis object across all points — treat it as read-only.
MERRA-2 mapping (merra2_mappings()):
| variable | family / signal | unit | converter |
|---|---|---|---|
T2M |
Temperature / air_2m | °C | kelvin_to_celsius (K→°C) |
U10M |
Wind / u_10m | m/s | none — direct 10 m component |
V10M |
Wind / v_10m | m/s | none — direct 10 m component |
PS |
Pressure / surface | hPa | pa_to_hpa (Pa→hPa) |
PRECTOT |
Precipitation / total | mm | none — see flux caveat |
SWGDN |
Radiation / ghi | W/m² | none |
Notes & caveats:
- Grid → point is nearest-cell: the chosen cell can differ from the exact station coordinates, more so on coarse grids (MERRA-2 ≈ 0.5° × 0.625°). Longitude is matched with a circular distance, so a station in −180…180 resolves correctly against a 0…360 grid and vice-versa.
- MERRA-2
PRECTOTis a flux (kg m⁻² s⁻¹ ≡ mm s⁻¹). To store the family’s canonical mm per record multiply by the record interval in seconds (3600 for the hourly product). No converter is wired (the divisor depends on the granule’s step) — supply your own fixed-stepconvert, mirroring the ERA5ssrdcaveat above. - ERA5 / ERA5-Land: use
NetcdfImport::era5_mappings()as themappings(itsprovider_keys are the ERA5 short namest2m,sp,tp, …); the K→°C, Pa→hPa, m→mm converters and thessrdaccumulation caveat apply unchanged. Legacy-CDS granules with a size-2expverdimension are handled automatically (per-timestep coalesce, seeread_series_columnar_points). - Feed
.ncfiles from trusted sources only (GES DISC, the Copernicus CDS/ADS, your own pipeline). The byte-level parse surface for a granule is libnetcdf/libhdf5 — not this importer — and the importer’s own validation (allocation caps, dimension/coordinate sanity checks, loud failures on malformed metadata) cannot vouch for the underlying C libraries against an adversarial file. - Build: the weather
.gcliblinks staticlibnetcdf+libhdf5(+ zlib), built bydeps/netcdf.sh/deps/hdf5.sh. It is native-only (no WASM target).
Derived & analytics layer
Pure, on-demand helpers that compute over the base signals. None of them store
anything unless you write the result back into a WeatherSignal yourself.
Note: comfort indices and astronomy are implemented in pure GCL for v1 (matching the library’s “GCL helper first, native later” philosophy), so they are testable without a native rebuild. They are candidates for promotion to native
stdprimitives once proven.
Comfort (abstract)
Human thermal-comfort indices. Inputs use the library’s canonical units: temperature °C, wind m/s, relative humidity %.
static heat_index(temp_c: float, rh: float): float— NWS heat index (“feels like” in heat). Below 10 °C (50 °F) the index is undefined, so the air temperature is returned unchanged (mirroringwind_chilland NWS practice — previously the Steadman line was extrapolated below the air temperature). From 50 °F it uses the simple Steadman form while the simple (Steadman-averaged) value stays below 80 °F (~26.7 °C), then the Rothfusz regression (with low/high-humidity adjustments) — matching the NWS reference algorithm.static wind_chill(temp_c: float, wind_ms: float): float— NWS wind chill. Defined fortemp_c <= 10and wind> 4.8 km/h; returnstemp_cunchanged outside that domain.static apparent_temp(temp_c: float, rh: float, wind_ms: float): float— Australian apparent temperature (Steadman 1994), including the vapour-pressure and wind terms.static utci(temp_c: float, mrt_c: float, wind_ms: float, rh: float): float— Universal Thermal Climate Index, the operational 6th-order polynomial approximation (Bröde et al., 2012).mrt_cis the mean-radiant temperature; wind is clamped to[0.5, 17]m/s.static saturation_vapour_pressure(temp_c: float): float/static vapour_pressure(temp_c: float, rh: float): float— Magnus/Tetens-form helpers (hPa) with the Australian BOM over-water constant set (6.105 / 17.27 / 237.7 — the exact constants of the BOM apparent-temperature formula). The fit is for saturation over water (roughly 0…50 °C); below 0 °C it returns the supercooled-water value, which exceeds saturation over ice by up to ~10 % — do not use it as an over-ice e_s. Insideutciit substitutes for the reference implementation’s Hardy (1998) ITS-90 formulation, a deliberate simplification worth ~0.05 K of UTCI.
var hi = Comfort::heat_index(32.0, 70.0); // ~40.5 °C
var utci = Comfort::utci(0.0, 0.0, 5.0, 60.0); // ~ -14 °C (wind chill)
Astronomical (abstract)
Solar position, sunrise/sunset and moon phase via the NOAA algorithm (accurate to
~a degree / a minute). Independent of the solar library.
static sun_position(location: geo, t: time): SunPosition— Solarelevation/azimuth(degrees) — see SunPosition.static sunrise(location: geo, day: time): time?/static sunset(location: geo, day: time): time?— Sunrise/sunset instant for the UTC calendar day containingday— honored for every longitude (the neighbouring solar days are probed so the returned instant always falls inside the requested UTC day);nullwhen no such event occurs in that day. Null is not only a polar (day/night) phenomenon: consecutive events are spaced 1440 ± a few minutes apart in UTC, so at longitudes where the event falls near 00:00 UTC, the rare day on which it drifts across midnight contains no event at all — at perfectly ordinary mid-latitudes (e.g. 45°N 82.5°E, 2023-09-07 has no sunrise). Handlenullat every latitude. Note that for |longitude| > ~90° the requested UTC day’s sunset can precede its sunrise in UTC order (e.g. Tokyo on 2023-06-21: sunset ~10:00Z before sunrise ~19:26Z) — don’t assumesunrise < sunset.static moon_phase(t: time): float— Synodic phase fraction:0= new,0.5= full, approaching1= back to new.static moon_illumination(t: time): float— Illuminated fraction of the disk (0= new,1= full).static julian_day(t: time): float,static solar_geom(jd: float): Tuple<float, float>(declination °, equation-of-time min),static norm360(deg: float): float— building blocks.
WeatherStation exposes sun_position/sunrise/sunset bound to the station’s own location.
SunPosition
| field | type | description |
|---|---|---|
elevation |
float |
Degrees above the horizon (negative = below) |
azimuth |
float |
Degrees clockwise from true north |
Conditions (abstract)
Maps provider condition codes to the canonical WeatherCondition,
and derives one heuristically when a provider supplies none.
static from_wmo(wmo: int, is_day: bool): WeatherCondition— WMO weather code (Open-Meteo).is_daypicks the day/night variant for clear / partly-cloudy.static from_vc_icon(icon: String): WeatherCondition— Visual Crossing icon string.static from_pictocode(pictocode: int, is_day: bool): WeatherCondition— meteoblue daily pictocode (1…17), per meteoblue’s “Symbols and Pictograms” table; snow-shower and light variants collapse into their base condition, the snow+rain mix (code 11) reads as sleet (5=fog, 6=rain, 7=showers, 8=thunder, 9/10=snow, 11=sleet, 12/14/16=rain, 13/15/17=snow; unknown codes fall back to cloudy). Note: codes 5–17 previously mapped to wrong conditions (e.g. 5 returned cloudy, 6–8 fog, 12–13 thunder); storedConditionFamilycodes (Conditions::code/from_code) are unaffected, but data previously ingested viafrom_pictocodewas misclassified and needs re-ingestion to correct.- For OpenWeatherMap condition codes, see
OpenWeatherMapService::from_owm_code. static auto_detect(is_day: bool, temp_c: float?, cloud_pct: float?, precip_mm: float?, snow_cm: float?, wind_ms: float?, visibility_m: float?): WeatherCondition— heuristic fallback (the GCL port of solarleb’sautoDetectWeatherIcon). Priority: reported snow > precipitation > fog > strong wind > cloud-cover state. When the provider reports no snowfall series (snow_cm == null) but does report a temperature, precipitation phase is partitioned by surface air temperature (classical rain/snow climatologies: predominantly snow below ~0.5 °C, mixed phase/sleet up to ~2 °C, liquid above); passingtemp_c: nullreproduces the temperature-blind behavior exactly (all precipitation reads as rain/showers). Null inputs are treated as “not reported”. Prefer a real provider value; use this only when none is supplied.static code(c: WeatherCondition): int/static from_code(code: int): WeatherCondition— storage codec used byConditionFamily.codeis exhaustive over the current 13-member vocabulary and throws on an unmapped enum value ("Conditions::code: no storage code for <c>") — a newWeatherConditionmember must be wired into both before it can be persisted, instead of silently storing under a wrong code.from_codestill falls back tocloudyfor unknown stored codes.
ConditionFamily
Special “categorical” family: a coded weather-condition series over time
(nodeTime<int> of Conditions::code), so it is not a WeatherFamily and is
reached via WeatherStation::conditions() rather than signal_for.
static new(): ConditionFamilyset(t: time, c: WeatherCondition)— record the observed condition att.get(t: time): WeatherCondition?— exact-match lookup.resolve(t: time): WeatherCondition?— at-or-before lookup.latest(): WeatherCondition?— most recently recorded condition.
var s = WeatherStation::get_or_create(geo{49.6, 6.1}, "lux");
s->conditions()->set(time::now(), Conditions::from_wmo(61, true)); // rain
var now = s->conditions()->latest();
Skill (abstract)
Forecast verification: how forecast error grows with lead time.
static error_by_lead(signal: node<WeatherSignal>, from: time, to: time, lead_bucket: duration): Array<ForecastSkillScore>— For every(target, issued)forecast in[from, to]that has an exact observation attarget, computeerror = forecast - observedandlead = target - issued, and fold errors into per-lead-bucket Gaussians. Returns oneForecastSkillScoreper non-empty lead bucket, ascending by lead. Forecasts issued after their target (negative lead) are skipped.
ForecastSkillScore
| field | type | description |
|---|---|---|
lead |
duration |
Lower edge of the lead bucket (0, lead_bucket, 2·lead_bucket, …) |
count |
int |
Number of (target, issued) pairs in the bucket |
bias |
float |
Mean forecast - observed (positive = runs high) |
mae |
float |
Mean absolute error |
rmse |
float |
Root-mean-square error |
var sig = station->signal_for(TemperatureFamily, "air_2m");
var skill = Skill::error_by_lead(sig, time::min, time::max, 6_hour);
for (_, s in skill) {
println("lead ${s.lead.to(DurationUnit::hours)}h: rmse=${s.rmse} bias=${s.bias} n=${s.count}");
}
Spatial (abstract)
k-nearest, within-radius and inverse-distance-weighting over a nodeGeo index
(e.g. weather_station_by_geo). within bounds its work to the circle’s
bounding box (a nodeGeo Morton-range scan, O(range)); neighbors necessarily
visits the whole index (it returns every entry), and nearest/idw_weights
build on it with an O(n) scan — both flagged for native nodeGeo::nearest
promotion.
static neighbors<T>(index: nodeGeo<T>, point: geo): Array<GeoNeighbor<T>>— every entry paired with its distance topoint, ascending.static nearest<T>(index: nodeGeo<T>, point: geo, k: int): Array<GeoNeighbor<T>>— theknearest entries.static within<T>(index: nodeGeo<T>, area: GeoCircle, max: int): Array<GeoNeighbor<T>>— entries insidearea, nearest-first, capped atmax. Scans only the circle’s bounding box (Morton-range, post-filtered to exact circle membership), falling back to a full scan for the rare circle straddling the antimeridian.static idw_weights<T>(index: nodeGeo<T>, point: geo, k: int, power: float): Array<Tuple<geo, float>>— normalized inverse-distance weights (∝ 1 / distance^power, summing to 1) for theknearest neighbours. An exact coincidence (distance 0) takes the full weight. Compute once from geometry, reuse across every signal and timestamp.
GeoNeighbor (volatile)
| field | type | description |
|---|---|---|
location |
geo |
The indexed location |
value |
T |
The stored value at that location |
distance |
float |
Great-circle distance (metres) to the query point |
var knn = Spatial::nearest(weather_station_by_geo, geo{49.6, 6.1}, 3);
var weights = Spatial::idw_weights(weather_station_by_geo, geo{49.6, 6.1}, 3, 2.0);
spatial::GeoBox (private, volatile)
The shared geographic bounding box — the single bbox semantics for the whole library: point-in-box tests, nodeGeo Morton range scans, upstream bbox API URLs and grid tiling all go through it. Declared private because std already exports core::GeoBox; same-module code uses the bare name, everything else (including the rest of the weather lib) reaches it as spatial::GeoBox.
Longitude convention — read before building one. The box spans from west going east to east (GeoJSON bbox convention). west/east are kept exactly as given (west = sw.lng, east = ne.lng) and never reordered: passing swapped longitudes is interpreted as a box that crosses the antimeridian (west > east), not as a mistake to auto-correct. Latitudes, by contrast, are order-agnostic (south = min, north = max). Every bbox entry point across the library (LocalStations, StationDiscovery implementations, the weather.com grid tiling) inherits these semantics, so dateline-crossing boxes are supported end to end.
Fields: south/north/west/east: float. Methods:
static from_degrees(sw_lat: float, sw_lng: float, ne_lat: float, ne_lng: float): GeoBox— validating constructor; throws when any latitude is outside[-90, 90]or any longitude outside[-180, 180](NaN fails too).static from_corners(sw: geo, ne: geo): GeoBox— same semantics from two corner points.contains(g: geo): bool— inclusive membership; on a crossing box the longitude test islng >= west || lng <= east.crosses_antimeridian(): bool—west > east.split(): Array<GeoBox>— non-crossing sub-boxes for axis-aligned consumers (Morton range scans, upstream bbox APIs):[this]for a normal box, else the western lobe[west..180]followed by the eastern lobe[-180..east].sw(): geo/ne(): geo— corner accessors, defined only on non-crossing boxes — they throw on a crossing box (callsplit()and take each lobe’s corners instead).geoclamps latitude to the web-mercator domain (~±85.05°), matching the nodeGeo index domain.
LocalStations (abstract)
Read-only spatial queries over the already-persisted weather_station_by_geo
index. These resolve existing WeatherStation nodes only — they never create or
mutate stations; to discover and register new stations from a provider, use
StationDiscovery::register_bbox.
static in_bbox(g: geo, sw: geo, ne: geo): bool— inclusive rectangle membership test withspatial::GeoBoxcorner semantics: latitudes are order-agnostic (south = min, north = max), longitudes are taken as given —sw.lng() > ne.lng()means an antimeridian-crossing box, not a mistake to auto-correct.static within_bbox(sw: geo, ne: geo, max: int?): Array<node<WeatherStation>>— every station inside the inclusive box[sw,ne](corner semantics perspatial::GeoBox), ordered ascending by great-circle distance to the box centre, capped atmax(null= uncapped;max <= 0⇒ empty, matching theStationDiscoverycontract). Runs onenodeGeoMorton-range scan per non-crossingsplit()lobe — a Morton slice is a superset of the rectangle, so every visited entry is post-filtered with the lobe’scontains— O(range), not O(total stations).static near(point: geo, k: int): Array<node<WeatherStation>>— thekstations nearest topoint, ascending by great-circle distance (a thin projection ofSpatial::nearest). Fewer thankstations indexed ⇒ all of them are returned, ordering preserved.
// Nearest 5 stations already in the graph to a point:
var nearby = LocalStations::near(geo { 49.6, 6.1 }, 5);
// Every station inside a bounding box, nearest-to-centre first, capped at 20:
var inbox = LocalStations::within_bbox(geo { 49.0, 6.0 }, geo { 50.0, 7.0 }, 20);
// Inclusive rectangle membership test (corners in any order):
var hit = LocalStations::in_bbox(geo { 49.5, 6.5 }, geo { 49.0, 6.0 }, geo { 50.0, 7.0 });
Enums
WeatherCondition
Canonical weather-condition vocabulary (the Visual Crossing icon set, collapsed: showers / snow-showers / thunder variants fold into their base, and only the clear / partly-cloudy states keep a day vs night distinction).
clear_day, clear_night, partly_cloudy_day, partly_cloudy_night, cloudy,
fog, wind, rain, showers, sleet, snow, thunder, hail.
Produced by the Conditions mappers and stored by
ConditionFamily.
OpenMeteoVariable
Open-Meteo hourly variables that can be requested. Each enum value carries its Open-Meteo query name as its string value.
Selected values by category:
- Air temperature:
temperature_2m,apparent_temperature,dew_point_2m,wet_bulb_temperature_2m,relative_humidity_2m - Pressure:
pressure_msl,surface_pressure,vapour_pressure_deficit,total_column_integrated_water_vapour - Precipitation / clouds:
precipitation,precipitation_probability,rain,showers,snowfall,snow_depth,weather_code,cloud_cover,cloud_cover_low,cloud_cover_mid,cloud_cover_high,visibility - Wind (10/80/120/180 m):
wind_speed_10m,wind_speed_80m,wind_speed_120m,wind_speed_180m,wind_direction_10m,wind_direction_80m,wind_direction_120m,wind_direction_180m,wind_gusts_10m - Upper-air temperatures:
temperature_80m,temperature_120m,temperature_180m - Soil:
soil_temperature_0cm,soil_temperature_6cm,soil_temperature_18cm,soil_temperature_54cm,soil_moisture_0_to_1cm,soil_moisture_1_to_3cm,soil_moisture_3_to_9cm,soil_moisture_9_to_27cm,soil_moisture_27_to_81cm - Radiation / solar:
shortwave_radiation,direct_radiation,diffuse_radiation,direct_normal_irradiance,global_tilted_irradiance,terrestrial_radiation, plus*_instantvariants,uv_index,uv_index_clear_sky,is_day,sunshine_duration - Evapotranspiration:
evapotranspiration,et0_fao_evapotranspiration - Convective / boundary layer:
cape,lifted_index,convective_inhibition,freezing_level_height,boundary_layer_height
Common Use Cases
Seed a station from scratch
var hits = OpenMeteoService::geocode("Berlin", 1, "en", "DE", null, null);
var hit = hits[0];
var station = WeatherStation::get_or_create(geo{hit.latitude, hit.longitude}, "berlin");
station->enable(Array<type>{ TemperatureFamily, WindFamily, RadiationFamily, PrecipitationFamily });
var provider = OpenMeteoProvider {
variables: Array<OpenMeteoVariable>{
OpenMeteoVariable::temperature_2m,
OpenMeteoVariable::apparent_temperature,
OpenMeteoVariable::precipitation,
OpenMeteoVariable::precipitation_probability,
OpenMeteoVariable::wind_speed_10m,
OpenMeteoVariable::wind_direction_10m,
OpenMeteoVariable::wind_gusts_10m,
OpenMeteoVariable::shortwave_radiation,
OpenMeteoVariable::direct_normal_irradiance,
OpenMeteoVariable::diffuse_radiation,
}
};
provider.feed_history(station, time::now() - 365_day, time::now()); // archive over [from, to]
provider.feed_forecast(station); // native 16-day horizon
Read the latest observed value and the latest forecast
var sig = station->signal_for(TemperatureFamily, "air_2m");
// latest measured point
var observed = sig->observed.last();
println("latest temperature: ${observed} °C");
// latest forecast for a target 48 h from now
var target = time::now() + 2_day;
var fc = sig->latest_forecast(target);
println("forecast for now+2d: ${fc} °C");
// forecast as it was issued at a specific run
var issued = time::now() - 6_hour;
var fc_old = sig->forecast_as_of(target, issued);
println("forecast as of 6h ago for now+2d: ${fc_old} °C");
Enable calendar rollups on a signal
Call materialize once after creating a station; all subsequent set_observed calls (including those from Ingest::feed) maintain the buckets incrementally:
var sig = station->signal_for(TemperatureFamily, "air_2m");
sig->materialize(
Array<CalendarUnit>{ CalendarUnit::day, CalendarUnit::month },
TimeZone::"Europe/Berlin"
);
// after feeding data, query daily stats
var day = '2026-04-21T00:00:00Z';
var g = sig->daily!!.getAt(day);
println("daily avg: ${g?.avg()} °C min: ${g?.min} max: ${g?.max}");
Backfill rollups over existing history
WeatherSignal::rebuild_rollups folds the signal’s existing observed series over a window into its own persistent daily/monthly/yearly rollup fields (allocating them like materialize if absent). It is persistent (the result lands on disk, reachable from the station index) and idempotent — it resets the targeted buckets in the window before folding, so re-running it never double-counts. Use it to backfill rollups after a bulk import (when materialize was not called before feeding), then read a bucket straight off the signal’s daily/monthly/yearly field:
var sig = station->signal_for(RadiationFamily, "ghi");
sig->rebuild_rollups(
Array<CalendarUnit>{ CalendarUnit::month },
TimeZone::"Europe/Berlin",
time::now() - 365_day,
time::now()
);
// read back from the signal's own persistent monthly rollup
var bucket = '2026-04-01T00:00:00Z';
var g = sig->monthly!!.getAt(bucket);
println("avg GHI for ${bucket.format("%Y-%m", null)}: ${g?.avg()} W/m²");
Refresh forecasts on a schedule
Forecast sets are kept in full (all issues), so running feed_forecast every few hours is cheap and keeps latest_forecast current:
fn refresh_all_forecasts() {
var provider = OpenMeteoProvider {
variables: Array<OpenMeteoVariable>{
OpenMeteoVariable::temperature_2m,
OpenMeteoVariable::wind_speed_10m,
OpenMeteoVariable::wind_direction_10m,
}
};
for (_, station in weather_station_by_id) {
provider.feed_forecast(station); // native 16-day horizon, no range arg
}
}
Add a custom data provider
Define a mapping table from your provider’s variable names to the unified family/signal model, then call Ingest::feed:
// Provider that sends temperatures in Fahrenheit
var my_mappings = Array<VarMapping>{
VarMapping{ provider_key: "air_temp_F", family: TemperatureFamily, signal: "air_2m", unit: "°C", convert: f_to_c },
VarMapping{ provider_key: "gust_ms", family: WindFamily, signal: "gust", unit: "m/s", convert: null },
VarMapping{ provider_key: "rain_mm", family: PrecipitationFamily, signal: "total", unit: "mm", convert: null },
};
var now = time::now();
var my_points = Array<WeatherPoint>{
WeatherPoint{
t: now - 1_hour,
historical: true,
issued_at: null,
values: Map<String, float>{ "air_temp_F": 68.0, "gust_ms": 5.2, "rain_mm": 0.3 }
}
};
Ingest::feed(station, my_points, my_mappings);
Test with a canned response
The per-provider *_scatter streaming functions are network-free — feed a canned response into a real station via an IngestFeeder, then read back from the disk-backed signal node (the real streaming path, end-to-end):
var fake = OpenMeteoResponse{
latitude: 0.0, longitude: 0.0, generationtime_ms: 0.0,
utc_offset_seconds: 0, timezone: "GMT", timezone_abbreviation: "GMT",
elevation: 0.0, hourly_units: Map<String, String>{},
hourly: Map<String, any?>{
"time": Array<String>{ "2026-04-21T00:00" },
"temperature_2m": Array<float?>{ 10.0 }
}
};
var station = WeatherStation::get_or_create(geo{ 0.0, 0.0 }, "test");
var feeder = IngestFeeder::new(station, OpenMeteoService::mappings());
// window_from/window_to null = unbounded (no branch-window clamp)
OpenMeteoService::openmeteo_scatter(fake, feeder, true, null, null);
var t0 = '2026-04-21T00:00:00Z';
Assert::equalsd(
station->signal_for(TemperatureFamily, "air_2m")->observed.getAt(t0)!!,
10.0, 1e-9
);
Use a commercial API key
Open-Meteo’s paid tier requires an apikey query parameter and different host prefixes:
var key = System::getEnv("OPENMETEO_API_KEY");
var provider = OpenMeteoProvider {
archive_base_url: "https://customer-archive-api.open-meteo.com",
forecast_base_url: "https://customer-api.open-meteo.com",
api_key: key,
variables: variables
};
provider.feed_history(station, from, to);
provider.feed_forecast(station);
// geocoding helper also threads the instance api_key:
provider.geocode("Berlin", 10, "en", "DE", "https://customer-geocoding-api.open-meteo.com");
Point at a caching proxy
var provider = OpenMeteoProvider {
archive_base_url: "https://meteo-proxy.internal",
forecast_base_url: "https://meteo-proxy.internal",
variables: variables
};
provider.feed_history(station, from, to);
provider.feed_forecast(station);
Best Practices
Give every station a stable id
Stations without an id are not indexed by id, so weather_station_by_id misses them. Give every station a stable slug ("berlin", "sensor-0042") unless you only ever look up by geography.
Enable families before feeding
Ingest::feed calls signal_for, which auto-activates the family on first use even without a prior enable call. Calling enable explicitly at setup time documents your intent and lets is_enabled checks behave predictably at any point in the lifecycle.
Call materialize before the first feed
Rollup buckets are only filled by points arriving after materialize is called. If you call materialize after a bulk import, the rollups start empty. To backfill rollups over data already imported, call WeatherSignal::rebuild_rollups instead — it folds the existing observed series into the signal’s own persistent rollup fields and is idempotent.
materialize is idempotent: the first call’s tz is the one that sticks (rollup_tz is set-once), so a later call with a different timezone will not silently re-bucket existing rollups. Only CalendarUnit::hour, CalendarUnit::day, CalendarUnit::month, and CalendarUnit::year are materializable — passing any other unit throws. (The slots seasonal profile is always maintained on set_observed regardless of materialize, bucketed in rollup_tz when set, else UTC.)
set_observed adds each new timestamp’s value to the materialized rollup accumulators (Gaussian) and the overall profile. Because Gaussian cannot subtract, these accumulators are updated only the first time a given timestamp is written: re-feeding an existing timestamp corrects observed but leaves the rollup buckets and gaussian profile unchanged (no double-count, but they retain the first value). If you correct historical values and need exact stats afterward, rebuild from observed: WeatherSignal::rebuild_rollups fixes the calendar rollups (it resets the window’s buckets before re-folding, so the corrected values are reflected exactly), and WeatherSignal::rebuild_profile fixes the overall gaussian and the seasonal slots profile (a full rebuild — previously these two had no rebuild path at all).
Mark points correctly as historical or forecast
Only historical: true points update observed and the overall gaussian profile. Marking forecast data as historical pollutes the profile with speculative values; marking genuine past observations as historical: false puts them in the forecast matrix instead of the observed series.
Wind direction and vector components
Wind speed and direction are scalar quantities that wrap around at 0°/360°. The Open-Meteo connector converts speed + direction to meteorological u/v components inline before storing (via the shared Ingest::decompose_wind). If you build a custom mapping for wind from another provider, do the same decomposition:
// u = -speed * sin(dir_rad), v = -speed * cos(dir_rad)
var rad = dir_deg * MathConstants::pi / 180.0;
values.set("u_10m", -speed * sin(rad));
values.set("v_10m", -speed * cos(rad));
Use forecast_as_of to compare model runs
When you archive every forecast issue for a target time, forecast_as_of lets you answer “what was the forecast for 6 h from now, as published 12 h ago?” — useful for forecast skill evaluation.
Prefer batch ingestion
feed_history sends one archive request per started year of the historical window (≤1-year chunks), and feed_forecast usually sends one forecast request. Still prefer wide windows over calling feed_history in a tight loop with tiny windows — Open-Meteo rate-limits aggressive callers.
Geocoding edge cases
- Queries with 0–1 characters return no results.
- Server errors are turned into thrown exceptions by
geocode. get_or_createreturns an existing station when theidmatches (when non-null) first, then when thegeomatches. If the coordinate match has no id yet and you pass one, that id is adopted onto the node and indexed. If the coordinate match already carries a different id, the requested id is not indexed (a warning is logged) and the existing station is returned. Because every station is also keyed inweather_station_by_geo, two stations at the same coordinates collide — jitter the geo slightly if you need multiple stations at the same point.