8.3.101-stable Switch to dev
In this page
  1. Overview
  2. Installation
  3. Quick Start
  4. Module Variables
    1. weather_station_by_id: nodeIndex<String, node<WeatherStation>>
    2. weather_station_by_geo: nodeGeo<node<WeatherStation>>
  5. Types
    1. WeatherStation
    2. WeatherFamily (abstract)
      1. TemperatureFamily
      2. WindFamily
      3. RadiationFamily
      4. PrecipitationFamily
      5. PressureFamily
      6. HumidityFamily
      7. CloudFamily
      8. SnowFamily
      9. SoilFamily
      10. AirQualityFamily
      11. MarineFamily
      12. ConvectiveFamily
    3. WeatherSignal
    4. Ingest (abstract)
    5. is_finite(f: float): bool
    6. WeatherPoint (volatile)
    7. VarMapping (volatile)
    8. f_to_c(f: float): float
    9. CsvImport (abstract)
      1. CsvColumnMapping (volatile)
      2. CsvStationColumns (volatile)
    10. JsonImport (abstract)
      1. JsonFieldMapping (volatile)
      2. Import a records-array JSON document
    11. WeatherExport (abstract)
    12. Provider API
      1. WeatherProvider (abstract)
      2. ProviderCaps (volatile)
      3. Station-discovery contract
      4. Writing a custom provider
    13. OpenMeteoService
    14. OpenMeteoGeocodingResult (volatile)
    15. OpenMeteoGeocodingResponse (volatile)
    16. OpenMeteoResponse (volatile)
    17. NasaPowerService
      1. NasaPowerResponse (volatile)
      2. nasa_kpa_to_hpa(v: float): float
    18. PvgisService
      1. PvgisResponse / PvgisOutputs (volatile)
    19. MeteostatService
    20. VisualCrossingService
      1. VisualCrossingResponse / VisualCrossingDay (volatile)
    21. NoaaService
      1. NoaaMeasure / NoaaObsResponse / NoaaScatterStats (volatile)
    22. MeteoblueService
      1. MeteoblueResponse (volatile)
    23. SolcastService
      1. SolcastResponse / SolcastRecord (volatile)
    24. OpenWeatherMapService
      1. frac_to_pct(v: float): float
      2. mm_to_cm(v: float): float
      3. OwmResponse / OwmHour / OwmWeather (volatile)
    25. WeatherDotComService
      1. DiscoveredStation (volatile)
      2. PwsNearResponse / PwsNearLocation / PwsObsResponse (volatile)
      3. PwsHistoryGranularity / PwsImportMode (enums)
    26. Station discovery
      1. StationHit (volatile)
      2. StationDiscovery (abstract)
      3. WeatherDotComDiscovery
      4. MeteostatDiscovery
      5. Meteostat discovery response types (volatile)
    27. CamsAdapter
      1. kelvin_to_celsius(v: float): float · pa_to_hpa(v: float): float · m_to_mm(v: float): float
    28. NetcdfImport
  6. Derived & analytics layer
    1. Comfort (abstract)
    2. Astronomical (abstract)
      1. SunPosition
    3. Conditions (abstract)
      1. ConditionFamily
    4. Skill (abstract)
      1. ForecastSkillScore
    5. Spatial (abstract)
      1. GeoNeighbor (volatile)
      2. spatial::GeoBox (private, volatile)
    6. LocalStations (abstract)
  7. Enums
    1. WeatherCondition
    2. OpenMeteoVariable
  8. Common Use Cases
    1. Seed a station from scratch
    2. Read the latest observed value and the latest forecast
    3. Enable calendar rollups on a signal
    4. Backfill rollups over existing history
    5. Refresh forecasts on a schedule
    6. Add a custom data provider
    7. Test with a canned response
    8. Use a commercial API key
    9. Point at a caching proxy
  9. Best Practices
    1. Give every station a stable id
    2. Enable families before feeding
    3. Call materialize before the first feed
    4. Mark points correctly as historical or forecast
    5. Wind direction and vector components
    6. Use forecast_as_of to compare model runs
    7. Prefer batch ingestion
    8. Geocoding edge cases

Weather Integration

Weather data ingestion, time-series storage, calendar rollups, and forecast tracking for GreyCat.

Overview

The weather library provides a typed, graph-persistent foundation for weather-aware GreyCat applications:

  • Persistent weather stations located by id or by geo coordinates
  • Typed signal families: temperature, wind, radiation, precipitation, pressure, humidity, cloud, snow, soil, air-quality, and marine — each a first-class GCL type with named signals
  • Observed truth + forecast issue-time matrix per signal: the forecast is a nodeTime<nodeTime<float>> (outer = target time, inner = issued-at) so you can compare forecasts issued on different days for the same target hour
  • Opt-in calendar rollups: call materialize(units, tz) once on a signal to get incrementally maintained hourly/daily/monthly/yearly Gaussian<float> buckets
  • Persistent rollup backfill: WeatherSignal::rebuild_rollups(units, tz, from, to) folds the signal’s existing observed series into its own persistent hourly/daily/monthly/yearly rollup fields over a window (resetting the targeted buckets first, so it’s idempotent) — the disk-backed complement to materialize’s forward maintenance
  • Provider-agnostic ingest: Ingest::feed accepts a declarative Array<VarMapping> table — adding a new data provider is a one-file change
  • Data sources: connectors for Open-Meteo, NASA POWER, PVGIS, Meteostat, Visual Crossing, NOAA/GHCN, meteoblue, Solcast, and OpenWeatherMap; an offline adapter for Copernicus CAMS/ERA5 (CamsAdapter); a native NetCDF-4/HDF5 importer (NetcdfImport); plus the generic CSV (CsvImport) and JSON (JsonImport) importers — every connector funnels through the same Ingest::feed seam, and most handle wind u/v decomposition automatically

Typical use cases: energy forecasting, digital twins of outdoor assets, agriculture, smart-building HVAC optimization, anomaly detection against seasonal baselines.

Installation

@library("weather", "0.0.0");

If you want the tests too:

@library("weather");
@include("weather/test");

Quick Start

// 1. Geocode a city via Open-Meteo
var hits = OpenMeteoService::geocode("Esch-sur-Alzette", 1, "en", "LU", null, null);
if (hits.size() == 0) { throw "no geocoding result"; }
var hit = hits[0];

// 2. Get-or-create a station at that location
var station = WeatherStation::get_or_create(geo{hit.latitude, hit.longitude}, "esch");

// 3. Enable the families you care about (idempotent)
station->enable(Array<type>{ TemperatureFamily, WindFamily, RadiationFamily });

// 4. Feed OBSERVED history + the native forecast horizon from Open-Meteo
var now = time::now();
var provider = OpenMeteoProvider {
    variables: Array<OpenMeteoVariable>{
        OpenMeteoVariable::temperature_2m,
        OpenMeteoVariable::apparent_temperature,
        OpenMeteoVariable::wind_speed_10m,
        OpenMeteoVariable::wind_direction_10m,
        OpenMeteoVariable::shortwave_radiation,
        OpenMeteoVariable::direct_normal_irradiance,
        OpenMeteoVariable::diffuse_radiation,
    }
};
provider.feed_history(station, now - 60_day, now); // archive (ERA5) over [from, to]
provider.feed_forecast(station);                    // native 16-day forecast horizon

// 5. Read the latest observed temperature
var last_temp = station->signal_for(TemperatureFamily, "air_2m")->observed.last();
println("latest observed air_2m: ${last_temp} °C");

// 6. Read the latest forecast for a future target hour
var target = now + 3_day;
var fc = station->signal_for(TemperatureFamily, "air_2m")->latest_forecast(target);
println("latest forecast for now+3d: ${fc} °C");

Module Variables

weather_station_by_id: nodeIndex<String, node<WeatherStation>>

Primary index keyed by the application-defined id. Only populated for stations created with a non-null id.

weather_station_by_geo: nodeGeo<node<WeatherStation>>

Geographic index. Every station is registered here; use it for spatial lookups (nearest station, stations inside a bounding box, etc.).

Types

WeatherStation

Persistent weather station.

Fields:

field type description
location geo Station position
id String? Optional human-readable id (key in weather_station_by_id)
name String? Display name
timezone TimeZone? Caller-managed station metadata — not read by any ingest/bucketing path (rollup/profile bucketing is fixed per-signal via the explicit tz argument of materialize/rebuild_rollups, set-once rollup_tz). It is read by the export day-grouping paths — WeatherExport::to_csv(daily=true) and WeatherDotComProvider::export_archive split files by the station-local calendar day (UTC when unset) — and by the weather.com history feed for station-local chunk dates; see also the CSV-importer timezone note below
temperature node<TemperatureFamily>? null = family not yet activated
wind node<WindFamily>? null = family not yet activated
radiation node<RadiationFamily>? null = family not yet activated
precipitation node<PrecipitationFamily>? null = family not yet activated
pressure node<PressureFamily>? null = family not yet activated
humidity node<HumidityFamily>? null = family not yet activated
cloud node<CloudFamily>? null = family not yet activated
snow node<SnowFamily>? null = family not yet activated
soil node<SoilFamily>? null = family not yet activated
air_quality node<AirQualityFamily>? null = family not yet activated
marine node<MarineFamily>? null = family not yet activated
convective node<ConvectiveFamily>? null = family not yet activated
condition node<ConditionFamily>? Categorical condition series (special family — reached via conditions(), not the signal dispatch)

Methods:

  • static get_or_create(location: geo, id: String?): node<WeatherStation> — Return the existing station with that id (or at that geo) if one already exists; otherwise create, index, and return a new one. Safe to call repeatedly. When a station already occupies that exact coordinate but has no id yet and a non-null id is supplied, the id is adopted onto that node and registered in weather_station_by_id (previously the requested id was silently dropped, so later lookups by it failed).
  • static index(nws: node<WeatherStation>) — Register a station in both indexes. Called by get_or_create; use directly when constructing a station without the helper. The geo write is guarded like the id write: the first node indexed at a coordinate wins, so re-indexing or a distinct node sharing the coordinate cannot silently orphan the existing entry (callers replacing an entry must un_index the old node first).
  • static un_index(nws: node<WeatherStation>) — Remove a station from both indexes.
  • enable(families: Array<type>) — Activate the given families (idempotent). Pass type references, e.g. Array<type>{ TemperatureFamily, WindFamily }.
  • activate(fam: type) — Activate a single family (idempotent). Throws on an unknown family type.
  • is_enabled(fam: type): bool — Return true if the family sub-node is non-null (i.e. has been activated).
  • signal_for(fam: type, name: String): node<WeatherSignal> — Get-or-create the signal node for name inside fam. Auto-activates the family if needed. This is the single entry point used by Ingest::feed. Throws on an unknown family or signal name — and the signal name is validated before the family is activated, so an unknown name throws without persisting (or enabling) the family: a typo’d mapping whose throw is caught by the caller cannot leave a spuriously activated, empty family behind.
  • peek_signal(fam: type, name: String): node<WeatherSignal>? — Read-only counterpart of signal_for: the existing signal node for (fam, name), or null when the family is inactive or the signal was never created. Never creates or activates anything — safe for exports, diagnostics and other read paths (delegates to the family’s peek). An unknown family or signal name returns null instead of throwing.
  • conditions(): node<ConditionFamily> — Get-or-create this station’s categorical ConditionFamily (the special, non-signal family for coded weather conditions over time).
  • sun_position(t: time): SunPosition — Solar elevation/azimuth at this station’s location for instant t (computed on demand; see Astronomical).
  • sunrise(day: time): time? / sunset(day: time): time? — Sunrise/sunset instant for the UTC calendar day containing day, or null when no such event occurs in that day. Null is not only a polar (day/night) phenomenon: at longitudes where the event falls near 00:00 UTC, the rare day on which it drifts across midnight contains no event at all — handle null at every latitude. For |longitude| > ~90° the day’s sunset can precede its sunrise in UTC order — see Astronomical.

Example:

var station = WeatherStation::get_or_create(geo{49.49, 5.98}, "esch");
station->enable(Array<type>{ TemperatureFamily, PrecipitationFamily });

// write directly
station->signal_for(TemperatureFamily, "air_2m")->set_observed(time::now() - 1_hour, 14.7);

// read back
var temp = station->signal_for(TemperatureFamily, "air_2m")->observed.last();
println("last observed: ${temp} °C");

WeatherFamily (abstract)

Base type for all weather families. Each concrete family stores its canonical signals as explicit node<WeatherSignal>? fields and implements:

  • signal(name: String): node<WeatherSignal> — Get-or-create the signal by canonical name. Throws on an unknown name.
  • peek(name: String): node<WeatherSignal>? — Read-only lookup: the existing signal node, or null when it was never created. Never creates anything; an unknown name returns null instead of throwing.
  • signal_descriptors(): Array<Tuple<String, String>> — Every canonical (signal name, unit) pair the family can hold, in signal’s dispatch order. The single registry WeatherExport derives its column set from, so the export follows family.gcl automatically.

Concrete families:

TemperatureFamily

Temperature signals, all in °C.

signal name description
"air_2m" Air temperature at 2 m
"apparent" Apparent / feels-like temperature
"dew_point" Dew-point temperature
"wet_bulb" Wet-bulb temperature
"air_80m" Upper-air temperature at 80 m
"air_120m" Upper-air temperature at 120 m
"air_180m" Upper-air temperature at 180 m
  • static new(): TemperatureFamily

WindFamily

Wind component signals per measurement height, all in m/s. The stored truth is u/v (meteorological components); speed and direction are derived by the connector.

signal name description
"u_10m" / "v_10m" East/north wind component at 10 m
"gust" Wind gust
"u_80m" / "v_80m" East/north wind component at 80 m
"u_120m" / "v_120m" East/north wind component at 120 m
"u_180m" / "v_180m" East/north wind component at 180 m
  • static new(): WindFamily

RadiationFamily

Solar irradiance signals (W/m² unless noted).

signal name unit description
"ghi" W/m² Global horizontal irradiance
"dni" W/m² Direct normal irradiance
"dif" W/m² Diffuse horizontal irradiance
"global_tilted" W/m² Global tilted irradiance
"terrestrial" W/m² Terrestrial (extraterrestrial-normal) irradiance
"ghi_clear_sky" W/m² Clear-sky GHI
"dni_clear_sky" W/m² Clear-sky DNI
"dif_clear_sky" W/m² Clear-sky diffuse
"uv_index" index UV index
"is_day" Day/night flag (1 = day, 0 = night)
  • static new(): RadiationFamily

PrecipitationFamily

Precipitation signals.

signal name unit description
"total" mm Total precipitation accumulation
"rain" mm Rain accumulation
"showers" mm Convective-shower accumulation
"probability" % Precipitation probability
  • static new(): PrecipitationFamily

PressureFamily

Atmospheric pressure signals.

signal name unit description
"msl" hPa Mean-sea-level pressure
"surface" hPa Surface (station-level) pressure
  • static new(): PressureFamily

HumidityFamily

Humidity signals.

signal name unit description
"relative" % Relative humidity
"specific" g/kg Specific humidity
"vpd" kPa Vapour-pressure deficit
  • static new(): HumidityFamily

CloudFamily

Cloud cover fractions by layer (%) plus visibility and sunshine.

signal name unit description
"total" % Total cloud cover
"low" % Low-level cloud cover
"mid" % Mid-level cloud cover
"high" % High-level cloud cover
"visibility" m Horizontal visibility
"sunshine_duration" s Sunshine duration
  • static new(): CloudFamily

SnowFamily

Snow signals.

signal name unit description
"snowfall" cm Snowfall accumulation
"depth" m Snow depth on the ground
"swe" mm Snow-water equivalent
  • static new(): SnowFamily

SoilFamily

Soil / agriculture signals.

signal name unit description
"temperature" °C Soil temperature
"moisture" m³/m³ Volumetric soil moisture
"et0" mm Reference evapotranspiration (FAO)
  • static new(): SoilFamily

AirQualityFamily

Air-quality signals; concentrations in µg/m³ plus an aggregate index.

signal name unit description
"pm2_5" µg/m³ Particulate matter < 2.5 µm
"pm10" µg/m³ Particulate matter < 10 µm
"o3" µg/m³ Ozone concentration
"no2" µg/m³ Nitrogen-dioxide concentration
"so2" µg/m³ Sulfur-dioxide concentration
"co" µg/m³ Carbon-monoxide concentration
"aqi" index Air-quality index
"aod" index Aerosol optical depth
"pollen" grains/m³ Pollen concentration
  • static new(): AirQualityFamily

MarineFamily

Marine / sea-state signals.

signal name unit description
"wave_height" m Significant (wind-wave) height
"wave_period" s Wave period
"wave_direction" ° Wave direction (meteorological degrees)
"swell_height" m Swell height
"swell_period" s Swell period
"swell_direction" ° Swell direction
"sst" °C Sea-surface temperature
"sea_level" m Sea level / tide height
  • static new(): MarineFamily

ConvectiveFamily

Convective / atmospheric-instability signals.

signal name unit description
"cape" J/kg Convective available potential energy
"cin" J/kg Convective inhibition
"lifted_index" °C Lifted index
"freezing_level" m Freezing-level height
"boundary_layer_height" m Planetary boundary-layer height
  • static new(): ConvectiveFamily

WeatherSignal

One measurement stream on a family: observed truth, forecast issue-time matrix, optional calendar rollups, and a seasonal profile.

Fields:

field type description
unit String? Physical unit label (e.g. "°C", "W/m²")
observed nodeTime<float> Target-time → measured value
forecast nodeTime<nodeTime<float>> Target-time → (issued-at → value)
hourly nodeTime<Gaussian<float>>? Hourly rollup; null until materialize enables it
daily nodeTime<Gaussian<float>>? Daily rollup; null until materialize enables it
weekly nodeTime<Gaussian<float>>? Dormant placeholder (CalendarUnit has no week)
monthly nodeTime<Gaussian<float>>? Monthly rollup; null until materialize enables it
yearly nodeTime<Gaussian<float>>? Yearly rollup; null until materialize enables it
rollup_tz TimeZone? Calendar timezone set by materialize
max_forecast_issues int? Forecast-retention bound: null = keep all issues per target (default); N = keep the newest N. Set via limit_forecast_issues
slots nodeList<Gaussian<float>> Maintained seasonal profile: one Gaussian<float> per (day-of-year × hour-of-day) slot, slot index dayOfYear*24 + hourOfDay (dayOfYear is 0-based 0…365, so the index is 0…8783), bucketed in rollup_tz when set else UTC
gaussian Gaussian<float> Overall distribution across all observed points

Methods:

  • static new(unit: String?): WeatherSignal — Allocate a fresh signal. Called by family signal() methods; you normally don’t call it yourself.
  • materialize(units: Array<CalendarUnit>, tz: TimeZone) — Allocate the requested rollup granularities. Subsequent set_observed calls maintain them incrementally (O(1) per point). The whole units set is validated first: supported values are CalendarUnit::hour, CalendarUnit::day, CalendarUnit::month, CalendarUnit::year, and passing any other unit (minute/second/microsecond, and the absent week) throws before rollup_tz is armed or any rollup is allocated — so a bad unit leaves the signal completely untouched rather than half-materialized. Idempotent: rollup_tz is set-once, so the first call’s timezone wins — calling again with a different tz does not re-bucket already-materialized rollups. Note that calling materializeeven with an empty units array — fixes rollup_tz, which is also the timezone used to bucket the always-on slots seasonal profile. Call it before the first set_observed so the profile is bucketed in your chosen timezone from the start; calling it later mixes UTC-bucketed early slots with timezone-bucketed later ones — remediable with rebuild_profile, which re-buckets every slot in the armed rollup_tz.
  • rebuild_rollups(units: Array<CalendarUnit>, tz: TimeZone, from: time?, to: time?) — Backfill the requested rollup granularities (hour/day/month/year) from the existing observed series over [from?, to?] (a null bound ⇒ the first/last observed point) into the signal’s own persistent hourly/daily/monthly/yearly nodeTime<Gaussian<float>> fields. Allocates them like materialize if absent (and arms rollup_tz set-once, so bucketing always uses the fixed rollup_tz, not a differing tz passed on a later call). The window is snapped out to whole calendar-unit boundaries so the edge buckets cover a complete unit. Resets the targeted buckets in the window before folding, so re-running it (or running it after incremental materialize maintenance) never double-counts — Gaussians can’t subtract. Only one bucket Gaussian is live at a time (O(1) memory) and the result lands on disk, reachable from the station index. This is the persistent backfill complement to materialize (forward maintenance) — and the replacement for the removed transient calendar_reduce.
  • rebuild_profile() — Rebuild the overall gaussian and the seasonal slots profile from the existing observed series: both accumulators are reset, then every observed point is re-folded, re-bucketed in the current rollup_tz (else UTC) using the same slot encoding as set_observed. Always a full rebuild — there is no from/to window, because these two accumulators span all observed points and Gaussians cannot subtract (the calendar rollups keep their own windowed rebuild_rollups). Use it after corrected re-feeds (the accumulators are first-write-only) or after a late materialize(tz) left the slots key space mixing UTC- and tz-bucketed slots — the rebuild unifies the slot keys in the armed rollup_tz.
  • set_observed(t: time, value: float) — Write an observed (truth) point. Maintains the overall gaussian profile, the per-(day-of-year × hour-of-day) slots seasonal profile, and any materialized rollups incrementally. The profile, slots, and rollups are updated only on the first write of a given timestamp; re-feeding a timestamp corrects observed without touching the Gaussian accumulators (which cannot subtract), so there is no double-counting. To reflect corrections exactly afterwards, rebuild from observed: rebuild_rollups for the calendar rollups, rebuild_profile for gaussian/slots. The slot bucket and the rollup buckets are computed in rollup_tz when set, else UTC.
  • set_forecast(target: time, issued_at: time, value: float) — Write a forecast point. By default all issues for the same target are kept (keep-all); once limit_forecast_issues(N) has been set, the write prunes the target’s issues to the newest N.
  • latest_forecast(target: time): float? — Forecast value at the largest (most recent) issued-at timestamp for target. The target lookup is at-or-before (resolveAt): a target falling between two stored targets resolves to the closest preceding one. Returns null only when no stored target is at or before target.
  • forecast_as_of(target: time, cutoff: time): float? — Forecast for target as issued at or before cutoff. Returns the value from the most recent issue that is <= cutoff.
  • limit_forecast_issues(max: int?) — Bound the retained forecast issues per target to the newest max (the last N issues retention policy); null restores keep-all. max must be >= 1 or null: a max < 1 throws before any mutation (it would turn every subsequent set_forecast into a silent insert-then-delete of all issues). Applies immediately to every existing target and to all subsequent set_forecast writes. Use it on high-churn signals (forecasts overwrite constantly) to keep the issue-time matrix from growing without bound. See forecast-skill scoring.

Forecast issue-time matrix layout:

forecast: nodeTime<nodeTime<float>>
          ^                 ^
          target time       issued-at → value

This stores every issue for every target. latest_forecast always reads from the latest run; forecast_as_of lets you replay any past run.

Ingest (abstract)

Provider-agnostic scatter: routes WeatherPoints into station signals via a declarative mapping table.

Methods:

  • static feed(station: node<WeatherStation>, points: Array<WeatherPoint>, mappings: Array<VarMapping>) — A thin internal wrapper over the new streaming primitive IngestFeeder (in weather_import.gcl): the library’s own importers stream each value directly into the disk-backed signal node via IngestFeeder (built once per import; no intermediate Array<WeatherPoint> and, for importers without wind decomposition, no per-row Map), while feed(Array) is retained as a thin wrapper for callers who build their own small batch. For each WeatherPoint, look up each values key in mappings, apply any convert function, and call set_observed (when historical) or set_forecast (when not historical) on the target signal. Keys not covered by any mapping are silently skipped. The library-wide hindsight-forecast rule lives in IngestFeeder::feed_value (the single central guard every importer routes through): a forecast value whose target t precedes its issue time (t < issued) is hindsight, not a forecast — it is dropped (t == issued is kept; historical points are unaffected), so a “forecast” issued after its target can never win latest_forecast or inflate skill comparisons, whatever the provider. For historical points the feeder never reads the clock (issued is an ignored placeholder), so bulk historical imports are free of per-row time::now() calls; forecast callers should pre-resolve a single issued_at for the whole batch (a null issued_at falls back to a fresh time::now() per point, fragmenting the issue-time matrix). The family is auto-activated if needed. A mapping’s unit is never applied — families are the sole source of truth for units (see VarMapping below). A mapping that names a signal the family does not define is skipped with a warning (and dropped for the rest of the batch) rather than aborting the whole feed, so one bad mapping can’t lose an entire import. Non-finite values (NaN / ±Inf) are dropped centrally here — after the convert function runs — so they never reach a signal’s raw observed/forecast series. (Gaussian.add itself now self-rejects non-finite samples, so the overall/seasonal/rollup stats are protected regardless; the central drop additionally keeps the raw nodeTime series clean and stops a non-finite value from consuming a timestamp’s first-write slot — which would otherwise block a later finite value at the same timestamp from updating the accumulators.) (WeatherSignal.set_observed/set_forecast carry the same guard.) This central drop is what makes NetcdfImport’s documented “fill/missing → NaN → dropped by Ingest::feed” contract hold even when the NaN is produced by a convert fn rather than by the C reader.

Shared helpers — reusable static utilities every connector funnels through (so each provider no longer re-implements them inline):

  • static coerce_float(raw: any?): float? — Coerce a JSON numeric slot to a float. JSON numbers arrive as either an int or a float slot, and a bare as float does not widen an int slot, so the int case is widened explicitly. Returns null for a null or non-numeric slot, so callers skip it instead of throwing.
  • static decompose_wind(values: Map<String, float>, speed_key: String, dir_key: String, speed_scale: float) — Meteorological wind decomposition. Reads speed_key (multiplied by speed_scale, e.g. 1.0/3.6 for km/h → m/s, 1.0 for m/s) and dir_key (degrees); when both are present it replaces them with u_10m/v_10m under the “wind from” convention (u = -speed*sin(dir), v = -speed*cos(dir)). If exactly one is present the orphan is dropped with a warning. The raw speed/dir keys are always removed.
  • static recompose_wind(u: float?, v: float?): Tuple<float?, float?> — Inverse of decompose_wind: reconstructs the meteorological (speed_ms, dir_deg) pair from the u_10m/v_10m components under the same “wind from” convention (speed = sqrt(u²+v²), dir = atan2(-u, -v) in degrees normalized to [0, 360); a calm u = v = 0 vector yields direction 0). Null handling mirrors decompose_wind’s orphan rule — both-null returns (null, null), and a single non-null orphan is dropped with a warning and (null, null) returned. Used by WeatherExport/WeatherDotComProvider::export_archive to recover winddirAvg/windspeed* from the stored components.
  • static check_status(status_code: int, context: String) — Uniform HTTP status guard: throws a consistent error on any non-200 status. context must be a non-secret label (provider/endpoint name or a base URL without an api key) — never the full query string, which may embed a key.
  • static require_body(content: any?, context: String, error_msg: String?) — Uniform guard for a 200 response whose body is empty or failed to deserialize (Http<T> then returns content == null with the detail in error_msg): throws "<context>: 200 response with an empty or unparseable body[: <error_msg>]" instead of letting a connector skip the chunk silently (an invisible hole of up to a whole chunk). Same non-secret context rule as check_status.
  • static parse_time(cell: String, format: String?, tz: TimeZone?): time? — Parse a timestamp cell, returning null instead of throwing on a malformed value (the timestamp analogue of CsvImport::parse_cell). format null → ISO-8601. When tz is supplied the offset-less calendar fields are bound to it (Date::parse(...).to_time(tz)); otherwise the naive value is read in the host’s global timezone via time::parse.

is_finite(f: float): bool

Free function. Returns true for a finite float, false for NaN or ±Inf. Implemented as !isNaN(f - f) (f - f is 0 for any finite value but NaN for NaN and ±Inf). Used by Ingest::feed to drop non-finite samples centrally before they reach a signal’s raw observed/forecast series. (Gaussian.add itself also self-rejects non-finite samples, so the rollup folds in rebuild_rollups / set_observed are safe regardless.)

WeatherPoint (volatile)

Provider-agnostic input row.

Fields:

field type description
t time UTC timestamp of the measurement target
historical bool true → observed truth; false → forecast
issued_at time? Forecast issue time (null for historical points)
values Map<String, float> Provider-key → raw value

VarMapping (volatile)

One declarative mapping row connecting a provider variable to a family/signal.

Fields:

field type description
provider_key String Key in WeatherPoint.values
family type Target family type reference (e.g. TemperatureFamily)
signal String Signal name within the family (e.g. "air_2m")
unit String? Advisory label, never applied or stored. The family constructor (WeatherSignal::new) sets the canonical unit, so Ingest::feed ignores a mapping’s unit entirely. Families are the sole source of truth for units.
convert function? fn(float): float applied before storing; null for identity. The result is coerced via Ingest::coerce_float: an int return widens to float, while a non-numeric return warns once and skips that mapping’s entire stream (like a mapping naming an unknown signal)

f_to_c(f: float): float

Free function. Converts a Fahrenheit value to Celsius: (f - 32) * 5 / 9. Provided as a ready-to-use convert function for VarMapping.

CsvImport (abstract)

Provider-agnostic CSV importer. Turns already-parsed CSV rows into WeatherPoints and WeatherStations, and can feed a CSV file straight into a station’s signals. The pure row-level converters (rows_to_points / rows_to_stations) are the testable boundary; import_stations is a thin I/O wrapper, while feed streams the file row-by-row into the station’s signals (no raw-rows buffer, no points buffer).

Methods:

  • static to_var_mappings(cols: Array<CsvColumnMapping>): Array<VarMapping> — Convert each CsvColumnMapping to a VarMapping whose provider_key is the column name, so the per-row values Map keys line up with what Ingest::feed expects. Reuses the ingest seam.
  • static read_rows(path: String, separator: char?, header_lines: int?): Tuple<Array<String>, Array<Array<String>>> — Read a CSV file into a (header, rows) tuple of raw strings (.x = the column-name header line’s columns, .y = the data rows). It forces header_lines: 0 on the underlying CsvReader<Array<String>> so every line — including the header — is returned, then peels off the first header_lines lines itself, keeping the last of them as the column-name header (the line immediately before the data), which the reader would otherwise consume. header_lines null defaults to 1; pass 0 for a headerless file, or a larger count to discard a multi-line preamble (the real header line after the preamble is the one kept). Used internally by feed and import_stations, and reusable directly — e.g. to read a CAMS file whose #-comment preamble precedes the header line (note that CamsAdapter::feed_cams_radiation_csv now auto-detects that header itself).
  • static parse_cell(cell: String): float? — Non-throwing numeric parse. A blank, non-numeric sentinel (e.g. "M", "N/A", "--"), or otherwise unparseable cell returns null instead of throwing, so a single bad cell no longer aborts the whole import. Integer cells are coerced to real floats. It does not cover numeric missing-value codes (-9999, -999.9, …) — those parse as real values; declare them per column in CsvColumnMapping.sentinels. Used by both rows_to_points and rows_to_stations.
  • static rows_to_points(header: Array<String>, rows: Array<Array<String>>, mappings: Array<CsvColumnMapping>, ts_column: String, ts_format: String?, historical: bool, issued_at: time?, tz: TimeZone?): Array<WeatherPoint> — Pure converter. Locates ts_column in header and parses each row’s timestamp. A header name that is duplicated and actually used — the ts_column or a mapped value column — throws ("ambiguous import refused"): a silent last-wins pick could import the wrong column undetectably. Duplicates among columns no mapping references are tolerated. (Shared with the streaming feed, which resolves columns the same way.) When tz is null, the cell is read with time::parse(cell, ts_format) (host-global behavior; ts_format null → ISO8601). When tz is non-null, a naive (offset-less) timestamp is parsed in that timezone via Date::parse(cell, ts_format).to_time(tz), so a station-local timestamp lands on the correct UTC epoch. For each mapping whose column is present in the header, the cell is parsed with parse_cell (then checked against the mapping’s numeric sentinels); blank/sentinel/unparseable cells are skipped rather than aborting. Rows with a blank timestamp cell are skipped. issued_at is carried only on forecast (!historical) points — and on forecast imports a null issued_at resolves to a single time::now() for the whole batch (one consistent forecast-matrix issue key, matching CsvImport::feed and JsonImport). Ideal for testing without file access.
  • static rows_to_stations(header: Array<String>, rows: Array<Array<String>>, cols: CsvStationColumns): Array<node<WeatherStation>> — Pure converter. For each row reads lat/lng (via parse_cell) and optional id/name, then calls WeatherStation::get_or_create(geo { lat, lng }, id) (which indexes the node) and sets ->name when present. The same duplicate-header rule as rows_to_points applies: a duplicated header name that is actually used (lat/lng/id/name) throws instead of silently picking the last occurrence. Rows with a missing, non-numeric, or out-of-range lat/lng (outside the web-mercator geo::min/geo::max bounds; NaN included) are skipped. The first non-empty name wins — a later duplicate-coordinate row will not clobber an already-named station. Timezone is not applied — see the note below.
  • static feed(station: node<WeatherStation>, path: String, mappings: Array<CsvColumnMapping>, ts_column: String, ts_format: String?, separator: char?, header_lines: int?, historical: bool, tz: TimeZone?, issued_at: time?) — Full pipeline: peels the header off the front exactly like read_rows (keeping the last of the header_lines leading lines as the column-name header), then streams each data row — converted via row_to_point — straight into the station’s signals via IngestFeeder, so only one row (and its small values Map) is ever resident. tz is forwarded for naive-timestamp interpretation. Forecast points are stamped with issued_at (or time::now() when issued_at is null) as their issue time.
  • static import_stations(path: String, cols: CsvStationColumns, separator: char?, header_lines: int?): Array<node<WeatherStation>> — Reads a station-metadata CSV and creates/indexes one station per row via rows_to_stations.

Timezone note: TimeZone is a closed enum with no dynamic string-to-value lookup, so a free-text tz cell cannot be coerced into the enum without an exhaustive mapping. CsvStationColumns.tz is therefore accepted as metadata but not applied to WeatherStation.timezone; set the timezone explicitly on the returned node if you need it.

Header handling: the file-reading methods force header_lines: 0 on the underlying CsvReader<Array<String>> so every line (including the header) is returned, then peel off the first header_lines lines themselves — preserving the header text, which the CSV reader would otherwise silently consume. When header_lines is null it defaults to 1 (the natural single-header-line CSV), so a standard CSV needs no header_lines argument; pass header_lines: 0 explicitly for a headerless file. With header_lines > 1 the last of those leading lines (the column-name line immediately before the data) is kept as the header and the preamble lines above it are discarded — so a CAMS-style file whose comment preamble precedes its header line parses correctly when header_lines is set to (preamble lines + 1).

CsvColumnMapping (volatile)

Maps one CSV header column to a family/signal.

field type description
column String CSV header column name; becomes the VarMapping.provider_key
family type Target family type reference (e.g. TemperatureFamily)
signal String Signal name within the family (e.g. "air_2m")
unit String? Advisory unit label, carried into the VarMapping but never applied — the family owns the canonical unit
convert function? fn(float): float applied before storing; null for identity
sentinels Array<float>? Per-column numeric missing-value codes (e.g. -9999.0, -999.9), matched exactly (no epsilon) after parse_cell; a matching cell is skipped like a blank one. parse_cell itself only rejects blank/non-numeric cells

CsvStationColumns (volatile)

Header names locating station-metadata fields in a station CSV.

field type description
id String? Header of the station-id column
lat String Header of the latitude column (mandatory)
lng String Header of the longitude column (mandatory)
name String? Header of the station-name column
tz String? Header of a timezone column (metadata only — not applied, see note)

Usage — import a measurements CSV into a station:

var s = WeatherStation::get_or_create(geo { 49.6, 6.12 }, "esch");
var mappings = Array<CsvColumnMapping> {
    CsvColumnMapping { column: "temp",     family: TemperatureFamily, signal: "air_2m",   unit: "°C", convert: null },
    CsvColumnMapping { column: "humidity", family: HumidityFamily,    signal: "relative", unit: "%",  convert: null }
};
// CSV: timestamp,temp,humidity   (ISO8601 timestamps, 1 header line, comma-separated)
// header_lines null defaults to 1; tz null keeps host-global timestamp parsing;
// issued_at null stamps forecast points with time::now() (here historical=true).
CsvImport::feed(s, "files/esch.csv", mappings, "timestamp", null, ',', null, true, null, null);

Usage — import a station-metadata CSV:

// CSV: station_id,latitude,longitude,label
// ST-1,49.6,6.12,Esch
// ST-2,48.85,2.35,Paris
var cols = CsvStationColumns { id: "station_id", lat: "latitude", lng: "longitude", name: "label", tz: null };
var stations = CsvImport::import_stations("files/stations.csv", cols, ',', 1);
// stations are now registered in weather_station_by_id / weather_station_by_geo

JsonImport (abstract)

Generic importer for weather JSON documents shaped as an array of record objects (one record per timestamp) — the common shape of weather REST APIs. The importer streams each value directly into the disk-backed signal node (no intermediate Array<WeatherPoint>); feed_json adds the JSON-string parse + navigation.

Mappings are expressed as JsonFieldMapping rows whose path is a field-path relative to each record: dot-separated keys with optional [i] array indices (e.g. "main.temp", "hourly[0].ghi", "a.b[1][2]").

Methods:

  • static resolve_path(root: any?, path: String): any? — Walk a dotted field-path within an already-parsed JSON value (Map<String, any?> / Array<any?> tree). Each .-separated segment may carry one or more trailing [i] array indices. Descends through map keys and array indices; returns null if any step is missing, an index is out of bounds, or a type mismatches. An empty path ("") returns root, but a path with an empty segment — a trailing or double dot such as "main." or "a..b" — is malformed and returns null, as is a segment with junk after a ] index (e.g. "values[0]x").
  • static to_var_mappings(ms: Array<JsonFieldMapping>): Array<VarMapping> — Adapt JSON field mappings to the provider-agnostic VarMapping rows consumed by IngestFeeder. Each VarMapping.provider_key is set to the mapping’s path, because the importer feeds each value under its mapping path.
  • Importer streams internally (no exposed pure converter). For each record, time_field is resolved: an int value is treated as epoch seconds (time::new(<int>, DurationUnit::seconds)); a float value is treated as float-valued epoch seconds and truncated to integer seconds (time::new((v as float) as int, DurationUnit::seconds)); any other value is read as a String and parsed with time_format (ISO 8601 when null). String timestamps are bound to tz when supplied; with a null tz, offset-less strings bind to the host’s global timezone — pass TimeZone::"UTC" for offset-less UTC sources (same semantics as CsvImport). Each mapping’s path is resolved to a number (int coerced to float; null skipped) and fed under that path. The time_field and every mapping path are tokenized once up front, so a large pull never re-splits a path string per record. The forecast issue-time is hoisted once for the whole batch (issued_at ?? time::now() when !historical, else null). Records that are null or have a missing time field are skipped.
  • static feed_json(station: node<WeatherStation>, json_text: String, records_path: String, time_field: String, time_format: String?, tz: TimeZone?, mappings: Array<JsonFieldMapping>, historical: bool, issued_at: time?) — Full pipeline: Json<any> {}.parse(json_text), navigate to the records array via records_path, then stream each value into the station’s family signals via IngestFeeder. issued_at is forwarded for forecast points (null → time::now()). A records_path that is missing or resolves to a non-array (e.g. a provider error payload) emits a warning and imports nothing; an empty records array stays silent (a valid empty pull).

JsonFieldMapping (volatile)

One declarative mapping row for the generic JSON importer.

field type description
path String Field-path relative to each record: dot-separated keys with optional [i] array indices
family type Target family type reference (e.g. TemperatureFamily)
signal String Signal name within the family (e.g. "air_2m")
unit String? Advisory unit label, carried into the VarMapping but never applied — the family owns the canonical unit
convert function? fn(float): float applied before storing; null for identity

Import a records-array JSON document

var station = WeatherStation::get_or_create(geo { 49.594, 6.119 }, "my-station");

// JSON shaped as { "list": [ { "ts": "...", "main": { "temp": .. }, "rad": { "ghi": .. } }, ... ] }
var json_text = "{\"list\":[{\"ts\":\"2026-04-21T00:00:00Z\",\"main\":{\"temp\":7.5},\"rad\":{\"ghi\":42}}]}";

var mappings = Array<JsonFieldMapping> {
    JsonFieldMapping { path: "main.temp", family: TemperatureFamily, signal: "air_2m", unit: "°C", convert: null },
    JsonFieldMapping { path: "rad.ghi", family: RadiationFamily, signal: "ghi", unit: "W/m²", convert: null }
};

// records_path = "list", time_field = "ts", ISO-8601 time (format null),
// tz null (the "Z"-suffixed stamps carry their own offset),
// historical = true, issued_at null (ignored for historical points).
JsonImport::feed_json(station, json_text, "list", "ts", null, null, mappings, true, null);

For epoch-seconds timestamps, point time_field at the numeric field and pass time_format = null — an int time value is interpreted as Unix seconds automatically, and a float value (e.g. 1700000000.0) is truncated to integer seconds. To test without the JSON-string parse, build Array<any?> records by hand and stream them through a feeder: var feeder = IngestFeeder::new(station, JsonImport::to_var_mappings(mappings)); JsonImport::records_scatter(records, time_field, time_format, tz, mappings, feeder, historical, issued_at); then read back from station->signal_for(...)->observed.getAt(t).

WeatherExport (abstract)

Neutral, round-trippable station export — the counterpart to CsvImport / JsonImport. Writes any station’s canonical signals to a wide CSV (one column per present signal) or a record-major JSON document, both re-importable through the same import seams. An abstract type with static methods only (no instances). The weather.com archive-exact 33-column layout (WeatherDotComProvider::export_archive) is a specialization that reuses this scaffolding (observed_union, fmt_cell, read_signal_at, the per-day grouping loop).

Scope and guarantees:

  • Observed series only — forecasts are excluded by design. The wide one-row-per-instant layout cannot carry the (target, issued) two-key forecast matrix, so the export is lossy for forecasts; read the per-signal forecast nodes directly if forecast retention matters.
  • The column set is derived, not hand-listed: canonical_columns() walks canonical_families(), whose descriptors come from each family’s own signal_descriptors() — all 69 canonical (family, signal) pairs, in stable family-major order. Adding a signal to family.gcl extends the export automatically. A signal name declared by more than one family is qualified <family_key>_<signal> (today: totalprecipitation_total and cloud_total), so a full export never carries two identical headers; qualification is decided on the static registry, never on the data.
  • Export never mutates the graph: signals are reached through the read-only WeatherStation::peek_signal / family peek paths, so exporting can never activate a family or allocate a signal node.
  • Empty export ⇒ no file for both formats: a station with nothing to export (no present signal, or no observed point in the window) writes no file at all (to_json no longer writes {"observations": []}).
  • Timestamps carry microsecond precision: the CSV time column and the JSON time field are ISO-8601 UTC with 6 fractional digits (%Y-%m-%dT%H:%M:%S.%fZ), so sub-second points neither shift nor collide on re-import. Re-import must use ts_format/time_format null (ISO mode) — an explicit "%Y-%m-%dT%H:%M:%SZ" format no longer parses them.

Methods:

  • static observed_union(sigs: Array<node<WeatherSignal>>, from: time?, to: time?): Array<time> — the sorted, de-duplicated UNION of observed timestamps across sigs within [from,to] (a null bound is open on that side). One entry per distinct instant, chronological — the row index for any export.
  • static fmt_cell(v: float?): String — render a float for a cell, or "" when null. Integer-valued floats drop the decimal point (17.0"17"); other values use the default float rendering when it parses back exactly through the importers’ number parser, and escalate to 17-significant-digit scientific notation when the default rendering would be lossy (e.g. 0.1 + 0.2) — keeping the round-trip exact for every value the runtime can re-parse exactly. (Also used by WeatherDotComProvider::export_archive cells.)
  • static canonical_families(): Array<ExportFamily> — the canonical families in stable export order, each as an ExportFamily { family: type, key: String, descriptors: Array<Tuple<String, String>> } (the key matches the WeatherStation field name and qualifies ambiguous column names; descriptors is the family’s signal_descriptors()). The single registry the export derives from.
  • static canonical_columns(): Array<ExportColumn> — every canonical (family, signal) pair as an ExportColumn { family: type, signal: String, column: String } in stable family-major order; column equals signal except for shared names, which are qualified <family_key>_<signal>.
  • static present_signals(station): Array<Tuple<String, node<WeatherSignal>>> — the canonical signals that actually exist (active family with a non-empty observed series), as (column-name, node) pairs in canonical order — .x is the column name, which may be family-qualified, not always the bare signal name. Strictly read-only (peek_signal).
  • static read_signal_at(sig: node<WeatherSignal>?, t: time): float? — a signal’s observed value at t, or null when the signal is null or has no point at t.
  • static csv_mappings(): Array<CsvColumnMapping> — the import mapping that exactly inverts to_csv: one CsvColumnMapping per canonical column, keyed by the same column name to_csv writes, no convert (u/v are stored truth, re-imported verbatim). Use with CsvImport::feed / read_rows (ts_column "time", ts_format null = ISO 8601) to round-trip a neutral CSV export.
  • static json_mappings(): Array<JsonFieldMapping> — the JSON twin of csv_mappings: one JsonFieldMapping per canonical column (each column is a flat top-level field on every record object). Use with JsonImport::feed_json (records_path "observations", time_field "time", time_format null) to round-trip a neutral JSON export.
  • static to_csv(station, path: String, from: time?, to: time?, daily: bool) — wide CSV export: header = "time" + each present canonical column name (canonical order), one row per observed_union timestamp over [from,to]. Timestamps are ISO-8601 UTC with microsecond precision (round-trippable independent of the host tz). daily = false writes one file at path; daily = true treats path as a directory and writes <path>/<id-or-export>_YYYYMMDD.csv grouped by the station-local calendar day (station.timezone ?? UTC). No file on an empty export.
  • static to_json(station, path: String, from: time?, to: time?) — record-major JSON export: a single document { "observations": [ { "time": "<iso8601 UTC, µs precision>", "<column>": <float>, ... }, ... ] }, one object per observed_union timestamp (absent points omit the field). Re-imports directly via JsonImport::feed_json with records_path = "observations", time_field = "time", time_format null and json_mappings() — the exact inverse of import. No file on an empty export.
var station = weather_station_by_id.get("my-station")!!;

// Wide neutral CSV (single file) — re-import with WeatherExport::csv_mappings().
WeatherExport::to_csv(station, "./export/my-station.csv", null, null, false);
var dst = WeatherStation::get_or_create(geo { 49.6, 6.1 }, "round-trip");
// ts_format null = ISO mode — the µs-precision "Z" timestamps carry their own
// offset, so no tz binding is needed.
CsvImport::feed(dst, "./export/my-station.csv", WeatherExport::csv_mappings(),
    "time", null, ',', 1, true, null, null);

// Record-major JSON — re-import with JsonImport over records_path "observations".
WeatherExport::to_json(station, "./export/my-station.json", null, null);

Provider API

The uniform provider contract every connector’s public surface implements: one configured WeatherProvider instance per connection, carrying its own credentials/endpoints as fields so the method surface stays uniform. Reference implementations: OpenMeteoProvider, MeteostatProvider, WeatherDotComProvider.

WeatherProvider (abstract)

GreyCat has no method override (a subtype only implements an abstract method or inherits a concrete one), so every feed/capability method is abstract and every provider implements all of them — a provider that does not serve a mode implements it as this.unsupported("<method>"):

  • abstract id(): String — stable lower-case key (e.g. "openmeteo"), for logs and provenance.
  • abstract capabilities(): ProviderCaps — what this provider serves (cheap introspection; no try/catch needed).
  • abstract feed_current(station: node<WeatherStation>) — pull the latest/current observation(s).
  • abstract feed_history(station: node<WeatherStation>, from: time, to: time) — backfill OBSERVED history over [from, to].
  • abstract feed_forecast(station: node<WeatherStation>) — pull the provider’s native FORECAST horizon.
  • abstract station_discovery(): StationDiscovery? — upstream station discovery, or null when the provider has no station network.
  • unsupported(method: String) — the one concrete, inherited helper (never overridden): throws the uniform "<id>: <method> unsupported". The convention for every mode a provider does not serve, so the failure message is centralized and identical across connectors.

ProviderCaps (volatile)

Cheap capability introspection so callers branch without try/catch: current: bool, history: bool, forecast: bool, discovery: bool.

var p = MeteostatProvider::new("RAPIDAPI_KEY");
if (p.capabilities().forecast) {
    p.feed_forecast(station);     // gated: never hits an "unsupported" throw
}
var disc = p.station_discovery(); // null when the provider has no station network
if (disc != null) {
    var nodes = disc.register_bbox(sw, ne, null);
}

Station-discovery contract

station_discovery() returns a StationDiscovery whose max parameter follows one binding contract across every implementation: null = uncapped (provider default); max <= 0 = empty result (an explicit zero/negative cap caps to nothing — it is never an “uncapped” sentinel; implementations short-circuit before any network call). Bounding boxes follow spatial::GeoBox corner semantics (see Spatial): latitudes are order-agnostic, longitudes are taken as given — sw.lng() > ne.lng() means an antimeridian-crossing box.

Writing a custom provider

type MyProvider extends WeatherProvider {
    api_key: String?;

    fn id(): String {
        return "myprovider";
    }

    fn capabilities(): ProviderCaps {
        return ProviderCaps { current: false, history: true, forecast: false, discovery: false };
    }

    fn feed_current(_station: node<WeatherStation>) {
        this.unsupported("feed_current");
    }

    fn feed_history(station: node<WeatherStation>, from: time, to: time) {
        // fetch [from, to] from the upstream API, then stream each point
        // through the shared ingest seam, e.g.:
        //   Ingest::feed(station, points, my_mappings);
    }

    fn feed_forecast(_station: node<WeatherStation>) {
        this.unsupported("feed_forecast");
    }

    fn station_discovery(): StationDiscovery? {
        return null;
    }
}

OpenMeteoService

Connector for the Open-Meteo family of APIs.

The public entry point is OpenMeteoProvider (a WeatherProvider): it carries the archive_base_url/forecast_base_url/api_key/variables as fields, so they are off every method signature, and the history-vs-forecast distinction is the method you call. Open-Meteo is a gridded model — it serves OBSERVED history (archive/ERA5) and a native FORECAST horizon, but has no “current” snapshot product (so feed_current is unsupported) and no station network (so station_discovery() is null). Forward geocode(…) stays a provider helper (geocoding is not station discovery). The old single ranged feed_from_openmeteo(station, from, to, …) that straddled now is gone: feed_history is archive-only over the literal [from, to] (a future to never spills into a forecast call — the archive simply returns nulls there), and feed_forecast has no range argument (the native horizon [now, now+15d] is the range).

var p = OpenMeteoProvider::new();          // or OpenMeteoProvider { api_key: "…", variables: … }
p.feed_history(station, from, to);          // archive (ERA5) over [from, to]
p.feed_forecast(station);                   // native 16-day forecast horizon
// p.capabilities() => { current:false, history:true, forecast:true, discovery:false }
// p.station_discovery() => null  (gridded: no station network)
var hits = p.geocode("Esch", null, null, "LU", null);  // forward geocoding helper

OpenMeteoProvider delegates to the OpenMeteoService statics below (which remain the testable, lower-level surface). When variables is null, OpenMeteoProvider::default_variables() is requested — the canonical family set (temperature, humidity, precipitation, pressure, wind speed+direction+gust, cloud, radiation GHI/DNI/DIF, snow, soil).

Provider methods (OpenMeteoProvider):

  • static new(): OpenMeteoProvider — all-defaults constructor (public endpoints, no api key, default_variables()).
  • static default_variables(): Array<OpenMeteoVariable> — the default hourly variable set requested when variables is null.
  • id(): String"openmeteo".
  • capabilities(): ProviderCaps{ current: false, history: true, forecast: true, discovery: false }.
  • feed_current(station)unsupported (Open-Meteo has no current-snapshot product; use feed_forecast for the near-term horizon).
  • feed_history(station, from, to) — backfill OBSERVED history over the literal [from, to] from the archive (ERA5) endpoint, streaming via IngestFeeder (delegates to feed_archive_range). Never spills into the forecast endpoint — a future to simply returns archive nulls there. Throws if from > to. The archive lags ~5 days behind real time, so a window ending near now logs a warning about the possible recent-past gap.
  • feed_forecast(station) — pull the native FORECAST horizon (issued now): the forecast endpoint over [now, now+15d], clamped to Open-Meteo’s 16-day cap (delegates to feed_forecast_horizon). No range argument — the horizon is the range.
  • station_discovery(): StationDiscovery?null (gridded: no station network).
  • geocode(name, count, language, country_code, geocoding_base_url): Array<OpenMeteoGeocodingResult> — forward geocoding helper threading this instance’s api_key into OpenMeteoService::geocode.

Methods (OpenMeteoService):

  • static geocode(name: String, count: int?, language: String?, country_code: String?, base_url: String?, api_key: String?): Array<OpenMeteoGeocodingResult> — City search. Defaults: count=10, language="en", base_url="https://geocoding-api.open-meteo.com". country_code is an ISO-3166-1 alpha-2 filter (e.g. "LU"). When api_key is non-null it is appended as &apikey=<key> — needed only for commercial endpoints. The name (after Unicode normalization/casefolding), country_code, and api_key are URL-encoded, so names with spaces or non-ASCII characters (e.g. "New York", "Düsseldorf") are handled correctly. Throws on non-200 responses or when the server returns error: true.
  • static mappings(): Array<VarMapping> — Return the declarative mapping table from Open-Meteo variable names to unified family/signal pairs. It maps temperature (temperature_2m→air_2m, dew_point_2m→dew_point, apparent_temperature→apparent), precipitation (precipitation→total, precipitation_probability→probability), radiation (shortwave_radiation→ghi, direct_normal_irradiance→dni, diffuse_radiation→dif), wind (u_10m/v_10m, wind_gusts_10m→gust), humidity (relative_humidity_2m→Humidity/relative, vapour_pressure_deficit→Humidity/vpd), pressure (pressure_msl→Pressure/msl, surface_pressure→Pressure/surface), cloud (cloud_cover→Cloud/total, cloud_cover_low/mid/high→Cloud/low|mid|high), snow (snowfall→Snow/snowfall, snow_depth→Snow/depth), and soil (soil_temperature_0cm→Soil/temperature, soil_moisture_0_to_1cm→Soil/moisture). The humidity, pressure, cloud, snow, and soil variables were previously fetched but silently dropped; they are now mapped. Wind speed/direction are converted to u/v components inline as each record is streamed, emitting "u_10m"/"v_10m" as provider keys — those map to the WindFamily signals "u_10m" and "v_10m" (reached via signal("u_10m") / signal("v_10m")). The table still maps only a curated subset of OpenMeteoVariable (e.g. wet_bulb_temperature_2m is in the enum but not currently mapped); on a provider_key collision the last VarMapping wins.
  • Importer streams internally (no exposed pure converter); the per-hour mapping and the wind_speed_10m + wind_direction_10mu_10m/v_10m decomposition described above happen inline as each record is fed. Rows outside the requested branch window are skipped: requests are date-granular (whole UTC days), so a response also carries edge-day hours before from/after to — and, on a forecast pull, today’s hours before now — which are no longer ingested. The window clamp is value-precise: the preceding-hour-mean radiation keys are clamped on their re-stamped interval-start instant (t − 1h), all other values on the provider stamp t — so the last in-window radiation hour (carried by the row at t = to + 1h) survives and no radiation lands before from. A response reporting utc_offset_seconds != 0 logs a warning (timestamps are parsed as UTC; requests must not carry &timezone).
  • static feed_archive_range(feeder: IngestFeeder, lat: float, lng: float, from: time, to: time, variables: Array<OpenMeteoVariable>, archive_base_url: String?, api_key: String?) — IO loop (archive endpoint): backfill OBSERVED rows over [from, to] into feeder from Open-Meteo’s archive (ERA5) endpoint. Chunks the window into ≤1-year requests (end-inclusive, +1-day step to avoid overlap); each chunk’s rows are clamped to the branch window [from, to]. The request pins wind_speed_unit=ms, so wind_speed_10m-derived u_10m/v_10m and wind_gusts_10m arrive in true m/s (the provider default is km/h, which would store every wind value 3.6× too high). The request date windows (start_date/end_date) are formatted in UTC, and each response’s HTTP status is checked — a non-200 throws (the URL is omitted from the message because it may carry an api key), as does a 200 chunk whose body is empty or unparseable (shared Ingest::require_body guard). archive_base_url defaults to the public archive endpoint. The non-_instant radiation columns (shortwave_radiation, direct_radiation, diffuse_radiation, direct_normal_irradiance, global_tilted_irradiance, terrestrial_radiation) are means over the preceding hour stamped at the interval end by the provider; the converter re-stamps them one hour earlier (interval start) so the same physical hour lands on the same instant as CAMS/Solcast/PVGIS. (OpenMeteoProvider::feed_history is the public entry point.)
  • static feed_forecast_horizon(feeder: IngestFeeder, lat: float, lng: float, now: time, variables: Array<OpenMeteoVariable>, forecast_base_url: String?, api_key: String?) — IO loop (forecast endpoint): pull the FORECAST horizon (issued now) into feeder from Open-Meteo’s forecast endpoint. Plans the fetchable window with forecast_window (the native horizon [now, now+15d] clamped to Open-Meteo’s 16-day horizon — 16 days including today, the last valid end_date is UTC-today + 15 days), returning early when nothing is fetchable. Same wind_speed_unit=ms pin, UTC date windows, status/body guards, and radiation re-stamping as feed_archive_range. forecast_base_url defaults to the public forecast endpoint. (OpenMeteoProvider::feed_forecast is the public entry point.)
  • Pure request/planning helpers (no IO — the unit-testable seams geocode/feed_archive_range/feed_forecast_horizon delegate to):
    • static build_api_key_param(api_key: String?): String — the &apikey=<url-encoded key> suffix, or "" for null.
    • static build_geocode_url(name: String, count: int?, language: String?, country_code: String?, base_url: String?, api_key: String?): String — the geocoding search URL (free-text params URL-encoded).
    • static geocode_decode(body: OpenMeteoGeocodingResponse?): Array<OpenMeteoGeocodingResult> — unwrap a geocoding body: empty array for a null body / missing results; throws on an error: true payload with the provider’s reason.
    • static build_hourly_param(variables: Array<OpenMeteoVariable>): String — comma-join the requested variables for the hourly= query value.
    • static build_archive_url(base_url: String?, lat: float, lng: float, start: time, end: time, variables: Array<OpenMeteoVariable>, api_key: String?): String / static build_forecast_url(…) (same parameters) — one chunk’s archive/forecast URL: whole-UTC-day start_date/end_date, wind_speed_unit=ms pinned.
    • static chunk_windows(from: time, to: time, span: duration): Array<Tuple<time, time>> — split [from, to] into end-inclusive windows of at most span, each next window starting one day past the previous end.
    • static forecast_window(from: time, to: time, now: time): Tuple<time, time>? — the forecast portion of [from, to] clamped to the 16-day horizon; null (with a warning when skipping) when no fetchable forecast hours remain.

OpenMeteoGeocodingResult (volatile)

Deserialized geocoding result.

Required fields: id: int, name: String, latitude: float, longitude: float.

Optional fields: elevation: float?, feature_code: String?, country_code: String?, country_id: int?, country: String?, timezone: String?, population: int?, postcodes: Array<String>?, admin1..4: String?, admin1_id..admin4_id: int?.

OpenMeteoGeocodingResponse (volatile)

Envelope returned by the geocoding API. Either results: Array<OpenMeteoGeocodingResult>? on success, or error: bool? / reason: String? on failure.

OpenMeteoResponse (volatile)

Internal deserialization shape for the archive and forecast endpoints. hourly is a Map<String, any?>? (and hourly_units a Map<String, String>?) — nullable so a degraded/empty 200 body deserializes and is skipped by the scatter instead of throwing at parse time. The "time" entry is Array<String>, every other entry is Array<float?> (or Array<int?> for integer-valued variables such as precipitation_probability and cloud_cover) keyed by the Open-Meteo variable name. Integer columns are coerced to float as each record is streamed. You normally don’t handle this yourself; feed_archive_range / feed_forecast_horizon (driven by OpenMeteoProvider) consume it internally.

NasaPowerService

Connector for the NASA POWER hourly point API — no API key, history-only, hourly data from 2001-01-01 onward (the often-quoted 1981 start applies to the daily API).

Methods:

  • static default_parameters(): Array<String> — The PARAM list requested from NASA POWER: T2M, T2MDEW, ALLSKY_SFC_SW_DWN, ALLSKY_SFC_SW_DNI, ALLSKY_SFC_SW_DIFF, WS10M, WD10M, PRECTOTCORR, RH2M, PS. Wind is requested as raw speed (WS10M) + direction (WD10M); they are decomposed inline into the u_10m/v_10m components the WindFamily stores.
  • static mappings(): Array<VarMapping> — Declarative mapping from NASA POWER parameter names to unified family/signal pairs. Emits u_10m/v_10m provider keys (from the WS10M/WD10M decomposition) mapped to WindFamily. PS (kPa) → Pressure/surface (hPa via nasa_kpa_to_hpa).
  • Importer streams internally (no exposed pure converter). NASA POWER data is PARAM-major (param → {ts → value}); the importer drives off one parameter’s timestamp keys (the old TS-major by_ts pivot is gone) and feeds each PARAM’s value per "YYYYMMDDHH" timestamp (parsed as UTC) straight into the signals. -999/-999.0 fill cells are skipped, integer values coerced to float, and WS10M+WD10M decomposed to u_10m/v_10m inline.
  • static feed_from_nasa_power(station: node<WeatherStation>, from: time, to: time, base_url: String?) — Full pipeline: pulls GET /api/temporal/hourly/point over [from, to] (chunked into ≤1-year requests, end-inclusive with a +1-day step to avoid overlap), always historical: true, streaming each response into the station’s signals via IngestFeeder. base_url defaults to https://power.larc.nasa.gov. Throws if from > to, on a non-200 response, or on a 200 with an empty or unparseable body (Ingest::require_body).
  • Pure request helpers (no IO): static build_parameters_param(params: Array<String>): String — comma-join the PARAM list for the parameters= query value; static build_hourly_point_url(base_url: String, lat: float, lng: float, start: time, end: time, parameters: String): String — one chunk’s hourly point URL (YYYYMMDD UTC day bounds, time-standard=UTC, community=RE, format=JSON).

NasaPowerResponse (volatile)

Deserialization shape: NasaPowerResponse { properties: NasaPowerProperties }; NasaPowerProperties { parameter: Map<String, Map<String, any?>> }. parameter is keyed by PARAM name; each inner map is keyed by "YYYYMMDDHH" UTC strings → numeric values (NASA’s -999 sentinel marks missing data and is skipped by the converter).

nasa_kpa_to_hpa(v: float): float

Free function. Converts NASA POWER surface pressure (PS, reported in kPa) to hPa (v * 10). Wired as the convert function for the PS mapping.

var station = WeatherStation::get_or_create(geo{ 49.594, 6.119 }, "esch");
station->enable(Array<type>{ TemperatureFamily, WindFamily, RadiationFamily, PressureFamily });
// NASA POWER is history-only — request a past window
NasaPowerService::feed_from_nasa_power(station, time::now() - 365_day, time::now() - 1_day, null);
var last_temp = station->signal_for(TemperatureFamily, "air_2m")->observed.last();

PvgisService

Connector for the PVGIS v5.3 (Photovoltaic Geographical Information System, JRC) hourly radiation API. History-only (no forecast) and key-free (public, rate-limited to roughly 30 requests/second).

Methods:

  • static mappings(): Array<VarMapping> — Declarative mapping from PVGIS hourly keys to unified family/signal pairs: the synthetic ghi_sumRadiationFamily/"ghi" (see below), Gd(i)RadiationFamily/"dif", T2mTemperatureFamily/"air_2m". Wind (WS10m) is intentionally not mapped — see the limitation note below — and Gb(i) is not ingested on its own (see the note below). A components=1 response (the form feed_from_pvgis requests) carries no G(i) — the global irradiance is replaced by its components — so the importer reconstructs ghi per record as Gb(i)+Gd(i)+Gr(i) and feeds it under the synthetic ghi_sum key (a record missing one component skips the reconstruction with a warning, once per response).
  • Importer streams internally (no exposed pure converter; the streaming path is Map-free and clamps to [from, to] inline). Each hourly record’s time field is parsed with the "%Y%m%d:%H%M" format (note the colon; PVGIS stamps records at the satellite scan minute, 10 minutes past the hour) explicitly as UTC — PVGIS seriescalc stamps are UTC, so a non-UTC host global timezone can no longer shift the epoch — then floored to the hour so the same physical hour lands on the same instant as CAMS/Solcast (interval-START convention) instead of forming a disjoint :10 twin series; integer-valued cells are coerced to real floats as the record is fed.
  • static feed_from_pvgis(station: node<WeatherStation>, from: time, to: time, base_url: String?) — Full pipeline: builds the seriescalc URL for the station’s location, requests the JSON time-series, and streams it into the station’s signals via IngestFeeder (always historical: true). The endpoint only accepts whole-year bounds and returns every hour of the requested years, so to bound memory this requests one year at a time (startyear == endyear == y, looping over the UTC years of from/to, like NasaPowerService) instead of building one giant multi-year response, and clamps each year’s points to the requested [from, to] window inline as it streams. base_url defaults to https://re.jrc.ec.europa.eu. Throws if from > to, on a non-200 response, or on a 200 with an empty or unparseable body (Ingest::require_body).

No wind direction: PVGIS reports only wind speed (WS10m) and no direction. WindFamily stores meteorological u/v components, which cannot be computed without a direction. Wind therefore cannot be ingested from PVGIS, so WS10m is left unmapped and Ingest::feed silently skips it.

Gb(i) is not ingested as dni: PVGIS Gb(i) is the beam component on the requested plane (here horizontal, angle=0) — not beam-normal irradiance; the two differ by a 1/cos(solar zenith) factor, unbounded near sunrise/sunset. CAMS (BNI) and Solcast (dni) feed true beam-normal into Radiation/dni, so PVGIS contributes no dni to keep the canonical series physically consistent — Gb(i) only enters the Gb(i)+Gd(i)+Gr(i) sum behind the synthetic ghi_sum key (with angle=0 that sum is the global horizontal irradiance; Gr(i) is ~0 on the horizontal plane but is summed regardless).

PvgisResponse / PvgisOutputs (volatile)

Internal deserialization shape. PvgisResponse { outputs: PvgisOutputs? }; PvgisOutputs { hourly: Array<Map<String, any?>>? } — both nullable so a 200 with a non-seriescalc body is skipped instead of null-derefing the feed. Each hourly record is a Map<String, any?> because PVGIS keys (G(i), Gb(i), …) are not valid GCL field names; numeric cells arrive as int or float and are coerced to float as each record is streamed.

MeteostatService

Connector for Meteostat, offering two ingestion paths: a LIVE JSON path (point/hourly via RapidAPI, requires a key) and a BULK CSV path (free, no auth).

The public entry point is MeteostatProvider (a WeatherProvider): it carries the rapidapi_key/base_url as fields, so they are off every method signature, and the history-vs-current distinction is the method you call (no historical flag). It serves current + observed history and exposes station discovery; Meteostat has no forecast product, so feed_forecast is unsupported.

var p = MeteostatProvider::new("RAPIDAPI_KEY");      // or MeteostatProvider { rapidapi_key: "…", base_url: "…" }
p.feed_history(station, from, to);                    // observed hourly over [from, to]
p.feed_current(station);                              // most recent observed 24h
p.feed_bulk_csv(station, "/path/to/<station>.csv");   // gunzip'd bulk dump (observed)
var disc = p.station_discovery()!!;                   // MeteostatDiscovery
var nodes = disc.register_bbox(sw, ne, null);         // stations in a bbox -> WeatherStation nodes

MeteostatProvider delegates to the MeteostatService statics below (which remain the testable, lower-level surface). The feed_bulk_csv(station, path) instance method is provider-specific (bulk file formats differ per provider) and is not on the WeatherProvider contract; the gunzip prerequisite is unchanged (see the bulk recipe below).

Model substitution caveat: the point/hourly endpoint fills missing records with model data (&model=true, the provider default, pinned explicitly in build_hourly_url), and the date range is day-granular — so a window ending today (every feed_current) returns the full current UTC day, including model predictions for hours that have not happened yet. The importer clamps historical records to t <= now (warn-free, expected on every feed_current), so those future model hours never land in observed as truth; past gaps may still be model-filled (Meteostat’s documented gap-filling behavior).

Methods:

  • static mappings(): Array<VarMapping> — Declarative mapping from the provider keys emitted by the importer to unified family/signal pairs. All unit conversions and the wind u/v decomposition happen inline during ingest (the LIVE streaming path and the BULK converter), so these mappings carry no convert function. Keys: temp→Temperature/air_2m (°C), dwpt→Temperature/dew_point (°C), rhum→Humidity/relative (%), prcp→Precipitation/total (mm), u_10m/v_10m→Wind/u_10m,v_10m (m/s), gust→Wind/gust (m/s), pres→Pressure/msl (hPa), snow_m→Snow/depth (m), tsun_s→Cloud/sunshine_duration (s). coco (condition code) is deserialized but intentionally not ingested — there is no Conditions:: mapper for Meteostat’s coco vocabulary.
  • LIVE JSON importer streams internally (no exposed pure converter). Timestamps arrive as "YYYY-MM-DD HH:MM:SS" (UTC, because the endpoint is queried with tz=UTC) and are parsed with "%Y-%m-%d %H:%M:%S". The unit conversions below are applied and integer JSON fields (rhum, wdir, snow) coerced to float as each record is streamed.
  • static feed_live(station: node<WeatherStation>, from: time, to: time, rapidapi_key: String, historical: bool, base_url: String?) — Full LIVE pipeline. Chunks [from, to] into ≤30-day requests (the Meteostat hourly endpoint serves at most 30 inclusive calendar dates per request; end-inclusive with a +1-day step to avoid overlap), issuing GET <base_url>/point/hourly?lat=<>&lon=<>&start=<YYYY-MM-DD>&end=<YYYY-MM-DD>&tz=UTC per chunk with headers x-rapidapi-key and x-rapidapi-host: meteostat.p.rapidapi.com, and streams each response into the station’s signals via IngestFeeder (one feeder reused across chunks). base_url defaults to https://meteostat.p.rapidapi.com. Throws if from > to, on a non-200 response, or on a 200 with an empty or unparseable body (Ingest::require_body).
  • Pure request helpers (no IO — feed_live delegates to them): static chunk_windows(from: time, to: time): Array<Tuple<time, time>> — the ≤30-day window tiling (end-inclusive, +1-day step); static build_hourly_url(base_url: String, lat: float, lon: float, start: time, end: time): String — one chunk’s point/hourly URL (%Y-%m-%d UTC bounds, tz=UTC, and &model=true — the provider’s default model-substitution pinned explicitly: missing records are filled with model data); static rapidapi_headers(rapidapi_key: String): Map<String, String> — the x-rapidapi-key/x-rapidapi-host header pair.
  • static meteostat_bulk_to_points(rows: Array<Array<String>>, historical: bool): Array<WeatherPoint>Internal pure converter for the BULK hourly CSV, kept as the unit-test boundary over small canned rows (the production feed_bulk_csv path streams row-by-row instead of building this array). The CSV is headerless and positional — column order: date(YYYY-MM-DD), hour(HH), temp, dwpt, rhum, prcp, snow, wdir, wspd, wpgt, pres, tsun, coco. The timestamp is rebuilt from date+hour (parsed with "%Y-%m-%dT%H"). Cells parsed with the non-throwing CsvImport::parse_cell (int coerced to float); blank / sentinel / unparseable cells and short rows are skipped rather than aborting the bulk file, and feed_bulk_csv enables trim to strip stray \r/spaces. Same unit conversions as the LIVE path.
  • static feed_bulk_csv(station: node<WeatherStation>, path: String, historical: bool) — Reads a decompressed bulk CSV at path via CsvReader<Array<String>> (headerless, comma-separated) and streams it row-by-row into the station’s signals via IngestFeeder — no raw-rows buffer and no points buffer; only the current row’s small values Map is ever resident.

Bulk download recipe (no native gzip): Meteostat serves bulk files gzip-compressed at https://data.meteostat.net/hourly/<station>.csv.gz (the new host; legacy bulk.meteostat.net stops updating Jan 2026). GreyCat’s std library has no gzip decompressor, so download and gunzip externally first, then pass the resulting .csv to feed_bulk_csv:

curl -sL https://data.meteostat.net/hourly/10637.csv.gz | gunzip > 10637.csv

Unit conversions (both LIVE and BULK paths): wind wspd/gust wpgt km/h → m/s (÷3.6); wind decomposed to u/v from speed (m/s) + direction wdir; snow depth snow mm → m (÷1000, emitted as snow_m); sunshine tsun minutes → seconds (×60, emitted as tsun_s — the BULK path consumes positional column 11; column 12 coco is skipped). Types: MeteostatResponse { data: Array<MeteostatRecord> } (volatile) and MeteostatRecord (volatile) with time: String plus nullable measurement fields (integer fields typed any?, coerced to float).

VisualCrossingService

Connector for the Visual Crossing Timeline API (a single endpoint serving both history and forecast). Handles wind u/v decomposition automatically and normalizes the provider’s native units: wind speed/gust km/h → m/s (÷3.6) and snow depth cm → m.

Methods:

  • static cm_to_m(v: float): float — Centimetres → metres (v / 100). Wired as the convert function for snow depth in mappings().
  • static km_to_m(v: float): float — Kilometres → metres (v * 1000). Wired as the convert function for visibility in mappings() (unitGroup=metric reports visibility in km; the canonical signal stores metres).
  • static mappings(): Array<VarMapping> — Declarative mapping from Visual Crossing hourly keys to unified family/signal pairs (temp→Temperature/air_2m, feelslike→Temperature/apparent, dew→Temperature/dew_point, humidity→Humidity/relative, precip→Precipitation/total, precipprob→Precipitation/probability, pressure→Pressure/msl, cloudcover→Cloud/total, solarradiation→Radiation/ghi, uvindex→Radiation/uv_index, visibility→Cloud/visibility via km_to_m, snow→Snow/snowfall, snowdepth→Snow/depth via cm_to_m). Wind speed/direction → "u_10m"/"v_10m" and gust → "gust" (m/s) inline as each record is streamed.
  • Importer streams internally (no exposed pure converter). It iterates days[].hours[], timestamps each hour from datetimeEpoch (time::new(.., DurationUnit::seconds)), coerces integer JSON columns to float, converts windspeed/windgust km/h → m/s and decomposes speed + direction to u/v (removing the raw wind keys) inline. The forecast issue time is resolved once per import by feed_from_visualcrossing (time::now() when !historical, else null), so every chunk’s forecast targets share one issued-at.
  • static feed_from_visualcrossing(station: node<WeatherStation>, location: String, from: time, to: time, api_key: String, historical: bool, base_url: String?) — Full pipeline: chunks [from, to] into ≤1-year requests, building a Timeline URL per chunk (dates "%Y-%m-%d"; location may be "lat,lng" or free-text and is URL-encoded), sending each GET, checking status, and streaming each response into the station’s signals via IngestFeeder. base_url defaults to https://weather.visualcrossing.com. Throws if from > to or on a non-200 response — and also on a 200 chunk whose body is empty or unparseable ("Visual Crossing: 200 response with an empty or unparseable body (<d1>..<d2>)", with the std HttpResponse.error_msg deserialization detail appended when present, via the helper static require_body(body, d1, d2, error_msg)); previously such a chunk was silently skipped, leaving an invisible up-to-one-year hole. When historical=false, points flow through the forecast issue-time matrix; because the Timeline request is date-granular, today’s already-elapsed hours arrive too — rows whose target precedes the (batch-hoisted) issue time are dropped by the library-wide hindsight guard in the ingest seam (IngestFeeder::feed_value, see Ingest), so a hindsight “forecast” issued after its target can’t win latest_forecast or inflate skill comparisons.

Forecast-as-of: because Visual Crossing forecasts are written through set_forecast, every issue for a target time is retained. Feed the same forward window every few hours and latest_forecast(target) always returns the most recent run, while forecast_as_of(target, cutoff) replays the forecast as published at any past cutoff.

VisualCrossingResponse / VisualCrossingDay (volatile)

Internal deserialization shapes for the Timeline endpoint. VisualCrossingResponse holds latitude/longitude plus days: Array<VisualCrossingDay>; each VisualCrossingDay carries datetime plus hours: Array<Map<String, any?>> (numeric values may arrive as int or float — coerced to float by the converter).

NoaaService

Connector for NOAA / US National Weather Service data. Two independent paths: a LIVE path against api.weather.gov (US only, no API key, but a User-Agent header is mandatory), and a GHCN-Daily bulk CSV path (global daily summaries, no auth) served from S3 in long format.

Methods:

  • static mappings(): Array<VarMapping> — Declarative mapping for the LIVE path: temperature→Temperature/air_2m (°C), dewpoint→Temperature/dew_point (°C), relativeHumidity→Humidity/relative (%), u_10m/v_10m→Wind/u_10m,v_10m (m/s), barometricPressure→Pressure/surface (hPa, the station/surface pressure), seaLevelPressure→Pressure/msl (hPa, reduced to mean-sea-level), precipitationLastHour→Precipitation/total (mm). All conversions to canonical units happen inline as each measure is streamed (via noaa_value, driven by each measure’s WMO unitCode), so convert is null here. The unitCode == ladder is now a Map<String, function> lookup built once.
  • Importer streams internally (no exposed pure converter). Per GeoJSON feature it reads each NoaaMeasure via noaa_value, skipping any with a null value (NWS reports many quantities as null). Unit-aware: rather than hard-coding km/h÷3.6 and Pa÷100, each measure is converted to the family’s canonical unit based on the WMO unitCode it reports — handling temperature (degC/degF/K → °C), pressure (Pa/hPa/kPa → hPa), wind speed (km_h-1/m_s-1 → m/s), percent, degree_(angle), and precip length (mm/m → mm). A measure whose unitCode is missing or unrecognized is skipped with a warning (never silently mis-scaled by a wrong factor, and never assumed to already be canonical). Wind speed (already normalized to m/s) + direction are then decomposed into u_10m/v_10m (raw keys removed). Timestamps parsed with Ingest::parse_time(ts, "%Y-%m-%dT%H:%M:%S%z", null). The internal streaming converter (noaa_scatter) returns a NoaaScatterStats (count: int — features processed, oldest: time? — oldest parsed timestamp) so feed_from_noaa can detect a truncated full page.
  • static noaa_value(key: String, measure: NoaaMeasure?, converters: Map<String, function>): float? — Read one optional NWS measure to its canonical-unit float, honoring the per-measure WMO unitCode (see the unit list above). converters is the unit_converters() table, built once per import and reused for every measure. Returns null for a null measure/value, and for a missing or unrecognized unitCode warns and returns null rather than guessing.
  • static put_noaa(values: Map<String, float>, key: String, measure: NoaaMeasure?, converters: Map<String, function>) — Resolve one measure via noaa_value and store it in values under key only when non-null.
  • static build_observations_url(base_url: String, station_id: String, start: time, end: time): String — Pure request construction (no IO): the station observations URL for one [start, end] window, bounds formatted as UTC %Y-%m-%dT%H:%M:%SZ.
  • static next_page_end(stats: NoaaScatterStats, chunk_start: time, page_end: time, query: String): time? — Pure re-page decision (no IO) for one fetched page: returns the next end bound to re-request, or null when re-paging must stop (a non-full page, the window start reached, a full page with zero parseable timestamps, or a non-shrinking window — the latter two warn instead of looping).
  • static feed_from_noaa(station: node<WeatherStation>, station_id: String, from: time, to: time, user_agent: String, base_url: String?) — Full LIVE pipeline. Chunks [from, to] into ≤7-day windows, building <base_url>/stations/<station_id>/observations?start=<ISO>&end=<ISO> per chunk, sending a GET with the required User-Agent header (pass a contact string; api.weather.gov rejects requests without one), checking status, and streaming each response as historical into the station’s signals via IngestFeeder (one feeder reused across chunks). The NWS endpoint silently truncates an over-full window to its newest 500 features (HTTP 200, no pagination Link), so each window is re-paged: start stays fixed and end walks down past the oldest feature seen until a page returns fewer than 500 features (or the window start is reached) — sub-hourly (ASOS) stations no longer silently lose the oldest ~5 days of each chunk. On a full page with no parseable timestamps, or a non-shrinking window, it warns and stops rather than looping. base_url defaults to https://api.weather.gov. Throws if from > to, on a non-200 response, or on a 200 with an empty or unparseable body (Ingest::require_body — previously such a page silently imported nothing).
  • static ghcn_mappings(): Array<VarMapping> — Mapping for the GHCN-Daily path. Because the GHCN stream already pivots and scales into canonical signal names, keys line up 1:1: air_2m→Temperature/air_2m (°C), total→Precipitation/total (mm), snowfall→Snow/snowfall (cm), depth→Snow/depth (m). The recognized GHCN-Daily LONG format elements (ID,DATE,ELEMENT,DATA_VALUE,…, one row per (DATE,ELEMENT)) are scaled and renamed during ingest: TAVG (tenths °C, ÷10) → air_2m; PRCP (tenths mm, ÷10) → total; SNOW (mm → cm, ÷10) → snowfall; SNWD (mm → m, ÷1000) → depth. TMAX/TMIN are skipped — there are no max/min signals in the unified contract. Rows with a non-empty Q_FLAG (column 5 — the value failed a NOAA quality-assurance check, e.g. D/G/I/L/O/X) are skipped. GHCN dates (YYYYMMDD) are parsed explicitly as UTC, so daily points always land at UTC midnight regardless of the host’s global timezone.
  • static feed_ghcn_csv(station: node<WeatherStation>, path: String, historical: bool)Streams the CSV at path (header_lines: 1) one row at a time through a GhcnAccumulator, which coalesces consecutive same-DATE rows into one point and flushes on each date change. Real by-station files are not date-sorted — they mirror the .dly layout (year-month block, then ELEMENT, then day), so same-date rows are rarely contiguous and in practice one single-element point flushes per row. Correctness is order-independent: each value lands independently at its (t, signal) via feed_value, and only the current date’s small values Map is ever live (no full rows buffer, no by_date Map-of-Maps over the whole history). Source files: https://noaa-ghcn-pds.s3.amazonaws.com/csv/by_station/<ID>.csv.

NoaaMeasure / NoaaObsResponse / NoaaScatterStats (volatile)

NoaaMeasure { value: any?; unitCode: String? } — one measured quantity (value is any? because NWS reports some quantities as JSON integers; a null value means the station didn’t report it and is skipped). NoaaObsResponse { features: Array<NoaaFeature>? } (nullable, so a degraded body deserializes instead of throwing); NoaaFeature { properties: NoaaObsProps }; NoaaObsProps carries the ISO-8601 timestamp plus optional temperature, dewpoint, windSpeed, windDirection, relativeHumidity, barometricPressure, seaLevelPressure, precipitationLastHour (each a NoaaMeasure?). NoaaScatterStats { count: int; oldest: time? } — one page’s processed-feature count and oldest parsed timestamp, returned by the streaming converter and consumed by next_page_end.

MeteoblueService

Connector for the meteoblue packages API. meteoblue exposes data as packages (e.g. basic-1h) that map roughly 1:1 to weather families; this connector requests a single <package> and ingests its data_1h block. Handles wind u/v decomposition automatically and supports meteoblue’s opt-in HMAC-SHA256 URL signing.

Methods:

  • static mappings(): Array<VarMapping> — Declarative mapping from meteoblue data_1h keys to unified family/signal pairs: temperature→Temperature/air_2m (°C), felttemperature→Temperature/apparent (°C), relativehumidity→Humidity/relative (%), precipitation→Precipitation/total (mm), sealevelpressure→Pressure/msl (hPa), gust→Wind/gust (m/s), ghi_instant→Radiation/ghi (W/m²), dni_instant→Radiation/dni (W/m²), dif_instant→Radiation/dif (W/m²), totalcloudcover→Cloud/total (%). Wind speed/direction (windspeed/winddirection) are converted to "u_10m"/"v_10m" (m/s) inline as each record is streamed. meteoblue’s snowfraction is a dimensionless 0…1 ratio (fraction of precipitation falling as snow), not a snowfall depth in cm, so it is intentionally not mapped. meteoblue key names vary by package, so the table maps the common ones; on a provider_key collision the last VarMapping wins.
  • Importer streams internally (no exposed pure converter). It iterates the data_1h columns (same shape as Open-Meteo’s hourly), parses each "YYYY-MM-DD HH:MM" timestamp with "%Y-%m-%d %H:%M" as UTC — guaranteed because the request forces tz=UTC; without it the provider auto-detects the location’s local timezone, which would shift every ingested point by the UTC offset — coerces integer columns to real floats, and decomposes windspeed+winddirection into u_10m/v_10m (removing the raw keys) inline. The forecast issue time is hoisted once for the whole batch.
  • static build_query_path(lat: float, lng: float, package: String, api_key: String): String — Pure request construction (no IO): the path+query "/packages/<package>?lat=..&lon=..&apikey=..&format=json&tz=UTC" (package/api_key URL-encoded; tz=UTC sits inside the string that gets HMAC-signed).
  • static build_signed_url(base_url: String?, lat: float, lng: float, package: String, api_key: String, shared_secret: String?): String — Pure URL builder (no IO): base_url ?? "https://my.meteoblue.com" + build_query_path(...), HMAC-SHA256-signing exactly that path+query (including tz=UTC) and appending &sig=<hex> when shared_secret is non-null.
  • static feed_from_meteoblue(station: node<WeatherStation>, package: String, api_key: String, shared_secret: String?, base_url: String?, historical: bool) — Full pipeline: builds the (optionally signed) URL via build_signed_url, sends a single GET (no date-range chunking is possible — the package URL carries no date range), checks status, and streams the response into the station’s signals via IngestFeeder. Throws on a non-200 response and on a 200 with an empty or unparseable body (Ingest::require_body — a single-request connector would otherwise silently import nothing).

Optional HMAC signing: when shared_secret is non-null, the connector signs the path+query string (the exact build_query_path output, including tz=UTC) with Crypto::sha256_hmac_hex(query_path, shared_secret) and appends the result as &sig=<hex> (meteoblue’s signed-URL scheme). When shared_secret is null, the request is sent unsigned (plain apikey auth).

MeteoblueResponse (volatile)

Internal deserialization shape. data_1h is a Map<String, any?> — the "time" entry is Array<String>, every other entry an Array<any?> aligned by index (numeric values arrive as int or float, coerced to float by the converter); metadata/units are optional passthrough maps.

SolcastService

Connector for the Solcast radiation & weather API — a single endpoint serving three time horizons under Bearer-token (Authorization: Bearer <api_key>) auth. The horizon is selected by the SolcastKind enum (live, forecast, historic) rather than a magic string, so a typo’d mode is a compile error, not a late HTTP 404 with a silently-wrong historical flag.

Methods:

  • static mappings(): Array<VarMapping> — Declarative mapping from Solcast output parameters to unified family/signal pairs: ghi→Radiation/ghi (W/m²), dni→Radiation/dni (W/m²), dhi→Radiation/dif (W/m²), air_temp→Temperature/air_2m (°C), relative_humidity→Humidity/relative (%), surface_pressure→Pressure/surface (hPa), cloud_opacity→Cloud/total (%), plus u_10m/v_10m→Wind/u_10m,v_10m (m/s) from the wind decomposition. Wind speed is already m/s, so no conversion is applied.
  • Importer streams internally (no exposed pure converter). It picks forecasts when present, else estimated_actuals, decomposes wind_speed_10m (m/s) + wind_direction_10m into u_10m/v_10m and coerces integer JSON columns to floats inline. The forecast issue time is hoisted once for the whole batch.
  • static period_duration(period: String?): duration — Pure parser (no IO) for the record’s ISO-8601 period field ("PT<minutes>M"); a null/malformed/non-positive value falls back to PT60M (the period the URL requests).
  • static kind_path(kind: SolcastKind): String — Pure helper: the URL path segment for a kind (live"live", forecast"forecast", historic"historic").
  • static build_radiation_url(base_url: String?, kind: SolcastKind, lat: float, lng: float, hours: int?, start: time?, end: time?): String — Pure request construction (no IO): the radiation_and_weather URL for kind; hours is the live/forecast window-length selector, start/end (ISO-8601 UTC instants) key the historic endpoint — each appended only when supplied.
  • static historic_windows(from: time, to: time, span: duration): Array<Tuple<time, time>> — Pure chunking helper (no IO): split [from, to] into edge-to-edge instant windows of at most span (31 days for Solcast’s historic cap); a boundary-instant re-feed is idempotent.
  • static feed_from_solcast(station: node<WeatherStation>, kind: SolcastKind, hours: int?, from: time?, to: time?, api_key: String, base_url: String?) — Full pipeline: builds the URL(s) via build_radiation_url for kind (SolcastKind::live, ::forecast or ::historic), sends with an Authorization: Bearer <api_key> header, checks status, and streams each response into the station’s signals via IngestFeeder (one feeder shared across requests). Window selection is per kind: live/forecast are bounded by the hours window-length selector (passing from/to throws); historic requires from < to (passing hours throws), requests the range as ISO-8601 UTC start/end query params, and chunks ranges longer than Solcast’s 31-day historic cap into ≤31-day edge-to-edge windows — one request per window, all streamed through one feeder. historical is derived as kind != SolcastKind::forecast, so live/historic write observed truth while forecast routes through the forecast issue-time matrix. base_url defaults to https://api.solcast.com.au. Throws on a non-200 response and on a 200 with an empty or unparseable body (Ingest::require_body).

Timestamp parsing: Solcast stamps each record’s period_end as ISO-8601 with 7 fractional digits + a Z suffix (e.g. "2026-04-21T00:30:00.0000000Z"). time::parse does not accept the fractional+Z form, so the converter slices the leading 19 chars (String.slice(0, 19)) and parses them with "%Y-%m-%dT%H:%M:%S" as UTC. period_end is the end of the averaging interval; the connector re-stamps every point at the interval start (period_end - period, with the record’s period field parsed by period_duration, PT60M fallback) to match the library’s interval-start convention (same as CAMS) — observed and forecast target times shift accordingly.

GTI is not stored: Solcast can return gti (global tilted irradiance), but the frozen RadiationFamily has only ghi/dni/dif. gti is deliberately left unmapped and dropped.

SolcastResponse / SolcastRecord (volatile)

SolcastResponse { estimated_actuals: Array<SolcastRecord>?; forecasts: Array<SolcastRecord>? }live/historic populate estimated_actuals, forecast populates forecasts. SolcastRecord carries period_end: String plus the measured quantities; fields that may arrive as int OR float (ghi, dni, dhi, gti, relative_humidity, wind_direction_10m, cloud_opacity) are any? and coerced.

OpenWeatherMapService

Connector for the OpenWeatherMap One Call 3.0 API (appid key required). Forecast-only: it consumes the hourly block of GET /data/3.0/onecall, which is always a forecast. Requests use units=metric (temperature °C, wind speed m/s, pressure hPa) and exclude=current,minutely,daily,alerts — every block except the consumed hourly is excluded so the response carries no dead payload.

Methods:

  • static mappings(): Array<VarMapping> — Declarative mapping from OWM keys to unified family/signal pairs. Wind speed/direction are decomposed to u/v inline as each record is streamed. OWM pop is a 0…1 fraction, scaled to a percentage via the frac_to_pct converter before being stored in Precipitation/probability (%). OWM snow."1h" is mm, converted to cm via the mm_to_cm converter before Snow/snowfall (cm). uvi → Radiation/uv_index (index) and visibility → Cloud/visibility (metres — the stored unit) map directly, no conversion.
  • static from_owm_code(id: int, is_day: bool): WeatherCondition — Map an OWM condition code (weather[0].id) to the canonical WeatherCondition — the OWM analogue of Conditions::from_wmo/from_vc_icon/from_pictocode: 2xx → thunder; 3xx (drizzle) and 500–504 → rain; 511 (freezing rain) → sleet; 520–531 → showers; 600–602 and 620–622 → snow; 611–613 and 615/616 (“rain and snow” — a rain+snow mix reads as sleet, matching the pictocode-11 convention) → sleet; 771 (squall) and 781 (tornado) → wind; remaining 701–762 (obscuration) → fog; 800 → clear_day/clear_night and 801/802 → partly_cloudy_* via is_day; 803/804 and unknown codes → cloudy. No connector auto-feeds ConditionFamily — record conditions yourself via station->conditions()->set(t, OpenWeatherMapService::from_owm_code(id, is_day)).
  • Importer streams internally (no exposed pure converter). Each hourly entry is streamed: dt (epoch seconds) → time::new(dt, DurationUnit::seconds); wind_speed (m/s, no conversion) + wind_deg decomposed into u_10m/v_10m; wind_gustgust; integer columns coerced to float; the nested rain/snow objects (keyed by "1h") flattened into rain_1h/snow_1h. Issue time hoisted once for the whole batch.
  • static build_onecall_url(base_url: String?, lat: float, lng: float, api_key: String): String — Pure request builder (no IO): the One Call URL with units=metric and exclude=current,minutely,daily,alerts pinned (base_url null ⇒ https://api.openweathermap.org; the key is URL-encoded).
  • static feed_from_openweathermap(station: node<WeatherStation>, api_key: String, base_url: String?) — Full pipeline: reads lat/lng from the station, pulls the build_onecall_url query, and streams the hourly block into the station’s signals via IngestFeeder (always historical: false). Throws on a non-200 response — and the thrown message never echoes the query, so the appid api key is not leaked. A 200 response with an empty/unparseable body also throws rather than silently doing nothing (the shared Ingest::require_body guard). Whole-number JSON fields (pressure/humidity/clouds/wind_deg, etc.) are coerced consistently from int to float. Observed history requires the separate /data/3.0/onecall/timemachine endpoint, out of scope here.

frac_to_pct(v: float): float

Free function. Scales an OWM pop probability fraction (0…1) to a percentage (v * 100). Wired as the convert for the pop mapping.

mm_to_cm(v: float): float

Free function. Converts OWM hourly snow (snow."1h", mm) to centimetres (v / 10). Wired as the convert for the snow_1h mapping.

OwmResponse / OwmHour / OwmWeather (volatile)

OwmResponse { hourly: Array<OwmHour>? } (only hourly is consumed, always treated as a forecast). OwmHour carries dt: int (epoch seconds); every other scalar numeric field (temp, feels_like, dew_point, pressure, humidity, clouds, uvi, visibility, wind_deg, wind_gust, pop) is any? — OWM sends whole numbers as JSON integers, so they are coerced via Ingest::coerce_float — except wind_speed: float?; rain/snow are Map<String, any?>? keyed by "1h"; weather: Array<OwmWeather>? carries the categorical condition codes. OwmWeather { id: int } — one entry of an hour’s weather array; only the OWM condition code is consumed, mapped via from_owm_code.

WeatherDotComService

Connector for the Weather.com / Wunderground PWS (Personal Weather Station) API — the data source behind a Wunderground member’s own stations. Three capabilities behind one mapping table: (1) a live API path (nearby-station discovery, current observations, dated history), (2) a bounding-box discovery + import built on top of it, and (3) archive CSV import/export of the on-disk <ID>/<ID>_YYYYMMDD.csv files the legacy Python downloader produced. Pure GCL — no native code.

Auth is a single apiKey=<key> query param (issued to Wunderground members who run a registered PWS). Every request pins format=json&units=m&numericPrecision=decimalnumericPrecision=decimal is mandatory: the default rounds temperatures/wind to integers and silently degrades the data. Unit-independent fields (solarRadiation W/m², uv, humidity %, winddir°) sit at the top level; unit-dependent fields (temp °C, windSpeed/windGust km/h, pressure hPa, precipTotal mm, dewpt) live under a nested metric object. The API returns two record shapes — instantaneous (observations/current, observations/all/1day) and High/Low/Avg aggregate (history/*, observations/hourly/7day, and the CSV archive) — both normalized into the same canonical keys before mapping.

Field mapping (mappings()):

canonical key family / signal notes
solarRadiation (*High in aggregate) Radiation / ghi W/m², no conversion
uv (uvHigh) Radiation / uv_index index
temp (tempAvg) Temperature / air_2m °C
dewpt (dewptAvg) Temperature / dew_point °C
heatIndex (heatindexAvg) Temperature / apparent °C
humidity (humidityAvg) Humidity / relative %
windSpeed+winddir (windspeedAvg+winddirAvg) Wind / u_10m,v_10m km/h→m/s (÷3.6), decomposed to u/v
windGust (windgustHigh) Wind / gust km/h→m/s
pressure (pressureMax) Pressure / msl hPa
precipTotal Precipitation / total mm

Methods:

The public surface is WeatherDotComProvider (a WeatherProvider — see Provider API above); the WeatherDotComService statics below are the connector internals it composes (live HTTP, the bbox grid/quadtree geometry helpers, the CSV archive importers). Discovery now lives on WeatherDotComDiscovery (see Station discovery below); the archive-exact export lives on the provider as WeatherDotComProvider::export_archive (built on the neutral WeatherExport — see WeatherExport below).

WeatherDotComProvider (the instance API):

  • static new(api_key: String?): WeatherDotComProvider — one configured PWS connection. base_url (null ⇒ https://api.weather.com) and history_granularity (null ⇒ hourly) are plain fields; set them via the record literal WeatherDotComProvider { api_key: ..., base_url: ..., history_granularity: PwsHistoryGranularity::daily }.
  • id(): String"weatherdotcom"; capabilities(): ProviderCaps{ current: true, history: true, forecast: false, discovery: true }.
  • feed_current(station) — pulls the station’s trailing-24 h rapid observations and streams them as observed. Requires station->id. Threads this connection’s api_key/base_url. An idle station (HTTP 204 ‘no data’ — the documented PWS response for an offline station) is skipped with a warning, not an error.
  • feed_history(station, from, to) — backfills [from,to] from /v2/pws/history/{hourly,daily} (chunked into ≤31-day windows) at this provider’s history_granularity (default hourly). Requires station->id. Chunk dates are formatted in the station’s timezone (station.timezone, UTC fallback) because the history endpoints interpret startDate/endDate as the station’s local apparent day — formatting in UTC clipped up to ~half a day at the range edges for stations far from UTC. A 204 ‘no data’ chunk is warned and skipped (the remaining chunks continue).
  • feed_forecast(station)unsupported (PWS has no forecast product); throws weatherdotcom: feed_forecast unsupported.
  • require_key(): String — the configured api key, or throws weatherdotcom: api_key required (a loud, non-secret failure used by every call).
  • station_discovery(): StationDiscovery? — the default discoverer for this connection: a WeatherDotComDiscovery with a 5 km uniform grid. For custom step_km/delay/max_tiles, build a WeatherDotComDiscovery directly and pass it to import_bbox.
  • import_bbox(disc: WeatherDotComDiscovery, sw: geo, ne: geo, mode: PwsImportMode, from: time?, to: time?): Array<node<WeatherStation>> — discover every PWS in [sw,ne] via disc.register_bbox (register as WeatherStation nodes), then ingest each per mode: discover_only returns them untouched, current pulls trailing-24 h obs, history backfills [from,to] at history_granularity. History args and the provider key are validated up front — before the slow, throttled discovery sweep spends its rate budget — except for discover_only, which needs no provider key (the discoverer carries its own). The discoverer’s delay is passed through to the per-station ingest phase too (the most rate-limit-sensitive one). Replaces the old 13-param import_bbox/import_bbox_adaptive statics.
  • export_archive(station: node<WeatherStation>, dir: String, from: time?, to: time?) — serializes the station’s stored signals back to the archive-exact layout <dir>/<ID>/<ID>_YYYYMMDD.csv, one row per observed instant in [from,to] (null bound ⇒ unbounded), grouped into files by the station-local calendar day (station.timezone ?? UTC). High=Low=Avg are written from the single stored value; winddirAvg/windspeed* are recomposed from the u/v components via Ingest::recompose_wind and windgust* from the gust signal; columns the model does not store (windchill*, precipRate, pressureTrend) are left blank; qcStatus is a constant 1. Requires station->id (throws otherwise). Reuses the neutral WeatherExport scaffolding (observed_union/fmt_cell/read_signal_at); the output is byte-for-byte identical to the former WeatherDotComService::export_csv_archive. (api_key/base_url are unused — export is offline.)

WeatherDotComService connector internals (statics):

  • static mappings(): Array<VarMapping> — the canonical-key → family/signal table above, shared by the live API, the bbox importer, and the CSV archive importer. Wind speed/direction are decomposed into u_10m/v_10m in the scatter, so only the components appear here.
  • static near(location: geo, api_key: String, base_url: String?): Array<DiscoveredStation> — one GET /v3/location/near?geocode=LAT,LON&product=pws call → up to 10 nearest PWS, de-parallelized from the response’s parallel-array shape into clean records. base_url defaults to https://api.weather.com. (Wrapped by WeatherDotComDiscovery.)
  • static feed_current(station, api_key, base_url?): bool / static feed_history(station, from, to, granularity, api_key, base_url?): bool — the HTTP workers the provider’s feed_current/feed_history thread credentials into; require station->id (throw otherwise), and feed_history throws if from > to. Both return true when data was ingested and false on an HTTP 204 no-data skip (feed_history: false when every chunk was empty), so batch callers can count skips without try/catch; any other non-200, or a 200 with an unparseable body, still fails loudly.
  • static history_chunk_date(t: time, tz: TimeZone?): String — one history-chunk bound as a YYYYMMDD calendar day in the station’s timezone (tz null ⇒ UTC fallback) — the PWS history endpoints read startDate/endDate as the station-local apparent day.
  • static redact_key(msg: String): String — strip an embedded apiKey=<value> from a message before logging (transport errors may echo the full request URL; no warn/error in this lib may ever carry an API key).
  • static require_history_args(mode, from, to) / static ingest_discovered(stations, api_key, base_url, delay, mode, granularity, from, to) — the validation + per-station ingest the provider’s import_bbox composes after discovery. ingest_discovered is batch-isolated (the sweep before it was paid — it never aborts mid-way): a station with no data (204) counts as skipped; a station whose feed throws is warned (api key redacted via redact_key), counted as failed, and the batch continues. An ok/skipped/failed summary is logged at the end; it throws only when every station failed.
  • static feed_from_csv_archive(path: String) — imports the on-disk archive into the station/signal model. The path kind is auto-dispatched: a single .csv file is imported directly; a directory is walked recursively for *.csv (so both a single <ID>/ station folder and the whole archive root work). The station id is parsed from each filename (<ID>_<YYYYMMDD>.csv), the location from each file’s first lat/lon, and each row is scattered as an observed aggregate point (reusing the live-history scatter). Idempotent (per-timestamp set_observed dedup).
  • static archive_columns(): Array<String> / static join_semicolons(cells) / static archive_row_to_obs(row, index_of) — archive-layout helpers reused by both the importer and the provider’s export_archive: the canonical 33-column header, the ;-join, and the row → synthetic-aggregate-observation adapter.
  • The archive export now lives on the provider as WeatherDotComProvider::export_archive(station, dir, from, to) (see Provider API above), reusing the neutral WeatherExport scaffolding (observed_union/fmt_cell/read_signal_at) and Ingest::recompose_wind. The old WeatherDotComService::export_csv_archive static has been removed.
  • Pure helpers (no IO — exposed for testing): near_decode(body), grid_points(sw,ne,step_km) with tile_box(pts, box, step_km) / grid_axis(v0, v1, step) (the bbox geometry used by WeatherDotComDiscovery: corners go through spatial::GeoBox, so an antimeridian-crossing box is tiled as its two non-crossing lobes instead of the 340° complement; each axis is SW-anchored with ceil semantics — the last row/column lands exactly on the north/east boundary, so the NE edge is never left up to a full step uncovered; throws when step_km <= 0), in_bbox(g,lat0,lat1,lng0,lng1) (axis-aligned only — for crossing boxes use spatial::GeoBox::contains), build_near_url/build_obs_url/build_history_url, obs_values(obs, aggregate), obs_time(obs), scatter(body, feeder, aggregate) with the per-observation scatter_obs(obs, feeder, aggregate).
  • Quadtree helpers are reserved: quad_split/should_subdivide/box_smaller_side_km/box_half_diagonal_km (over PwsBox) have no production caller yetwithin_bbox is currently a uniform grid sweep. They are intentionally kept (with their truth-table tests) for the planned adaptive-densification mode that subdivides only saturated /near cells; do not treat them as dead code.
  • Archive/export internals: feed_csv_file_into(path, file_name, feeders) (one archive file into the per-station feeder map), archive_signals(station): PwsArchiveSignals (the 11 mapped archive signal nodes resolved once per export pass — PwsArchiveSignals is a @volatile holder of 11 nullable signal-node slots; a null slot = family inactive / signal absent, its cells stay blank), archive_signal_list(sigs) (the non-null slots in canonical archive order), and export_signals(station) (retained as a thin wrapper over the two).

/near is a point query, not a bbox query. It returns the ≤10 nearest stations to a single geocode, so “every station in a box” requires tiling + union + clip — WeatherDotComDiscovery::within_bbox does this over a uniform step_km grid (the cos(lat) longitude correction included). Choose step_km small enough that no cluster of >10 stations sits between grid nodes.

Rate limits. A free PWS/contributor key allows ~30 calls/min and ~1500 calls/day. A 5 km grid over a country is hundreds of /near calls, and import_bbox adds one (current) or ⌈days/31⌉ (history) calls per station — use the discoverer’s delay/max_tiles, cache discovery, and prefer wide-window history. Read 429 at runtime.

qcStatus policy. Records are dropped only on an explicit QC failure (qcStatus == 0). qcStatus == -1 (“not checked” — the bulk of historical PWS data and the entire pre-2020 CSV archive) and qcStatus == 1 (“passed”) are kept — dropping < 1 would silently discard nearly the whole archive.

Export is functional, not byte-identical. Because the lib stores one value per (signal, time), export writes High=Low=Avg from that value and re-formats numbers — it round-trips the data (re-importing an export reproduces the signals) but won’t reproduce the rare rows where the original High/Low/Avg diverged, legacy number formatting, or the unstored columns.

Example — discover + import a Lebanon bbox, then archive round-trip:

// 0. One configured PWS connection (history_granularity defaults to hourly).
var p = WeatherDotComProvider::new(my_api_key);

// 1. Per-station feeds (station->id required).
p.feed_current(station);            // trailing-24 h observations
p.feed_history(station, from, to);  // backfill at history_granularity

// 2. Discover every PWS in a Lebanon bbox and pull the last 24 h of observations.
//    Tiling knobs (step_km / delay / max_tiles) live on the discoverer.
var sw = geo { 33.05, 35.10 };
var ne = geo { 34.69, 36.62 };
var disc = WeatherDotComDiscovery { api_key: my_api_key, step_km: 5.0, delay: 2_s };
var stations = p.import_bbox(disc, sw, ne, PwsImportMode::current, null, null);
println("registered ${stations.size()} PWS stations");

// 3. Bulk-import the historical CSV archive (a station folder OR the whole root).
WeatherDotComService::feed_from_csv_archive("./data/archive/weather");

// 4. Export one station back to the identical on-disk archive format
//    (`export_archive` is an instance method on the provider).
var st = weather_station_by_id.get("IBEIRU11");
if (st != null) {
    p.export_archive(st!!, "./export/weather", null, null);
}

DiscoveredStation (volatile)

One station from /v3/location/near, de-parallelized: station_id: String, name: String?, location: geo, distance_km: float?, qc_status: int? (-1 not-checked / 0 fail / 1 pass), update_time_utc: time? (last-obs time), probe: Map<String, float>? (reserved for a future data-aware discovery filter; currently unset). WeatherDotComDiscovery::hit_of maps this to a provider-agnostic StationHit.

PwsNearResponse / PwsNearLocation / PwsObsResponse (volatile)

Internal deserialization shapes. PwsNearResponse { location: PwsNearLocation? }; PwsNearLocation holds the parallel arrays stationId[], stationName[], latitude[], longitude[], distanceKm[], qcStatus[], updateTimeUtc[] (index i is the same station across all arrays). PwsObsResponse { observations: Array<Map<String, any?>>? } — each observation is free-shaped (values arrive as int or float, coerced via Ingest::coerce_float; the nested metric object is read with an as cast).

PwsHistoryGranularity / PwsImportMode (enums)

PwsHistoryGranularity { hourly, daily } is the WeatherDotComProvider.history_granularity field — it selects the feed_history endpoint (hourly observations vs daily summaries). PwsImportMode { discover_only, current, history } selects what WeatherDotComProvider::import_bbox does after discovery (register only / pull current obs / backfill history).

Station discovery

A provider-agnostic station-discovery surface that turns a point or a bounding box into StationHits and (optionally) materializes them into the station graph. WeatherDotComDiscovery wraps the Weather.com PWS connector (the point /near endpoint plus the grid geometry helpers); MeteostatDiscovery composes the Meteostat RapidAPI /stations/nearby + /stations/meta endpoints. The pure mappers (hit_of, collect_near_hits, collect_hits, nearby_candidates, merge_candidates, hit_from_meta) and pure builders (build_nearby_url, lobe_center, lobe_radius_m) carry no IO, so they unit-test against canned data with no network. All bbox geometry goes through the shared spatial::GeoBox (antimeridian-aware — see Spatial).

StationHit (volatile)

One discovered station, provider-agnostic: id: String?, name: String? (either may be null when the provider omits it), location: geo (always present), distance_m: float? (great-circle distance in metres from the query point for point queries; null for bbox sweeps).

StationDiscovery (abstract)

The discovery contract. Two provider-specific network paths plus one shared registration convenience.

max contract (binding for every implementation): null = uncapped (provider default); max <= 0 = empty result. An explicit zero/negative cap caps to nothing — it is never an “uncapped” sentinel — and implementations short-circuit on it before any network call.

bbox corner contract: [sw,ne] boxes follow spatial::GeoBox semantics — latitudes are order-agnostic (south = min, north = max); longitudes are taken as given, the box spanning sw.lng() going east to ne.lng(), so sw.lng() > ne.lng() means an antimeridian-crossing box (not a mistake to auto-correct).

  • abstract near(point: geo, max: int): Array<StationHit> — the stations nearest point, capped at max (max <= 0 ⇒ empty), nearest-first when the provider reports distances.
  • abstract within_bbox(sw: geo, ne: geo, max: int?): Array<StationHit> — every station the provider knows inside [sw,ne], capped at max per the contract above.
  • register_bbox(sw: geo, ne: geo, max: int?): Array<node<WeatherStation>> — runs within_bbox, then registers each hit into the station graph via WeatherStation::get_or_create(location, id) (id-keyed, so re-runs are idempotent); a hit name is copied onto a freshly created station only when the station has no name yet. Returns the registered nodes in discovery order.

WeatherDotComDiscovery

Weather.com / Wunderground PWS discoverer. Fields: api_key: String (PWS key), base_url: String? (defaults to https://api.weather.com), step_km: float (uniform grid spacing for within_bbox), delay: duration? (rate-limit between /near calls), max_tiles: int? (caps grid nodes; loud warn on truncation, no silent cap).

  • static hit_of(ds: DiscoveredStation): StationHit — pure map: a DiscoveredStation (km distance) to a StationHit (metre distance, km×1000).
  • static collect_near_hits(found: Array<DiscoveredStation>, max: int): Array<StationHit> — pure: map a decoded /near candidate list to hits, capped at max (max <= 0empty, per the StationDiscovery contract); near_decode already orders nearest-first.
  • static collect_hits(sw: geo, ne: geo, tiles: Array<Array<DiscoveredStation>>): Array<StationHit> — pure: union+dedup by station_id across the already-fetched per-tile candidate lists, clip each survivor to the [sw,ne] spatial::GeoBox (the nearest-N /near bleeds past the box edges; the box may cross the antimeridian), emit one StationHit per survivor with distance_m = null (bbox sweeps carry no caller-meaningful distance). No network — within_bbox fetches the tiles and passes them here.
  • near(point: geo, max: int): Array<StationHit> — one /v3/location/near point query (≤10 nearest PWS), mapped + capped (max <= 0 ⇒ empty with no network call).
  • within_bbox(sw: geo, ne: geo, max: int?): Array<StationHit> — a uniform step_km grid sweep (one /near per node; an antimeridian-crossing box is swept one non-crossing spatial::GeoBox lobe at a time), deduped+clipped by collect_hits; max_tiles caps grid nodes, delay rate-limits, max truncates the final hit list. Throws when step_km <= 0 (a zero/negative step would tile forever). A failed tile is warned and skipped — the already-fetched (rate-limited, paid) tiles are kept and the sweep continues, an end-of-sweep ok/failed summary is logged, and the call throws only when every tile failed.

MeteostatDiscovery

Meteostat (RapidAPI) station discoverer over the two real /stations endpoints (/stations/nearby returns id + localized name + distance only — no coordinates; /stations/meta?id= returns the full record with the nested location). There is no server-side /stations/bounds query, so coordinates are always resolved with a /stations/meta follow-up per candidate: near issues 1 + N HTTP requests (one nearby + one meta per candidate), and within_bbox is emulated — one covering /stations/nearby probe per non-crossing spatial::GeoBox lobe (lobe centre + circumradius, so 1–2 probes), candidates deduped by id across lobes, one meta per unique candidate, then clipped to the box.

Fields: rapidapi_key: String, base_url: String? (defaults to https://meteostat.p.rapidapi.com), radius_m: int (bounds the near search, metres), nearby_limit: int? (per-lobe bbox probe limit, default 100; throws if explicitly <= 0; a saturated probe warns about possible truncation — raise nearby_limit or shrink the box). The RapidAPI key travels in the request headers, never in a URL.

  • static pick_name(names: Map<String, String?>?): String? — pure: a display name from the localized { "<lang>": name } object — en when present, else the first non-null localization.
  • static nearby_candidates(resp: MeteostatNearbyResponse?, max: int?): Array<MeteostatNearbyRecord> — pure: the usable candidates of a nearby response (non-null id only — an id-less record cannot be meta-resolved), response order (nearest-first), capped at max per the StationDiscovery contract.
  • static merge_candidates(batches: Array<Array<MeteostatNearbyRecord>>): Array<MeteostatNearbyRecord> — pure: union the per-lobe batches, deduplicating by station id (first occurrence wins).
  • static hit_from_meta(rec: MeteostatNearbyRecord, meta: MeteostatMetaRecord?, keep_distance: bool): StationHit? — pure: combine a nearby candidate with its meta record into a StationHit, or null when the meta record carries no usable coordinates; keep_distance keeps the nearby distance on point queries (bbox sweeps pass false so distance_m stays null).
  • build_nearby_url(point: geo, limit: int, radius_m: int): String — pure builder for /stations/nearby?lat&lon&limit&radius.
  • static lobe_center(lobe: spatial::GeoBox): geo / static lobe_radius_m(lobe: spatial::GeoBox): int — pure: a non-crossing lobe’s centre and a covering nearby-probe radius (larger centre→corner distance + 5% margin).
  • fetch_meta(id: String): MeteostatMetaRecord? — one /stations/meta?id= lookup (throws on non-200 / unparseable body; the batch loop catches per item).
  • locate_hits(cands, clip, max) — the shared coordinate-resolution loop (batch semantics): one meta call per candidate; a failed or coordinate-less item is warned and skipped, a summary is logged, and it throws only when every attempted lookup failed outright.
  • near(point: geo, max: int): Array<StationHit> — the max nearest stations: one /stations/nearby call + one /stations/meta per candidate (max <= 0 ⇒ empty with no network call); distance_m kept from the nearby response.
  • within_bbox(sw: geo, ne: geo, max: int?): Array<StationHit> — the emulated bbox sweep described above, truncated to max; bbox hits carry distance_m = null.

Meteostat discovery response types (volatile)

Pinned to the real RapidAPI shapes: MeteostatNearbyResponse { data: Array<MeteostatNearbyRecord>? } with MeteostatNearbyRecord { id: String?, name: Map<String, String?>?, distance: float? } (metres; no coordinates in nearby responses), and MeteostatMetaResponse { data: MeteostatMetaRecord? } (a single record, not an array) with MeteostatMetaRecord { id, name, location: MeteostatStationLocation?, timezone } and MeteostatStationLocation { latitude: float?, longitude: float?, elevation: int? }. All fields are nullable so a partial record degrades gracefully. (These replace the former flat MeteostatStationRecord/MeteostatStationsResponse, whose flat shape never matched the live nested JSON.)

CamsAdapter

Offline adapter for Copernicus climate data: the CDS reanalyses (ERA5 / ERA5-Land) and the ADS CAMS Radiation Service (McClear / CAMS-rad). There is no live synchronous connector — these datasets are NetCDF/GRIB files served through the Copernicus CDS (cds.climate.copernicus.eu) and ADS (ads.atmosphere.copernicus.eu) async download queue (the cdsapi Python client). This adapter ships only the column→signal mapping tables, a pure CAMS-radiation row converter (the unit-test boundary), and a streaming CSV-file feed that reuses the CsvImport / IngestFeeder seam.

Offline recipe (cdsapi → CSV → feed):

  1. Request + download the dataset via cdsapi (the request is queued, then the file is downloaded once ready) — e.g. an ERA5-Land or CAMS-radiation NetCDF/GRIB.
  2. Extract it to CSV with a NetCDF tool (cdo, xarray, or ncdump).
  3. Feed the CSV: for the CAMS Radiation Service use CamsAdapter::feed_cams_radiation_csv(station, path, header_lines); for ERA5-Land use the generic CsvImport::feed(...) with CamsAdapter::era5_land_mappings().

Native alternative (skip the CSV step): the downloaded granules are NetCDF-4/HDF5, so for MERRA-2 / ERA5 / ERA5-Land you can read them directly with NetcdfImport::feed(...) instead of extracting to CSV with cdo/xarray/ncdump. The convert functions and family routing are identical (both funnel through the IngestFeeder streaming seam); use NetcdfImport::era5_mappings() (this table adapted to Array<VarMapping>) for ERA5 / ERA5-Land.

GLOBE vs ESA clarification: “GLOBE” refers to the NASA/NSF GLOBE Program citizen-science observation network (globe.gov) — distinct from ESA. ESA-origin climate data reaches this library via Copernicus CDS (ERA5 reanalysis) and ESA CCI / Sentinel satellite products (NetCDF, offline only); all of these are file-based and ingested through this same offline CSV recipe.

Methods:

  • static cams_radiation_mappings(): Array<CsvColumnMapping> — Column→signal table for the CAMS Radiation Service CSV (the ;-separated McClear/CAMS-rad product): GHI→Radiation/ghi, BNI (beam-normal irradiation = DNI)→Radiation/dni, DHI (diffuse horizontal)→Radiation/dif, all W/m².
  • static era5_land_mappings(): Array<CsvColumnMapping>Example table for a user-extracted ERA5-Land CSV using the common ERA5 short names (adjust to your export’s column names). See the table below.
  • static cams_normalize_header(header: Array<String>): Array<String> — Normalize a parsed CAMS header row: strips a leading '#' (plus surrounding whitespace) from every cell, so the verbatim CAMS comment-header line ("# Observation period;TOA;...;GHI;...") indexes cleanly by column name.
  • static cams_period_to_interval(period_cell: String): Tuple<time, time>? — Parse a CAMS Observation period cell ("<start>/<end>") to its (start, end) instants: both sides sliced to 19 chars (dropping fractional seconds) and parsed with "%Y-%m-%dT%H:%M:%S" as UTC. Returns null when either side is missing, short, or unparseable.
  • static cams_period_to_time(period_cell: String): time? — The interval’s start instant (the interval-start stamping convention). Delegates to cams_period_to_interval, so it now requires both <start>/<end> sides — a lone instant returns null.
  • static cams_radiation_rows_to_points(header: Array<String>, rows: Array<Array<String>>, historical: bool): Array<WeatherPoint>Internal pure converter (network/file-free; the tested boundary, over small canned rows). The header is normalized first (leading '#' stripped via cams_normalize_header), so a verbatim CAMS comment-header works. Each row’s Observation period interval is parsed via cams_period_to_interval: the start instant stamps the point, and each GHI/BNI/DHI Wh/m² cell is divided by the period length in hours to store a mean W/m² for every CAMS product step (1-min/15-min/hourly/daily/monthly — identity for hourly). values is built via the non-throwing CsvImport::parse_cell (integer cells coerced to real floats, blank/sentinel/unparseable cells skipped); one WeatherPoint per row. Rows with a malformed period side, a zero/negative period length, or ragged rows shorter than the period column are skipped (not fatal). The production feed_cams_radiation_csv path streams instead of building this array.
  • static feed_cams_radiation_csv(station: node<WeatherStation>, path: String, header_lines: int?) — Full pipeline: streams the ;-separated CSV at path row-by-row into the station’s signals via IngestFeeder (Map-free, one feed_value per GHI/BNI/DHI cell), using CsvImport::to_var_mappings(cams_radiation_mappings()). The feed always writes observed history (CAMS Radiation is a historical product). With header_lines null the header is auto-detected: the # preamble is scanned until the row whose first cell normalizes to exactly Observation period (the verbatim "# Observation period;TOA;..." comment-header line), which becomes the header; pass header_lines (preamble lines + the 1 column-name line) only as an override for a non-standard export. Each row’s Wh/m² cells are normalized to a mean W/m² (÷ period hours, per row) exactly like cams_radiation_rows_to_points.

ERA5-Land example mapping (era5_land_mappings()):

column family / signal unit converter
t2m Temperature / air_2m °C kelvin_to_celsius (K→°C)
d2m Temperature / dew_point °C kelvin_to_celsius (K→°C)
u10 Wind / u_10m m/s none — ERA5 provides u/v directly (no decomposition)
v10 Wind / v_10m m/s none — direct component
sp Pressure / surface hPa pa_to_hpa (Pa→hPa)
tp Precipitation / total mm m_to_mm (m→mm)
ssrd Radiation / ghi W/m² none — see SSRD caveat

Unit caveats:

  • K → °C (ERA5 t2m/d2m): subtract 273.15 (kelvin_to_celsius).
  • Pa → hPa (ERA5 sp): divide by 100 (pa_to_hpa).
  • m → mm (ERA5 tp, metres of water equivalent): multiply by 1000 (m_to_mm).
  • CAMS irradiation is Wh/m² over the period, not instantaneous W/m². Both cams_radiation_rows_to_points and feed_cams_radiation_csv normalize it automatically per row: each GHI/BNI/DHI cell is divided by the row’s observation-period length in hours (identity for the hourly product), so the stored series is a mean W/m² for every CAMS step. Rows with a zero/negative or malformed period are skipped.
  • Precipitation accumulation stamping differs per provider (no automatic re-stamp is applied — unlike radiation, the library declares no accumulation-stamping convention): Visual Crossing’s precip is the amount falling in the hour starting at the stamp, while Open-Meteo (precipitation), OpenWeatherMap (rain."1h"), Meteostat (prcp) and NOAA (precipitationLastHour) are accumulations for the hour ending at the stamp. Mixing two of these into one station’s total signal offsets hourly totals by 1 h depending on the source — prefer a single precipitation provider per station.
  • ERA5 ssrd is J/m² accumulated over the model step. To recover a mean W/m² divide by the accumulation period in seconds (÷3600 for the hourly product). No converter is wired because the divisor depends on the export’s accumulation step — pre-scale during extraction or supply your own convert.

kelvin_to_celsius(v: float): float · pa_to_hpa(v: float): float · m_to_mm(v: float): float

Free functions wired as convert functions in era5_land_mappings(): Kelvin → Celsius (v - 273.15), Pascal → hectopascal (v / 100), and metre → millimetre (v * 1000).

NetcdfImport

Native NetCDF-4 / HDF5 importer for gridded reanalysis granules — MERRA-2 (GES DISC), ERA5, and ERA5-Land. It reads the granule directly, the native alternative to the CamsAdapter offline “download → extract to CSV → feed” recipe. This is the weather library’s only native © component; the .gclib statically links netcdf-c + HDF5.

Scope (v1): local files only (no OPeNDAP / Earthdata streaming), NetCDF-4 / HDF5 only (not classic NetCDF-3 or GRIB), nearest grid cell (no interpolation), and standard gregorian / proleptic-gregorian CF calendars only (noleap, 360_day, … are rejected with an error).

Methods:

  • static native read_series_columnar_points(path: String, var_names: Array<String>, lat_var: String, lon_var: String, time_var: String, lats: Array<float>, lons: Array<float>): Array<NetcdfColumns> — The batched native core every other entry funnels through: opens path once, finds the grid cell nearest each (lats[p], lons[p]) point, and for each name in var_names reads that point’s single cell across the whole time axis — never the full cube. Batching matters because all file work (open, HDF5 metadata parse, per-variable attribute resolution, chunk decompression) is serialized process-wide behind the non-thread-safe HDF5 library, and nearby points share the same compressed chunks — N single-point calls repeat that work N times. lats/lons must be non-empty, equal-sized, and finite (a NaN target would silently match cell 0 — rejected loudly instead). An extra (non time/lat/lon) dimension is pinned to its first index — with a warn log when its size > 1 — except a legacy-CDS expver dimension (case-insensitive), whose indices are coalesced per timestep to the first non-fill value (ECMWF’s documented ERA5/ERA5T merge), so granules spanning the final/preliminary boundary keep their recent data. Applies CF unpacking (real = raw · scale_factor + add_offset) and CF time decoding (the time variable’s units = "<seconds|minutes|hours|days> since <base>"; the seconds field may be omitted, and a udunits UTC-offset suffix is honored). _FillValue / missing_value (and a variable absent from the granule) become NaN, dropped on the GCL side; a variable declaring neither attribute has the NetCDF default fill for its type (e.g. 9.96921e36 for floats) treated as missing (resolved via nc_inq_var_fill; NC_NOFILL variables get no sentinel). A packing/sentinel attribute that is present but not readable as a number (e.g. a text scale_factor) raises an error rather than silently importing unscaled values. Returns one NetcdfColumns per point, aligned with lats/lons; all entries share the same axis Array object (identical for every point by construction) — treat it as read-only. The native side allocates only O(npoints · nvars · ntime) primitive i64/f64 — no per-timestep WeatherPoint/Map/key-string churn.
  • static read_series_columnar(path: String, var_names: Array<String>, lat_var: String, lon_var: String, time_var: String, lat: float, lon: float): NetcdfColumns — Single-point convenience over read_series_columnar_points (same contract, one target, one NetcdfColumns). Prefer the batched form (or feed_many) when importing several stations from the same granule: each call re-opens the file and repeats the serialized HDF5 work.
  • static feed(station: node<WeatherStation>, path: String, mappings: Array<VarMapping>, lat_var: String, lon_var: String, time_var: String, historical: bool) — Full pipeline (signature unchanged): reads station’s geo, takes the nearest cell, and streams the per-variable columns into the station’s family signals via IngestFeeder — one value at a time, no intermediate Array<WeatherPoint> and (since MERRA-2/ERA5 supply U10M/V10M directly, so no wind decomposition) zero per-row Map. When historical is false, one issue time (time::now()) is hoisted for the whole batch, like every other importer. mappings provider_key must be the NetCDF variable name. Use merra2_mappings() for MERRA-2 or era5_mappings() for ERA5 / ERA5-Land. Delegates to feed_many with a single station.
  • static feed_many(stations: Array<node<WeatherStation>>, path: String, mappings: Array<VarMapping>, lat_var: String, lon_var: String, time_var: String, historical: bool) — Batched feed: reads all stations’ nearest cells in a single granule open (via read_series_columnar_points), then streams each station’s columns through its own IngestFeeder. Use this for fleet backfills over granule sets — per-station semantics are identical to feed, but the serialized per-open HDF5 work is paid once instead of once per station.
  • static merra2_mappings(): Array<VarMapping> — MERRA-2 single-level (e.g. M2T1NXSLV) variable→family table; see below.
  • static era5_mappings(): Array<VarMapping> — ERA5 / ERA5-Land variable→family table: wraps CsvImport::to_var_mappings(CamsAdapter::era5_land_mappings()), so each provider_key is the ERA5 short name (t2m, d2m, u10, v10, sp, tp, ssrd). Use this — not the raw CamsAdapter::era5_land_mappings() table, which is Array<CsvColumnMapping> and does not type-check against feed (GCL generics are invariant). The ssrd accumulated-J/m² caveat applies unchanged.

NetcdfColumns (@volatile): a transient native-build result holding axis: Array<time> (the shared timestep axis) and columns: Array<Array<float>> (one float column per requested variable; column v aligns with var_names[v], row i with axis[i]; a cell is NaN for a fill/missing value or an absent variable). Results from read_series_columnar_points share one axis object across all points — treat it as read-only.

MERRA-2 mapping (merra2_mappings()):

variable family / signal unit converter
T2M Temperature / air_2m °C kelvin_to_celsius (K→°C)
U10M Wind / u_10m m/s none — direct 10 m component
V10M Wind / v_10m m/s none — direct 10 m component
PS Pressure / surface hPa pa_to_hpa (Pa→hPa)
PRECTOT Precipitation / total mm none — see flux caveat
SWGDN Radiation / ghi W/m² none

Notes & caveats:

  • Grid → point is nearest-cell: the chosen cell can differ from the exact station coordinates, more so on coarse grids (MERRA-2 ≈ 0.5° × 0.625°). Longitude is matched with a circular distance, so a station in −180…180 resolves correctly against a 0…360 grid and vice-versa.
  • MERRA-2 PRECTOT is a flux (kg m⁻² s⁻¹ ≡ mm s⁻¹). To store the family’s canonical mm per record multiply by the record interval in seconds (3600 for the hourly product). No converter is wired (the divisor depends on the granule’s step) — supply your own fixed-step convert, mirroring the ERA5 ssrd caveat above.
  • ERA5 / ERA5-Land: use NetcdfImport::era5_mappings() as the mappings (its provider_keys are the ERA5 short names t2m, sp, tp, …); the K→°C, Pa→hPa, m→mm converters and the ssrd accumulation caveat apply unchanged. Legacy-CDS granules with a size-2 expver dimension are handled automatically (per-timestep coalesce, see read_series_columnar_points).
  • Feed .nc files from trusted sources only (GES DISC, the Copernicus CDS/ADS, your own pipeline). The byte-level parse surface for a granule is libnetcdf/libhdf5 — not this importer — and the importer’s own validation (allocation caps, dimension/coordinate sanity checks, loud failures on malformed metadata) cannot vouch for the underlying C libraries against an adversarial file.
  • Build: the weather .gclib links static libnetcdf + libhdf5 (+ zlib), built by deps/netcdf.sh / deps/hdf5.sh. It is native-only (no WASM target).

Derived & analytics layer

Pure, on-demand helpers that compute over the base signals. None of them store anything unless you write the result back into a WeatherSignal yourself.

Note: comfort indices and astronomy are implemented in pure GCL for v1 (matching the library’s “GCL helper first, native later” philosophy), so they are testable without a native rebuild. They are candidates for promotion to native std primitives once proven.

Comfort (abstract)

Human thermal-comfort indices. Inputs use the library’s canonical units: temperature °C, wind m/s, relative humidity %.

  • static heat_index(temp_c: float, rh: float): float — NWS heat index (“feels like” in heat). Below 10 °C (50 °F) the index is undefined, so the air temperature is returned unchanged (mirroring wind_chill and NWS practice — previously the Steadman line was extrapolated below the air temperature). From 50 °F it uses the simple Steadman form while the simple (Steadman-averaged) value stays below 80 °F (~26.7 °C), then the Rothfusz regression (with low/high-humidity adjustments) — matching the NWS reference algorithm.
  • static wind_chill(temp_c: float, wind_ms: float): float — NWS wind chill. Defined for temp_c <= 10 and wind > 4.8 km/h; returns temp_c unchanged outside that domain.
  • static apparent_temp(temp_c: float, rh: float, wind_ms: float): float — Australian apparent temperature (Steadman 1994), including the vapour-pressure and wind terms.
  • static utci(temp_c: float, mrt_c: float, wind_ms: float, rh: float): float — Universal Thermal Climate Index, the operational 6th-order polynomial approximation (Bröde et al., 2012). mrt_c is the mean-radiant temperature; wind is clamped to [0.5, 17] m/s.
  • static saturation_vapour_pressure(temp_c: float): float / static vapour_pressure(temp_c: float, rh: float): float — Magnus/Tetens-form helpers (hPa) with the Australian BOM over-water constant set (6.105 / 17.27 / 237.7 — the exact constants of the BOM apparent-temperature formula). The fit is for saturation over water (roughly 0…50 °C); below 0 °C it returns the supercooled-water value, which exceeds saturation over ice by up to ~10 % — do not use it as an over-ice e_s. Inside utci it substitutes for the reference implementation’s Hardy (1998) ITS-90 formulation, a deliberate simplification worth ~0.05 K of UTCI.
var hi = Comfort::heat_index(32.0, 70.0);          // ~40.5 °C
var utci = Comfort::utci(0.0, 0.0, 5.0, 60.0);     // ~ -14 °C (wind chill)

Astronomical (abstract)

Solar position, sunrise/sunset and moon phase via the NOAA algorithm (accurate to ~a degree / a minute). Independent of the solar library.

  • static sun_position(location: geo, t: time): SunPosition — Solar elevation/azimuth (degrees) — see SunPosition.
  • static sunrise(location: geo, day: time): time? / static sunset(location: geo, day: time): time? — Sunrise/sunset instant for the UTC calendar day containing day — honored for every longitude (the neighbouring solar days are probed so the returned instant always falls inside the requested UTC day); null when no such event occurs in that day. Null is not only a polar (day/night) phenomenon: consecutive events are spaced 1440 ± a few minutes apart in UTC, so at longitudes where the event falls near 00:00 UTC, the rare day on which it drifts across midnight contains no event at all — at perfectly ordinary mid-latitudes (e.g. 45°N 82.5°E, 2023-09-07 has no sunrise). Handle null at every latitude. Note that for |longitude| > ~90° the requested UTC day’s sunset can precede its sunrise in UTC order (e.g. Tokyo on 2023-06-21: sunset ~10:00Z before sunrise ~19:26Z) — don’t assume sunrise < sunset.
  • static moon_phase(t: time): float — Synodic phase fraction: 0 = new, 0.5 = full, approaching 1 = back to new.
  • static moon_illumination(t: time): float — Illuminated fraction of the disk (0 = new, 1 = full).
  • static julian_day(t: time): float, static solar_geom(jd: float): Tuple<float, float> (declination °, equation-of-time min), static norm360(deg: float): float — building blocks.

WeatherStation exposes sun_position/sunrise/sunset bound to the station’s own location.

SunPosition

field type description
elevation float Degrees above the horizon (negative = below)
azimuth float Degrees clockwise from true north

Conditions (abstract)

Maps provider condition codes to the canonical WeatherCondition, and derives one heuristically when a provider supplies none.

  • static from_wmo(wmo: int, is_day: bool): WeatherCondition — WMO weather code (Open-Meteo). is_day picks the day/night variant for clear / partly-cloudy.
  • static from_vc_icon(icon: String): WeatherCondition — Visual Crossing icon string.
  • static from_pictocode(pictocode: int, is_day: bool): WeatherCondition — meteoblue daily pictocode (1…17), per meteoblue’s “Symbols and Pictograms” table; snow-shower and light variants collapse into their base condition, the snow+rain mix (code 11) reads as sleet (5=fog, 6=rain, 7=showers, 8=thunder, 9/10=snow, 11=sleet, 12/14/16=rain, 13/15/17=snow; unknown codes fall back to cloudy). Note: codes 5–17 previously mapped to wrong conditions (e.g. 5 returned cloudy, 6–8 fog, 12–13 thunder); stored ConditionFamily codes (Conditions::code/from_code) are unaffected, but data previously ingested via from_pictocode was misclassified and needs re-ingestion to correct.
  • For OpenWeatherMap condition codes, see OpenWeatherMapService::from_owm_code.
  • static auto_detect(is_day: bool, temp_c: float?, cloud_pct: float?, precip_mm: float?, snow_cm: float?, wind_ms: float?, visibility_m: float?): WeatherCondition — heuristic fallback (the GCL port of solarleb’s autoDetectWeatherIcon). Priority: reported snow > precipitation > fog > strong wind > cloud-cover state. When the provider reports no snowfall series (snow_cm == null) but does report a temperature, precipitation phase is partitioned by surface air temperature (classical rain/snow climatologies: predominantly snow below ~0.5 °C, mixed phase/sleet up to ~2 °C, liquid above); passing temp_c: null reproduces the temperature-blind behavior exactly (all precipitation reads as rain/showers). Null inputs are treated as “not reported”. Prefer a real provider value; use this only when none is supplied.
  • static code(c: WeatherCondition): int / static from_code(code: int): WeatherCondition — storage codec used by ConditionFamily. code is exhaustive over the current 13-member vocabulary and throws on an unmapped enum value ("Conditions::code: no storage code for <c>") — a new WeatherCondition member must be wired into both before it can be persisted, instead of silently storing under a wrong code. from_code still falls back to cloudy for unknown stored codes.

ConditionFamily

Special “categorical” family: a coded weather-condition series over time (nodeTime<int> of Conditions::code), so it is not a WeatherFamily and is reached via WeatherStation::conditions() rather than signal_for.

  • static new(): ConditionFamily
  • set(t: time, c: WeatherCondition) — record the observed condition at t.
  • get(t: time): WeatherCondition? — exact-match lookup.
  • resolve(t: time): WeatherCondition? — at-or-before lookup.
  • latest(): WeatherCondition? — most recently recorded condition.
var s = WeatherStation::get_or_create(geo{49.6, 6.1}, "lux");
s->conditions()->set(time::now(), Conditions::from_wmo(61, true)); // rain
var now = s->conditions()->latest();

Skill (abstract)

Forecast verification: how forecast error grows with lead time.

  • static error_by_lead(signal: node<WeatherSignal>, from: time, to: time, lead_bucket: duration): Array<ForecastSkillScore> — For every (target, issued) forecast in [from, to] that has an exact observation at target, compute error = forecast - observed and lead = target - issued, and fold errors into per-lead-bucket Gaussians. Returns one ForecastSkillScore per non-empty lead bucket, ascending by lead. Forecasts issued after their target (negative lead) are skipped.

ForecastSkillScore

field type description
lead duration Lower edge of the lead bucket (0, lead_bucket, 2·lead_bucket, …)
count int Number of (target, issued) pairs in the bucket
bias float Mean forecast - observed (positive = runs high)
mae float Mean absolute error
rmse float Root-mean-square error
var sig = station->signal_for(TemperatureFamily, "air_2m");
var skill = Skill::error_by_lead(sig, time::min, time::max, 6_hour);
for (_, s in skill) {
    println("lead ${s.lead.to(DurationUnit::hours)}h: rmse=${s.rmse} bias=${s.bias} n=${s.count}");
}

Spatial (abstract)

k-nearest, within-radius and inverse-distance-weighting over a nodeGeo index (e.g. weather_station_by_geo). within bounds its work to the circle’s bounding box (a nodeGeo Morton-range scan, O(range)); neighbors necessarily visits the whole index (it returns every entry), and nearest/idw_weights build on it with an O(n) scan — both flagged for native nodeGeo::nearest promotion.

  • static neighbors<T>(index: nodeGeo<T>, point: geo): Array<GeoNeighbor<T>> — every entry paired with its distance to point, ascending.
  • static nearest<T>(index: nodeGeo<T>, point: geo, k: int): Array<GeoNeighbor<T>> — the k nearest entries.
  • static within<T>(index: nodeGeo<T>, area: GeoCircle, max: int): Array<GeoNeighbor<T>> — entries inside area, nearest-first, capped at max. Scans only the circle’s bounding box (Morton-range, post-filtered to exact circle membership), falling back to a full scan for the rare circle straddling the antimeridian.
  • static idw_weights<T>(index: nodeGeo<T>, point: geo, k: int, power: float): Array<Tuple<geo, float>> — normalized inverse-distance weights (∝ 1 / distance^power, summing to 1) for the k nearest neighbours. An exact coincidence (distance 0) takes the full weight. Compute once from geometry, reuse across every signal and timestamp.

GeoNeighbor (volatile)

field type description
location geo The indexed location
value T The stored value at that location
distance float Great-circle distance (metres) to the query point
var knn = Spatial::nearest(weather_station_by_geo, geo{49.6, 6.1}, 3);
var weights = Spatial::idw_weights(weather_station_by_geo, geo{49.6, 6.1}, 3, 2.0);

spatial::GeoBox (private, volatile)

The shared geographic bounding box — the single bbox semantics for the whole library: point-in-box tests, nodeGeo Morton range scans, upstream bbox API URLs and grid tiling all go through it. Declared private because std already exports core::GeoBox; same-module code uses the bare name, everything else (including the rest of the weather lib) reaches it as spatial::GeoBox.

Longitude convention — read before building one. The box spans from west going east to east (GeoJSON bbox convention). west/east are kept exactly as given (west = sw.lng, east = ne.lng) and never reordered: passing swapped longitudes is interpreted as a box that crosses the antimeridian (west > east), not as a mistake to auto-correct. Latitudes, by contrast, are order-agnostic (south = min, north = max). Every bbox entry point across the library (LocalStations, StationDiscovery implementations, the weather.com grid tiling) inherits these semantics, so dateline-crossing boxes are supported end to end.

Fields: south/north/west/east: float. Methods:

  • static from_degrees(sw_lat: float, sw_lng: float, ne_lat: float, ne_lng: float): GeoBox — validating constructor; throws when any latitude is outside [-90, 90] or any longitude outside [-180, 180] (NaN fails too).
  • static from_corners(sw: geo, ne: geo): GeoBox — same semantics from two corner points.
  • contains(g: geo): bool — inclusive membership; on a crossing box the longitude test is lng >= west || lng <= east.
  • crosses_antimeridian(): boolwest > east.
  • split(): Array<GeoBox> — non-crossing sub-boxes for axis-aligned consumers (Morton range scans, upstream bbox APIs): [this] for a normal box, else the western lobe [west..180] followed by the eastern lobe [-180..east].
  • sw(): geo / ne(): geo — corner accessors, defined only on non-crossing boxes — they throw on a crossing box (call split() and take each lobe’s corners instead). geo clamps latitude to the web-mercator domain (~±85.05°), matching the nodeGeo index domain.

LocalStations (abstract)

Read-only spatial queries over the already-persisted weather_station_by_geo index. These resolve existing WeatherStation nodes only — they never create or mutate stations; to discover and register new stations from a provider, use StationDiscovery::register_bbox.

  • static in_bbox(g: geo, sw: geo, ne: geo): bool — inclusive rectangle membership test with spatial::GeoBox corner semantics: latitudes are order-agnostic (south = min, north = max), longitudes are taken as givensw.lng() > ne.lng() means an antimeridian-crossing box, not a mistake to auto-correct.
  • static within_bbox(sw: geo, ne: geo, max: int?): Array<node<WeatherStation>> — every station inside the inclusive box [sw,ne] (corner semantics per spatial::GeoBox), ordered ascending by great-circle distance to the box centre, capped at max (null = uncapped; max <= 0 ⇒ empty, matching the StationDiscovery contract). Runs one nodeGeo Morton-range scan per non-crossing split() lobe — a Morton slice is a superset of the rectangle, so every visited entry is post-filtered with the lobe’s contains — O(range), not O(total stations).
  • static near(point: geo, k: int): Array<node<WeatherStation>> — the k stations nearest to point, ascending by great-circle distance (a thin projection of Spatial::nearest). Fewer than k stations indexed ⇒ all of them are returned, ordering preserved.
// Nearest 5 stations already in the graph to a point:
var nearby = LocalStations::near(geo { 49.6, 6.1 }, 5);

// Every station inside a bounding box, nearest-to-centre first, capped at 20:
var inbox = LocalStations::within_bbox(geo { 49.0, 6.0 }, geo { 50.0, 7.0 }, 20);

// Inclusive rectangle membership test (corners in any order):
var hit = LocalStations::in_bbox(geo { 49.5, 6.5 }, geo { 49.0, 6.0 }, geo { 50.0, 7.0 });

Enums

WeatherCondition

Canonical weather-condition vocabulary (the Visual Crossing icon set, collapsed: showers / snow-showers / thunder variants fold into their base, and only the clear / partly-cloudy states keep a day vs night distinction).

clear_day, clear_night, partly_cloudy_day, partly_cloudy_night, cloudy, fog, wind, rain, showers, sleet, snow, thunder, hail.

Produced by the Conditions mappers and stored by ConditionFamily.

OpenMeteoVariable

Open-Meteo hourly variables that can be requested. Each enum value carries its Open-Meteo query name as its string value.

Selected values by category:

  • Air temperature: temperature_2m, apparent_temperature, dew_point_2m, wet_bulb_temperature_2m, relative_humidity_2m
  • Pressure: pressure_msl, surface_pressure, vapour_pressure_deficit, total_column_integrated_water_vapour
  • Precipitation / clouds: precipitation, precipitation_probability, rain, showers, snowfall, snow_depth, weather_code, cloud_cover, cloud_cover_low, cloud_cover_mid, cloud_cover_high, visibility
  • Wind (10/80/120/180 m): wind_speed_10m, wind_speed_80m, wind_speed_120m, wind_speed_180m, wind_direction_10m, wind_direction_80m, wind_direction_120m, wind_direction_180m, wind_gusts_10m
  • Upper-air temperatures: temperature_80m, temperature_120m, temperature_180m
  • Soil: soil_temperature_0cm, soil_temperature_6cm, soil_temperature_18cm, soil_temperature_54cm, soil_moisture_0_to_1cm, soil_moisture_1_to_3cm, soil_moisture_3_to_9cm, soil_moisture_9_to_27cm, soil_moisture_27_to_81cm
  • Radiation / solar: shortwave_radiation, direct_radiation, diffuse_radiation, direct_normal_irradiance, global_tilted_irradiance, terrestrial_radiation, plus *_instant variants, uv_index, uv_index_clear_sky, is_day, sunshine_duration
  • Evapotranspiration: evapotranspiration, et0_fao_evapotranspiration
  • Convective / boundary layer: cape, lifted_index, convective_inhibition, freezing_level_height, boundary_layer_height

Common Use Cases

Seed a station from scratch

var hits = OpenMeteoService::geocode("Berlin", 1, "en", "DE", null, null);
var hit = hits[0];
var station = WeatherStation::get_or_create(geo{hit.latitude, hit.longitude}, "berlin");
station->enable(Array<type>{ TemperatureFamily, WindFamily, RadiationFamily, PrecipitationFamily });

var provider = OpenMeteoProvider {
    variables: Array<OpenMeteoVariable>{
        OpenMeteoVariable::temperature_2m,
        OpenMeteoVariable::apparent_temperature,
        OpenMeteoVariable::precipitation,
        OpenMeteoVariable::precipitation_probability,
        OpenMeteoVariable::wind_speed_10m,
        OpenMeteoVariable::wind_direction_10m,
        OpenMeteoVariable::wind_gusts_10m,
        OpenMeteoVariable::shortwave_radiation,
        OpenMeteoVariable::direct_normal_irradiance,
        OpenMeteoVariable::diffuse_radiation,
    }
};
provider.feed_history(station, time::now() - 365_day, time::now()); // archive over [from, to]
provider.feed_forecast(station);                                    // native 16-day horizon

Read the latest observed value and the latest forecast

var sig = station->signal_for(TemperatureFamily, "air_2m");

// latest measured point
var observed = sig->observed.last();
println("latest temperature: ${observed} °C");

// latest forecast for a target 48 h from now
var target = time::now() + 2_day;
var fc = sig->latest_forecast(target);
println("forecast for now+2d: ${fc} °C");

// forecast as it was issued at a specific run
var issued = time::now() - 6_hour;
var fc_old = sig->forecast_as_of(target, issued);
println("forecast as of 6h ago for now+2d: ${fc_old} °C");

Enable calendar rollups on a signal

Call materialize once after creating a station; all subsequent set_observed calls (including those from Ingest::feed) maintain the buckets incrementally:

var sig = station->signal_for(TemperatureFamily, "air_2m");
sig->materialize(
    Array<CalendarUnit>{ CalendarUnit::day, CalendarUnit::month },
    TimeZone::"Europe/Berlin"
);

// after feeding data, query daily stats
var day = '2026-04-21T00:00:00Z';
var g = sig->daily!!.getAt(day);
println("daily avg: ${g?.avg()} °C  min: ${g?.min}  max: ${g?.max}");

Backfill rollups over existing history

WeatherSignal::rebuild_rollups folds the signal’s existing observed series over a window into its own persistent daily/monthly/yearly rollup fields (allocating them like materialize if absent). It is persistent (the result lands on disk, reachable from the station index) and idempotent — it resets the targeted buckets in the window before folding, so re-running it never double-counts. Use it to backfill rollups after a bulk import (when materialize was not called before feeding), then read a bucket straight off the signal’s daily/monthly/yearly field:

var sig = station->signal_for(RadiationFamily, "ghi");
sig->rebuild_rollups(
    Array<CalendarUnit>{ CalendarUnit::month },
    TimeZone::"Europe/Berlin",
    time::now() - 365_day,
    time::now()
);
// read back from the signal's own persistent monthly rollup
var bucket = '2026-04-01T00:00:00Z';
var g = sig->monthly!!.getAt(bucket);
println("avg GHI for ${bucket.format("%Y-%m", null)}: ${g?.avg()} W/m²");

Refresh forecasts on a schedule

Forecast sets are kept in full (all issues), so running feed_forecast every few hours is cheap and keeps latest_forecast current:

fn refresh_all_forecasts() {
    var provider = OpenMeteoProvider {
        variables: Array<OpenMeteoVariable>{
            OpenMeteoVariable::temperature_2m,
            OpenMeteoVariable::wind_speed_10m,
            OpenMeteoVariable::wind_direction_10m,
        }
    };
    for (_, station in weather_station_by_id) {
        provider.feed_forecast(station); // native 16-day horizon, no range arg
    }
}

Add a custom data provider

Define a mapping table from your provider’s variable names to the unified family/signal model, then call Ingest::feed:

// Provider that sends temperatures in Fahrenheit
var my_mappings = Array<VarMapping>{
    VarMapping{ provider_key: "air_temp_F", family: TemperatureFamily, signal: "air_2m",   unit: "°C", convert: f_to_c },
    VarMapping{ provider_key: "gust_ms",    family: WindFamily,         signal: "gust",     unit: "m/s", convert: null },
    VarMapping{ provider_key: "rain_mm",    family: PrecipitationFamily, signal: "total",   unit: "mm",  convert: null },
};

var now = time::now();
var my_points = Array<WeatherPoint>{
    WeatherPoint{
        t: now - 1_hour,
        historical: true,
        issued_at: null,
        values: Map<String, float>{ "air_temp_F": 68.0, "gust_ms": 5.2, "rain_mm": 0.3 }
    }
};

Ingest::feed(station, my_points, my_mappings);

Test with a canned response

The per-provider *_scatter streaming functions are network-free — feed a canned response into a real station via an IngestFeeder, then read back from the disk-backed signal node (the real streaming path, end-to-end):

var fake = OpenMeteoResponse{
    latitude: 0.0, longitude: 0.0, generationtime_ms: 0.0,
    utc_offset_seconds: 0, timezone: "GMT", timezone_abbreviation: "GMT",
    elevation: 0.0, hourly_units: Map<String, String>{},
    hourly: Map<String, any?>{
        "time": Array<String>{ "2026-04-21T00:00" },
        "temperature_2m": Array<float?>{ 10.0 }
    }
};
var station = WeatherStation::get_or_create(geo{ 0.0, 0.0 }, "test");
var feeder = IngestFeeder::new(station, OpenMeteoService::mappings());
// window_from/window_to null = unbounded (no branch-window clamp)
OpenMeteoService::openmeteo_scatter(fake, feeder, true, null, null);
var t0 = '2026-04-21T00:00:00Z';
Assert::equalsd(
    station->signal_for(TemperatureFamily, "air_2m")->observed.getAt(t0)!!,
    10.0, 1e-9
);

Use a commercial API key

Open-Meteo’s paid tier requires an apikey query parameter and different host prefixes:

var key = System::getEnv("OPENMETEO_API_KEY");

var provider = OpenMeteoProvider {
    archive_base_url: "https://customer-archive-api.open-meteo.com",
    forecast_base_url: "https://customer-api.open-meteo.com",
    api_key: key,
    variables: variables
};
provider.feed_history(station, from, to);
provider.feed_forecast(station);

// geocoding helper also threads the instance api_key:
provider.geocode("Berlin", 10, "en", "DE", "https://customer-geocoding-api.open-meteo.com");

Point at a caching proxy

var provider = OpenMeteoProvider {
    archive_base_url: "https://meteo-proxy.internal",
    forecast_base_url: "https://meteo-proxy.internal",
    variables: variables
};
provider.feed_history(station, from, to);
provider.feed_forecast(station);

Best Practices

Give every station a stable id

Stations without an id are not indexed by id, so weather_station_by_id misses them. Give every station a stable slug ("berlin", "sensor-0042") unless you only ever look up by geography.

Enable families before feeding

Ingest::feed calls signal_for, which auto-activates the family on first use even without a prior enable call. Calling enable explicitly at setup time documents your intent and lets is_enabled checks behave predictably at any point in the lifecycle.

Call materialize before the first feed

Rollup buckets are only filled by points arriving after materialize is called. If you call materialize after a bulk import, the rollups start empty. To backfill rollups over data already imported, call WeatherSignal::rebuild_rollups instead — it folds the existing observed series into the signal’s own persistent rollup fields and is idempotent.

materialize is idempotent: the first call’s tz is the one that sticks (rollup_tz is set-once), so a later call with a different timezone will not silently re-bucket existing rollups. Only CalendarUnit::hour, CalendarUnit::day, CalendarUnit::month, and CalendarUnit::year are materializable — passing any other unit throws. (The slots seasonal profile is always maintained on set_observed regardless of materialize, bucketed in rollup_tz when set, else UTC.)

set_observed adds each new timestamp’s value to the materialized rollup accumulators (Gaussian) and the overall profile. Because Gaussian cannot subtract, these accumulators are updated only the first time a given timestamp is written: re-feeding an existing timestamp corrects observed but leaves the rollup buckets and gaussian profile unchanged (no double-count, but they retain the first value). If you correct historical values and need exact stats afterward, rebuild from observed: WeatherSignal::rebuild_rollups fixes the calendar rollups (it resets the window’s buckets before re-folding, so the corrected values are reflected exactly), and WeatherSignal::rebuild_profile fixes the overall gaussian and the seasonal slots profile (a full rebuild — previously these two had no rebuild path at all).

Mark points correctly as historical or forecast

Only historical: true points update observed and the overall gaussian profile. Marking forecast data as historical pollutes the profile with speculative values; marking genuine past observations as historical: false puts them in the forecast matrix instead of the observed series.

Wind direction and vector components

Wind speed and direction are scalar quantities that wrap around at 0°/360°. The Open-Meteo connector converts speed + direction to meteorological u/v components inline before storing (via the shared Ingest::decompose_wind). If you build a custom mapping for wind from another provider, do the same decomposition:

// u = -speed * sin(dir_rad), v = -speed * cos(dir_rad)
var rad = dir_deg * MathConstants::pi / 180.0;
values.set("u_10m", -speed * sin(rad));
values.set("v_10m", -speed * cos(rad));

Use forecast_as_of to compare model runs

When you archive every forecast issue for a target time, forecast_as_of lets you answer “what was the forecast for 6 h from now, as published 12 h ago?” — useful for forecast skill evaluation.

Prefer batch ingestion

feed_history sends one archive request per started year of the historical window (≤1-year chunks), and feed_forecast usually sends one forecast request. Still prefer wide windows over calling feed_history in a tight loop with tiny windows — Open-Meteo rate-limits aggressive callers.

Geocoding edge cases

  • Queries with 0–1 characters return no results.
  • Server errors are turned into thrown exceptions by geocode.
  • get_or_create returns an existing station when the id matches (when non-null) first, then when the geo matches. If the coordinate match has no id yet and you pass one, that id is adopted onto the node and indexed. If the coordinate match already carries a different id, the requested id is not indexed (a warning is logged) and the existing station is returned. Because every station is also keyed in weather_station_by_geo, two stations at the same coordinates collide — jitter the geo slightly if you need multiple stations at the same point.