http

HTTP/HTTPS client. Runs on libcurl, and verifies TLS against the same trust chain the runtime resolved at startup, so --ca-path and --unsecure apply here exactly as they do to the rest of GreyCat.

@library("http", "0.0.0");

One-shot requests

Http<T> is generic over what the response body deserializes into. Without a generic parameter, or with String, the body is handed back as raw text; with anything else an application/json body is parsed into T.

var body = Http<String>::get("https://example.com/", null);

var repo = Http<Repo>::post(
    "https://api.example.com/repos",
    Repo { name: "greycat" },
    Map<String, String> { "authorization": "Bearer ${token}" },
);

Http<any>::getFile("https://example.com/data.csv", "data/local.csv", null);

send gives access to the status line and the response headers, and takes the options a bare verb cannot express:

var res = Http<String>::send(HttpRequest {
    method: HttpMethod::POST,
    url: "https://api.example.com/things",
    headers: Map<String, String> { "content-type": "application/json" },
    body: "{\"name\":\"thing\"}",
    timeout: 30s,
    max_response_size: 1_000_000,
});
println("${res.status_code} ${res.headers.get("content-type")}");

max_response_size bounds the body in bytes and aborts the transfer past it, which is what keeps a chunked or Content-Length-less response from growing without bound. null or 0 means no limit.

Time limits

Three fields bound how long a request may take. Each is a duration?, and a null one falls back as follows.

Field	Bounds	When null
connect_timeout	Establishing the connection, TLS handshake included	timeout, or 5 seconds when that is null too
idle_timeout	No byte moving either way, the wait for the first response byte included	5 seconds when timeout is null too, else none
timeout	The whole exchange, from connecting to the last body byte	None

A request that sets none of them connects within 5 seconds and fails once the server goes 5 seconds without sending anything, while a download that keeps moving completes however long it takes. Setting timeout alone turns that into a hard deadline, which bounds the connect as well and switches the idle limit off. For an API that computes for a while before it answers, raise idle_timeout and keep connect_timeout short, so a dead host fails fast and a slow one that is working gets waited for:

var res = Http<String>::send(HttpRequest {
    method: HttpMethod::GET,
    url: "https://archive.example.com/v1/series?year=2024",
    connect_timeout: 10s,
    idle_timeout: 2min,
});

idle_timeout is checked about once a second, so it fires up to a second after running out. A request body still being sent counts as traffic, so a slow upload is not idle. A chunked reader never applies the 5 second default, since a stream is quiet between two events, but honours an explicit idle_timeout.

A request ended by a limit throws with reason=net_recv_timeout, and limit= names the field whose limit fired:

http: request to archive.example.com:443 failed (reason=net_recv_timeout, code=28, limit=idle_timeout)

Retries

A request is sent once unless it sets max_attempts, the number of tries including the first. Retrying is opt-in because a try that failed may still have reached the server, so a POST sent twice can take effect twice.

var res = Http<Forecast>::send(HttpRequest {
    method: HttpMethod::GET,
    url: "https://api.example.com/v1/forecast?lat=49.6&lon=6.1",
    idle_timeout: 1min,
    max_attempts: 3,
    retry_delay: 2s,
});

Retried	Not retried
A refused connection (net_connect)	A host that does not resolve (net_resolve)
A connection dropped or broken mid-exchange (net_send, net_recv, an empty or truncated reply)	A TLS failure, a malformed URL, too many redirects
A time limit that fired (net_recv_timeout)	response too large, a failed write to response_file
Status 429, 502, 503, 504	Any other status

The first retry waits retry_delay (1 second when null), and each further one waits twice as long as the one before, capped at 30 seconds. A response carrying Retry-After in seconds sets the next wait instead, under the same cap. Each retry is logged as a warning:

http: request to api.example.com:443 failed (reason=status, code=503), attempt 1 of 3, retrying in 2s

When every try fails, the error of the last one gets attempts=:

http: request to api.example.com:443 failed (reason=net_connect, code=7, attempts=3)

A status is a response rather than an error, so send hands back the last 503 as it would without retries, and the status check stays the caller’s. get, post and put set no max_attempts and so never retry. A file body is sent again from its first byte, and a response_file holds the body of the try that succeeded, not what the failed ones wrote. A chunked reader always sends once.

A retry blocks the task for the whole wait, and time limits apply to each try, so the worst case of a request is max_attempts times its limits plus the waits.

The status decides where the body lands

T names the shape of a successful body. A failing request answers with something else, so a non-2xx body is never decoded into T: it comes back as text in error_msg, and content stays null.

var res = Http<Order>::send(request);
if (res.status_code >= 200 && res.status_code < 300) {
    println(res.content?.total);
} else {
    println("HTTP ${res.status_code} ${res.error_msg}");
}

Success is strictly 2xx. The two fields are never both set, and a failure carrying no body at all leaves error_msg null too.

A JSON error envelope arrives as its JSON text, so reading a field out of it is the caller’s step and only happens when the detail is wanted:

var detail = Json::parse(res.error_msg ?? "{}");

The untyped forms are the exception: Http, Http<any> and Http<String> name no schema for a failed body to contradict, so they carry it in content as well, decoded exactly as a 2xx would be.

Streaming responses

chunked returns a reader that yields the body as it arrives instead of buffering it whole, for text/event-stream, newline-delimited JSON and other long-lived streams.

var reader = Http<String>::chunked(HttpRequest {
    method: HttpMethod::GET,
    url: "https://api.example.com/v1/stream",
    timeout: 60s,
});
while (reader.can_read()) {
    println(reader.read());
}
reader.close();

Framing follows the response Content-Type. An event stream is read as Server- Sent Events, so one read returns the data of one event with the prefix removed and comment lines skipped; anything else is read line by line. Either way the transfer encoding of the wire is not observable. An event stream ends on data: [DONE], which is reported as the end of the stream rather than handed out.

On a reader, max_response_size caps the bytes held between two reads rather than the whole body, so an endless stream is read at any length as long as the caller keeps up.

HttpReader holds a live connection, so it can neither be persisted nor resumed: read it to completion, or close it, from the task that opened it.

Unix sockets

unix_socket connects through a Unix domain socket instead of resolving the URL’s authority, which is how a local daemon is reached: Docker, podman, systemd, or an application behind nginx.

var res = Http<any>::send(HttpRequest {
    method: HttpMethod::GET,
    url: "http://localhost/v1.43/containers/json?all=true",
    unix_socket: "/var/run/docker.sock",
});

The socket decides only where the connection goes. The URL still supplies the request line, its scheme still decides whether TLS is negotiated over the socket, and its host still becomes the Host header, which is why localhost is the convention for an authority nothing routes on. A URL that is a bare path gets http://localhost put in front of it, so one nothing reads can be left out:

url: "/v1.43/containers/json",

chunked takes the field too, so a daemon’s event stream is read exactly like any other stream.

The socket belongs to the connection rather than to the URL, so a redirect is followed over the same socket whatever host it names.

Bodies on disk

A body too large to hold in memory travels as a file. body_file streams the request body straight off the disk, and response_file writes the response body to one as it arrives, so neither passes through the heap. A 64 MB upload whose echo comes back to a file costs the same resident memory as an empty request.

var res = Http<any>::send(HttpRequest {
    method: HttpMethod::PUT,
    url: "https://api.example.com/objects/report.parquet",
    headers: Map<String, String> { "content-type": "application/vnd.apache.parquet" },
    body_file: http::FileBody { path: "/data/report.parquet" },
    timeout: 10min,
});
println("${res.status_code} ${res.headers.get("etag")}");

offset and size send a byte range rather than the whole file, which is what a multipart part is. Both default to the whole file, and a range reaching past the end is refused before the request opens rather than stalling the peer mid-transfer.

body_file: http::FileBody { path: "/data/report.parquet", offset: 8_388_608, size: 8_388_608 },

response_file leaves content null, since a body is either returned or written and never both. The status and headers arrive as they do for any other response, which is what tells a downloaded file apart from a downloaded error page:

var res = Http<any>::send(HttpRequest {
    method: HttpMethod::GET,
    url: "https://api.example.com/objects/report.parquet",
    response_file: http::FileSink { path: "/data/downloaded.parquet" },
    timeout: 10min,
});
if (res.status_code != 200) {
    File::delete("/data/downloaded.parquet");
    throw "download failed: HTTP ${res.status_code}";
}

Parent directories are created. append adds to the file instead of truncating it. Setting body and body_file on the same request throws rather than picking one.

FileBody and FileSink are always qualified, as http::FileBody and http::FileSink.

For the two common cases there are shortcuts, both returning the response:

var up = Http<any>::put_file(url, "/data/report.parquet", headers);
var down = Http<any>::get_file(url, "/data/downloaded.parquet", null);

getFile predates all of this and stays as it is. It reports no status, so a 404 writes the error body to the target file and looks like a success; prefer get_file, which returns one.