s3

An S3 client for GreyCat. Talks to AWS S3 and to anything implementing its API (MinIO, Ceph, Backblaze B2, Cloudflare R2, Wasabi), signing every request with AWS Signature Version 4.

Pure GCL on top of the http library, with xml for response bodies.

@library("s3", "0.0.0");

http and xml come along with it and do not need declaring.

var credentials = sigv4::Credentials::from_env();
if (credentials == null) {
    throw "no AWS credentials in the environment";
}
var client = s3::Client::aws("eu-west-1", credentials);

client.put_object(s3::PutObjectRequest {
    bucket: "reports",
    key: "2026/08/summary.json",
    body: Json::to_string(summary),
    content_type: "application/json",
});

for (_, entry in client.list_objects_all("reports", "2026/08/")) {
    println("${entry.key} ${entry.size}");
}

Every type is private, so it is reached by its qualified name: s3::Client, s3::PutObjectRequest, sigv4::Credentials. Bare S3Object would collide with std’s own, and the qualified form says which library a call belongs to.

Which client

std ships a native S3 already. This library complements it rather than replacing it, and for the six things that one does it is the better choice: it is native code, and it needs no dependency.

	std’s S3	this library
Objects to and from a file	yes	yes
Objects to and from memory	no	yes
Content type, user metadata	no	yes
Listing	first 1000 keys	every key, following continuation tokens
delimiter, common prefixes	no	yes
Copy, batch delete, head	no	yes
Presigned URLs	no	yes
Multipart upload	no	yes
Non-AWS endpoints	host and path style	host and path style, plus timeouts and extra headers

Reach for std’s when moving whole files is all you need. Reach for this one for anything above that line.

Connecting

// AWS, buckets addressed as hostnames.
var aws = s3::Client::aws("eu-west-1", credentials);

// Anything else, buckets addressed as path segments.
var minio = s3::Client::compatible("http://localhost:9000", "us-east-1", credentials);

The difference is force_path_style, and it is not cosmetic. Virtual-host addressing puts the bucket in the hostname (reports.s3.amazonaws.com), which needs a wildcard DNS entry per bucket. Self-hosted implementations do not have one, so they need the bucket in the path. compatible sets it; aws clears it.

The region is signed, so it has to be the bucket’s own. A mismatch is rejected as AuthorizationHeaderMalformed or SignatureDoesNotMatch, never silently redirected. S3-compatible servers that have no notion of regions accept any value; us-east-1 is the convention.

The remaining fields are optional:

var client = s3::Client {
    endpoint: "https://s3.eu-west-1.amazonaws.com",
    region: "eu-west-1",
    credentials: credentials,
    force_path_style: false,
    timeout: 60s,
    max_response_size: 100_000_000,
    extra_headers: Map<String, String> { "x-amz-request-payer": "requester" },
};

extra_headers is merged over what this library sets, last write winning, and is signed along with everything else.

Credentials

sigv4::Credentials { access_key_id: "...", secret_access_key: "...", session_token: null }

sigv4::Credentials::from_env() reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN, returning null when the first two are not both set. session_token is for temporary credentials from STS or an instance role, and is signed rather than merely sent alongside.

Objects

client.put_object(s3::PutObjectRequest {
    bucket: "reports", key: "notes.txt",
    body: "plain text",
    content_type: "text/plain",
    metadata: Map<String, String> { "origin": "greycat" },
});

var got = client.get_object("reports", "notes.txt");
println(got.body);
println(got.metadata.get("origin"));

client.copy_object("reports", "notes.txt", "archive", "notes.txt");
client.delete_object("reports", "notes.txt");

body is a GreyCat String, which is a byte string: arbitrary binary round-trips through it unchanged, including NUL.

metadata entries travel as x-amz-meta-<name> and come back on get_object and head_object with that prefix stripped. An object carrying none reads as an empty map, never null.

head_object returns null for an object that does not exist, which is what makes it the way to test for one. Every other call throws on a non-2xx, so get_object on a missing key is an error, not a null.

if (client.head_object("reports", "notes.txt") == null) {
    // not there
}

Files

put_object takes file instead of body to send a path, and get_object_to_file writes the response to disk as it arrives. Neither holds the object in memory, so both are bounded by the filesystem rather than the heap. body on the result of get_object_to_file is empty; the bytes are on disk.

client.put_object(s3::PutObjectRequest { bucket: "backups", key: "db.tar.gz", file: "data/db.tar.gz" });
client.get_object_to_file("backups", "db.tar.gz", "data/restored.tar.gz");

body and file are mutually exclusive; file wins if both are set.

Listing

var page = client.list_objects(s3::ListObjectsRequest {
    bucket: "reports",
    prefix: "2026/",
    max_keys: 100,
});

S3 returns at most a thousand keys per request, whatever max_keys asks for. A truncated page carries next_continuation_token, fed back on the next request:

var token: String? = null;
var more = true;
while (more) {
    var page = client.list_objects(s3::ListObjectsRequest { bucket: "reports", continuation_token: token });
    for (_, entry in page.objects) {
        println(entry.key);
    }
    token = page.next_continuation_token;
    more = page.is_truncated && token != null;
}

list_objects_all(bucket, prefix) runs that loop and hands back everything. It holds the whole listing in memory, so page manually for a bucket whose size is unknown.

Directories

S3 has no directories: a key is a flat string that happens to contain slashes. delimiter emulates them by rolling up everything past the next separator into common_prefixes:

var page = client.list_objects(s3::ListObjectsRequest {
    bucket: "reports", prefix: "2026/", delimiter: "/",
});
page.common_prefixes;   // "2026/07/", "2026/08/" -- the subdirectories
page.objects;           // keys directly under 2026/, if any

Batch delete

var result = client.delete_objects("reports", Array<String> { "a.txt", "b.txt" });
for (_, err in result.errors) {
    println("${err.key}: ${err.code} ${err.message}");
}

Up to a thousand keys per call. A partial failure comes back as a successful request: individual keys fail inside a 200 and land in errors rather than throwing, so a caller that ignores errors will not notice.

Presigned URLs

A presigned URL carries its own authorization in the query string, so whoever redeems it needs no credentials and sets no header. That is what makes it shareable with a browser or a third party.

var download = client.presign_get("reports", "summary.json", 15min);
var upload = client.presign_put("uploads", "incoming.bin", 1hour);

Signing is local: neither call touches the network, and neither checks that the object exists. The URL is valid the moment it is returned and refused after expires, whose ceiling is 7 days for SigV4.

Multipart upload

upload_file is the whole thing in one call. It splits the file into ranges, streams each off the disk, and aborts the upload if any part fails, so a broken run leaves no billable parts behind:

client.upload_file("backups", "db.tar.gz", "data/db.tar.gz", "application/gzip", null);

Under one part size it sends a single PUT instead, so it is safe to call for a file of any size. part_size defaults to 8 MiB and must be at least 5 MiB, which is the smallest part S3 accepts for anything but the last one.

The steps are also available individually, for parts that do not come from a file or that are uploaded from more than one place:

var id = client.create_multipart("backups", "assembled.bin", null);
var parts = Array<s3::CompletedPart> {};
parts.add(s3::CompletedPart { part_number: 1, etag: client.upload_part("backups", "assembled.bin", id, 1, first) });
parts.add(s3::CompletedPart { part_number: 2, etag: client.upload_part("backups", "assembled.bin", id, 2, second) });
client.complete_multipart("backups", "assembled.bin", id, parts);

An upload left open keeps its parts, and they are billable. Nothing expires them but a lifecycle rule, so a failure that skips abort_multipart leaks storage indefinitely. list_parts shows what an upload is holding.

Errors

A non-2xx throws, carrying the status and the server’s own diagnosis out of the S3 error envelope:

s3: get_object missing.txt failed, HTTP 404 NoSuchKey: The specified key does not exist.

The exceptions are deliberate, and each is documented above: head_object answers null for a missing object, create_bucket succeeds on a bucket that is already yours, bucket_exists answers a bool, and delete_objects reports per-key failures in errors.

ETags

An ETag is the object’s MD5 only when it was stored in one part. A multipart object gets a digest of its part digests followed by -<part count>, which is not the MD5 of anything and cannot be compared against a locally computed one. Quotes are stripped before the value reaches you.

Signing on its own

sigv4 is usable without the client, for an S3 API this library does not cover:

var request = sigv4::Request {
    method: "GET",
    canonical_uri: sigv4::Signer::canonical_uri("/reports/summary.json"),
    query: Map<String, String> { "versions": "" },
    headers: Map<String, String> { "host": "s3.eu-west-1.amazonaws.com" },
    payload_hash: Crypto::sha256hex(""),
};
sigv4::Signer { region: "eu-west-1", credentials: credentials }.sign(request, time::now());
// request.headers now carries authorization, x-amz-date and x-amz-content-sha256

sign mutates headers in place. presign leaves them alone and returns a query string instead.

Tests

The suite runs against MinIO in a container, which verifies signatures the way AWS does and enforces the same semantics, so it is what decides whether the library works.

greycat test

It needs a container engine on the path; the helper reads CONTAINER_ENGINE and falls back to docker. The nine signer unit tests run without one; the rest skip straight to a failure.