# Sierpe — complete documentation
# A self-hosted Stellar indexer. Register a contract, get its complete history behind an honest REST API.
# Release v1.5.2 (2026-08-21) · image ghcr.io/zkcaleb-dev/sierpe:v1.5.2
# Authoritative API contract: https://raw.githubusercontent.com/zkCaleb-dev/sierpe/main/docs/openapi.yaml

---

# Installation
> Deploy the container on Railway, Docker Compose, or any platform that runs OCI images.


Sierpe is distributed as a container image and as release binaries. All it
needs is an **empty Postgres database** it can own — Sierpe manages its own
schema and migrations.

## Choosing an image

| Tag | Contents |
|---|---|
| `ghcr.io/zkcaleb-dev/sierpe:v1.5.2` | Slim: static, distroless, multi-arch. Indexes from the RPC and clamps honestly at the retention wall |
| `ghcr.io/zkcaleb-dev/sierpe:v1.5.2-full` | Slim plus `stellar-core`, to heal history below RPC retention. **linux/amd64 only** — see [the archive leg](/docs/archive-leg/) |

Start with the slim image. Move to `-full` when you need history older
than the roughly seven days an RPC serves.

## Requirements

- An **empty Postgres database** reachable via `DATABASE_URL` — Sierpe
  owns the schema and runs its own migrations; do not point it at a
  database shared with another application. The bundled compose runs
  Postgres 16.
- Outbound HTTPS to public Stellar RPC endpoints.
- Storage grows with the contracts you register, not with the chain: a
  typical project (a handful of contracts) fits Railway's smallest paid
  tier.

## Docker Compose

This is the complete file — save it as `docker-compose.yml`, nothing else
is needed (it matches the one
[in the repository](https://github.com/zkCaleb-dev/sierpe/blob/main/docker-compose.yml)):

```yaml
services:
  sierpe:
    image: ghcr.io/zkcaleb-dev/sierpe:v1.5.2
    # image: ghcr.io/zkcaleb-dev/sierpe:v1.5.2-full   # archive leg: heals history below RPC retention (amd64)
    restart: unless-stopped
    depends_on:
      db:
        condition: service_healthy
    environment:
      DATABASE_URL: postgres://sierpe:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD}@db:5432/sierpe?sslmode=disable
      NETWORK: ${NETWORK:-testnet}
      ADMIN_TOKEN: ${ADMIN_TOKEN:?set ADMIN_TOKEN (min 16 chars)}
      # RPC_URLS: https://your-rpc-1,https://your-rpc-2   # required on mainnet
    ports:
      - "8080:8080"

  db:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: sierpe
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD}
      POSTGRES_DB: sierpe
    volumes:
      - sierpe-pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U sierpe -d sierpe"]
      interval: 5s
      timeout: 3s
      retries: 12

volumes:
  sierpe-pgdata:
```

```bash
export POSTGRES_PASSWORD=$(openssl rand -hex 16)
export ADMIN_TOKEN=$(openssl rand -hex 24)
docker compose up -d
curl localhost:8080/health
```

Set `NETWORK=mainnet` and `RPC_URLS` for mainnet.

## Railway

**One click**: [Deploy on Railway](https://railway.com/deploy/sierpe?referralCode=tuEgvN)
creates the Postgres and the Sierpe service already wired together, with
`ADMIN_TOKEN` and the Basic Auth password generated for your instance.
Open the generated domain, sign in with the `HTTP_BASIC_AUTH` value from
the Variables tab, paste `ADMIN_TOKEN` into the UI's admin box, register
your first contract. Every variable in the template carries a
description of what it does.

The manual route, if you prefer to assemble it yourself — no GitHub
account or build step needed:

1. **New Project → Deploy PostgreSQL.** A fresh Railway Postgres is
   empty, which is what Sierpe needs; do not reuse a database another app
   owns. Note the service name on the card (default `Postgres`).
2. **+ New → Docker Image**, type `ghcr.io/zkcaleb-dev/sierpe:v1.5.2`.
   The first deploy fails until the variables exist — expected.
3. **Variables** tab of the new service:
   - `DATABASE_URL` = `${{Postgres.DATABASE_URL}}` — the reference works
     as-is; no `sslmode` parameter is needed on Railway's internal network.
   - `NETWORK` = `testnet` (or `mainnet`, plus `RPC_URLS`)
   - `ADMIN_TOKEN` = a random string, 16+ characters (`openssl rand -hex 24`)
   - `HTTP_BASIC_AUTH` = `user:password` — set it if you will give the
     service a public domain (next step). Skip it if only other services
     in the same project will reach it over private networking.
4. **Settings → Deploy → Healthcheck Path**: `/health`. Not `/ready`,
   which returns 503 while catching up and would fail the deploy.
5. **Settings → Networking → Generate Domain**, and when asked for the
   port, answer **`8080`**. Sierpe listens on its own `HTTP_PORT`
   (default 8080) and deliberately ignores the `PORT` Railway injects —
   honouring it would silently move the listener of every deployment
   that did not pin `HTTP_PORT`. Private
   networking stays on by default: other services in the project reach
   it at `http://sierpe.railway.internal:8080`.
6. Deploy, then open `https://YOUR-APP.up.railway.app/status`. A fresh
   database starts at the current tip, so `ready` flips to `true` within
   seconds — history arrives per contract, through the backfill.

No volume is needed on the Sierpe service: all state lives in Postgres.
The `-full` image is the one exception — it wants a few GB of scratch
disk for captive core; see [the archive leg](/docs/archive-leg/).

## AWS (ECS Fargate + RDS)

The shape a small team would run: one Fargate task, a managed Postgres,
nothing public.

- **RDS for PostgreSQL 16**, private subnets, not publicly accessible.
  Create an empty database and a role that **owns** it
  (`CREATE ROLE sierpe LOGIN PASSWORD '…'; CREATE DATABASE sierpe OWNER sierpe;`).
  Owner is enough — Sierpe runs its own migrations; it never needs
  superuser.
- **`DATABASE_URL` must end in `?sslmode=require`.** RDS enforces TLS by
  default on PostgreSQL 15+, and the compose file's `sslmode=disable` is
  for the bundled local Postgres only. Use `require`, not `verify-full`:
  the image carries the public root CAs (it talks HTTPS to the RPC) but
  not Amazon's RDS CA, so full verification would fail. This applies to
  every managed Postgres with a private CA (RDS, Supabase, Neon…).
- **Task definition**: image `ghcr.io/zkcaleb-dev/sierpe:v1.5.2`,
  container port `8080`, `NETWORK` as plain environment, `DATABASE_URL`
  and `ADMIN_TOKEN` as ECS `secrets` from Secrets Manager. 0.5 vCPU /
  1 GiB is a sound start (see sizing below). No volume, no EFS. Use the
  `awslogs` driver; logs are structured JSON with secrets redacted.
- **Health**: the image is distroless (no shell, no curl), so use the
  load balancer's target-group check, not a container `CMD` check.
  Path `/health`, success code 200. **Never `/ready`** here — it returns
  503 while catching up, and an ECS health check on it would kill a
  healthy task mid-backfill.
- **Networking**: private subnets with a NAT gateway — the task needs
  outbound HTTPS for the Stellar RPC and for the `ghcr.io` image pull.
  An **internal** ALB gives your backend a stable name inside the VPC.
  Only if you truly need a public endpoint: internet-facing ALB + ACM
  certificate and set `HTTP_BASIC_AUTH`.
- **Exactly one task**: `desiredCount: 1`, `minimumHealthyPercent: 0`,
  `maximumPercent: 100`, so a deploy never runs two copies at once (why:
  next section).

## Any container platform

```bash
docker run -d -p 8080:8080 \
  -e DATABASE_URL=postgres://user:pass@host:5432/sierpe \
  -e NETWORK=testnet \
  -e ADMIN_TOKEN=$(openssl rand -hex 32) \
  ghcr.io/zkcaleb-dev/sierpe:v1.5.2
```

## Operating it on any cloud — the facts that matter

Answers to what an operator (or their assistant) has to decide, stated
from the code rather than guessed.

- **Run one instance per database in steady state.** There is no
  leader election. A second instance is harmless to the data — the
  cursor only ever moves forward and every insert is idempotent — but
  it is pure waste: each copy ingests every ledger (double RPC load)
  and two backfill workers re-walk each other's chunks. A brief overlap
  during a rolling deploy is fine; two replicas as a steady state is
  not high availability, just double the work. Restarts are safe at any
  moment: the cursor and the data commit in one transaction, so a killed
  task resumes exactly where it stopped.
- **Shutdown**: `SIGTERM` is handled; the HTTP server drains for up to
  5 seconds and the loop stops between commits. First boot runs the
  embedded migrations in well under a minute.
- **Database connections**: pgx defaults — at most `max(4, CPUs)`
  pooled connections; `pool_max_conns=2` in the URL lowers it for tiny
  plans. PostgreSQL **14 or newer** (the driver's floor); 16 or 17 for a
  new install. Behind a transaction-mode pooler without prepared-statement
  support, append `default_query_exec_mode=simple_protocol` — Sierpe
  names that fix in its last log line when it dies that way.
- **Memory**: the slim image idles at tens of MB. The ceiling is the
  backfill, which buffers RPC responses of up to 64 MB each and shrinks
  its batch when the network is busier than that; plan 512 MB, and
  1 GiB if you register many contracts at once.
- **TLS to Postgres**: `sslmode=require` for any managed provider with a
  private CA (RDS, Cloud SQL, Supabase…); `verify-full` as-is where the
  certificate chains to public roots (Azure Flexible Server, Neon,
  PlanetScale); `disable` only for a Postgres on a private network you
  control. For `verify-full` against a private CA, mount the provider's
  CA file into the container and add `sslrootcert=/path/to/ca.pem` to
  the URL — the driver honours it, no derived image needed.
- **Testnet resets**: when the network is reset (the tip jumps back by
  millions of ledgers), the loop detects it and **stops with zero
  writes** rather than mixing two chains. Recovery is deliberate and
  manual: drop and recreate the empty database, redeploy, re-register
  your contracts. Everything Sierpe holds is re-derivable from the chain.
- **Defaults you do not need to set**: on testnet the RPC pool is
  `https://soroban-testnet.stellar.org`; history archives default to the
  SDF public archives on **both** networks. Mainnet has no free public
  RPC, so `RPC_URLS` is required there.
- **Behind a proxy**: TLS termination in front is fine. Path prefixes
  are not — the embedded UI and the API assume they live at `/`.
- **Health checks inside the container**: the image declares
  `HEALTHCHECK CMD ["/sierpe", "healthcheck"]` (since 1.5.2), so Docker,
  Swarm and the self-hosted PaaS family get a health signal despite the
  distroless base. Kubernetes and cloud load balancers ignore it and
  probe `/health` over the network, which is equally fine.
- **The `-full` image runs as root** today (its stellar-core base has no
  unprivileged user). Clusters enforcing the restricted Pod Security
  profile will reject it; the slim image runs as UID 65532 and passes.
  See [Where it runs](/docs/platforms/) for the per-platform picture.

## Configuration

Boot configuration comes from environment variables; everything else
(contracts, their kinds) is data managed at runtime through the admin API.

| Variable | Required | Meaning |
|---|---|---|
| `DATABASE_URL` | yes | Postgres connection string; Sierpe owns this database |
| `NETWORK` | yes | `testnet` or `mainnet` |
| `ADMIN_TOKEN` | yes | Bearer token for the admin surface. At least 16 characters with 6 distinct ones, enforced at boot (`openssl rand -hex 24` is fine) |
| `RPC_URLS` | mainnet | Comma-separated failover pool, in preference order; testnet defaults to the public SDF endpoint |
| `HTTP_PORT` | no | API port, default 8080 |
| `START_LEDGER` | no | First ledger for a fresh database (default: current tip) |
| `HTTP_BASIC_AUTH` | no | `user:password`; when set, every request needs these credentials except `/health` and `/ready`. For public-domain deployments |
| `STELLAR_CORE_BINARY` | no | Path to a stellar-core binary; enables the [archive leg](/docs/archive-leg/). Pre-set in the `-full` image |
| `HISTORY_ARCHIVE_URLS` | no | History archives for the archive leg. Defaults to the SDF public archives |
| `CAPTIVE_STORAGE_PATH` | no | Disposable scratch space for captive core buckets. Defaults to the OS temp dir |

Secrets are redacted from all logs. Verify the deployment with
`GET /health` and `GET /status`; `/ready` returns 503 while catching up —
wire it to your platform's readiness probe. Then open `/` in a browser:
the [embedded UI](/docs/management-ui/) covers the whole surface.

## First-run troubleshooting

Every one of these was hit by a real first deployment; the fixes are
exact.

| Symptom | Cause | Fix |
|---|---|---|
| Boot error naming `DATABASE_URL`, `NETWORK` or `ADMIN_TOKEN` | Variables are unprefixed — `SIERPE_DATABASE_URL` is not read | Use the exact names from the table above |
| `/ready` returns 503, `/health` returns 200 | Normal while catching up to the tip | Wait; watch `/status` — `ready` flips when the cursor reaches the tip |
| `401` on `POST /v1/contracts` | Missing bearer, or `HTTP_BASIC_AUTH` is set and the client sent only one credential | Send `Authorization: Bearer $ADMIN_TOKEN`; with Basic Auth enabled the admin token is also accepted as the Basic password |
| `404 contract does not exist` on registration | Contract not found on the configured network — wrong `NETWORK`, a typo, or an asset whose SAC was never deployed | Check the id on that network; deploy the SAC first for classic assets |
| A fresh database starts at the tip, not in the past | By design — history arrives via each contract's backfill, not by replaying the whole chain | Register contracts with `"from"`; use `START_LEDGER` only when you need the live cursor itself to begin earlier |
| Right after registering, coverage shows `indexedFromLedger` above `indexedToLedger` | An intentionally empty window: the backfill anchors slightly past the live cursor | It closes on its own within a minute; not an error |

## Exposing it safely

The default shape is **private networking**: do not give the instance a
public domain, and let your backend reach it over your platform's
internal network (on Railway, `http://sierpe.railway.internal:8080`).
Management surfaces do not face the internet — the same rule of thumb you
apply to RabbitMQ or Postgres.

If you do need a public domain, set `HTTP_BASIC_AUTH=user:password`, which
gates everything except the orchestrator probes.

---

# Where it runs
> Every deployment target we evaluated — what works, what needs one setting, what fails and why — plus the constraints that let you judge a platform we did not list.


Sierpe is one long-running process next to a Postgres. That shape rules
some platforms in and others out, and the reasons are always the same
handful. This page states them once, then walks every target we
evaluated. If your platform is not here, the constraints are enough to
judge it.

## The constraints that decide everything

- **It polls.** The ingestion loop asks the Stellar RPC for the next
  ledger every few seconds, forever. A platform that scales to zero,
  sleeps on idle, or only allocates CPU while an HTTP request is in
  flight stops indexing — silently, because `/health` stays green.
- **One replica.** There is no leader election. A second copy is
  harmless to the data (the cursor only moves forward, every insert is
  idempotent) but doubles the RPC load and re-walks backfill chunks.
  Brief overlap during a rolling deploy is fine.
- **It listens on `HTTP_PORT`** (default 8080) and ignores `PORT`. This
  is deliberate: Railway injects `PORT` with a value of its own, and
  honouring it would change the listening port of every Railway
  deployment that did not pin `HTTP_PORT`. Platforms that *assign* a
  random port at runtime (Heroku web dynos) therefore fail; platforms
  that let you *declare* the port work.
- **The image is distroless**: static binary, non-root (UID 65532), no
  shell, no curl. Health checks must be HTTP from outside, or the
  built-in `sierpe healthcheck` (declared as the image `HEALTHCHECK`
  since 1.5.2). `linux/amd64` + `linux/arm64`; the `-full` variant is
  amd64 only, needs a few GB of writable scratch disk, and currently
  runs as root.
- **No disk.** The slim image writes nothing: read-only root filesystem
  is fine, no volume needed, ≥512 MB memory.
- **Postgres ≥ 14** (the driver's floor) over plain TCP with a password
  in the URL. No Unix sockets, no cloud proxies, no IAM auth. It uses
  prepared statements by default and a transaction-scoped advisory lock
  at boot; `sslmode` is honoured, and `verify-full` works against a
  provider's private CA only if you mount the CA file and add
  `sslrootcert=/path/ca.pem` to the URL.
- **Outbound HTTPS** to the RPC and to `ghcr.io` for the image pull.

## App platforms (PaaS)

| Platform | Verdict | What to set |
|---|---|---|
| **Railway** | Works | Image route; domain target port `8080`; healthcheck `/health`. [Guide](/docs/install/#railway) |
| **Render** | Works with config (paid) | Starter or larger; `HTTP_PORT=10000` or rely on port detection; health `/health`, never `/ready`. **Free tier fails** — spins down after 15 min idle |
| **Fly.io** | Works with config | `internal_port = 8080`, `auto_stop_machines = "off"`, `min_machines_running = 1`, memory ≥512 MB, check `/health`. The default `fly launch` toml reintroduces autostop — watch it |
| **DigitalOcean App Platform** | Works with config | `http_port: 8080`, health `/health`, 512 MB+ instance, do not enable inactivity sleep. No persistent disk → no `-full` |
| **Koyeb** | Works with config (paid) | Expose 8080, health `/health`, min=max=1. **Free instance fails** — sleeps after 1 h idle, not disableable. Free Koyeb Postgres burns its 5 compute-hours in a day |
| **Northflank** | Works with config | Port 8080; liveness `/health`, readiness `/ready` (the one PaaS that maps readiness correctly); nf-compute-20 or larger |
| **Zeabur** | Works with config (Dev plan) | Port 8080 declared. **Free plan fails** — sleeps on idle. No health-check configuration exists |
| **Sevalla** | Works with config | Port 8080; liveness `/health`, readiness `/ready` |
| **Porter** | Works with config | BYO cloud; `port: 8080`, `healthCheck /health`, 1 replica |
| **Heroku** | **Fails** as a web dyno | Dynos must bind to a random `$PORT`; Sierpe listens on `HTTP_PORT` by design (see above). Worker dyno runs it but the API is unreachable |
| **Coolify** | Works with config | Docker Image resource, port 8080. Since 1.5.2 the image `HEALTHCHECK` makes Coolify's in-container check pass; on older tags disable health checks |
| **Dokploy** | Works with config | Docker provider, port 8080. Since 1.5.2 the Swarm health check can use the image's own; before, leave it unset |
| **CapRover** | Works with config | Container HTTP Port `8080` (default is 80 — forgetting it is a 502) |
| **Dokku** | Works with config | `ports:set http:80:8080`; startup check `/health` runs from the host, so distroless was never a problem here |

## Hyperscalers

| Platform | Verdict | What to set |
|---|---|---|
| **AWS ECS Fargate** | Works | [Guide](/docs/install/#aws-ecs-fargate--rds) |
| **AWS EKS / GKE / AKS** | Works | Plain Kubernetes — see below |
| **AWS Elastic Beanstalk (Docker)** | Works with config | `Dockerrun.aws.json` v1 with `ContainerPort 8080`; single instance or min=max=1; ALB health `/health` |
| **AWS Lightsail Containers** | Works with config | Power Micro (1 GB) or larger, scale 1, endpoint port 8080, health `/health` |
| **AWS App Runner** | **Fails** | Closed to new customers; ECR-only images; CPU throttled between requests so the poll loop starves while `/health` stays green |
| **AWS Lambda** | **Fails** | Invocation-driven, frozen between calls, 15 min cap |
| **Google Cloud Run** | Works with config | `--no-cpu-throttling --min-instances 1 --max-instances 1 --port 8080`; startup probe `/health`; Cloud SQL via private IP + `sslmode=require` (the built-in Cloud SQL socket is Unix — unusable). Filesystem is RAM → no `-full` |
| **Google GKE Autopilot** | Works | Kubernetes below; ephemeral storage up to 10 GiB covers `-full` on amd64 nodes |
| **Google Compute Engine (COS)** | Works with config | `docker run --restart=always --network host`; the container-VM UI path is deprecated, use cloud-init |
| **Azure Container Apps** | Works with config | `--target-port 8080 --min-replicas 1 --max-replicas 1`; probes on `/health` only. Azure Flexible Server chains to **public** roots: `sslmode=verify-full` works as-is |
| **Azure Container Instances** | Works with config | Declared port 8080, `restartPolicy: Always`, liveness `/health` |
| **Azure App Service for Containers** | Works with config | B1+, **Always On**, `WEBSITES_PORT=8080`, health `/health` |

## Serverless and edge — no

Vercel, Netlify, Cloudflare Workers/Containers, Deno Deploy, Replit and
Glitch all fail on the platform model: they run code in response to
requests and freeze or stop it in between. No change to Sierpe can make
a request-driven runtime keep polling. Cloudflare Containers can be
kept awake with a hand-rolled cron keepalive, without any guarantee.

## Self-hosted and bare metal

| Target | Verdict | Notes |
|---|---|---|
| **Kubernetes** (any distro) | Works | `replicas: 1`, `strategy: Recreate` (or RollingUpdate with readiness on `/health`, not `/ready`, or a long backfill wedges the rollout); liveness `/health`; `readOnlyRootFilesystem: true`, `runAsNonRoot: true` (UID 65532). Pin `-full` to amd64 nodes with an `emptyDir` for scratch |
| **Docker Swarm** | Works | `replicas: 1`, `order: stop-first`; the image `HEALTHCHECK` (1.5.2) gives Swarm its health signal |
| **Nomad** | Works | `count = 1`, `canary = 0`, service check `/health` — never `/ready` as the deployment check |
| **Podman + systemd (Quadlet)** | Works | `.container` unit with `ReadOnly=true`, `Restart=always`; `loginctl enable-linger` for rootless |
| **systemd + release binary** | Works | `DynamicUser=yes`, `ProtectSystem=strict` — it writes nothing |
| **VPS** (Hetzner, OVH, DigitalOcean, Linode, Vultr) | Works | ≥1 GB for Sierpe alone, ≥2 GB with Postgres on the same box; 512 MB tiers OOM during backfill. ARM plans (Hetzner CAX, Oracle A1) run the slim image only |
| **Oracle Cloud Free Tier** | Works with config | A1 (arm64, slim only); note Oracle reclaims idle Always-Free instances — an indexer at the tip is light enough to trip it |
| **Raspberry Pi 4/5** | Works with config | **64-bit OS required** (32-bit pulls fail with "no matching manifest"); ≥2 GB; Postgres on SSD, never SD. No `-full` |
| **NAS** | Works on x86 (Synology +/xs, Unraid, TrueNAS SCALE, QNAP x86) and arm64 QNAP; **fails on 32-bit ARM NAS** | Use the compose/YAML route; skip the exec-based health check UI |
| **Mac (Apple Silicon)** | Works for development | Slim runs natively; `-full` only via Rosetta (OrbStack/Colima `--vz-rosetta`; Docker Desktop needs the Apple Virtualization backend). Laptops sleep — not a server |
| **Windows** | Works via Docker Desktop/WSL2; **no native binary** | WSL2 shuts down on idle and on sleep — not a server |
| **Proxmox** | Works | Binary in an unprivileged LXC (cleanest), or a VM with compose |

## Postgres providers

The three things that decide a provider: whether it is real Postgres,
whether a transaction-mode pooler sits in the path, and what signs its
TLS certificate.

| Provider | Verdict | The URL detail |
|---|---|---|
| **Amazon RDS / Aurora** | Works | `sslmode=require` (private CA); Aurora: the **writer** endpoint |
| **RDS Proxy** | Works | Multiplexes prepared statements since 2023; `verify-full` works (public ACM cert) |
| **Google Cloud SQL** | Works with config | Public or private IP + `sslmode=require`; instance must **not** be set to "require trusted client certificates" |
| **AlloyDB** | Works with config | Public IP + `require`; keep managed pooling off or in session mode |
| **Azure Flexible Server** | Works | `sslmode=verify-full` — public roots. Avoid the built-in PgBouncer on 6432 unless `max_prepared_statements` > 0 |
| **Supabase** | Works with config | **Session pooler** (`…pooler.supabase.com:5432`) or direct (IPv6 only). The **transaction pooler on 6543 fails** unless you append `default_query_exec_mode=simple_protocol` (works since 1.5.2) |
| **Neon** (incl. former Vercel Postgres) | Works | Direct or `-pooler` host both fine; `verify-full` works. Free-tier autosuspend drops connections after 5 idle minutes — a stalled RPC can trigger it; the pool reconnects |
| **Railway Postgres** | Works | Private URL as-is; public proxy with `sslmode=require` |
| **Render Postgres** | Works | Internal URL as-is; external with `require` |
| **Fly Managed Postgres** | Works | Pooled URL defaults to session mode |
| **DigitalOcean Managed** | Works with config | Direct port + `require`; connection pools only in **session** mode |
| **Heroku Postgres** | Works with config | `DATABASE_URL` + `require`; skip the pooled URL |
| **Crunchy Bridge** | Works | `require`; its PgBouncer (5431) supports prepared statements |
| **Timescale Cloud** | Works | `require`; both pools fine (PgBouncer ≥ 1.21) |
| **PlanetScale Postgres** | Works | 5432 or 6432; `verify-full` works |
| **EDB Cloud Service** | Works | `verify-full` (Let's Encrypt); session-mode pooler |
| **Prisma Postgres** | Works with config | The **direct** string, not the pooled one |
| **Xata** | Unverified | SQL proxy behaviour with prepared statements unknown |
| **YugabyteDB** | Works with config | ≥ v2025.1 (advisory locks); port 5433 |
| **CockroachDB** | Unverified | Transaction-scoped advisory locks are supported; `hashtext()` and serialization retries untested |
| **Aurora DSQL** | **Fails** | IAM-only auth, no advisory locks, no `text[]` columns, `numeric` ≤ 38 digits (i128 needs 39) |
| **Spanner (PG interface)** | **Fails** | Requires the PGAdapter sidecar and IAM; no advisory locks |
| **PgBouncer** (self-hosted) | Works | Session mode as-is; transaction mode on ≥ 1.21 with `max_prepared_statements` > 0, else `simple_protocol` |
| **Pgpool-II / Odyssey / pgcat** | Works with config | Pgpool: no load balancing for the writer. Odyssey: `pool_reserve_prepared_statement yes`. pgcat: `prepared_statements = true` |

When a pooler is the problem, Sierpe says so: a boot that dies on a
prepared-statement error (SQLSTATE 26000 / 42P05) logs the
`default_query_exec_mode=simple_protocol` fix in its last line.

## How this page was made

Each cluster was researched against the platforms' current documentation
by an assistant that only knew the constraints above, then checked
against the code. Three deployment paths (Compose, Railway, AWS) were
additionally driven end-to-end by a fresh-context assistant given nothing
but this site. Two product changes came out of it — the simple-protocol
fix and the built-in healthcheck — and one decision not to change
anything (`PORT`). Verdicts marked *unverified* are exactly that; if you
run Sierpe somewhere not listed, [tell us](https://github.com/zkCaleb-dev/sierpe/discussions).

---

# Quickstart
> Register a contract and query its complete history in five minutes.


This assumes a running instance — see [Installation](/docs/install/).

## 1. Register a contract

POST the contract id. Sierpe reads its on-chain spec, classifies it (SAC
by executable, wasm events from `contractspecv0`), and starts a
descending backfill while following the tip:

```bash
curl -X POST localhost:8080/v1/contracts \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"contract_id": "CBMLLYBH...", "from": "genesis",
       "kinds": ["events", "state", "movements"]}'
```

`kinds` picks what gets derived — `events`, `state`, `transfers`,
`trustlines`, `movements`. Omitting it gives `events` and `state`
(`transfers` too for a SAC). **If you care about deposits — anything
entering or leaving the contract — include `movements`**; see step 5.
`from` takes `"genesis"` or a ledger number. Adding a kind later reopens
the history walk for it, so nothing is lost by starting small.

Registration is idempotent. `DELETE` stops indexing but keeps the data;
re-registering resumes where it left off. If `HTTP_BASIC_AUTH` is set,
add `-u user:password` to every request in this guide.

## 2. Watch it work

```bash
curl localhost:8080/v1/contracts/CBMLLYBH...
```

The response includes the classification, discovered event names, backfill
progress and **derived coverage** — the exact ledger ranges Sierpe can
answer for.

## 3. Query events

Filters follow the proposed `getEvents` v2 semantics — positional topic
filters, opaque cursors:

```bash
curl "localhost:8080/v1/contracts/CBMLLYBH.../events?topic0=<base64-scval>&limit=100"
```

Every page declares `coverage` and a `scanStatus`:

| scanStatus | Meaning |
|---|---|
| `COMPLETE` | The full requested range was scanned |
| `HAS_MORE` | More results — follow the `cursor` |
| `WAITING_FOR_LEDGERS` | Part of the range isn't indexed yet |
| `OLDEST_REACHED` | You hit the oldest ledger Sierpe has |

The cursor encodes the whole query, so pagination never drifts: passing a
cursor *and* different filters is a 400, by design.

## 4. Query contract state

Current snapshot of storage entries, or the full change history of any
entry with provenance:

```bash
curl "localhost:8080/v1/contracts/CBMLLYBH.../state?key=<base64-scval>"
curl "localhost:8080/v1/contracts/CBMLLYBH.../state/history?startLedger=..."
```

## 5. See what moved in and out

With the `movements` kind, every token transfer that names your contract
as sender or recipient lands here — whoever emitted it. A payment of any
asset to your contract is emitted by the **asset's own SAC**, and you do
not need to register that SAC:

```bash
curl "localhost:8080/v1/contracts/CBMLLYBH.../movements?role=recipient"
```

Each row carries `role`, `transferType`, the exact amount in raw token
units, `tokenContractId` (the asset's real identity) and the
counterparty. Two caveats the response itself repeats: movements are
evidence of token events, **not a balance**, and amounts from different
`tokenContractId` values must never be summed — different tokens,
different scales.

The backfill derives movement history from before the contract was
registered, so yesterday's deposits appear too.

---

# The embedded UI
> A management interface baked into the binary — status, contracts, and a data explorer, with no build system and no external assets.


Open `/` in a browser. That is the whole setup.

Sierpe serves a management interface from the binary itself: **one
self-contained HTML page**, no build system, no external assets, no
separate service to deploy or keep in sync with the API. It covers the
same surface the REST API does.

## What it gives you

- **Live instance status** — cursor position, tip lag, the archive leg
  verdict, open gaps.
- **The contract list** — every registration with its classification,
  kinds, declared coverage and counts.
- **A data explorer** — a tab per kind: events, transfers, movements, state and its
  history, trustlines and theirs. Filters and cursor pagination included,
  so you page through real data instead of composing curl calls.
- **Registration and unregistration** — behind an admin-token field that
  the page holds **only in memory**. Nothing is written to local storage;
  reload and the token is gone.

## Access

Reads work without credentials, matching the open-reads access model of
the API. Only mutations ask for the admin token.

If the instance faces a public domain, set `HTTP_BASIC_AUTH` — the browser
prompts natively and the UI inherits those credentials with no
configuration of its own. See [the access model](/docs/api/#access-model).

## Why it is built this way

The appliance rule applies to its own interface: if you had to build,
host or configure the UI separately, it would stop being an appliance.
Baking one static page into the binary keeps deployment a single
container, and keeps the UI incapable of drifting out of sync with the
API version it ships with.

---

# API reference
> The v1 REST surface — contracts, events, state, and the honesty contract every response follows.


The authoritative specification is
[docs/openapi.yaml](https://github.com/zkCaleb-dev/sierpe/blob/main/docs/openapi.yaml)
in the repository. This page is the map.

## Admin surface (bearer-authenticated)

| Method & path | Purpose |
|---|---|
| `POST /v1/contracts` | Register a contract (or reconcile an existing registration); classification and backfill start automatically |
| `GET /v1/contracts/:id` | Detail: classification, discovered events, backfill progress, coverage |
| `DELETE /v1/contracts/:id` | Stop indexing; data is kept, re-registration resumes |

## Read surface

| Method & path | Purpose |
|---|---|
| `GET /v1/contracts` | List every registration with its classification and kinds |
| `GET /v1/contracts/:id/events` | Events with getEvents-v2-style filters and cursors |
| `GET /v1/contracts/:id/state` | Current storage snapshot, paginated by key + durability |
| `GET /v1/contracts/:id/state/history` | Change history of storage entries, with provenance |
| `GET /v1/contracts/:id/transfers` | Decoded token movements in chain order |
| `GET /v1/contracts/:id/trustlines` | Current trustline holders of the SAC asset |
| `GET /v1/contracts/:id/trustlines/history` | Trustline changes with before/after balances |
| `GET /v1/contracts/:id/movements` | Token transfers this contract took part in, whoever emitted them |

### Token transfers

SEP-41 movements (transfer, mint, burn, clawback) decoded into structured
rows: `from`/`to` addresses, the exact i128 amount, the SEP-0011 asset and
the CAP-67 muxed destination id. Filter by `account` (either side of the
movement, exclusive with `from`/`to`), `from`, `to`, `type`, and a ledger
range. SAC registrations derive transfers by default; custom SEP-41 tokens
opt in through `kinds`.

### Trustlines

For a registered SAC, Sierpe attributes the classic trustline changes of
the asset it wraps: live holders at `/trustlines`, and chain-order changes
with before/after balances at `/trustlines/history`. Opt in through
`kinds`. Native XLM has no trustlines, so the kind observes issued assets
only.

### Movements

Transfers answer "what did this token emit". Movements answer the other
question — **"what came into and went out of my contract"** — and they are
different questions: a payment to your contract is emitted by the asset's
own SAC, not by your contract. Register the `movements` kind and every
token transfer naming your contract as sender or recipient lands here,
without registering the asset's SAC at all.

Query parameters: `role` (`recipient` | `sender`; omit for both),
`token` (the emitting contract id — the asset's real identity, never its
SEP-0011 string), `type` (`transfer` | `mint` | `burn` | `clawback`),
`startLedger`, `endLedger`, `limit` (1–1000, default 100) and `cursor`.
"Deposits" in the everyday sense are `role=recipient` — that includes
mints to the contract, since a mint is value arriving too.

Each row: `id` (shared by the two rows of a self-transfer — key on `id` + `role`), `role`, `transferType`, `tokenContractId`,
`counterparty` (absent on mints and burns), `amount` (exact i128 in raw
token units, as a string), `ledger`, `ledgerClosedAt`. The page carries
the usual `cursor`, `scanStatus`, `coverage` and a `note`.

Because ingestion downloads whole ledgers, the descending backfill derives
movement history from **before** the contract was registered — the thing
dynamic-source indexers cannot do.

One honest caveat, stated by the API itself in a `note` field: movements
are **not a balance**. Value can also arrive without any SEP-41 transfer
event, and amounts are raw base units of different tokens — never sum
across `tokenContractId`.

## Operational surface

| Path | Purpose |
|---|---|
| `/health` | Liveness |
| `/ready` | Readiness — 503 while catching up |
| `/status` | Cursor position, tip distance, per-contract summary |
| `/metrics` | Prometheus metrics ([documented](https://github.com/zkCaleb-dev/sierpe/blob/main/docs/METRICS.md)) |

## The honesty contract

Every paginated response carries:

- **`coverage`** — the ledger ranges this instance can actually answer
  for, derived from backfill progress and the live cursor. Since 1.5.0
  coverage is declared **per (contract, kind)**: a walk only vouches for
  the kinds it actually derived, and a kind added later reopens the walk
  instead of silently claiming history it never looked at.
- **`scanStatus`** — `COMPLETE`, `HAS_MORE`, `WAITING_FOR_LEDGERS` or
  `OLDEST_REACHED`.
- **`cursor`** — opaque, encodes the full query. Cursors are bound to
  their endpoint and cannot drift across filters.

Event ids follow the `getEvents` format: `{toid}-{event_index}`,
zero-padded, stable across replays.

## Access model

Reads are open; admin mutations require the `ADMIN_TOKEN` bearer. That
suits the default deployment shape — private networking, where nothing
outside your platform's internal network reaches the instance.

If you do expose a public domain, set `HTTP_BASIC_AUTH=user:password` and
**every** request needs those credentials — the UI, the API and
`/metrics` — leaving only `/health` and `/ready` open for orchestrator
probes. Browsers prompt natively; clients send the standard header
(`curl -u user:password …`). Admin mutations still need the bearer on top.

---

# The archive leg
> How Sierpe reaches history no RPC serves anymore — and why it verifies itself before writing a single healed ledger.


Stellar RPCs retain roughly **seven days** of events. The slim image stops
honestly at that wall: the range it cannot serve is recorded as a
**declared gap**, and the API says so.

The `-full` image variant closes the wall. It bundles `stellar-core` and
**heals** those gaps by replaying the missing ledgers from the public
history archives — register a contract with `from: "genesis"` and its
complete history converges even where no RPC reaches.

## The equivalence gate

Replaying archives is only useful if the result is *identical* to what the
RPC would have served. Before the first heal, Sierpe proves it: the
captive replay must come out **byte-equivalent to your RPC** on a
checkpoint-aligned range both can serve.

Two parts of the ledger meta are unstable run to run even on identical
core builds, so they are normalized before comparing: diagnostic events
are stripped, and ledger-entry-change units are canonically ordered within
each operation.

If the replay diverges, healing is **disabled** —
`sierpe_archive_equivalence_failures_total` increments (alert on it) and
the gaps stay recorded. Sierpe would rather show you an honest hole than
fill it with unverified data.

`/status` reports the verdict:

```text
archive: off | unverified | verified | equivalence_failed
```

## How healing progresses

Gaps are walked downward in atomic 2000-ledger chunks. Each chunk lowers
the heal watermark on the gap row *and* the clamped backfill frontier in
the same transaction — so **declared coverage grows exactly as fast as
healed data lands**, never ahead of it.

Watch `sierpe_gaps_healed_total`, `sierpe_healed_ledgers_total` and
`open_gaps` draining in `/status`.

## Enabling it

Deploy the `-full` tag; `STELLAR_CORE_BINARY` is pre-set:

```bash
docker pull ghcr.io/zkcaleb-dev/sierpe:v1.2.0-full
```

| Variable | Meaning |
|---|---|
| `STELLAR_CORE_BINARY` | Path to a stellar-core binary; enables the leg. Pre-set in `-full` |
| `HISTORY_ARCHIVE_URLS` | Archives to replay from. Defaults to the SDF public archives |
| `CAPTIVE_STORAGE_PATH` | Disposable scratch space for buckets. Defaults to the OS temp dir |

Before enabling it, know that:

- The `-full` image is **linux/amd64 only** (SDF publishes stellar-core
  for amd64); it runs under emulation on ARM hosts.
- Budget more CPU and a few GB of scratch disk for bucket downloads.
- The slim image stays multi-arch and distroless for archive-less
  deployments — if you only need the last seven days plus everything
  since, you do not need this.

---

# Observability
> Prometheus metrics, the Grafana dashboard, and the alerts that matter.


Sierpe is built on the principle that **silent data loss is the worst
failure mode**. Everything it cannot index is counted, exposed and
alertable.

## Metrics

`/metrics` exposes Prometheus metrics from a private registry. The full
catalogue — and which ones deserve alerts — is in
[docs/METRICS.md](https://github.com/zkCaleb-dev/sierpe/blob/main/docs/METRICS.md).
The headline signals:

- **`sierpe_tip_lag_seconds`** — age of the last committed ledger against
  wall clock. Sustained growth means falling behind.
- **`sierpe_open_gaps`** — unresolved coverage gaps. Any nonzero value is
  declared, unserved history.
- **Suppression counters** — `sierpe_suppressed_txs_total`,
  `_events_`, `_transfers_`, `_trustlines_`. Anything Sierpe could not
  read is counted, never silently dropped. **Alert if nonzero**: that is
  counted data loss.
- **`sierpe_archive_equivalence_failures_total`** — the archive replay did
  not match the RPC byte-for-byte, so healing is disabled. **Alert if
  nonzero** ([why](/docs/archive-leg/)).
- **Progress signals** — `sierpe_ledgers_ingested_total`,
  `sierpe_backfill_pending`, `sierpe_gaps_healed_total`,
  `sierpe_healed_ledgers_total`.

## Grafana

A ready-made dashboard ships in the repository at
[deploy/grafana/sierpe-dashboard.json](https://github.com/zkCaleb-dev/sierpe/blob/main/deploy/grafana/sierpe-dashboard.json)
— eight panels covering the signals above.

## Status page

A [Gatus](https://github.com/TwiN/gatus) configuration is provided at
`deploy/gatus/config.yaml`, including a check that turns the status page
red when open gaps exist — your users see data honesty, not just uptime.

## Integrity guarantees worth knowing

- The ingestion loop verifies `PreviousLedgerHash` continuity on **every**
  ledger, including the first one after a restart.
- Cursor and data commit in the same transaction — a crash can never
  leave them disagreeing.
- On testnet resets, divergence is detected and the process stops loudly
  with zero writes rather than mixing two histories.

---

# AI assistants & agents
> Machine-readable docs at /llms.txt, how to point an agent at the API, and why the honesty contract is what makes agent answers trustworthy.


Sierpe's API was designed for consumers that cannot shrug — dashboards,
backends, and increasingly, AI agents. This page covers both directions:
teaching an assistant about Sierpe, and letting an agent query a running
instance.

## Machine-readable documentation

The site publishes its documentation in the
[llms.txt](https://llmstxt.org/) convention:

| URL | Contents |
|---|---|
| [`/llms.txt`](/llms.txt) | Index: what Sierpe is, the facts assistants get wrong, links to every page |
| [`/llms-full.txt`](/llms-full.txt) | Every documentation page concatenated into one plain-text file |

Both are generated at build time from the same source files as the pages
you are reading, so they cannot drift. Paste `/llms-full.txt` into any
assistant's context — or point tools that fetch `llms.txt` automatically
at the site root.

For the API itself, give the agent the authoritative contract, not prose
about it:

```text
https://raw.githubusercontent.com/zkCaleb-dev/sierpe/main/docs/openapi.yaml
```

Swagger-literate agents can generate correct calls from that file alone.

## Letting an agent query your instance

An agent needs three things: the base URL, credentials, and one paragraph
of ground rules. A system-prompt block that works:

```text
You can query a Sierpe instance (a self-hosted Stellar contract indexer)
at https://YOUR-INSTANCE. Reads use GET; if HTTP_BASIC_AUTH is set, send
those credentials. Endpoints: /v1/contracts (list),
/v1/contracts/{id}/{events|state|transfers|trustlines|movements}.

Rules you must follow:
- Check `coverage` and `scanStatus` on every response. An empty page with
  partial coverage means "not indexed here", NOT "it never happened" —
  say which, based on the declared range.
- Page with the `cursor` value; never combine a cursor with other filters.
- Movements are evidence of token events, not a balance. Never sum
  amounts across different tokenContractId values — they are raw base
  units of different tokens.
- Amounts are exact integers in raw token units (i128); do not round.
```

Registering contracts (POST/DELETE) needs the `ADMIN_TOKEN` bearer. Only
hand that to an agent if you want it registering contracts on its own;
read-only agents do not need it.

## Why this works better than most APIs

Agents fail loudest when an API leaves silence ambiguous — an empty list
that could mean "nothing exists" or "we did not look". Sierpe never
leaves that ambiguous, by design:

- **Coverage per (contract, kind)** states exactly which ledger range the
  instance can vouch for, so an agent can qualify its answer instead of
  guessing.
- **`scanStatus`** distinguishes a complete scan from a truncated page,
  a range beyond the tip, and history this instance cannot serve.
- **In-band caveats**: the movements endpoint carries its "this is not a
  balance" note in the response itself, where an agent will actually
  read it — not in documentation it never fetched.

The honesty contract was built so that dashboards do not lie. It turns
out to be exactly what keeps language models from lying, too.

## Roadmap

An MCP server — exposing a running instance as tools an assistant can
call natively, instead of raw HTTP — is a natural next step we are
exploring. It is not committed yet; if you would use one,
[say so in Discussions](https://github.com/zkCaleb-dev/sierpe/discussions).

---

# Architecture
> The pipeline, the data model, and the design decisions behind the appliance.


The full design document lives at
[docs/DESIGN.md](https://github.com/zkCaleb-dev/sierpe/blob/main/docs/DESIGN.md);
the study behind it — 29 principles distilled from production indexers,
each with its source — at
[docs/KNOWLEDGE.md](https://github.com/zkCaleb-dev/sierpe/blob/main/docs/KNOWLEDGE.md).

## The pipeline

```text
source  →  ingest  →  process  →  store  →  serve
 (RPC       (single-   (classify,   (Postgres,  (REST API,
  pool)      writer     extract      atomic      honest
             loop)      events &     commits)    paging)
                        state)
```

- **Source** is a seam: the RPC pool follows the tip, and a captive
  stellar-core replay source serves bounded history-archive ranges behind
  the same interface — that seam is what made [the archive
  leg](/docs/archive-leg/) a drop-in addition rather than a rewrite.
- **Ingest** is a single-writer loop. One writer means hash-chain
  continuity can be *verified*, not assumed.
- **Store** owns its Postgres: embedded migrations under an advisory
  lock, cursor and data in one transaction.
- **Serve** never invents data: what the store hasn't got, the API
  declares as a gap.

## Backfill

Registration triggers a **descending** backfill: from the tip backwards
in atomic 2000-ledger chunks, each with its own watermark. Newest data
arrives first — usually what you want — and progress survives restarts
exactly. At the RPC retention wall, the unserved range is persisted as a
gap *before* the clamp commits, so nothing is ever silently missing. With
the [archive leg](/docs/archive-leg/) enabled, those recorded gaps are
later healed from the public archives — and the clamped frontier is
lowered in the same transaction that lands the healed data.

## Design rules the code enforces

- If the user has to touch code, it's a design bug.
- Sierpe owns its database; consumers use the API, never the tables.
- Exactly-once by construction (atomic cursor+data), not by deduplication.
- Systematic distrust: failed transactions skipped and counted, spec
  parse failures degrade to `opaque` classification instead of erroring.
- No CGO — a single static binary, trivially containerized.

---

# Why Sierpe


Every team building on Stellar eventually hits the same wall: the RPC
retains about **seven days** of events. If your frontend depends on
contract events, you either build and babysit a custom indexer, pay for a
hosted service, or lose your own history.

The existing landscape offers three shapes, and all of them ask something
from you:

| Shape | Examples | What it asks of you |
|---|---|---|
| Hosted service | Mercury, stellarindexer.com | Your data lives in someone else's infra, usually behind a subscription |
| Composable toolkit | CDP building blocks, flowctl/nebu | You assemble and operate a pipeline |
| Framework | SubQuery | You fork a template and write indexing code |

Sierpe takes the fourth slot, the one nobody occupied: the **self-hosted
appliance**. Like Postgres or Prometheus, it is a server you deploy, not a
codebase you adopt. If you have to write code to use it, that's a bug.

## What makes it different

**History past the retention wall.** Registering a contract *today* gets
you its complete past, not just its future. Sierpe backfills as far as the
RPC serves, records what it cannot reach as a declared gap, and — with the
[archive leg](/docs/archive-leg/) enabled — heals those gaps by replaying
the public history archives, but only after the replay proves itself
byte-equivalent to your RPC. History below retention, or an honest gap.
Never an unverified guess.

**Honesty as an API contract.** Distributed ingestion loses data in
silence; Sierpe refuses to. Every paginated response declares its
`coverage` and a `scanStatus` (`COMPLETE`, `HAS_MORE`,
`WAITING_FOR_LEDGERS`, `OLDEST_REACHED`). Gaps are persisted, queryable
and alertable — never papered over.

**More than events.** Storage entry change history with provenance plus a
current snapshot, decoded SEP-41 token movements, and the classic
trustlines of the asset a SAC wraps — data most event indexers ignore.

**Forward-compatible by design.** The events API follows the semantics of
the proposed `getEvents` v2 RPC endpoint (positional topic filters, opaque
cursors, scan status), so an integration built against Sierpe speaks
tomorrow's standard.

**Integrity paranoia.** A single-writer loop verifies ledger hash-chain
continuity permanently — including across restarts — commits cursor and
data in the same transaction (exactly-once by construction), detects
testnet resets, and would rather stop loudly than write a lie.

## What Sierpe is not

- **Not a hosted service** — you run it. That's the point.
- **Not an analytics platform** — the [embedded UI](/docs/management-ui/)
  explores and operates; it does not aggregate or chart your data.
- **Not a chain-wide indexer** — it indexes the contracts you register.
- **Not a framework** — there is nothing to fork and no SDK to learn.

## The cost story

The volume Sierpe stores is proportional to the contracts you register,
not to the chain. A typical project runs the container plus a small
Postgres for **under $10/month** on Railway or any VPS.