Operations

How to run FDS outside a development checkout: what it needs, every setting it reads, and how the database schema is managed. Nothing here is specific to any one installation.

Shape of a deployment

FDS is one container image serving HTTP on port 8000. It is stateless: everything it knows is in its database or in the object stores it brokers access to. A deployment therefore has three parts, and FDS provides only the first.

FDS speaks plain HTTP and does not terminate TLS; how requests reach it is your decision. Two things about its behaviour bear on that:

• The identifiers in its JSON-LD output are built from the host the request arrived with, so the hostname clients use becomes part of the catalogue's public identity. Choose one you intend to keep.
• When something in front of FDS rewrites requests, it believes X-Forwarded-* headers only from senders listed in FORWARDED_ALLOW_IPS. Setting FDS_BASE_URL is the more robust answer, because it stops FDS deriving its own address from requests at all.

Any web front end is a separate deployable that calls the API.

Container images

Two images are published to the GitHub Container Registry on every push to main and on every release tag:

FDS serves JSON, never HTML. A catalogue people can browse, and the readable landing page a DOI has to resolve to, come from a web front end running alongside FDS. The reference UI is published so that a deployment has one without building it first, and so there is a working example to base your own on. Only a deployment serving machines alone can do without it.

Each carries four kinds of tag:

Pin a deployment to sha-<short> or a version, never to main or latest: a rollback is only possible if the tag you rolled back from still names the build you were running.

Database

FDS runs on PostgreSQL and has no other backend. Point it at one with FDS_DB_HOST and friends, or give it a complete URL in FDS_DB_URL.

Schema migrations

The schema is created and changed only by the Alembic migrations shipped with the code, in alembic/versions/. A migration describes how the table layout changes from one release to the next. It carries no data and nothing about any particular installation; the connection comes from the environment at run time. The only per-installation state is a one-row alembic_version table recording which migration that database has reached.

The image runs alembic upgrade head before starting the server, so a new database is built and an existing one brought up to date on every start. To run it separately, for example before swapping in a new image:

alembic upgrade head

Every migration is reversible. Rolling back to an older release that expects an older schema means downgrading first, then deploying the older image:

alembic downgrade <revision>

Back up the database before either operation on data you care about.

Moving to a new hostname

FDS stores nothing about its own address. The URLs in its responses, including every JSON-LD @id, are built from the incoming request's host, and a record created locally carries no origin (only records ingested from another catalogue do). So a change of hostname needs no change to the database. What it does need is outside FDS:

• Records held elsewhere. Harvesters, other catalogues and citations hold the old URLs. Keeping the old hostname answering with redirects for as long as you can is the usual remedy.
• DOIs. Update the landing URL on each DOI record at the registrar; that indirection is what a DOI is for.

The same reasoning applies in the other direction. When ingesting records from another catalogue, set their origin to the most persistent identifier that catalogue has (a re3data record, a DOI, an identifier it advertises in its own DCAT catalogue) and use the same value every time; its URL is the fallback, and it goes stale if that catalogue moves.

Settings

Every setting is an environment variable. .env.example in the repository lists them with their defaults.

Application

Database

Identity

Storage

Storage providers

Each entry describes one store. A distribution gets credentials from the entry whose type matches its URL's scheme and whose endpoint_url equals its own. FDS refuses to start if two entries share a type and endpoint_url, since only the first could ever be chosen.

type is required in every entry, s3 included. For gcs and azure, endpoint_url only matches distributions: FDS never calls it. An Azure URL that names its account, as in abfs://container@account.dfs.core.windows.net/path, gets credentials only when that account is the provider's storage_account.

[
  {
    "type": "s3",
    "endpoint_url": "https://s3.echo.stfc.ac.uk",
    "region": "us-east-1",
    "sts_role_arn": "arn:aws:iam::123456789012:role/fds"
  },
  {"type": "gcs", "endpoint_url": null}
]

Serving

Read by the web server and the tracing library rather than by FDS itself.

Health

GET /health returns 200 with the running version when the process is up. It does not touch the database; use it for container health checks and uptime probes, and monitor the database directly.