Operator guide

For the contribution engineer who declares streams, hands the venue its address and secret, and watches the event. It assumes you know SRT, streamid, passphrases and TS on UDP; it spends its words on how the relay exposes them. New to the relay? The quickstart is ten minutes and comes first, and the four things everything here is made of are in Concepts.

Everything here can be done in the operator UI or with the control API; the examples use the API (RELAY=http://<host>:8091) because it is exact, and say where the same thing lives in the UI. The API reference is API in the UI’s navigation.

Setting up an event

A camera with a primary and a backup feed from the venue, two encoders pulling from you, and an engineer monitoring:

curl -X POST "$RELAY/streams" -H 'content-type: application/json' -d '{
  "name": "event/cam1",
  "tags": { "production": "event" },
  "inputs": [
    {
      "protocol": "srt", "route": "event/cam1", "listeners": ["srt-5001"],
      "users": [
        { "name": "venue-a", "passphrase": "a-long-venue-secret-1" },
        { "name": "venue-b", "passphrase": "a-long-venue-secret-2" }
      ],
      "codecs": ["h264", "hevc", "aac"]
    },
    { "protocol": "srt", "route": "event/cam1-b", "role": "backup", "listeners": ["srt-5001"] }
  ],
  "output": {
    "legs": [
      { "name": "enc-p", "role": "primary", "passphrase": "encoder-primary-secret" },
      { "name": "enc-b", "role": "backup", "passphrase": "encoder-backup-secret", "alertIfDisconnected": true },
      { "name": "mon", "role": "monitor", "maxConnections": 4 }
    ]
  }
}'

What that declared:

• Two inputs — event/cam1.primary and event/cam1.backup — and a source that switches between them (primary first). Declared inputs listed in one call join the stream’s switch in that order.
• Named sources on the primary. Each user pushes with streamid=#!::r=event/cam1,u=venue-a and their own passphrase (10–80 characters), so each can be revoked alone. With users declared and no anonymous passphrase, a source that names no user is refused. The backup has no grants, so any source may push it in the clear — give it users too for a real event.
• A codec policy on the primary: anything other than H.264, HEVC or AAC is dropped, with the reason in the events.
• Named consumers on the output. Each dials the output’s port with streamid=#!::r=event/cam1,u=<name> and its passphrase. enc-b raises an alert while nobody is connected as it (enc-p does too, by default, because its role is primary); mon allows four connections at once, the others one.

Other input settings you will reach for: active windows (accept a source only between from and to; enforceWindow: true kicks one still connected when the window closes, otherwise it runs on), acceptance rules (validation: required video, frame rates, profiles, levels, audio languages, bitrate range — reject drops a source that breaks them, flag admits it and marks the input degraded), and TR 101 290 (below).

What to send the venue is the input’s address, route, user and passphrase — the venue push sheet is a sheet to fill in and send. The input’s Ingest panel (open it in Inputs) shows the exact streamid for each user, with a copy button.

The tech check. Have the venue push days early. In Inputs, the input goes present and its row shows who is connected and from where. Open it: the Media panel shows the PIDs and codecs actually arriving and the bitrate; the eye button monitors it in the browser, sound included. Then stop: with no windows declared the input stays declared, ready for the day.

Failover: backups, slates and pins

A stream’s source is a switch over its inputs. Its policy:

Setting	Default	What it does
prefer	first	first: always serve the first serving input, so a returning primary takes back over. sticky: stay on the current input while it serves; move only when it fails.
holdDownMs	0	After a switch, a better-ranked input does not take over for this long, so a flapping primary cannot bounce the consumers. Losing the serving input always switches.
switchOn	none	Losing the source always triggers a switch. Add degraded to also leave a source the health checks or acceptance rules have flagged.
bufferDelayMs	0	Delays every input through the switch by this much so a switch decided on loss can still land on the new source’s keyframe. Costs that much latency.

curl -X PATCH "$RELAY/streams/event%2Fcam1" -H 'content-type: application/json' -d '{
  "source": { "kind": "switch",
    "policy": { "prefer": "first", "holdDownMs": 10000, "switchOn": ["degraded"] },
    "of": [ { "kind": "input", "input": "event/cam1.primary" },
            { "kind": "input", "input": "event/cam1.backup" } ] }
}'

A switch cuts on a video keyframe: picture and sound change together. In the quickstart’s test, losing the primary put the backup on air about six seconds later (the time to decide the primary is gone), and a returning primary took back over within a second. The events record each cut with its reason (failover, revert).

Slates. A slate is what a switch serves while none of its inputs can: an uploaded MP4. Upload it once, then name it on the switch:

curl -X POST "$RELAY/assets?name=Holding%20slate&tags=use:slate" \
  -H 'content-type: video/mp4' --data-binary @holding.mp4      # -> "id": "holding-slate"

curl -X PATCH "$RELAY/streams/event%2Fcam1" -H 'content-type: application/json' -d '{
  "source": { "kind": "switch", "slate": { "asset": "holding-slate", "when": "no-input" },
    "of": [ { "kind": "input", "input": "event/cam1.primary" },
            { "kind": "input", "input": "event/cam1.backup" } ] }
}'

Assets in the UI lists uploads with thumbnails; the stream’s canvas picks a slate from them.

Taking control. On the stream’s page, make active puts an input on air by hand. It is held: it overrides the policy until it stops serving or you press resume policy. A pin goes further: the pinned input is on air whenever it serves, coming back to it after a failure, until you unpin it.

curl -X POST "$RELAY/streams/event%2Fcam1/switches/main/active" \
  -H 'content-type: application/json' -d '{ "input": "event/cam1.backup" }'            # hold
curl -X POST "$RELAY/streams/event%2Fcam1/switches/main/active" \
  -H 'content-type: application/json' -d '{ "input": "event/cam1.backup", "pin": true }' # pin
curl -X DELETE "$RELAY/streams/event%2Fcam1/switches/main/active"                        # resume
curl -X DELETE "$RELAY/streams/event%2Fcam1/switches/main/pin"                           # unpin

The root switch’s id is main. A stream can nest switches and muxes (tracks from several inputs composed into one stream, PIDs and languages picked per track); the canvas on the stream’s page builds those, and its changes are listed before you apply them.

Inputs of different protocols. The switch passes its live input’s track ids through as the output’s PIDs. An SRT, TS or RIST input’s tracks carry their TS PIDs; an RTMP or MoQ input’s, and an input at another site (carried over MoQ), number from 0 or 1. Mixed in one switch as they come, every cut moves the output to new PIDs, and a consumer already connected sits on the old ones with nothing on them. Renumber every input of such a switch by kind (video from 256, audio from 288), so the output keeps its PIDs whichever is on air:

"of": [ { "kind": "input", "input": "event/cam1.primary", "pids": { "byKind": {} } },
        { "kind": "input", "input": "event/cam1.moq",     "pids": { "byKind": {} } } ]

The canvas does this for you when an input you add keys differently from the ones there; the API leaves it to you.

Watching it live

• Dashboard: every production at a glance — streams, health, inbound and outbound bitrate.
• Inputs: each input’s state (declared, present, degraded, absent), who is connected, the bitrate, and the streams it feeds. Open one for its four panels:
  • Ingest — the streams it feeds (links), where to dial, the listeners and codecs it admits;
  • Media — the PIDs as the engine sees the TS (or the tracks, for other protocols), the bitrate over the last minute, and link health for bonded, RIST or ST 2022-7 sources;
  • TR 101 290 — see below;
  • Recording — the policy, whether it is writing, and what it holds.
• Monitor: the eye button on any input, stream or output plays it in the browser, with per-channel audio meters. Monitor wall shows every stream at once.
• Events: the audited record of everything — sources accepted, rejected and dropped, switches, consumers attached and leaving, recordings. Newest first, searchable. Times show in the zone picked at the top of the page (local, UTC, or a broadcast centre’s zone); the API speaks UTC.

Acceptance rules in flag mode mark a source that breaks them degraded (out_of_spec in the events) without dropping it; a switch with switchOn: ["degraded"] then leaves it. In reject mode the source is dropped.

TR 101 290 is opt-in per input, for transport streams (SRT, UDP-TS, RIST). Turn it on in the input’s TR 101 290 panel (monitored), or with PATCH /inputs/<id> { "tr101290": true }. It costs a little on every video frame, and a source that is not 101 290 compliant only reports noise, so monitor the feeds you care about. The panel shows priority 1 (the stream cannot be decoded reliably: sync loss, PAT/PMT missing, continuity errors), priority 2 (it decodes but is out of spec: CRC, PCR timing, PTS) and priority 3 (service information), each for the last minute and since the source connected, with the counters that moved. A P1 error raises a 101 290 P1 badge on the input and an event, cleared a minute after the last one. Turning it on restarts a TS, RIST or pull input’s ingest (a moment’s drop); an SRT input a source dials takes it from the source’s next connection.

Prometheus ($RELAY/metrics): input presence and bitrate, consumers per output and its alarms, active streams, and for monitored inputs relay_input_tr101290_errors_total{input,counter,priority} and relay_input_tr101290_p1_active.

Consumers

An output’s legs are its named consumers. A consumer dials the output’s port (shown in Outputs and on the output’s Output panel) with streamid=#!::r=<output>,u=<leg> and the leg’s passphrase. Per leg: a maxConnections cap (excess connections are kicked), and alertIfDisconnected, which raises an alert while nobody is connected as it (on by default for a leg whose role is primary). The Outputs view lists each leg with its connections, from where and for how long.

• Kick a consumer by identity (DELETE /outputs/<name>/consumers?user=enc-b) or one connection of a shared identity (the × beside it).
• Move an output to another stream with its Stream selector, or detach it: its consumers stay connected throughout, and carry whatever it is attached to next. That is how a consumer is re-plumbed without being reconfigured.
• Push: an output can dial out instead — push: { host, port, streamId } sends the stream to a remote SRT listener (bonded links too).

Expiring a stream nobody removed

A stream that should go away when its event is over can carry a safety net in case nobody removes it: expireAfterIdleMs on the stream (on create, or PATCH /streams/<name>; Expire when idle in the stream’s settings on the canvas).

curl -X PATCH "$RELAY/streams/event-a%2Fcam1" -H 'content-type: application/json' \
  -d '{"expireAfterIdleMs": 1800000}'

Only consumers count. Until the first one connects nothing happens — a stream declared weeks ahead simply waits — and while any is connected nothing happens either. When the last one leaves the clock starts; a consumer connecting stops it, and it starts again from the next last disconnect. The stream’s page shows the setting, and expires in … while the clock runs, on its Streams row too (expiresAt on the API). reset beside it gives the stream its whole period again (POST /streams/<name>/expiry/reset), without waiting for a consumer.

At the deadline the relay removes the stream, the outputs attached to it, and any of its inputs no other stream reads; recordings stay. One stream_expired event says so, with how long it had been idle and what it removed. A relay restart carries on counting rather than starting over: a stream that had consumers when the relay stopped is timed from the restart, since the restart dropped them.

Recordings

Recording is a policy on an input (as it arrives), an output (what it carries) or a stream (as served), or a group default for its streams’ inputs. Two tiers:

• ts-dump — chunked TS files on the relay, in chunkSeconds chunks (default 15 minutes), deleted expiryDays after their last write (default 3 days). Downloadable, and playable in the browser with seek.
• recorder — an engine media store (TAMS): play it, and cut from it in the Cut UI.

curl -X PATCH "$RELAY/inputs/event%2Fcam1.primary" -H 'content-type: application/json' \
  -d '{ "recording": { "tier": "ts-dump", "chunkSeconds": 300, "expiryDays": 7 } }'

A recording starts when the source arrives and runs until it goes; each is a set of numbered chunks. Recordings lists every owner. Open one for its recordings, newest first, each collapsed to a header (when it started, how many chunks, its size, and play for the whole of it); open a recording for its chunks — when each started and was last written, download, delete. The retention control keeps a recording past its expiry, or deletes it sooner. When an owner is removed, you choose whether its recordings go with it; kept ones are listed under No longer attached until they expire.

When something goes wrong

• A source has frozen but not disconnected, or the wrong encoder has the route: kick it (DELETE /inputs/<id>/source) so a replacement can connect. On a UDP-TS input, a kick blocks the sender for a short time, then lets it back.

• A passphrase has leaked: replace that user’s grant. The others keep working; a connected source with the old passphrase stays until kicked.

  curl -X PATCH "$RELAY/inputs/event%2Fcam1.primary" -H 'content-type: application/json' -d '{
    "users": [ { "name": "venue-a", "passphrase": "a-new-venue-secret-1" },
               { "name": "venue-b" } ]
  }'

  A user named without a passphrase keeps the one it has.

• The primary keeps flapping: set holdDownMs so a returning primary has to stay up before it takes back over, or prefer: "sticky".

• Something unexplained happened: the Events page has it, with the reason in the relay’s words — a source rejected for its codec, a window, a passphrase, or a rule; a switch and why; a consumer kicked over its cap.

Groups: defaults for a whole event

A group names a set of streams — by tag, or a list — and carries defaults for their inputs: a schedule (active windows), a recording policy, TR 101 290 monitoring. An input’s own setting always wins; one without it follows its group.

curl -X POST "$RELAY/groups" -H 'content-type: application/json' -d '{
  "name": "event",
  "members": { "match": { "tags": { "production": "event" } } },
  "activeWindows": [ { "from": "2026-11-01T18:00:00Z", "to": "2026-11-01T23:00:00Z" } ],
  "recording": { "tier": "ts-dump", "chunkSeconds": 300, "expiryDays": 7 }
}'

Every input of every stream tagged production: event now accepts sources only during the event and records while they are present — including streams declared later with that tag.

Several sites

Relays at several sites form a cluster: each site runs its own relay and keeps its own streams, inputs and outputs, and sees every other site’s, read-only, on its pages (“At site”). A stream lives at one site, its home; it can still be served at the others, and its inputs can be at other sites.

Joining. The first site is configured with just its own name and address; every later one with a seed (any site already in), that site’s key fingerprint, and a one-time join token from the cluster’s registrar (the first site, unless handed over). Under norsk-ctl these are the template’s CLUSTER_* parameters; in a relay’s own config:

"cluster": {
  "site": "venue-b",
  "control_url": "https://venue-b.example:8444",
  "media_host": "venue-b.example",
  "https_port": 8444,
  "seed": "https://venue-a.example:8443",
  "seed_fingerprint": "12e2:f42f:1f7b:dabb:aed0:7351:73eb:0504",
  "join_token": "<from POST /cluster/join-tokens at the registrar>"
}

The fingerprint is the seed’s site key, as its Sites page shows it, not its TLS certificate’s. A site keeps its key and membership across restarts; the token is only needed once. Sites lists every site, whether this one is following it, and since when.

A stream served at another site. A consumer can dial a stream’s output at any site both the stream and the output may be served at (sites on each; unset means everywhere). That site stands in for the stream: it pulls it from the home over MoQ when a consumer arrives, and hangs up when the last one leaves. keepWarm on the stream names sites it is carried to with no consumer there, so the first one does not wait.

An input at another site. A stream’s switch can list an input declared at another site — a backup at a second venue, say — by its id, like a local one. The home stands in for it and carries it over MoQ while that site says it has a source, so a failover to it is as quick as to a local backup:

# at venue-b
curl -X POST "$RELAY_B/inputs" -H 'content-type: application/json' -d '{
  "id": "event/cam1.far", "protocol": "srt", "route": "event/cam1-far",
  "listeners": ["srt-5001"], "role": "backup", "tags": {} }'

# at venue-a, the stream's home
curl -X PATCH "$RELAY_A/streams/event%2Fcam1" -H 'content-type: application/json' -d '{
  "source": { "kind": "switch",
    "of": [ { "kind": "input", "input": "event/cam1.primary", "pids": { "byKind": {} } },
            { "kind": "input", "input": "event/cam1.far",     "pids": { "byKind": {} } } ] } }'

The far input arrives over MoQ, so renumber the switch by kind (see Inputs of different protocols above). If its site goes quiet, the input reads as absent at the home, as a local one whose source stopped would.

While the home has that input on air, the input’s own site serves its consumers of the stream from the input itself, rather than by way of the home: it follows the home’s choice and makes none of its own, so when the home switches away, so does it. This holds for an input the home’s switch lists directly, not one inside a nested switch or a mux.

Certificates. Media between sites is MoQ over QUIC, so it is TLS. A site whose MoQ certificate is self-signed (norsk-ctl’s own, say) hands it to the site it serves along with the media grant, over the connection its site key vouches for; that site trusts it for that transit alone. A CA-issued certificate is checked against the system’s CAs as usual.

What another site serves. A consumer at another site gets the stream laid out as at its home: the same programme, PIDs and audio languages. Data tracks (SCTE-35, subtitles) are not carried between sites yet.

Latency between sites. Each site sets how long it waits for media from other sites (TRANSIT_LATENCY_MS, the relay’s relay.transit_latency_ms; 100 ms by default): media that comes later is dropped. A lossy or distant link needs more. Sites shows, for each stream served here from elsewhere, the budget, how many objects came too late, and the latest one came (hover): raise the budget above that if objects are being dropped.

Limits.

• One site per host. Two sites on one host reach each other through the host’s own address, and on a docker bridge that UDP may not come back: transit then never connects.
• Every site’s containers must resolve the other sites’ names (their control_url and media_host). A docker daemon set to public DNS servers will not know private names; use addresses, or a resolver that does.
• A stream’s sites (“Served at”) and keepWarm (“Kept warm at”) are set on its page with its other settings; an output’s sites (“Dialled at”) where its other settings are.
• Events shows every site’s events in a cluster, each with its site, and can be filtered by site; a site that does not answer is named.
• Moving an output to a stream homed at another site hands the output to that site: it owns it from then on, and consumers stay connected everywhere.
• If the registrar is down, membership is frozen (streaming carries on). Take over as registrar on Sites (POST /cluster/registrar/takeover) at a site that cannot reach it makes that site the registrar; the old one follows it when it is back.