Health
Provider health checks and how they gate pairs, features, and safe write behavior.
Health checks decide what CrossWatch is allowed to do.
If health is bad, CrossWatch skips risky work.
What you’ll see
auth_failed: the pair is skipped.
down: writes are skipped to that provider.
feature unsupported: a feature becomes a no-op.
If your run is skipping everything
Re-auth the provider.
Check server URL and tokens (media servers).
Retry the run after fixing auth.
Related:
Safety model: Guardrails
Provider expectations: Provider contract
This doc describes how provider health is gathered and how it gates orchestrator behavior.
Core code: cw_platform/orchestrator/_pairs.py (_collect_health_for_run)
Provider contract: InventoryOps.health(cfg, emit=...)
Why health exists
Health is the orchestrator’s “don’t do something dumb” preflight.
It prevents:
running sync with invalid auth (guaranteed failure + noise)
treating a down provider as “everything removed”
making delete propagation decisions on missing data
Health is best-effort:
if a provider doesn’t implement
health(), the orchestrator assumes it’s OK (less safe).
When health is collected
At the start of _run_pairs(ctx):
The orchestrator determines which providers are used by enabled pairs.
For each provider, it calls
ops.health(cfg, emit=ctx.emit)(orops.health(cfg)fallback).It stores the result in
ctx.health_map.
This happens once per run, not per feature.
Scope during health
During health, the orchestrator sets a temporary scope:
CW_PAIR_MODE=healthCW_PAIR_SCOPE=healthCW_PAIR_FEATURE=health
This ensures any provider-level state files created during health don’t collide with real pair scopes.
Expected health response shape
There is no strict schema, but the orchestrator expects these fields if present:
ok(bool)status(string):okdownauth_faileddegraded(optional)
features(dict[str,bool]) (optional)api(dict) (optional)latency_ms(int) (optional)
Example:
If fields are missing, the orchestrator falls back conservatively.
How health gates runs
Auth failed
If either side of a pair has:
status == "auth_failed"
Then:
the pair is skipped entirely
features do not run
emits:
pair:skip reason=auth_failed
Down
If a provider is:
status == "down"
Then behavior depends on mode:
One-way
If source is down:
feature run is skipped (no meaningful plan)
If destination is down:
plan is computed, but writes are skipped and items are recorded unresolved
Two-way
Writes to down side are skipped (unresolved)
Observed deletions are disabled for both sides
because missing snapshots can look like deletes
Feature support
If health contains features[feature] == False:
feature run becomes a no-op and emits
feature:unsupported
This is in addition to provider-declared ops.features().
Emitting health events
After calling health, the orchestrator emits a structured event:
healthwith provider name + status + basic details
This lets the UI show:
“SIMKL auth failed”
“TRAKT rate limited”
“PLEX ok”
Providers can also emit their own sub-events during health, especially:
api:hitsamples (requests made during health)
Practical tips for provider authors
A good health implementation should:
verify credentials (fast, one request if possible)
detect rate-limits separately from auth errors
return per-feature capability flags if certain features are unavailable
emit
api:hitsamples for every network request
Also: don’t do heavy snapshot calls in health. That belongs in build_index.
Troubleshooting patterns
“Everything skipped with auth_failed”
provider token invalid or expired
connected app revoked
wrong base URL (for Jellyfin/Emby)
wrong protocol switch (HTTP/HTTPS) causing cookie invalidation in UI, etc.
“Two-way suddenly wants to delete everything”
health didn’t mark provider as down, but snapshot was empty
drop guard should catch it if checkpoints exist
if checkpoints missing, implement provider
activities()and return an updated timestamp
“Feature says unsupported but provider supports it”
health
featuresmap is returning false accidentallyor
ops.features()says false
Related pages
Provider health contract: Provider contract
Drop guard and delete suppression: Guardrails
Observed deletions gating: Two-way sync
Health event format: Eventing
Last updated