Chunking

How write batches are split and throttled to reduce rate limits and failure blast radius.

Chunking splits big writes into smaller batches.

It helps when a provider is strict or rate-limited.

When to enable chunking

Enable it if you see:

rate limit errors
timeouts on large batches
“one bad item breaks the whole batch”

Safe starting point

Start with apply_chunk_size: 25
Add apply_chunk_pause_ms: 200 if you still hit limits

Runtime config keys: Runtime

This doc describes how the orchestrator decides batch sizes for writes, and how it slows down to avoid rate-limit pain.

Code: cw_platform/orchestrator/_chunking.py + orchestrator/_applier.py

Why chunking exists

Providers vary wildly:

Some accept big batches (Plex-ish).
Some want small batches (Trakt-ish).
Some accept batches but rate-limit aggressively (Simkl-ish depending on endpoint).

Chunking gives you:

faster progress visibility
less blast radius per API call
fewer “one bad item ruins 500” failures

Where chunking is applied

Chunking happens in the applier:

_applier._apply_chunked(...)

The pipeline passes:

chunk_size
chunk_pause_ms

If chunking is off, applier calls provider once per add/remove.

How chunk size is chosen

The orchestrator computes chunk size via:

effective_chunk_size(ctx, provider_name) in _chunking.py

Inputs:

ctx.apply_chunk_size (base size)
ctx.apply_chunk_size_by_provider (overrides)
provider name normalization (uppercased)

Resolution rule:

If per-provider override exists and > 0 → use it
Else if base size > 0 → use base size
Else → 0 (no chunking)

Config keys

In config.json:

{
  "runtime": {
    "apply_chunk_size": 50,
    "apply_chunk_pause_ms": 200,
    "apply_chunk_size_by_provider": {
      "TRAKT": 10,
      "SIMKL": 25
    }
  }
}

Accepted alias keys (same behavior):

apply_chunk_sizes_by_provider
apply_chunk_sizes

Provider keys are uppercased internally.

Chunk pause (throttle between chunks)

If runtime.apply_chunk_pause_ms > 0:

applier sleeps after each chunk

This is a simple “be polite” delay. It’s not adaptive and doesn’t look at headers.

Rate-limit awareness (telemetry warnings)

Chunking itself doesn’t read rate-limit headers. Instead, the orchestrator can emit warnings if rate remaining is low.

At end of run:

ctx.emit_rate_warnings()
uses thresholds from telemetry.warn_rate_remaining

Default thresholds are provider-specific and conservative.

This is a warning system only; it doesn’t auto-throttle.

If you want adaptive rate limiting, you’d implement it in providers (based on 429 / Retry-After, etc.).

Provider side: recommended behavior under chunking

Providers should treat each chunk as independent and return:

confirmed_keys when possible
unresolved as a list of items with IDs for failures

If a provider throws an exception for a chunk:

applier retries that chunk up to 3 times with exponential backoff
if it still fails, it treats that chunk as failed and returns ok:false at aggregate level

This is why chunking can help:

one failing chunk doesn’t nuke the entire run.

Common chunking setups

Conservative (good for Trakt/Simkl)

base 25
Trakt override 10
pause 150–300ms

Aggressive (fast, but riskier)

base 100
no pause

“Debug mode”

base 1
pause 0

Helps isolate a single bad item and see which exact key fails.

Gotchas

Chunking increases total calls → can increase rate-limit pressure if set too low.
If provider doesn’t return confirmed_keys, it can be harder to map successes/failures precisely.
If you chunk too large, one provider error can still fail a lot of items.

How chunking aggregates results: Applier
Where chunk config is parsed: Runtime
Progress events: Eventing

PreviousHealth NextCaching layers

Last updated 14 hours ago

Good afternoon

hashtagWhen to enable chunking

hashtagSafe starting point

hashtagWhy chunking exists

hashtagWhere chunking is applied

hashtagHow chunk size is chosen

hashtagConfig keys

hashtagChunk pause (throttle between chunks)

hashtagRate-limit awareness (telemetry warnings)

hashtagProvider side: recommended behavior under chunking

hashtagCommon chunking setups

hashtagConservative (good for Trakt/Simkl)

hashtagAggressive (fast, but riskier)

hashtag“Debug mode”

hashtagGotchas

hashtagRelated pages