> For the complete documentation index, see [llms.txt](https://wiki.crosswatch.app/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://wiki.crosswatch.app/crosswatch/advanced-and-api/rate-limiting-and-retries.md).
# Rate limiting and retries
CrossWatch uses several layers of request control.
That reduces API errors and avoids unnecessary load.
{% hint style="info" %}
Only tune provider limits when logs show repeated 429s, 5xx errors, or timeouts.
{% endhint %}
{% tabs %}
{% tab title="Standard users" %}
Most installs should keep the defaults.
Only tune rate limits when a provider returns repeated `429`, temporary `5xx`, or timeout errors.
{% hint style="success" %}
Start by reducing request speed.
Change chunk sizes and delays only if needed.
{% endhint %}
### What CrossWatch does by default
CrossWatch can apply:
* client-side request pacing
* separate read and write limits
* automatic retries
* exponential backoff
* `Retry-After` handling
* request batching and write delays
Supported providers usually use this shape:
```json
"rate_limit": {
"get_per_sec": 10,
"post_per_sec": 1
}
```
* `get_per_sec` controls reads.
* `post_per_sec` controls writes.
* Any non-`GET` method uses the write bucket.
`1` means about one request per second.
`0.5` means about one request every two seconds.
`3.33` means about one request every 300 milliseconds.
Setting a bucket to `0` disables local pacing for that bucket.
It does not disable requests.
### Quick links
### Default provider pacing
* **Trakt** — `3.33` reads/sec, `1` write/sec, chunks of `100`
* **SIMKL** — `10` reads/sec, `1` write/sec, chunks of `500`
* **Nuvio** — `100` reads/sec, `100` writes/sec
* **MDBList** — `10` reads/sec, `1` write/sec, chunks of `500`, write delays of `600ms`
* **PublicMetaDB** — `20` reads/sec, `3` writes/sec, plus hourly ratings quotas
AniList and TMDb use retry handling, but do not expose configurable per-second pacing.
Plex, Emby, and Jellyfin use feature-specific pacing instead of provider-wide requests-per-second settings.
### When to change settings
Change settings when you see:
* HTTP `429`
* temporary `500`, `502`, `503`, or `504`
* repeated provider timeouts
Most of the time, increasing `max_retries` is not the first fix.
Reducing pressure works better.
### Safe tuning order
When writes hit `429`:
1. Reduce `post_per_sec`.
2. Reduce batch or chunk size.
3. Increase the write delay.
When reads hit `429`:
1. Reduce `get_per_sec`.
2. Increase the schedule interval.
3. Avoid overlapping jobs on one account.
When you see repeated `5xx` or timeouts:
1. Reduce batch or chunk size.
2. Add or increase write delays.
3. Reduce request rates.
### Conservative examples
Trakt:
```json
"trakt": {
"timeout": 15,
"max_retries": 5,
"rate_limit": {
"get_per_sec": 2,
"post_per_sec": 0.5
},
"watchlist_batch_size": 50,
"ratings_chunk_size": 50,
"history_chunk_size": 50
}
```
MDBList:
```json
"mdblist": {
"timeout": 15,
"max_retries": 3,
"rate_limit": {
"get_per_sec": 5,
"post_per_sec": 0.5
},
"watchlist_batch_size": 250,
"ratings_chunk_size": 250,
"history_chunk_size": 250,
"ratings_write_delay_ms": 1000,
"history_write_delay_ms": 1000,
"ratings_max_backoff_ms": 10000,
"history_max_backoff_ms": 10000
}
```
Use conservative settings when several jobs, profiles, or installs share one account.
### Related
* [Limitations](/getting-started/limitations.md)
* [Best practices](/getting-started/best-practices.md)
* [Chunking](/blueprint-architecture/orchestrator/chunking.md)
{% endtab %}
{% tab title="Power users" %}
### Request control layers
CrossWatch uses several layers of request control to reduce API errors and prevent unnecessary load on media servers and tracking providers.
Rate limiting is not limited to a single requests-per-second setting.
Depending on the provider, CrossWatch can apply:
* client-side request pacing
* separate limits for reads and writes
* automatic retries
* exponential backoff
* `Retry-After` handling
* request batching
* write delays
* orchestrator chunk pauses
* scheduler jitter
* rate-limit diagnostics
The default settings should work for most installations.
Changes are normally only required when a provider returns HTTP `429`, temporary `5xx`, or repeated timeout errors.
### Request pacing
Supported providers use a client-side limiter that spaces requests before they are sent.
Two request buckets are available:
* `get_per_sec` controls `GET` requests
* `post_per_sec` controls `POST`, `DELETE`, and other non-`GET` requests
Example:
```json
"rate_limit": {
"get_per_sec": 3.33,
"post_per_sec": 1
}
```
A value of `1` allows about one request per second.
A value of `0.5` allows about one request every two seconds.
A value of `3.33` allows about one request every 300 milliseconds.
Setting a value to `0` disables client-side pacing for that request bucket.
It does not disable the requests themselves.
### Providers with configurable request pacing
#### Trakt
Default settings:
```json
"trakt": {
"timeout": 10,
"max_retries": 5,
"rate_limit": {
"get_per_sec": 3.33,
"post_per_sec": 1
},
"watchlist_batch_size": 100,
"ratings_chunk_size": 100,
"history_chunk_size": 100
}
```
Trakt uses separate read and write pacing.
The default write limit allows one write request per second.
Watchlist, ratings, and history changes are also split into chunks of `100` items.
#### SIMKL
Default settings:
```json
"simkl": {
"rate_limit": {
"get_per_sec": 10,
"post_per_sec": 1
},
"watchlist_batch_size": 500,
"ratings_chunk_size": 500,
"history_chunk_size": 500
}
```
SIMKL permits faster reads than writes.
Write operations are limited to one request per second by default.
#### Nuvio
Default settings:
```json
"nuvio": {
"rate_limit": {
"get_per_sec": 100,
"post_per_sec": 100
}
}
```
Nuvio allows up to `100` read and write requests per second.
#### MDBList
Default settings:
```json
"mdblist": {
"timeout": 10,
"max_retries": 3,
"rate_limit": {
"get_per_sec": 10,
"post_per_sec": 1
},
"watchlist_batch_size": 500,
"ratings_chunk_size": 500,
"history_chunk_size": 500,
"ratings_write_delay_ms": 600,
"history_write_delay_ms": 600,
"ratings_max_backoff_ms": 8000,
"history_max_backoff_ms": 8000
}
```
MDBList combines provider pacing with feature-specific write delays and backoff limits.
The default `600` millisecond write delay adds extra spacing between ratings and history writes.
#### PublicMetaDB
Default settings:
```json
"publicmetadb": {
"timeout": 15,
"max_retries": 3,
"rate_limit": {
"get_per_sec": 20,
"post_per_sec": 3
},
"ratings_submit_per_hour": 200,
"ratings_update_per_hour": 100
}
```
PublicMetaDB limits are capped by CrossWatch at:
* `20` `GET` requests per second
* `3` write requests per second
* `200` new rating submissions per hour
* `100` rating updates per hour
Values above these limits are reduced to the supported maximum.
### Providers using automatic retry handling
AniList and TMDb do not currently expose configurable requests-per-second controls.
They still use the shared request handler, which provides:
* automatic retries
* exponential backoff
* HTTP `429` handling
* temporary `5xx` handling
* connection and timeout retry handling
* rate-limit response header diagnostics
Plex, Emby, and Jellyfin use provider-specific and feature-specific pacing instead of a provider-level requests-per-second limiter.
### Automatic retries
CrossWatch automatically retries requests for these HTTP responses:
* `429`
* `500`
* `502`
* `503`
* `504`
Connection errors and request exceptions can also be retried.
The default retry delay starts at about `500` milliseconds.
It increases exponentially after each failed attempt.
Typical retry delays are:
* first retry — `0.5` seconds
* second retry — `1` second
* third retry — `2` seconds
* fourth retry — `4` seconds
The actual number of attempts depends on the provider `max_retries` setting.
Increasing `max_retries` does not prevent rate limiting.
It only controls how often CrossWatch retries a failed request.
Reducing request speed or batch size is usually more effective.
### Retry-After handling
When a provider returns HTTP `429` with a numeric `Retry-After` header, CrossWatch waits for at least that duration before retrying.
CrossWatch also reads common provider rate-limit headers:
* `X-RateLimit-Limit`
* `X-RateLimit-Remaining`
* `X-RateLimit-Reset`
* `RateLimit-Limit`
* `RateLimit-Remaining`
* `RateLimit-Reset`
These headers are used for diagnostics.
CrossWatch does not automatically change configured requests-per-second values based on remaining request count.
Only `Retry-After` directly changes the retry wait time.
### Batch and chunk sizes
A batch or chunk size controls how many media items are included in one operation.
It does not directly control requests per second.
Examples:
```json
"watchlist_batch_size": 100,
"ratings_chunk_size": 100,
"history_chunk_size": 100
```
Smaller batches can reduce processing time and lower the chance of provider timeouts.
Larger batches can finish a sync faster.
They can also produce larger payloads and longer provider processing times.
### Write delays
Write delays add an extra pause between provider write operations.
They are configured in milliseconds.
Example:
```json
"ratings_write_delay_ms": 1000
```
A value of `1000` adds a one-second pause between supported ratings writes.
Available write delay settings include:
```json
"plex": {
"watchlist_write_delay_ms": 100
}
```
```json
"jellyfin": {
"watchlist": {
"watchlist_write_delay_ms": 100
},
"history": {
"history_write_delay_ms": 100
}
}
```
```json
"emby": {
"watchlist": {
"watchlist_write_delay_ms": 100
},
"history": {
"history_write_delay_ms": 100
}
}
```
```json
"mdblist": {
"ratings_write_delay_ms": 600,
"history_write_delay_ms": 600
}
```
Write delays help when a provider accepts the first requests, then starts returning `429`, `502`, `503`, or timeout errors during larger runs.
### Orchestrator chunk pacing
CrossWatch can pause between groups of planned synchronization changes.
Default settings:
```json
"runtime": {
"apply_chunk_size": 100,
"apply_chunk_pause_ms": 50,
"apply_chunk_size_by_provider": {
"SIMKL": 500,
"MDBLIST": 500,
"PUBLICMETADB": 500
}
}
```
`apply_chunk_size` controls how many planned changes are processed before a pause is applied.
`apply_chunk_pause_ms` controls the pause between those groups.
Provider-specific values in `apply_chunk_size_by_provider` override the general chunk size.
This pacing is separate from the provider requests-per-second limiter.
### Scheduler jitter
Scheduled synchronizations can be distributed by adding a random delay to the calculated start time.
Example:
```json
"scheduling": {
"jitter_seconds": 300
}
```
A value of `300` adds a random delay between zero and five minutes.
Scheduler jitter helps when several jobs, profiles, or installs would otherwise start at the same time.
Avoid starting several large synchronization jobs against the same provider account at the exact same time.
### Important limitation
Request pacing is maintained by each active provider client session.
It is not a shared distributed quota across every CrossWatch process, container, profile, or component.
For example, two CrossWatch containers configured for one Trakt account can each send requests at their configured limit.
Their combined traffic can exceed the provider limit.
The same can happen when several provider profiles or synchronization jobs access the same remote account at once.
Configure lower values when multiple active sessions share one provider account.
### Diagnosing rate-limit problems
CrossWatch writes structured events for pacing and retry activity.
Relevant events include:
* `client_throttle_activity`
* `rate_limit_retry`
* `http_retry`
* `http_exception_retry`
* `http_exception_giveup`
* `http_request_failed`
A `rate_limit_retry` event can include:
* provider
* API feature
* HTTP method
* HTTP status
* current attempt
* retry budget
* wait duration
* `Retry-After` duration
* provider request limit
* remaining requests
* reset value
Client pacing messages are summarized to avoid a log entry for every short sleep.
You can adjust the logging frequency with environment variables:
```bash
CW_RATE_LIMIT_LOG_EVERY_S=60
CW_RATE_LIMIT_LOG_MIN_SLEEP_S=0.5
```
You can enable API request hit events with:
```bash
CW_API_HITS=1
```
API hit logging can generate a large amount of data.
Enable it only while investigating a problem.
### Recommended tuning order
When a provider returns HTTP `429` during write operations:
1. Reduce `post_per_sec`.
2. Reduce the provider batch or chunk size.
3. Increase the relevant write delay.
4. Add scheduler jitter.
5. Prevent simultaneous jobs from using the same provider account.
When a provider returns HTTP `429` during indexing or reading:
1. Reduce `get_per_sec`.
2. Increase the interval between scheduled synchronizations.
3. Reduce page sizes where supported.
4. Avoid running several pairs against the same provider profile at once.
When a provider returns repeated `5xx` or timeouts:
1. Reduce batch or chunk sizes.
2. Add or increase write delays.
3. Reduce request rates.
4. Check the provider service status.
5. Only increase `max_retries` after pacing has been adjusted.
### Conservative Trakt example
```json
"trakt": {
"timeout": 15,
"max_retries": 5,
"rate_limit": {
"get_per_sec": 2,
"post_per_sec": 0.5
},
"watchlist_batch_size": 50,
"ratings_chunk_size": 50,
"history_chunk_size": 50
}
```
This allows about two read requests per second and one write request every two seconds.
### Conservative MDBList example
```json
"mdblist": {
"timeout": 15,
"max_retries": 3,
"rate_limit": {
"get_per_sec": 5,
"post_per_sec": 0.5
},
"watchlist_batch_size": 250,
"ratings_chunk_size": 250,
"history_chunk_size": 250,
"ratings_write_delay_ms": 1000,
"history_write_delay_ms": 1000,
"ratings_max_backoff_ms": 10000,
"history_max_backoff_ms": 10000
}
```
Use conservative settings when an account is shared across several profiles, jobs, or CrossWatch installations.
{% endtab %}
{% endtabs %}
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://wiki.crosswatch.app/crosswatch/advanced-and-api/rate-limiting-and-retries.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.