> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stateset.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Sync Server troubleshooting

> Why an order is stuck, why a circuit breaker is open, and which failures you should not retry.

Most Sync Server incidents are one of four shapes. Work them in this order —
each rules out the layer below it.

## 1. Is the order actually stuck?

An order sitting in-flight is not necessarily failing. If its connector uses the
default `no_retry_after_ambiguous`
([dedupPolicy](/stateset-sync-server-connectors#deduppolicy--read-this-before-shipping-a-push-flow)),
an ambiguous submission **parks** for review rather than retrying — by design.

```bash theme={null}
# what is in flight, and for how long
curl "$SYNC_API/v1/orders?status=in_flight" \
  --header "Authorization: Bearer $SYNC_API_KEY"

# everything that happened to one order
curl "$SYNC_API/v1/lifecycle/$ORDER_REF" \
  --header "Authorization: Bearer $SYNC_API_KEY"
```

<Warning>
  Do not resubmit a parked order until you have checked the partner for it. The
  policy parked it precisely because the platform could not tell whether the
  remote accepted it — resubmitting on the assumption that it did not is how a
  parked order becomes a duplicate shipment.
</Warning>

## 2. Is a circuit breaker open?

The breaker opens after **10 consecutive failures** to an upstream and stays
open for 30 seconds, then admits a test request. A request failing with
"circuit breaker is open" is the platform protecting the upstream, not a bug in
your call.

```bash theme={null}
curl -s http://localhost:8080/metrics | grep circuit_breaker
```

<Note>
  An open breaker means the upstream failed ten times in a row — so the fix is
  almost never at your end. Check the partner's status before changing
  anything on yours. Breakers reset on restart, but restarting to clear one
  hides the outage that opened it.
</Note>

## 3. Is it the upstream's limits?

| Symptom                                   | Cause                             | What to do                                                  |
| ----------------------------------------- | --------------------------------- | ----------------------------------------------------------- |
| Bursts succeed, sustained load 429s       | Shopify's leaky-bucket rate limit | Reduce concurrency; the platform already backs off          |
| Intermittent failures under parallel load | NetSuite concurrency limits       | Lower parallelism for that tenant                           |
| Slow then timing out                      | 3PL connection timeouts           | Raise the timeout for that connector before raising retries |

## 4. Is it local — the database or the process?

| Symptom                     | Usually                                                                    |
| --------------------------- | -------------------------------------------------------------------------- |
| "connection pool exhausted" | Long-running queries holding connections, not pool size                    |
| Memory growing steadily     | An unbounded ingest without checkpointing                                  |
| Server will not start       | A migration that failed halfway, or a `GLIBC` mismatch in the built binary |

## Errors you should not retry

The [status-code mapping](/stateset-sync-server-grpc-flow#status-codes-and-what-they-mean)
is the retry policy. Two are worth repeating here:

* **`FAILED_PRECONDITION`** — the tenant has no configuration for that
  integration. Retrying forever will never fix it, and it almost always means
  onboarding was left incomplete. Alert on it rather than burying it in a queue.
* **`DEADLINE_EXCEEDED`** — does not tell you whether downstream systems
  accepted the order. Retry per target with the single-target RPCs, never by
  resubmitting the whole call.

## Next steps

<CardGroup cols={2}>
  <Card title="Connectors" icon="plug" href="/stateset-sync-server-connectors">
    dedupPolicy, and why a parked order is the safe outcome.
  </Card>

  <Card title="gRPC flow" icon="diagram-project" href="/stateset-sync-server-grpc-flow">
    The full status-code mapping and per-target retry.
  </Card>

  <Card title="API contract" icon="file-contract" href="/stateset-sync-server-api-contract">
    Envelopes and correlation ids — keep the requestId.
  </Card>

  <Card title="API basics" icon="key" href="/stateset-sync-server-api-basics">
    Authentication and tenancy.
  </Card>
</CardGroup>
