Load Throughput & Retries

This page explains how fast GAINS Connect takes in and loads your data, how to size batches, what a 429 means, and how to pace retries. Read it before you schedule large or frequent loads.

How long a load takes

A load goes through two stages, and only the first one happens while you wait for the response:

Accepting. GAINS Connect checks the records, creates the task and answers 202 or 207.
Loading. GAINS loads the batch at its next scheduled load for that kind of data.

The schedule is set in GAINS, separately for each kind of data. While a task is Queued, its StatusMessage gives the time of that load, for example Expected processing time is 2026-05-22 02:00:00.; a Received task waits until GAINS schedules one (see Task statuses). A task does not move before then, so the expected processing time is the earliest moment worth checking again. Ask your GAINS representative to change how often a kind of data is loaded.

Batch size

A request holds up to 1000 top-level records; see Sending a batch. GAINS Connect does not limit how many requests you send; it refuses a load only when its backlog is full.

Send full batches: 10 requests of 1000 records cost less than 100 requests of 100.

The ingest backlog and 429

When GAINS Connect is still taking in earlier batches, a load answers 429:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: application/json

{
  "Reason": "The service is busy ingesting previously submitted data. Retry this batch shortly."
}

After a 429:

Nothing from the request was saved and no task was created.
An Idempotency-Key on the request was released, so resend with the same key.
Wait at least the number of seconds in Retry-After, then send the same batch again.

Only data loads, including POST /custom/{ApiRoute}, can answer 429; deletes, POST /supply-transactions/link-order and the task endpoints never do.

Sending more requests in parallel does not help: slow down, or send fewer, fuller batches.

Retrying safely

Send an Idempotency-Key with every load. Resend with the same key only after a 429 or when no response arrived. After any other error response nothing was saved: correct the request if needed and resend under a new key. The full rule, and what to do when a resend answers that the key is a duplicate, are in Which key to resend with. POST /custom/{ApiRoute} ignores the key; see Custom Schema.

When a task fails

A task in Failed stopped on an unexpected error. Send the batch again under a new key. If the new task fails too, contact your GAINS representative with both TaskId values.

A task in RejectedFailedProcessing was loaded, but GAINS refused every record. Resending it unchanged gives the same result: read Batch.RejectedRecords and correct the records first. See Rejected Records.

Recommended client pattern

Send loads for one kind of data one after another. Wait for each response before sending the next batch. It keeps the backlog short, and you never have two attempts of the same batch in flight.
Respect the load order. Load what other records refer to first, and wait for those tasks to complete; see Initial Load & Load Order.
Back off on errors. Honour Retry-After on 429. When you get no response, or Decide whether to retry says to retry after a delay, wait before retrying and double the wait on each attempt, for example 30 seconds, 1, 2, 4 and 8 minutes, then stop and alert someone. Retry a 500 only once. Never retry 400, 403 or 422 unchanged.
Follow tasks without polling hard. Check a Queued task after its expected processing time, or receive the TaskStatusUpdate webhook; see Following a task.
Store the TaskId with your own record of the batch.

One batch, from send to loaded

Send the batch with a new Idempotency-Key.
If it is refused or gets no answer, use Decide whether to retry to choose whether to resend, and Which key to resend with to choose the key.
On 202 or 207, store the TaskId. After a 207, send the corrected rejected records as a new batch, under a new key.
Follow the task until it is Completed, RejectedFailedProcessing, Cancelled or Failed; see Following a task.

Loading Data: the 1000-record limit and the submit responses.
Initial Load & Load Order: which data to load first.
Idempotency: which key to resend with.
Errors & Retries: which statuses to retry.
Integration Tasks: following a task.