Inbound Endpoints & Integration Tasks

This API has various inbound integration endpoints covering master and transactional data sets.

Overview

Dat ais expected to be provided in hierarchical JSON structure allowing up to 1000 records per API call. This limit of 1000 refers to header level records (which means there could be 1000 headers with lines within each). For example, the structure of supply transactions has a hierarchy of header level data and nested line-item data:

The data goes through initial format and structure validation and if the payload is accepted, we respond with a task ID - this is important as it represents the task of importing the data in to GAINS as it flows through the relevant loader and then into the GAINS database. See below for more on Tasks.

By way of example:

  • Client pushes 500 records via API at 8am
  • Client pushed another 500 records via API at 8.20am
  • Both of the above will have a unique task ID assigned and they will be in queued status
  • Later in the data, based on GAINS configuration, the data is processed by the loader and ingested in to GAINS

Validation Checks at the API Level

Before the data is fed into it's respective loader, there will be some initial checks at the API level. These include:

  • Payload size checks - number of objects - to be within the limit
  • Structure checks - required fields and keys need to be present
  • Security checks / sanitary checks
  • Value range checks - certain values to fall within specific range/limit

Task Stages

Tasks can be in the following stages:

  • received: the data was submitted by the client and received by GAINS API
  • queued: the data has been queued for processing and has an expected processing time
  • processing: the data is being processed
  • completed: the data has come out of the loaders and we can indicate whether the dataset was accepted, rejected, etc. Example, 1000 lines total, 23 rejected with reason, 977 accepted
  • rejected: when the entire payload is rejected either before, during or after processing along with a reason for the reject
  • cancelled: based on client desire to cancel an unprocessed task

At the moment, all task monitoring and management is done via API - see Task endpoints.

Updated 20 days ago