# Automation API (/docs/integrations/api/overview)
This guide walks the whole loop with commands you can paste. The exact shape of every call is in
the [API reference](./reference/overview), generated from the contract the API publishes.
The contract it describes - field names, scope names, error codes - is English, so this guide and
the reference are not translated.
## What the Automation API is [#what-the-automation-api-is]
The machine door of the platform, at `https://api.ovidens.com`. An unattended system authenticates
with a credential of its own, pushes data into a dataset, runs a pipeline and pulls the result.
Its surface is deliberately narrow: read the workspace, write dataset rows, run a pipeline, export a
result. It cannot create or delete apps, change a pipeline, edit a Visualisation, invite people or
change workspace settings.
* **Use the Automation API** when a system acts on its own schedule with its own credential - a
nightly ERP export, a warehouse loader, a scheduled reconciliation.
* **Use MCP** when a PERSON's AI assistant should act as that person: a different door with a
different authorization model, in [Connect your own AI](../mcp). The choice between the two is
laid out in [Integrations](../../integrations).
Everything a connected system WRITES appears in the workspace's audit trail, attributed to the
integration by its credential's name rather than to the administrator who issued it. Reads are not
audited; an export leaves an export record instead.
## Getting a credential [#getting-a-credential]
A workspace administrator issues one on the **Access** page's **API credentials** tab. Ask for a
credential named after the system you are building, its **client id** (`ovc_...`) and **client
secret** (`ovs_...`) - the secret is shown once, at creation, and nothing can show it again - and
the **access** it should hold, granted as named scopes:
| Scope | What it grants |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `workspace:read` | List apps, dataset and output nodes, and their configuration. Read import status, and the status of runs of PUBLISHED configuration. |
| `data:write` | Stage, upload and commit a CSV import into a dataset node. |
| `pipeline:run` | Run an app's PUBLISHED configuration. |
| `pipeline:run_draft` | Run unpublished configuration nobody has reviewed, and see the runs of it - a draft run is invisible to a credential without this scope, including one an editor started in the app. Not granted by default. |
| `exports:read` | Generate and download an export of a run result. |
| `indicators:write` | Reserved. Nothing checks it today. |
| `admin:publish` | Reserved. Nothing checks it today. |
A new credential starts with `workspace:read`, `data:write`, `pipeline:run` and `exports:read`; the
other three are an explicit choice. Narrowing a credential's access - or revoking it - binds on its
very next request, including on a token it is already holding.
The administrator's side is [Credentials](./credentials).
## Minting a token [#minting-a-token]
OAuth 2.1, `client_credentials`. The authorization server is `https://id.ovidens.com`; the API is a
separate resource server that only validates tokens.
```bash title="POST /oauth/token"
curl -sS -X POST https://id.ovidens.com/oauth/token \
-u "ovc_YOUR_CLIENT_ID:ovs_YOUR_CLIENT_SECRET" \
-d "grant_type=client_credentials" \
-d "resource=https://api.ovidens.com" \
-d "scope=workspace:read data:write pipeline:run exports:read"
```
* **The credential goes in HTTP Basic** (`-u`). Form fields also work; Basic is the registered
method.
* **`resource=https://api.ovidens.com` is REQUIRED.** Omitting it, or naming any other resource, is
a `400` with `error: "invalid_target"`.
* **`scope` is optional.** Omit it for the credential's whole granted set. Naming a scope the
credential does not hold fails with `400 invalid_scope` and mints nothing.
* **Cache the token for its `expires_in`** rather than minting one per request; the token endpoint
is rate limited by source address.
* **A credential that has expired or is suspended is refused here too**, with the OAuth error
`invalid_client` and, once the secret itself was right, a `code` naming which:
`credential_expired` or `credential_attestation_overdue` (see [Error codes](#error-codes)). A
wrong secret never carries a `code`.
Present it on every API call as `Authorization: Bearer `. Three endpoints need no
token, so you can read them before you hold a credential:
`https://api.ovidens.com/.well-known/oauth-protected-resource` (where to authenticate),
`https://api.ovidens.com/v1/openapi.json` (the contract) and `https://api.ovidens.com/v1/docs` (a
redirect to this guide).
## Confirming it worked [#confirming-it-worked]
```bash title="GET /v1/whoami"
curl -sS https://api.ovidens.com/v1/whoami -H "Authorization: Bearer $TOKEN"
```
```json
{
"clientId": "ovc_7f3a9c1d2e4b6a8c0d5e7f91",
"name": "ERP nightly import",
"tenantId": "3f1c9a52-4d7e-4b60-9a1f-2c8d5e7b0a34",
"workspaceName": "Contoso",
"scopes": ["workspace:read", "data:write", "pipeline:run"],
"credentialScopes": ["workspace:read", "data:write", "pipeline:run", "exports:read"],
"tokenExpiresAt": "2026-08-02T09:30:00.000Z",
"credentialExpiresAt": "2027-06-30T09:30:00.000Z",
"attestationDueAt": "2026-10-01T09:30:00.000Z"
}
```
This is the one call that needs a valid token and no scope, so it separates a bad secret from a
missing permission. `scopes` is what THIS token can do (what you asked for, intersected with what
the credential holds now); `credentialScopes` is what the administrator granted. On an unexpected
`403 insufficient_scope`: in `credentialScopes` but not in `scopes` means your token asked for too
little, so mint a new one; in neither means the administrator has to widen the credential.
The last two fields say when the CREDENTIAL stops working unless a person acts. Every credential
expires (`credentialExpiresAt`, at most a year after it was issued or last rotated), and a workspace
administrator must confirm every 90 days that it still needs its access (`attestationDueAt`); a
credential whose confirmation is more than 14 days overdue is suspended. A job that reads `whoami`
can warn its owner well before either date instead of discovering it as a refusal.
## Discovery [#discovery]
```bash title="Discovery"
curl -sS https://api.ovidens.com/v1/apps -H "Authorization: Bearer $TOKEN"
curl -sS https://api.ovidens.com/v1/apps/sales_analytics/datasets -H "Authorization: Bearer $TOKEN"
curl -sS https://api.ovidens.com/v1/apps/sales_analytics/outputs -H "Authorization: Bearer $TOKEN"
```
Every app, node and column is addressed by its **`technicalId`**, a stable snake\_case identifier
that survives a rename. There is no numeric id on this door.
**A dataset node's declared column schema IS the integration contract.** `listDatasets` reports, per
column, its technical id, its type, whether a value is required, its permitted values when it is an
enumeration, and which CSV value options apply. Your file's header must name exactly those columns.
Build the file from this response rather than from a sample, and a schema change becomes a diff you
detect instead of an import rejected at three in the morning.
**A cell of a `dropdown` or `multi_select` column may carry the option's LABEL or its ID**, and
either way what the platform stores is the id. `enumValues` lists the labels; beside it, `options`
reports the same options in the same order, each with the stable `id` and the `description` its
author wrote. A `multi_select` cell packs several of them separated by `,` or `;`. Write the label
when a person authors the file and it should read like a spreadsheet; write the `id` when a system
generates it, because a label can be renamed and an id cannot. Anything that is neither is refused
with the row named.
`listOutputs` is the mirror image: the output nodes of the app's latest successful published run,
with the columns an export will carry. An app that has never published a successful run answers
`200` with `publishedRun: null` and an empty list - a normal state to poll through, not an error.
## The import loop [#the-import-loop]
```text
createImport -> PUT the file -> commitImport -> poll getImport -> getImportErrors
```
**1. Stage the import.** The dialect is validated and fully resolved here, before a byte moves.
```bash title="POST /v1/apps/sales_analytics/datasets/customers/imports"
curl -sS -X POST \
https://api.ovidens.com/v1/apps/sales_analytics/datasets/customers/imports \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"mode":"replace","filename":"customers.csv","csv":{"delimiter":";","decimalSeparator":","}}'
```
`mode` is `replace` (the file becomes the dataset's contents) or `append`. The response carries the
`importId`, the `uploadUrl` and both expiry times.
**2. Upload the file** with a plain `PUT`. Send no `Authorization` header - the URL carries its own
signature, and a bearer token makes the upload fail.
```bash title="PUT {uploadUrl}"
curl -sS -X PUT --upload-file customers.csv -H "Content-Type: text/csv" "$UPLOAD_URL"
```
**3. Commit.** Validates the file against the declared columns and, if every row parses, applies it.
```bash title="POST /v1/apps/sales_analytics/datasets/customers/imports/imp_8c2f4a1d9e63/commit"
curl -sS -X POST \
https://api.ovidens.com/v1/apps/sales_analytics/datasets/customers/imports/imp_8c2f4a1d9e63/commit \
-H "Authorization: Bearer $TOKEN"
```
* **Validation is all or nothing.** One unparsable row applies nothing and leaves the dataset as it
was.
* **Committing the same import id twice is safe.** The first commit answers `202`; a repeat answers
`200` with the import's current record and writes nothing, so retrying a timed-out commit is
correct rather than a second write.
* **Only one import per dataset node validates at a time**; a second is `409 import_in_progress`.
**4. Poll the import** until `status` leaves `validating`:
```bash title="GET /v1/apps/sales_analytics/datasets/customers/imports/imp_8c2f4a1d9e63"
curl -sS \
https://api.ovidens.com/v1/apps/sales_analytics/datasets/customers/imports/imp_8c2f4a1d9e63 \
-H "Authorization: Bearer $TOKEN"
```
`applied` reports `rowsParsed`, `rowsApplied` and `rowsRejected`. `failed` reports `errorCode`,
`likelyCause`, a grouped `errorGroups` summary and, when the header was wrong, a `structure` block
naming the expected, detected, unknown and missing columns.
A file with no data rows - a header alone, or no bytes at all - fails with `errorCode:
"no_data_rows"` in either mode and leaves the dataset exactly as it was; a file whose every row is
invalid fails with `row_validation_failed` and leaves it as well. An import never empties a
dataset, because an empty file is far more often a failed export than an intent; check the export
that produced it before you retry.
A row longer than 2,000,000 characters fails the import with `errorCode: "row_too_long"` and
leaves the dataset as it was. The file is read row by row and one row is never held whole past that
length, so the limit is per row, not per file: shorten or split the longest values on the row the
message names, then stage the file again.
This read needs only `workspace:read`, so a monitoring job can watch an integration without being
able to feed it. If the calling token ALSO holds `data:write` and the import is still awaiting its
file, the response carries a fresh `uploadUrl`. **That is how you recover an expired upload URL** -
re-read the import, do not stage a new one.
**5. Get the full error report** when the grouped summary is not enough: the same path with
`/errors` appended answers a short-lived `downloadUrl` for a CSV of every rejected row. There is no
report for an import that applied cleanly.
Codes that matter here: `upload_missing` (nothing staged, or the staged file expired - stage a fresh
import), `file_too_large`, `import_in_progress`, `app_locked`, and the four `csv_dialect_*` codes.
## The run loop [#the-run-loop]
```bash title="POST /v1/apps/sales_analytics/runs"
curl -sS -X POST https://api.ovidens.com/v1/apps/sales_analytics/runs \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"target":"published"}'
```
`target` is required. `published` needs `pipeline:run`; `draft` runs unpublished configuration and
needs `pipeline:run_draft`, which a credential does not hold by default. **This door never falls
back between the two**: an app with no configuration of the target you asked for is refused with
`409 no_runnable_version`. The `202` echoes the target it accepted.
Only one run per app may be active. A second trigger answers `409 app_locked`, and when the blocker
is another run your credential may read, the `detail` carries `activeRunId`. **On `app_locked` WITH
an `activeRunId`, poll that run instead of starting another; without one, back off.** That is also
the recovery when a `POST` timed out but succeeded: this endpoint is not idempotent, so a blind
retry starts a second run.
An editor previewing unpublished work in the app locks it the same way, and that run is a draft one:
it blocks your trigger, but without `pipeline:run_draft` you cannot read it, so no `activeRunId` is
named and backing off is the answer. You are never denied the id of a run you could have polled.
Then poll:
```bash title="GET /v1/apps/sales_analytics/runs/4821"
curl -sS https://api.ovidens.com/v1/apps/sales_analytics/runs/4821 \
-H "Authorization: Bearer $TOKEN"
```
The run is finished when `status` is no longer `running`. Every node starts `pending`, so a poll
taken mid-run reports that for most of the array; on failure, `error` names the node and the
message. `GET /v1/apps/sales_analytics/runs` is the paged history, and every run reports
`triggeredBy` - an object naming the credential when a system started it, `null` when a person or
the workspace assistant did.
Both reads need `workspace:read`, and both show runs of published configuration. A run whose
`target` is `draft` is a preview of work nobody has reviewed: it is listed, counted in `total` and
readable by its id only for a credential that also holds `pipeline:run_draft`. Without that scope,
the run id of a draft answers `404 not_found`, the same as an id that names nothing - so a run you
started yourself is always readable, because a credential that cannot run a draft cannot have
started one.
## The export loop [#the-export-loop]
```bash title="POST /v1/apps/sales_analytics/outputs/revenue_by_region/exports"
curl -sS -X POST \
https://api.ovidens.com/v1/apps/sales_analytics/outputs/revenue_by_region/exports \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"runId":4821,"compression":"gzip"}'
```
Every field is optional and the body itself may be omitted, which means "the latest published run,
default dialect".
An **empty** payload declared as `application/json` is refused by the JSON parser as a
`400 invalid_request` before the endpoint sees it, so dropping only the `-d` fails. Send either no
body and no `Content-Type`, or an explicit `{}` with the header. This is a property of the HTTP
layer, so it applies to any call you strip the body from - if your client sets a JSON content type
for every request, send `{}`.
```bash title="POST /v1/apps/sales_analytics/outputs/revenue_by_region/exports (defaults)"
curl -sS -X POST https://api.ovidens.com/v1/apps/sales_analytics/outputs/revenue_by_region/exports \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{}'
```
**Whether you name a `runId` is the decision that matters.** With none, the file comes from the
latest successful published run, and that answer can move between two otherwise identical requests -
right for "the current numbers", wrong for a reproducible load. **Naming the `runId` you got from
`startRun` pins the file to an immutable run snapshot**, so the same load a month later produces the
same bytes. The `202` echoes the resolved `runId` and a `runPinned` flag.
Then poll and download:
```bash title="GET /v1/apps/sales_analytics/exports/exp_5b1e7d0a3c92"
curl -sS https://api.ovidens.com/v1/apps/sales_analytics/exports/exp_5b1e7d0a3c92 \
-H "Authorization: Bearer $TOKEN"
```
When `status` is `ready` the response carries a `downloadUrl`; fetch it with no `Authorization`
header. `rowLimitReached` says the file was cut off at the export size limit and holds only part of
the output; `runPinned` says whether the run was yours or resolved. An export writes nothing to the
audit trail, but it does leave an export record the administrator can see: which app, which output,
which run, how many rows, how large.
Codes that matter here: `no_published_run` (nothing published successfully yet - run the pipeline),
`run_data_unavailable` (the run exists but its outputs have been removed by retention - run it
again), and `503 compute_unavailable`.
## The CSV dialect [#the-csv-dialect]
One dialect object describes both directions: what an import PARSES with, an export WRITES with. It
is validated at request time, so a bad option is a `400` with nothing uploaded or generated.
**Structural options are file-level only.**
| Option | Default | Notes |
| ----------- | ------- | -------------------------------------------------------------------------------------------------------------- |
| `delimiter` | `,` | One character. Must differ from every other single-character option. |
| `quote` | `"` | One character. |
| `escape` | `"` | One character, and in v1 it must EQUAL `quote`. A distinct escape character is refused rather than mis-parsed. |
| `encoding` | `utf-8` | `utf-8` or `windows-1252` on import. Export accepts `utf-8` only. |
| `hasHeader` | `true` | Only `true`. Every other option is validated against the declared column names. |
**Value options are file-level defaults with an optional per-column override**, under
`csv.columns.`. An override may name only value options, and only ones relevant
to that column's type: an irrelevant one is `csv_dialect_irrelevant_option`, an undeclared column is
`csv_dialect_unknown_column`.
| Option | Default | Applies to |
| -------------------- | -------------------- | --------------------------------------------------- |
| `decimalSeparator` | `.` | `integer`, `decimal`, `percent`, `currency` columns |
| `thousandsSeparator` | empty | the same numeric columns |
| `dateFormat` | `%Y-%m-%d` | `date` columns |
| `timestampFormat` | `%Y-%m-%dT%H:%M:%SZ` | `timestamp` columns |
**The supported date and timestamp tokens are `%Y`, `%m`, `%d`, `%H`, `%M` and `%S`.** Anything else
is `csv_dialect_unsupported` at request time. Month names are a later addition.
Two collisions are refused because they would produce plausible-looking WRONG data: a
`thousandsSeparator` equal to the `delimiter` (one value becomes two fields), and one equal to the
`decimalSeparator` (`1.5` would become `15`).
Beyond the separators, a value in a numeric column may carry a leading `+` or `-`, a parenthesised
negative (`(1234)` is `-1234`), surrounding whitespace, a `$`, `€`, `£`, `¥` or `%` symbol, and
scientific notation (`2E10`). A TRAILING minus sign is not recognised.
On the way OUT, the writer refuses what it cannot produce: `encoding` other than `utf-8`, any
decimal or thousands separator outside the set it emits, and a date or timestamp pattern STARTING
with a spreadsheet formula trigger (`=`, `+`, `-`, `@`, tab or carriage return) - a pattern is a
per-column constant, so it would land a live formula in every cell.
## Error codes [#error-codes]
Every failure answers the same body, whatever the endpoint:
```json
{
"type": "https://ovidens.com/problems/insufficient_scope",
"title": "The access token does not carry a scope this operation requires",
"status": 403,
"code": "insufficient_scope",
"requestId": "944debf6-805a-400c-bde6-4a1f437ee42d",
"detail": { "requiredScope": "data:write" }
}
```
**Branch on `code`, never on `title`.** The code is the contract; the title is the English sentence
beside it and may be corrected. `type` is a stable identifier, not a URL to fetch. `requestId` is
what to quote to support - it is also on every response as the `x-request-id` header.
**`detail` is keyed by the code, and every key in it is optional.** Read `code` first: the keys are
not shared between codes, and a given code may answer with no `detail` at all.
| Code | `detail` keys, when present |
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `insufficient_scope` | `requiredScope` |
| `app_locked` | `reason`, plus `activeRunId` when the blocker is a run this credential may read |
| `no_runnable_version` | `target` |
| `quota_exceeded` | `resource`, for the three platform-metered ceilings |
| `credential_expired` | `expiredAt` |
| `credential_attestation_overdue` | `attestationDueAt`, `suspendedAt` |
| `compute_unavailable` | `resource`, unless the pause names no ceiling |
| `file_too_large` | `bytes`, `max` |
| `rate_limited` | `retryAfterSeconds` |
| `csv_dialect_conflict` | `fields`, `character`, plus `column` for a per-column override |
| `csv_dialect_unsupported` | `field`; then `token` and `supportedTokens` for an unrecognised date pattern, or `reason` plus `quote` or `supported` for a restriction; plus `column` for a per-column override |
| `csv_dialect_unknown_column` | `column`, `declaredColumns` |
| `csv_dialect_irrelevant_option` | `column`, `option`, `columnType`, `relevantOptions` |
Every other code answers with no `detail`.
The contract declares the WHOLE code vocabulary on every failure response rather than a per-endpoint
subset, so this table, not the contract, is where you learn what a code means:
| Code | Status | What it means | What to do |
| -------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `invalid_request` | 400 | The body, a parameter or the content type is not valid. | Fix the request. Retrying it unchanged cannot work. |
| `invalid_token` | 401 | The token is missing, malformed, expired, or its credential no longer exists. | Mint a fresh token. If that also fails, the credential was revoked. |
| `invalid_client` | 401 | The client id or secret was not accepted at the token endpoint. | Check the credential. A rotated secret may have been retired. |
| `credential_expired` | 401 | The credential itself has expired. `detail.expiredAt` says when. | Stop and ask a workspace administrator to rotate the credential, which sets a new expiry and issues a new secret. No token will be accepted until then. |
| `credential_attestation_overdue` | 401 | The credential is suspended because its periodic confirmation is more than 14 days overdue. `detail` gives the due date and when the suspension began. | Stop and ask a workspace administrator to attest the credential on the Access page. It works again the moment they do; the secret does not change. |
| `invalid_target` | 400 | The `resource` indicator is missing or names something this grant may not request. | Send `resource=https://api.ovidens.com`. |
| `insufficient_scope` | 403 | The token's effective scope set is too narrow for this operation. | Call `whoami` and compare `scopes` with `credentialScopes`. |
| `not_found` | 404 | No such resource. Covers unknown, archived, wrong-type and other workspaces' identifiers alike. | Re-run discovery; do not guess identifiers. |
| `conflict` | 409 | A state conflict with no more specific code. | Re-read the resource and decide from its current state. |
| `quota_exceeded` | 422 | A workspace limit was reached. `detail.resource` names it for a metered ceiling; the open-import and open-export slot limits name none. | Tell the workspace administrator which limit. Retrying will not clear it. |
| `rate_limited` | 429 | Too many requests for this credential. | Wait `Retry-After` seconds. Do not tighten the loop. |
| `internal_error` | 500 | An unexpected fault. | Retry once with backoff, then escalate with the `requestId`. |
| `csv_dialect_conflict` | 400 | Two dialect options claim the same character. `detail.fields` names both. | Change one of them. |
| `csv_dialect_unsupported` | 400 | The dialect asks for something this version cannot do (an unknown strptime token, a distinct `escape`, a non-writable export option). | Read `detail.field`, then the rest of `detail` for the restriction it hit. |
| `csv_dialect_unknown_column` | 400 | A per-column override names a column the dataset does not declare. | Re-read `listDatasets`; the schema has changed. |
| `csv_dialect_irrelevant_option` | 400 | A per-column override is not relevant to that column's type. | Drop the override, or move it to the right column. |
| `upload_missing` | 409 | No staged file is available - none was uploaded, or the staged file expired. | Stage a fresh import and upload again. |
| `file_too_large` | 400 | The staged file exceeds the maximum upload size. | Split the file. |
| `app_locked` | 409 | A run or an agent job owns the app. `detail.activeRunId` names the run when it is one you may read. | Poll `activeRunId` if present; otherwise back off and retry. |
| `import_in_progress` | 409 | Another import for this dataset node is still validating. | Wait for it to finish, then commit. |
| `no_runnable_version` | 409 | The app holds no configuration of the requested `target`. | Ask the workspace to publish. The identical request then succeeds. |
| `compute_unavailable` | 503 | The platform is deliberately paused, or a platform-wide ceiling was reached. | Back off and retry. This is NOT a fault to escalate. |
| `no_published_run` | 409 | The app has no successful published run to export. | Run the pipeline, then export. |
| `run_data_unavailable` | 409 | The run exists but its output data has been removed by retention. | Run the pipeline again and export the new run. |
**Back-off.** `429` carries a `Retry-After` header and three `X-RateLimit-*` headers reporting the
allowance, what is left and when the window resets - respect them rather than guessing.
`503 compute_unavailable` means wait, not page someone; no workspace administrator can lift it.
`422 quota_exceeded` is the opposite: retrying never helps, somebody in the workspace has to act.
So are the two credential codes: `credential_expired` and `credential_attestation_overdue` are 401s,
but unlike `invalid_token` minting a new token cannot clear them, so a job that meets one should stop
and alert a person rather than retry. A queued import or export whose credential lapses before it
runs fails with the same code as its `errorCode`. One whose app was archived or moved into somebody's
personal space before it ran fails with `errorCode: "app_not_found"` and writes nothing - the same
answer a new request for that app gets. Retrying does not help; ask a workspace administrator where
the app went.
## Limits and lifetimes [#limits-and-lifetimes]
| Thing | Value |
| ----------------------------------- | ------------------------------------------------------------------------------------------------- |
| Requests per credential | 120 per minute by default; a workspace can be raised by an operator. `429` carries `Retry-After`. |
| Maximum upload | 5 GB per staged file. |
| Upload URL | Valid 1 hour. Re-read the import to mint a fresh one. |
| Staged file | Kept 7 days, then the import moves to `expired`. |
| Download URL (export, error report) | Valid 15 minutes. Re-read the record to mint a fresh one. |
| Generated export file | Kept 7 days. |
| Import and export records | Kept 90 days after the file is gone. |
| Credential | Expires at most 365 days after it was issued or last rotated (`credentialExpiresAt` on `whoami`). |
| Credential confirmation | Due every 90 days (`attestationDueAt`); suspended 14 days after a missed one. |
**An expired URL never means an expired object.** Every record read mints a fresh URL, so the
recovery is always "read the record again", never "start over".
## A worked example [#a-worked-example]
The whole loop as one script: mint a token, replace a dataset, run the published pipeline, export an
output and download the file, polling at each step and failing loudly.
```bash title="automation-loop.sh" lineNumbers
#!/usr/bin/env bash
set -euo pipefail
API="https://api.ovidens.com"
ISSUER="https://id.ovidens.com"
CLIENT_ID="${OVIDENS_CLIENT_ID:?set OVIDENS_CLIENT_ID}"
CLIENT_SECRET="${OVIDENS_CLIENT_SECRET:?set OVIDENS_CLIENT_SECRET}"
APP="sales_analytics"
DATASET="customers"
OUTPUT="revenue_by_region"
FILE="customers.csv"
# 1. Mint a token: HTTP Basic, the resource indicator, and the scopes this job needs.
TOKEN=$(curl -sS -X POST "$ISSUER/oauth/token" \
-u "$CLIENT_ID:$CLIENT_SECRET" \
-d "grant_type=client_credentials" \
-d "resource=$API" \
-d "scope=workspace:read data:write pipeline:run exports:read" \
| jq -r .access_token)
AUTH=(-H "Authorization: Bearer $TOKEN")
# 2. Prove the credential works before doing anything with it.
curl -sS "${AUTH[@]}" "$API/v1/whoami" | jq '{name, scopes, credentialScopes}'
# 3. Stage the import.
STAGED=$(curl -sS -X POST "${AUTH[@]}" -H "Content-Type: application/json" \
-d '{"mode":"replace","filename":"customers.csv","csv":{"delimiter":";"}}' \
"$API/v1/apps/$APP/datasets/$DATASET/imports")
IMPORT_ID=$(jq -r .importId <<<"$STAGED")
UPLOAD_URL=$(jq -r .uploadUrl <<<"$STAGED")
# 4. Upload the bytes. NO Authorization header.
curl -sS -X PUT --upload-file "$FILE" -H "Content-Type: text/csv" "$UPLOAD_URL"
# 5. Commit.
curl -sS -X POST "${AUTH[@]}" \
"$API/v1/apps/$APP/datasets/$DATASET/imports/$IMPORT_ID/commit" >/dev/null
# 6. Poll the import.
while :; do
IMPORT=$(curl -sS "${AUTH[@]}" "$API/v1/apps/$APP/datasets/$DATASET/imports/$IMPORT_ID")
STATUS=$(jq -r .status <<<"$IMPORT")
[ "$STATUS" = "validating" ] || [ "$STATUS" = "awaiting_upload" ] || break
sleep 5
done
if [ "$STATUS" != "applied" ]; then
echo "import $STATUS: $(jq -r '.errorCode // "unknown"' <<<"$IMPORT")" >&2
curl -sS "${AUTH[@]}" \
"$API/v1/apps/$APP/datasets/$DATASET/imports/$IMPORT_ID/errors" | jq -r .downloadUrl >&2
exit 1
fi
echo "applied $(jq -r .rowsApplied <<<"$IMPORT") rows"
# 7. Run the PUBLISHED pipeline. On 409 app_locked, adopt the active run.
RUN=$(curl -sS -X POST "${AUTH[@]}" -H "Content-Type: application/json" \
-d '{"target":"published"}' "$API/v1/apps/$APP/runs")
RUN_ID=$(jq -r '.runId // .detail.activeRunId // empty' <<<"$RUN")
[ -n "$RUN_ID" ] || { echo "no run: $RUN" >&2; exit 1; }
# 8. Poll the run.
while :; do
DETAIL=$(curl -sS "${AUTH[@]}" "$API/v1/apps/$APP/runs/$RUN_ID")
RUN_STATUS=$(jq -r .run.status <<<"$DETAIL")
[ "$RUN_STATUS" = "running" ] || break
sleep 10
done
[ "$RUN_STATUS" = "success" ] || { echo "run $RUN_ID failed" >&2; exit 1; }
# 9. Export the output, PINNED to the run we just made: that is what makes it reproducible.
BODY=$(jq -nc --argjson runId "$RUN_ID" '{runId: $runId, compression: "gzip"}')
EXPORT=$(curl -sS -X POST "${AUTH[@]}" -H "Content-Type: application/json" \
-d "$BODY" "$API/v1/apps/$APP/outputs/$OUTPUT/exports")
EXPORT_ID=$(jq -r .exportId <<<"$EXPORT")
# 10. Poll the export, then download (again, no Authorization header).
while :; do
RECORD=$(curl -sS "${AUTH[@]}" "$API/v1/apps/$APP/exports/$EXPORT_ID")
EXPORT_STATUS=$(jq -r .status <<<"$RECORD")
[ "$EXPORT_STATUS" = "generating" ] || break
sleep 5
done
[ "$EXPORT_STATUS" = "ready" ] || { echo "export $EXPORT_STATUS" >&2; exit 1; }
curl -sS -o "$OUTPUT.csv.gz" "$(jq -r .downloadUrl <<<"$RECORD")"
echo "downloaded $(jq -r .rowCount <<<"$RECORD") rows"
```
Next: the [API reference](./reference/overview) for the exact shape of any call, or the live
contract at [`https://api.ovidens.com/v1/openapi.json`](https://api.ovidens.com/v1/openapi.json).