# The generated pipeline (/docs/indicator-apps/pipeline) Every indicator app carries a **pipeline** that Ovidens generates from the app's own configuration. You do not build it and you cannot edit it - it is rebuilt whenever you change the app's structure - but you can read all of it, and that is the point. Open the editor and choose the **Pipeline** tab (or press P). ## What you are looking at [#what-you-are-looking-at] The pipeline has three kinds of node. * **Relations** are tables rather than statements. Each one carries a part of the app's own configuration as rows: its indicators and their cadence, the dimensions they are collected by, every member of every dimension, its rules, and the values collected against it. * **Your datapoints** are one node each - a step named after the datapoint, carrying just the cells collected for it. * **Engine steps** are single SQL statements over the tables before them. A few are the platform's own groundwork - the period calendar, the dimension member tree, the collection members, the stranded-data findings. The rest are **one step per thing you configured**: one for each formula datapoint's calculation, one for each datapoint's applicability and value rules, one for each consistency check. Each is a single statement you can read in full. Click a node to read it. ## Your datapoints, one step each [#your-datapoints-one-step-each] Every datapoint you **collect** - a number, a text, a yes/no or a single choice - has a step of its own. Open it and you see that datapoint's cells and nothing else: the period they were collected for, one column for each dimension you collect that datapoint by, and the value. **Each dimension column is named after your dimension.** Underneath, every cell of the app is stored with an address that says which dimension each of its members belongs to - so `region` means `region` in the stored cell, in this step, and in anything you write against it. These steps unpack that address into one column per dimension, in the order you assigned them. It is also what makes the SQL worth reading: a statement written against `region` still means `region` tomorrow. **A datapoint can be collected by as many dimensions as your workspace allows.** There is no separate storage limit underneath - the only ceiling is the one your workspace's settings state. Three columns come from Ovidens rather than from you, and they start with an underscore so they can never collide with a dimension of your own: `_period_type` (the cadence), `_period_start` (which period) and `_value` (the collected value itself). A dimension you call `value` is perfectly legal and gets a column called `value`, right beside `_value`. Two kinds of datapoint have no step here, each for its own reason. A **formula** datapoint has no collected cells at all - its figures come out of its own engine step, which carries the calculation you configured. A **file** datapoint's stored value is a pointer to the uploaded file rather than a value, so it is not something a table of results can carry; the files themselves stay on the datapoint, where you uploaded them. Reading one of these steps is cheap: it reads your collected cells directly rather than running the whole engine, so opening a datapoint's data is quick even on a large app. **Every node opens the same way a node opens in an analytics app** - the same panel, drawn the same way, down to the last control: a **Configuration** tab, and a **Data** tab showing that node's rows in the ordinary table. Nothing there is editable - the pipeline is Ovidens's, not yours - but everything the engine reads, and everything each step of it produced, is in front of you. Rows are read a page at a time as you scroll, and you can sort, filter and search them exactly as you would any other table; nothing is left out and nothing is summarised. What differs is the Configuration tab. A **relation** names which part of the app's configuration it carries. An **engine step** shows its statement **in full** - not a summary and not an excerpt - alongside the list of tables it reads, so you can read what produced the rows on the tab beside it. **The pipeline is Ovidens's, whole.** No step can be edited, handed over or taken on, in this editor or anywhere else. If the building blocks cannot express something you need, read the data into an **analytic app** with an [Indicator data](../analytical-apps/pipeline/indicator-data.mdx) node: that app has a real SQL editor with previews and typed output columns, and its statements are yours to write and keep. **This pipeline is not a picture of how your app is computed - it is what computes it.** Every number your app shows comes out of these steps: the value in a formula datapoint's cell, the pass or fail of a value rule and a consistency check, whether a conditional datapoint applies for a period. Open a formula datapoint's own step and you are reading the figures the collection grid shows you, at the coordinates it shows them at; open a step above it and you are reading what they were computed from. There is no second, private calculation anywhere behind them. **One object, one statement.** A calculation is not interpreted at run time - it is turned into SQL once, when you save it, and that statement is what runs. So the step you can read IS your calculation, and there is no engine between the two that could read it differently. That is also why a refusal reaches you at **Save** rather than as a finding you have to go looking for. A reference to a datapoint the app does not hold, a cadence that does not reconcile, a breakdown that cannot be lined up, a calculation that reads itself: each is refused while the statement is being generated, naming what did not fit. Nothing unsound is ever written, so there is no "findings" step to consult - the refusal happened before there was a step at all. ## Why Run is greyed out on an engine step [#why-run-is-greyed-out-on-an-engine-step] A step's SQL sits in the same editor an analytics app's SQL node uses, with the same **Run** control underneath it - and on an engine step that control is deliberately switched off, with the reason beside it. An engine step is not a query against tables that sit somewhere waiting to be read. It runs as one part of the app's own evaluation, over relations that are assembled for the length of that evaluation and then gone, so there is nothing for a stand-alone run to read. What that statement produced is on the **Data** tab beside it, which is a page of the real result rather than a rehearsal of it. The statements themselves are ordinary SQL for [DuckDB](https://duckdb.org) and nothing about them is special to Ovidens: select the text in the editor to take a copy into an editor of your own and read it at your own pace. ## If the tab is empty [#if-the-tab-is-empty] An app's pipeline is generated from its structure, and an app that existed before the pipeline did does not carry one yet. Saving any change to its structure generates it: add or edit a datapoint, a dimension, a group or a consistency check, and the tab fills. Every app created since carries its pipeline from the moment it exists. ## Stranded data [#stranded-data] A value is stored together with the structure it was collected against: which indicator, which dimension members, and which collection interval. Change that structure afterwards and the values already collected do not move with it - they are left behind, outside the grid the indicator now asks for, where nothing reads them. The Pipeline tab reports them, with the indicator and how many rows are affected. | What it says | What happened | What to do | | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------- | | Collected at a collection interval the datapoint no longer uses | The indicator's collection interval was changed after these values were entered | Set the interval back to reach them, or clear them | | Collected against a dimension the datapoint no longer has | A dimension was taken off the indicator, and these rows still name one of its members | Assign the dimension again, or clear them | | Collected before a dimension was added | A dimension was added after these values were entered, so they name no member for it | Enter the values again against the new dimension, or clear them | | Collected against a member that is no longer active | A dimension member was deleted or deactivated while values still named it | Make the member active again, or clear them | | Collected against a member at another level of the hierarchy | The indicator collects the dimension at one level, and these rows name an active member above or below it - usually because the collection level was changed after the values were entered | Collect the indicator at that level again, or clear them | When there is nothing to report, the tab says so. If the check could not run - the platform was busy, or your workspace has used up its compute allowance for the month - it says **that** instead, rather than reporting a clean result over data it did not read. **The check is taken each time you open the Pipeline tab**, so there is no refresh button to press. Nothing that could change the answer is reachable while the tab is open: values are entered on the data-entry grid and the structure is edited on the other tabs, so coming back here is what takes a fresh reading. **Clearing one of these values.** Where the value still has a cell on the data-entry grid - most often after a collection level or a member changed - empty the cell and save, exactly as you would correct any other value. That works even though entering a NEW value at the same place is refused: clearing is always allowed, because it is how you act on what this list tells you. The cell then reads as empty rather than disappearing from the grid. ## What needs compute, and what happens without it [#what-needs-compute-and-what-happens-without-it] Because the pipeline is what computes the app, reading it and changing it both use your workspace's compute allowance, and a workspace that has used the allowance up (or a platform-wide pause) will say so rather than answering with something it did not compute. * **Reading a step's Data tab** runs the engine for that step. Reading a **relation**'s rows does not - a relation is your own configuration, so its columns and rows are free. * **Changing the structure** does **not**. The pipeline is rebuilt from your configuration alone, and a calculation is checked by generating the statement for it - both of which happen in Ovidens itself. So a datapoint, a dimension, a group or a check can be saved while compute is unavailable, including one carrying a calculation. What you cannot do meanwhile is *read the results*. * **Entering a value** checks the app's rules against what you typed, so that needs compute as well. This is deliberate: accepting a value the platform could not check is worse than telling you to come back. The refusal always names itself and what to do about it; nothing is saved or shown on a half-answer. ## Why a cadence change can be refused [#why-a-cadence-change-can-be-refused] Because a value carries the interval it was collected at, changing an indicator's **collection interval** would strand every value already collected. When you edit the indicator, Ovidens refuses that change while such values exist and tells you how many rows are involved. The way forward is to clear those values first and then change the interval. There is deliberately no "change it anyway" option: it would be a button whose only effect is to lose data quietly. **Restoring an earlier version is not held to that check.** A restore puts the whole app back the way it was, collection intervals included, so it can strand values that an edit would have been refused for. It is deliberately not blocked - a rollback you are not allowed to perform is its own kind of trap - so what it strands is reported rather than prevented: check the **Stranded data** list above after restoring a version.