# Datasets (/docs/analytical-apps/pipeline/datasets)



A **dataset** node is an input table you fill with your own data. It is where your
information enters the pipeline, either by importing a file or by typing it in.
Everything downstream builds on what your datasets hold.

## Define columns and their types [#define-columns-and-their-types]

A dataset is organised into **columns**, and each column has a **data type** that
says what kind of value belongs in it. Choosing the right type keeps your data
clean: numbers stay numbers, dates show a date picker, and a dropdown only allows
the options you set. The available types include:

* **Text** and **Long text** for words and free-form notes,
* **Integer** and **Decimal** for whole and fractional numbers,
* **Currency** for money and **Percent** for percentages,
* **Boolean** for a simple yes/no,
* **Date**, **Time**, **Timestamp**, and **Year** for points in time,
* **Phone** and **Email** for those specific kinds of text,
* **Dropdown** for picking one option from a list, and **Multi-select** for
  picking several,
* **Attachment** for one or more uploaded files.

Give each column a clear name and the type that matches the values it will hold.

## Import a CSV or Excel file [#import-a-csv-or-excel-file]

The quickest way to fill a dataset is to import a spreadsheet. A step-by-step wizard
guides you through six steps: **Select file**, **Choose sheet**, **Parsing options**,
**Map columns**, **Preview & validate**, and **Confirm**. You can go **Back** at any
point without losing what you have set.

Importing a file is available to everyone who may edit the app's data - the same people
who can [edit its rows by hand](#edit-data-by-hand). Someone who may only view the app
cannot import.

### Select file [#select-file]

Drag a CSV or Excel file onto the wizard, or click to pick one. Ovidens detects the
delimiter and the encoding for you.

If you are starting from nothing, **Template** gives you a file that already has this
dataset's columns as headings. Choose CSV or Excel in the dialog, name the file, and
fill it in - upload it back and the mapping step has nothing left to guess.

If the file is too big or has more rows than a dataset may hold, the wizard says so
straight away and names the limit, so you can split the file rather than wait for a
failed import. Very large files are imported exactly as parsed instead of being
editable cell by cell in the preview.

A single row may hold up to 2,000,000 characters. A longer row stops the import with a
message naming its line, and nothing is imported, so shorten the longest values on that
line or split them across rows.

### Choose sheet [#choose-sheet]

An Excel workbook usually has several sheets, so pick the one holding your data. Each
is listed with its size, for example "1,204 rows · 12 cols". A CSV skips this
step.

### Parsing options [#parsing-options]

This step is how you deal with a file that is not perfectly clean. Everything here
has a sensible default, and the preview updates as you change it.

* **Delimiter** - the character that separates columns (comma, semicolon, tab).
* **Encoding** - the file's character set. Use Windows-1252 for older Excel exports
  whose accents come out garbled.
* **Header row** - which row holds the column names, or **No header** if the file
  starts straight into data.
* **Skip leading rows** and **Skip trailing rows** - ignore a title block or a set of
  notes at the top, and totals or footnotes at the bottom.
* **Empty/null tokens** - values that should count as empty, for example `N/A` or
  `-`. Type one and press Enter to add it.
* **Trim whitespace around values** - remove stray spaces from every value.

### Map columns [#map-columns]

Line up each column in your dataset with a column from your file. The wizard matches
what it can by name and labels how confident it is: an exact match, a name match, or
a fuzzy match. Anything you do not want, set to "leave empty".

Two shortcuts help with an awkward file:

* **Suggest with AI** asks Vera to work the mapping out from the column names. Each
  suggestion is marked "AI suggestion · needs review", so you confirm rather
  than trust it blindly. The wizard states plainly what leaves your workspace:
  "Column names and a few sample values are sent to our AI provider to suggest a
  mapping."
* **Reuse previous mapping** applies the mapping from the last import into this
  dataset, which is what you want for a file you refresh every month.

The wizard warns you if a required column is not mapped, or if columns in your file
are going unused.

### Preview & validate [#preview--validate]

Now the wizard checks every value against its column's type and shows you the totals:
how many rows are valid, how many have errors, and how many carry warnings. Warnings
cover things worth a second look rather than outright errors, such as a column that
is completely empty, a column with only one distinct value, or duplicate rows.

Click any issue to **jump to the cell** behind it and fix it in the preview. The
preview shows the first 500 rows, and says so; every row is still checked and
imported.

One switch decides what happens to bad rows:

* **Skip invalid rows** on: the invalid rows are left out and the valid ones are
  imported.
* Off: all rows must be valid, and the import is rejected if any row has errors.

If an import is blocked, **Download error report** lists exactly which rows failed,
including ones beyond the preview. A very long value is shortened in the report and ends
in "...", so the report stays a size a spreadsheet can open; the line number takes you
to the full row in your file.

**Deduplicate on** is optional: pick one or more columns as a key, and incoming rows
whose key already exists are skipped.

### Confirm [#confirm]

Finally, choose the **Import mode**: **Replace** overwrites all existing data, or
**Append** adds these rows to what is already there (offered only when there is data
to append to). Confirm, and Ovidens reports what happened, for example "Imported
1,180 rows · skipped 24".

A file needs at least one data row. A file that holds only a header row, or nothing at
all, is refused in both modes with the message "The file has no data rows, so there is
nothing to import", and the dataset keeps the rows it has. An empty file is usually a
mistake - an export that matched nothing, or a file saved too early - so Ovidens does not
treat it as an instruction to empty the dataset. The same holds with **Skip invalid
rows** on when every row of the file is invalid: the import is refused with a message
saying how many rows were invalid, rather than importing none. To empty a dataset on
purpose, delete its rows in the data table.

You cannot import while a pipeline run is in progress; the wizard says so and waits.

## Edit data by hand [#edit-data-by-hand]

You do not have to import everything. You can also type values directly into the
dataset and edit cells in place, which is handy for small tables or quick fixes.
You can set basic rules on a column too, such as making it required or requiring
its values to be unique, so your data stays consistent as you enter it.

This works from the very first row. A brand-new dataset that has never received any
data shows an empty table with the message "No data yet. Add the first row by hand,
or use Import to load a file." and an **Add first row** button. Choosing it opens
the editor with the first row already created; type into it and **Save**. From then
on the **+ New row** strip at the bottom of the grid adds further rows. No file
import is needed first, and importing a file stays available at any time from the
same toolbar. For someone who cannot import, the message leaves the import out:
"No data yet. Add the first row by hand."

There are three ways to add a row. The **+ New row** line at the bottom of the
table and the add-row control shown at the left of each row while you are editing
both create the new row straight away, with nothing to type first. Using **Row**
in the toolbar instead opens a form, and it only adds a row once you have filled
in at least one field - submitting it empty adds nothing. However you add a row,
it lands at the end of the table - a dataset keeps its rows in the order they were
created, so a new row is always appended rather than inserted between two existing
ones. The row counts as an unsaved change until you choose **Save**.

The only thing you need before you can type is at least one column. A dataset with
no columns defined yet shows the import drop zone instead, because importing a file
is also what defines the columns.

### Files in an attachment column [#attachments]

An **Attachment** column holds files - one or several per row. While you are editing,
hover over a cell in the column and choose **Add file** (on an empty cell) or the paperclip
(on a cell that already holds files). In the dialog that opens, click to pick files or drag
them onto the drop area, then confirm with the upload button, which names how many files it
sends. Each file can be up to 10 MB; a bigger file
is refused in the dialog with a message saying so, and nothing else you have changed is lost.

An uploaded file is not part of the row until you **Save**. Choosing **Cancel** instead
drops it, together with the rest of the unsaved changes.

Click the file icon in a cell to see all of its files with their names and sizes. From
there you can download a file (you are asked what to name it first), see an image in a
larger view, attach more files or remove one. To replace a file, remove it and add the new
one, then **Save**.

Anyone who can read the app's data can open and download its files. Only workspace
administrators can add or remove them. The same cells, with the same rules, appear in an
editable [table widget](../visualisation.mdx#table-attachments) on the app's dashboard.

A file stays downloadable when the app's configuration is edited or a new version is
published. Removing a file from a cell, or cancelling after uploading one, does not delete
the stored copy: it is kept, still counts towards your workspace's storage, and is not
cleaned up automatically today.

## Download the data [#download-the-data]

**Export** in the table toolbar downloads what the dataset holds. Every download
asks you to name the file first, so what lands in your downloads folder is
something you can recognise later rather than a technical id.

Downloading takes the rows out of the workspace, so it is a permission of its own,
separate from reading them. It comes with the **Contributor**, **Analyst**, **App builder** and **Admin** roles. Someone whose
access to an app lets them only view it can read its tables but is not offered
**Export**, here, on a table or pivot in the Visualisation, or under the results of
a SQL query, and a report offers them no **Export PDF**, because the PDF carries the
report's tables in full.

**The file matches what you are looking at.** If you have filtered the table,
searched it, or hidden columns, the dialog asks what the file should hold and
each answer carries its own number:

* **Rows** appears when a filter or a search is narrowing the table. **Current
  filter** holds exactly the rows on screen; **All rows** holds the whole
  dataset. The one on screen is chosen for you, so confirming without changing
  anything downloads what you were looking at.
* **Columns** appears when you have hidden at least one. **Visible** holds the
  columns you left on screen, in the order you put them in; **All** holds every
  column the dataset has.

The line under the file name always describes the choice you have made, so the
number you read there is the number of rows the file will have. Choosing
**All rows** on a very large dataset can also put it past what a spreadsheet
holds, and Excel is greyed out while that choice is selected.

One view cannot be downloaded on its own yet: with **failing rows only** turned
on, **Current filter** is offered greyed out and the download takes all rows.
Turn the toggle off to download what a column filter matches.

The name is filled in for you from the node's own name plus today's date - a
weekly pull of the same dataset no longer overwrites last week's. Edit it if you
want something else. The file type sits beside the box and is not part of the
name, so the two can never disagree.

Choose **CSV** or **Excel**:

* **CSV** is the interchange format. It is plain text, every value is written out
  as text, and it re-imports through the same wizard it came from.
* **Excel** carries types. A date arrives as a date, a number as a number and a
  yes/no column as TRUE or FALSE, so you can sort and total a column without
  retyping it. A percent column shows its `%` and a currency column its symbol,
  the same way they read on screen. Dates are written as `2026-08-29` so they mean
  the same thing in every region.

A spreadsheet holds about a million rows. If the dataset has more than that,
Excel is offered greyed out with the row count, and CSV takes it.

A run's results and a published version's pinned data download the same way, from
the same control, and their names say which one you took.