Back to docs

Dataset and Structure

title: Dataset and Structure

--- title: Dataset and Structure description: Dataset services and structure workflows. ---

Dataset and Structure

This section documents the dataset and structure packages.

@ekairos/dataset

The dataset package defines a dataset domain and services.

Key concepts:

  • dataset_datasets: metadata and state for a dataset.
  • dataset_records: per row data entries.
  • $files: file metadata compatible with InstantDB base file fields.

Services:

  • DatasetService.createDataset()
  • DatasetService.updateDataset()
  • DatasetService.addDatasetRecords()
  • DatasetService.batchAddDatasetRecords()
  • DatasetService.findDatasetByFileId()
  • DatasetService.getDatasetById()
  • DatasetService.updateDatasetSchema()
  • DatasetService.updateDatasetStatus()
  • DatasetService.clearDataset()
  • DatasetService.uploadDatasetOutputFile()

@ekairos/structure

Structure focuses on transforming unstructured data into structured datasets.

Typical flow: 1) Read inputs (documents, CSV, user input, logs). 2) Build a dataset schema definition. 3) Execute data extraction or transformation. 4) Store output as dataset records or dataset output file.

Structure can be used by agents as a long running workflow, or as a CLI tool in sandboxes.