/app/datasets lists every dataset your organization sells and every batch delivered for it.
Sources and datasets
A source is one connected system, such as your Xero organization or a folder of CSV exports. You set sources up in the SDK on your own machine. The portal only knows each source's name, its connector type and when it last ran.
A buyer licenses a dataset. There is one dataset for each pair of source and product, so one HubSpot source can back a records dataset and an episodes dataset.
| Product | Contents |
|---|---|
records |
De-identified rows from the source |
episodes |
Situation, action and outcome sequences built from the source's audit trail |
environment |
A replica of the system with synthetic data drawn from the real distributions |
Batches
A batch is one scrubbed delivery of a dataset for one period, written as YYYY-MM. The first batch carries the history. Each scheduled run after that adds a batch with the new records for its period.
Each batch moves through these statuses:
| Status | Meaning |
|---|---|
awaiting_upload |
The SDK registered the batch and is uploading the file. |
received |
The file arrived and its checksum matched. |
in_review |
We are checking the batch and its scrub report. |
accepted |
The batch passed review. Licenses with an active refresh term pay for it. |
rejected |
The batch failed review. We tell you why, and nothing is delivered to buyers. |
A batch can be rejected for reasons such as a failed leak check, a column that needs a role override, or records that stand out in the re-identification check. Fix the cause, then run the SDK again.
The report for each batch
Every batch carries the scrub report the SDK produced on your machine. The portal shows the same column roles, redaction counts and re-identification result you saw in datayield preview, so you and we review the same evidence.
Test batches
Batches sent with a dy_test_ key appear in the portal so you can check your setup, but they are never billable and are never delivered to buyers.