Dataset Files

Files can be used to provide a source of records in your datasets. You can create files, attach them to datasets, and sync them to import records.

Create File

Creating a file is the first step to using it as a data source for your datasets. You can create a file by making a POST request to the following endpoint:

Uploading File Content

There are multiple ways to upload file content to be used as a data source for your datasets.

Upload via JSON URL or Data URL

You can upload a file by providing a HTTP URL or a data URL in a JSON request body. This method is suitable for smaller files (up to 4.5MB).

or

Upload via Multipart/Form-Data

You can upload a file using multipart/form-data. This method is suitable for files up to 4.5MB.

Upload via Raw File Stream

You can upload a file by sending the raw file stream in the request body. This method is suitable for files up to 4.5MB.

Direct-to-Source Uploads

For larger files or more control over the upload process, you can obtain a pre-signed upload request by providing the file metadata in a JSON request body. You can then use the provided upload request to upload the file directly to the storage service.

The response will include an uploadRequest object with the necessary details to perform the upload.

You can then use this uploadRequest to upload the file directly to the storage service.

Attach Dataset File

Add a file to a dataset by creating an attachment between them. Specify the type of attachment.

Detach Dataset File

Remove a file from a dataset by deleting the attachment between them. You can pass an optional parameter to also delete all records associated with the file in the dataset.

Warning: This will permanently delete all records associated with the file in the dataset.

Sync a File to a Dataset

Files are not automatically synced to datasets when they are attached or updated. This is to give you control over when the data is imported and to avoid unnecessary processing.

You can trigger a sync of a file to a dataset by making a POST request to the following endpoint:

The response will contain the ID of the file that was synced. The processing of the file will happen asynchronously, and you can monitor the progress the dataset event log.