Skip to main content

Working with Data Buckets

Data Buckets can be managed using the DX Graph API, and files can be uploaded to and downloaded from a Data Bucket using either the DX Engine API or the DX Engine UI as described here. This is useful for uploading data files to be imported into a Collection or downloading data files that have been exported from a Collection.

The following examples handle a Data Bucket with code store-sales and name Store Sales.

Creating a Data Bucket

PUT https://io.conscia.ai/vue/_api/v1/buckets/store-sales
content-type: application/json
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

{
"dataBucketEntry": {
"name": "Store Sales"
}
}

Removing a Data Bucket

DELETE https://io.conscia.ai/vue/_api/v1/buckets/store-sales
content-type: application/json
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

Listing Data Buckets

GET https://io.conscia.ai/vue/_api/v1/buckets
content-type: application/json
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

Listing Files in a Data Bucket

The following lists the first 5 files in the store-sales Data Bucket that match the pattern store*.*.

GET https://io.conscia.ai/vue/_api/v1/buckets/store-sales/files?limit=5&pattern=store*.*
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

Deleting Files in a Data Bucket

The following removes all files in the store-sales Data Bucket that match the pattern store*.*.

DELETE https://io.conscia.ai/vue/_api/v1/buckets/store-sales/files?pattern=store*.*
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

Uploading Files

curl --request POST \
--url 'https://io.conscia.ai/vue/_api/v1/buckets/store-sales/upload' \
--header 'authorization: Bearer {{apiKey}}' \
--header 'X-Customer-Code: {{customerCode}}' \
--form 'file[]=@store-sales-01-2023.csv.gz' \
--form 'file[]=@store-sales-02-2023.csv.gz'

Downloading Files

When fetching a file from a Data Bucket, the filename query parameter is required. The filename is the name of the file that is contained in the Data Bucket. The following request streams the file to the client so it is important pipe the output to a file.

curl --request GET \
--url 'https://io.conscia.ai/vue/_api/v1/buckets/store-sales/download?filename=store-sales-01-2023.csv.gz' \
--header 'Accept: */*' \
--header 'Authorization: Bearer {{apiKey}}' \
--header 'User-Agent: httpyac' \
--header 'X-Customer-Code: hugo' > store-sales-01-2023.csv.gz