Working with Data Buckets
Data Buckets can be managed using the DX Graph API, and files can be uploaded to and downloaded from a Data Bucket using either the DX Engine API or the DX Engine UI as described here. This is useful for uploading data files to be imported into a Collection or downloading data files that have been exported from a Collection.
The following examples handle a Data Bucket with code store-sales
and name Store Sales
.
Creating a Data Bucket
PUT https://io.conscia.ai/vue/_api/v1/buckets/store-sales
content-type: application/json
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}
{
"dataBucketEntry": {
"name": "Store Sales"
}
}
Removing a Data Bucket
DELETE https://io.conscia.ai/vue/_api/v1/buckets/store-sales
content-type: application/json
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}
Listing Data Buckets
GET https://io.conscia.ai/vue/_api/v1/buckets
content-type: application/json
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}
Listing Files in a Data Bucket
The following lists the first 5 files in the store-sales
Data Bucket that match the pattern store*.*
.
GET https://io.conscia.ai/vue/_api/v1/buckets/store-sales/files?limit=5&pattern=store*.*
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}
Deleting Files in a Data Bucket
The following removes all files in the store-sales
Data Bucket that match the pattern store*.*
.
DELETE https://io.conscia.ai/vue/_api/v1/buckets/store-sales/files?pattern=store*.*
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}
Uploading Files
curl --request POST \
--url 'https://io.conscia.ai/vue/_api/v1/buckets/store-sales/upload' \
--header 'authorization: Bearer {{apiKey}}' \
--header 'X-Customer-Code: {{customerCode}}' \
--form 'file[]=@store-sales-01-2023.csv.gz' \
--form 'file[]=@store-sales-02-2023.csv.gz'
Downloading Files
When fetching a file from a Data Bucket, the filename
query parameter is required. The filename
is the name of the file that is contained in the Data Bucket. The following request streams the file to the client so it is important pipe the output to a file.
curl --request GET \
--url 'https://io.conscia.ai/vue/_api/v1/buckets/store-sales/download?filename=store-sales-01-2023.csv.gz' \
--header 'Accept: */*' \
--header 'Authorization: Bearer {{apiKey}}' \
--header 'User-Agent: httpyac' \
--header 'X-Customer-Code: hugo' > store-sales-01-2023.csv.gz