Skip to main content

Working with Data Buckets

Data Buckets can be managed using the DX Graph API, and files can be uploaded to and downloaded from a Data Bucket using either the DX Engine API or the DX Engine UI as described here. This is useful for uploading data files to be imported into a Collection or downloading data files that have been exported from a Collection.

The following examples handle a Data Bucket with code store-sales and name Store Sales.

Creating a Data Bucket​

PUT https://io.conscia.ai/vue/_api/v1/buckets/store-sales
content-type: application/json
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

{
  "dataBucketEntry": {
    "name": "Store Sales"
  }
}

Removing a Data Bucket​

DELETE https://io.conscia.ai/vue/_api/v1/buckets/store-sales
content-type: application/json
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

Listing Data Buckets​

GET https://io.conscia.ai/vue/_api/v1/buckets
content-type: application/json
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

Listing Files in a Data Bucket​

The following lists the first 5 files in the store-sales Data Bucket that match the pattern store*.*.

GET https://io.conscia.ai/vue/_api/v1/buckets/store-sales/files?limit=5&pattern=store*.*
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

Deleting Files in a Data Bucket​

The following removes all files in the store-sales Data Bucket that match the pattern store*.*.

DELETE https://io.conscia.ai/vue/_api/v1/buckets/store-sales/files?pattern=store*.*
Authorization: Bearer {{apiKey}}
X-Customer-Code: {{customerCode}}

Uploading Files​

curl --request POST \
  --url 'https://io.conscia.ai/vue/_api/v1/buckets/store-sales/upload' \
  --header 'authorization: Bearer {{apiKey}}' \
  --header 'X-Customer-Code: {{customerCode}}' \
  --form 'file[]=@store-sales-01-2023.csv.gz' \
  --form 'file[]=@store-sales-02-2023.csv.gz'

Downloading Files​

When fetching a file from a Data Bucket, the filename query parameter is required. The filename is the name of the file that is contained in the Data Bucket. The following request streams the file to the client so it is important pipe the output to a file.

curl --request GET \
  --url 'https://io.conscia.ai/vue/_api/v1/buckets/store-sales/download?filename=store-sales-01-2023.csv.gz' \
  --header 'Accept: */*' \
  --header 'Authorization: Bearer {{apiKey}}' \
  --header 'User-Agent: httpyac' \
  --header 'X-Customer-Code: hugo' > store-sales-01-2023.csv.gz