Skip to main content

Data Collections

In DX Graph, data resides in Data Collections. A Collection contains records of a certain user-defined type, such as customers, products, stores, categories

Collections can be used to manage and/or analyze various types of data such as:

  • Master data (e.g. Customers, Stores, Products, Promotions)
  • Lookup data (e.g. Customer Types, Regions, Countries, Colours). When editing a record in another Collection via DX Studio, a drop-down list may be presented that is populated by the data records in a Lookup Collection.
  • Categorization data (e.g. Customer Audiences/Segments, Product Categories). Categorization data can be configured to hierarchical (i.e. records may have a parent record).

Collections may also contains data that falls under more than one of the above types of data.

Schemas

Each Collection has a schema that allows for an extremely flexible record layout. A field can be of any type and structure.

Relationships

Collections may have relationships between them. There three types of relationships:

1-to-1

A 1-to-1 relationship is where a single value on a record must be the same as another records' Data Record Identifier (i.e. unique id).

1-to-Many

A 1-to-many relationship is where each value in an array of values on a record must be the same as another record's Data Record Identifier.

Dynamic

Dynamic Relationships are used when neither 1-to-1 or 1-to-many suffice.

Shadow Fields

A field can be designated as a Shadow Field which is a field whose value is calculated by using an expresion that references other fields from the same collection and/or related collections. The expression can be either a Javascript Expression or a JSONata expression. As well as specifying the expression, the referenced relationship(s) and field(s) need to specified to let the DX Graph's query engine know how to properly calculate the Shadow Fields.

Shadow Field expressions cannot reference other Shadow Fields.

For a Shadow Field to be used for searching and filtering, it must be materialized. Materialization is the process of persisting the Shadow Field values onto the records so that they do not need to be calculated at query-time. When querying a Collection, you can specify whether Shadow Fields are calculated (to see the freshest data) or if the materialized shadow field values are used.

Snapshots

At any given time, a Snapshot can be taken of a Collection. A Snapshot takes every record in a Data Collection and exports it to line-delimited JSON file in Snapshots Data Bucket.

Intel

If an Intel Data Table has been created in the Snapshots Data Bucket (which makes it a Snapshot Data Table), a periodic job will sync the latest snapshot file into the Data Table. This is useful when the data from a Collection is required for Computed Values.