6 docs tagged with "Extract, Transform, and Load (ETL) processes"

This straightforward recipe demonstrates a core Conscia behaviour - interacting with an API endpoint, and executing a task per-record against the results. In this example, we will be reading and writing from an Airtable, an innovative spreadsheet-type collaboration tool. We will read from a Table and for each unserved row, perform a unit of work on that input in Conscia, then write the output to another Table.

This recipe uses DX Engine and DX Graph to provide an entry point for CPQ (Configure, Price and Quote) capabilities common in custom products, B2B/B2B2C use cases, and more. The product in this example is a custom couch, whose options we will populate in DX Graph and access from DX Engine. Prices are cumulative, with some options adding to the total cost, and others reducing it. Logic for the flow through elements lives in DX Engine.

This recipe uses the processFileWithDxEngine Job to take price data from a DX Graph Bucket and process it with DX Engine. We are imagining our ERP is the source of truth for our product data, including pricelists, and we want to sync this data into our commerce engine.

In this recipe, we will show you how to use the DX Engine to stitch data from multiple sources. We will use GitHub Gists to store different parts of a product detail page. We will then use the DX Engine to fetch the data from the Gists and stitch it together into a single JSON object.

If you are managing content in DX Graph that is being indexed in a search engine, you can use DX Engine Orchestration to listen to record created or updated events which will trigger a search index update. This recipe shows using a DX Engine Orchestration flow to update an Algolia search index whenever a record appears or changes in a DX Graph Collection.

You can implement an orchestration flow in the DX Engine based on the following recipe that explains how to update the inventory quantity associated with a product variant in commercetools when inventory may be managed in an external system such as an ERP or even a Real-time Inventory Management platform like Fluent Commerce.