8 docs tagged with "Extract, Transform, and Load (ETL) processes"

This straightforward recipe demonstrates a core Conscia behaviour - interacting with an API endpoint, and executing a task per-record against the results. In this example, we will be reading and writing from an Airtable, an innovative spreadsheet-type collaboration tool. We will read from a Table and for each unserved row, perform a unit of work on that input in Conscia, then write the output to another Table.

Conscia Recipes are designed to be intentionally open-ended, providing a robust starting point for building custom workflows. However, just as Components can come togther to form more complex Flows or populate bespoke experiences, Recipes can be built using others as a foundation.

This recipe uses DX Engine and DX Graph to provide an entry point for CPQ (Configure, Price and Quote) capabilities common in custom products, B2B/B2B2C use cases, and more. The product in this example is a custom couch, whose options we will populate in DX Graph and access from DX Engine. Prices are cumulative, with some options adding to the total cost, and others reducing it. Logic for the flow through elements lives in DX Engine.

In the enterprise space, growth by merger is a common and successful strategy. However for the IT-facing teams, this kind of marriage can introduce duplicate systems - multiples of tax, content management, or search platforms servicing the frontend, but also multiple WMS, OMS and ERP systems in the backend. Customers could think they're placing one order, but multiple business units are receiving their own piece of the final delivery.

This recipe uses the processFileWithDxEngine Job to take price data from a DX Graph Bucket and process it with DX Engine. We are imagining our ERP is the source of truth for our product data, including pricelists, and we want to sync this data into our commerce engine.

In this recipe, we will show you how to use the DX Engine to stitch data from multiple sources. We will use GitHub Gists to store different parts of a product detail page. We will then use the DX Engine to fetch the data from the Gists and stitch it together into a single JSON object.

If you are managing content in DX Graph that is being indexed in a search engine, you can use DX Engine Orchestration to listen to record created or updated events which will trigger a search index update. This recipe shows using a DX Engine Orchestration flow to update an Algolia search index whenever a record appears or changes in a DX Graph Collection.

You can implement an orchestration flow in the DX Engine based on the following recipe that explains how to update the inventory quantity associated with a product variant in commercetools when inventory may be managed in an external system such as an ERP or even a Real-time Inventory Management platform like Fluent Commerce.