Skip to main content

Asynchronous Integration Patterns

When using Conscia to move data between various backend systems, there are several integration patterns that can be employed. Depending on your use cases, you may use one or more of these. Some of the common scenarios include the following:

  • Syndicate product data from ERP and/or PIM to Commerce Engine
  • Push order data from commerce engine to OMS in real-time
  • Update the search index with changes from the pricing or inventory service
  • Send an email notification to the customer when an order is submitted
  • Update the CRM when a new customer is added through the Commerce Engine

Here are the various ways in which Conscia supports these scenarios.

ScenarioUpdate TypeSourceDestinationConscia ProcessExample ScenarioDetails
1.1FullS3 Bucket, SFTP, or any other file systemConscia BucketConscia receives the file in the BucketCreate a data extract from the ERP and push it to Conscia for further processing.Customer pushes the file from source to Conscia bucket using Conscia API
1.2FullFile in Conscia BucketFile in Conscia BucketFile gets dropped to Conscia Bucket, Job processes file and outputs to another file in the Conscia buckets after some data transformation.The extract from ERP needs to be transformed before it is pushed to an external webservice. You want to see the output of the transformation before you send it to an external system.Take a file from the bucket, transform it and write the new file to a bucket using this job type
1.3FullFile in Conscia BucketExternal Webservice EndpointJob processes file and writes to an external webservice endpoint. There is an ability to transform data on the way out.Load product data directly into commerce engine, without first loading into the Graph for analysis and validation."Done in two steps 1) Load data to a file (see above). 2) Write data to a webservice using Call Webservice Job Type
1.4FullFile in Conscia BucketAzure Blob StorageFile gets dropped to Conscia Bucket, Job uploads file to a storage container in Azure.Upload product data to cloud storage service directly from Graph.Learn how to upload data to the Azure Blob Storage from a file here
1.5FullFile in Conscia BucketGraph CollectionFile gets dropped to Conscia Bucket, Job processes file and writes to a Graph.Download data from PIM and load it into the Graph for further enrichment and validation.Learn how to load data into the Graph from a file here
1.6FullExternal Webservice EndpointFileJob pulls data from a webservice endpoint and writes to a file in a bucket.Full product catalog download from PIMLearn how to pull data from a webservice endpoint to a file here
1.7FullExternal Webservice EndpointGraph CollectionJob pulls data from a webservice endpoint and writes to Graph.Full product catalog download from PIMLearn how to pull data from an external webservice and import it into the Graph collection here
1.8FullGraph CollectionFileJob exports data from Graph Collection and Writes to a File.Export invalid records from the Graph to a file so that the data owners can correct the issues in the source systems.Learn how to export data from the Graph Collection to a file here
1.9FullGraph CollectionExternal Webservice EndpointJob pulls from Graph Collection and Writes to the Webservice Endpoint.You want to validate the product information from the ERP before writing it to a downstream commerce engine or a search index.Two Steps: 1) Pull Data from Webservice to a file 2) Push from file to a webservice using the 'Process Data File with Webservice' job type
2.1PartialGraph Collection EventWebservice EndpointGraph 'Record Updated Trigger' initiates a Engine Job Type, which can then write the record to an external system.Update Search index if a product record gets updated in GraphDefine a Trigger for the 'Record Updated' Event Type. Specify the 'Call Engine' Job Type. Create a Engine template that implements the orchestration flow for updating the external system with the updated record. Note: If there is no need to transform the record first, you can use the 'Call Webservice' job type instead and call the external system endpoint directly.
2.2PartialExternal WebhookEngine ListenerExternal Webhook calls the Engine Listener which then processes the updated record.The record is updated in the Commerce Engine.Create a listener in Conscia, Configure a webhook in the commerce engine to call that listener, Use the listener to call a specific Orchestration template.
2.3PartialGraph APIGraphGraph API is called and the record within a Graph Collection is updated.An External process such as an inventory service calls the Graph API with a CRUD operation.Call Graph API directly.
2.4PartialEngineExternal Webservice EndpointThe Orchestration Template in the Engine writes a record to an external endpoint.Order record updated in commerce engine -> Webhook calls Conscia Listener -> Conscia Engine writes to OMS.External Webhook calls Conscia Listener -> Conscia Engine writes to an External Webservice.