Skip to main content

Using DX Engine to Update an Algolia Search Index on DX Graph Record Changes

If you are managing content in DX Graph that is being indexed in a search engine, you can use DX Engine Orchestration to listen to record created or updated events which will trigger a search index update. This recipe shows using a DX Engine Orchestration flow to update an Algolia search index whenever a record appears or changes in a DX Graph Collection.

Overall Orchestration Flow​

When operations such as Record creation or update occur, DX Graph raises internal events. You can configure DX Graph to trigger Jobs when such events occur. A DX Graph Job can invoke a DX Engine Orchestration Template. In this case, the following orchestration flow takes place:

  • A data record in a DX Graph Data Collection is updated
  • DX Graph issues an event
  • A Trigger listens for this event and executes a Job
  • A 'Call DX Engine' Job Type is used to take the Event payload, transform it into a data structure required by Algolia and update the Algolia index.

Here is the overall orchestration flow:

DX Graph to Algolia Flow

Mapping Out DX Engine Elements​

This recipe configures two DX Graph Triggers, one for each relevant operation: Record Creation and Update events:

  • OnDataRecordCreated: Triggered by DataRecordCreated events.
  • OnDataRecordUpdated: Triggered by DataRecordUpdated events.

In this recipe, both DX Graph Triggers invoke the same Job, which calls a DX Engine Update Search Index (update-search-index) Orchestration Template.

The update-search-index Orchestration Template includes two Orchestration Components:

  • Reformat Event for Algolia (reformat-event-for-algolia): Reformats the JSON representation of the DX Graph event to a format that Algolia can process.
  • Update Algolia (update-algolia): Passes the updated data to Algolia.

The Update Algolia Orchestration Component depends on a DX Engine Connection:

  • Algolia Connection (algolia-connection): Encapsulates Algolia connection details.

The Algolia Connection (algolia-connection) DX Engine Connection depends on the Algolia Token (algolia-token) Secret that contains the Algolia token.

DX Engine Configuration Details​

Retrieve your Application ID, API Key, and index identifier from Algolia.

The Algolia Token (algolia-token) Secret.​

Create the Algolia Token (algolia-token) Secret using the value of your Algolia token.

The Algolia Connection (algolia-connection) DX Engine Connection.​

Create the Algolia Connection (algolia-connection) DX Engine Connection.

  1. Use the Universal API Connector.
  2. Set Base URL to a Literal, replacing {algolia-application-id} with your Algolia application ID: https://{algolia-application-id}-dsn.algolia.net/1
  3. Add the X-Algolia-API-Key HTTP header using the algolia-token Secret.
  4. Add the X-Algolia-Application-Id HTTP header using your Algolia application ID as a Literal.
  5. Add the Content-Type HTTP header set to application/json as a Literal.

The Reformat Event for Algolia Orchestration Component​

Create the Reformat Event for Algolia (reformat-event-for-algolia) Orchestration Component.

  1. Create an Object Mapper component with named Reformat Event for Algolia with identifier reformat-event-for-algolia.
  2. Select the No Rules checkbox.
  3. Set Response Transform as follows:
_.assign(contextField('data').new,{'objectID': contextField('data').new.dataRecordIdentifier})

The Job definition passes event data from DX Graph to the DX Engine Template as Context, such as the following, which is the format of event data in DX Graph:

{
  "id": "413a2ac8-047e-493e-b2cd-0b55b9a59413",
  "type": "DataRecordCreated",
  "source": "",
  "time": "2023-06-13T13:42:20.951Z",
  "data": {
    "customerCode": "demo-customer",
    "dataCollectionCode":"demo-collection",
    "new": {
      "dataRecordIdentifier":"12345",
      "personId": "12345",
      "firstName": "John",
      "lastName": "Doe",
      "email": "john@acme.com",
      "phone": "123-456-7890"
    }
  }
}

The data that we want to pass to Algolia appears under the new key. The JavaScript expression above will cause the Component to respond with the following, using the objectID key to pass the unique Record identifier from DX Graph as required for the index entry in Algolia:

{
    "dataRecordIdentifier":"12345",
    "personId": "12345",
    "firstName": "John",
    "lastName": "Doe",
    "email": "john@acme.com",
    "phone": "123-456-7890",
    "objectID": "12345"
}

The Update Algolia (update-algolia) Orchestration Component​

Create a Component named Update Algolia with ID update-algolia:

  1. Use the Conscia - Universal API Connector.
  2. Select the No Rules checkbox.
  3. Use the Algolia Connection Connection.
  4. Set Method to PUT.
  5. Set Body to Component Response and select the Reformat Event for Algolia Component.
  6. Set URL Path to /indexes/{your-algolia-index}/${componentResponse('reformat-event-for-algolia').objectID} JS Expression, replacing {your-algolia-index} with your algolia index identifier.

This component will pass data from DX Graph to Algolia.

The Update Search Index (update-search-index) Orchestration Template​

Create the Update Search Index (update-search-index) Orchestration Template.

  • Add the Reformat Event for Algolia (reformat-event-for-algolia) and Update Algolia (update-algolia) Orchestration Components to the Template.

This template simply invokes the Components on which it depends.

Register the DX Graph Triggers​

Use Conscia's /triggers endpoint (eg https://io-staging.conscia.ai/vue/_api/v1/triggers)to register the DX Graph Triggers, replacing values as appropriate:

POST https://{{baseUrl}}/triggers
Content-Type: application/json
Authorization: Bearer {{dxGraphApiKey}}
X-Customer-Code: {{customerCode}}

{
"trigger": {
  "eventType": "DataRecordUpdated",
  "triggerCode": "recordUpdated",
  "criteria": "`event.data.dataCollectionCode === '{{collectionCode}}'`",
  "name": "On-Update Trigger",
  "description": "Trigger on DataRecordUpdate for {{collectionCode}}",
    "job": {
        "jobType": "callDxEngine",
        "params": {
            "customerCode": "{{customerCode}}",
            "templateCode": "{{templateCode}}",
            "context": "`event`",
            "environmentCode": "{{environmentCode}}",
            "token": "{{dxEngineApiKey}}"
        }
    }
  }
}

Note: Replace {{collectionCode}} with the identifier of the DX Graph Collection that contains the Records; or replace the value of the criteria key with true.

In this JSON:

  • customerCode is your Conscia customer identifier.
  • triggerCode is a unique identifier for the Trigger.
  • eventType is the type of event that causes this Trigger to fire.
  • criteria are criteria under which this Trigger creates the Job. The example condition creates the Job if the event occurred in the specified collection.
  • jobType is the Job Type of the Job. The callDxEngine Job Type invokes a DX Engine Orchestration Template (call GET /vue/_api/v1/job-types for a list of Job Types).
  • templateCode is the identifier of the DX Engine Orchestration Template to invoke.
  • context instructs the Job to pass the DX Graph event data (event.data) to the DX Engine Orchestration Template as Context.
  • dxGraphApiKey is your token used to connect to DX Graph.
  • dxEngineApiKey is your token used to connect to DX Engine.

To handle Record creation alongside update, call the same API with the same data, updating the configuration as follows:

POST https://{{baseUrl}}/triggers
Content-Type: application/json
Authorization: Bearer {{dxGraphApiKey}}
X-Customer-Code: {{customerCode}}

{
"trigger": {
  "eventType": "DataRecordCreated",
  "triggerCode": "recordCreated",
  "criteria": "`event.data.dataCollectionCode === '{{collectionCode}}'`",
  "name": "On-Create Trigger",
  "description": "Trigger on DataRecordCreated for {{collectionCode}}",
    "job": {
        "jobType": "callDxEngine",
        "params": {
            "customerCode": "{{customerCode}}",
            "templateCode": "{{templateCode}}",
            "context": "`event`",
            "environmentCode": "{{environmentCode}}",
            "token": "{{dxEngineApiKey}}"
        }
    }
  }
}

Now, when you create or update a Record in the DX Graph Collection, DX Engine updates the search index in Algolia accordingly.

References​