Merge Two Sources into a Single Data Model

Merge a content item from the CMS with a product record from a commerce engine

In the following example, we want to fetch data for the product detail page that requires product information from the commerce engine and related product content from a CMS. This is a typical use case in a headless ecommerce implementation. Frequently developers write glue code in the frontend to address this use case and end up making multiple API calls and stitching together data from the backends.

Let's say that we are using Storyblok as our headless CMS. Product related content items have a field (called productID) that contains a product SKU. We want to use that SKU to retrieve the product record from our product catalog in commercetools and then use that product record to render a product detail page. We will use the DX Engine to unify the product and content records.

1. Setup a productSku Context Field
2. Setup a Connection to Storyblok (using a Secret that points to Conscia's Storyblok sandbox)
3. Setup a Component to retrieve the Storyblok content record
4. Setup a Connection to commercetools (using a Secret that points to Conscia's commercetools sandbox)
5. Setup a Component to retrieve the commercetools product record
6. Setup a Component to merge the output of the Storyblok and commercetools Components

note

Your sandbox should have already been seeded with the following secrets:

Secret Code	Name
conscia-sandbox-storyblok-delivery-token	Storyblok Delivery Token for Conscia Sandbox
conscia-sandbox-storyblok-management-token	Storyblok Management Token for Conscia Sandbox
conscia-sandbox-commercetools-client-id	Commercetools Client ID for Conscia Sandbox
conscia-sandbox-commercetools-client-secret	Commercetools Client Secret for Conscia Sandbox

Setup "productSku" Context Field

In this section we will setup a Context Field called productSku that will be used to hold the productSku that we will pass to the DX Engine.

• Navigate to the Settings --> Context Fields page in the DX Engine UI.
• Click the + button to create a new Context Field.
• Fill in the Context Field form as follows:

Property	Value
Context Field Name	productSku
Display Name	Product SKU
Data Type	String

Setup a Webservice Connection to Storyblok

In this section we will setup a Connection to Storyblok that will be used to retrieve the content record that contains reference to the product SKU. We will use the Universal API Connector.

• Navigate to the Settings --> Connections page in the DX Engine UI.
• Click the + button to create a new Connection.
• Fill in the Connection form as follows:

Property	Get value from	Value
Connection Code		storyblok-webservice-connection
Connection Name		Storyblok Webserbice Connection
Connector		Universal API Connector
Base URL		https://api-us.storyblok.com/v2/cdn/stories
Method		GET
Query Parameters #1, Parameter		token
Query Parameters #1, Value	Secret	Storyblok Token for Conscia Sandbox

Setup a Component to retrieve the Storyblok content record via produtSku

• Navigate to Manage Experiences --> Experience Components
• Click the + button to create a new Component.
• Fill in the Component form as follows:

Property	Value
Component Code	storyblok-product-record
Component Name	Product Record from Storyblok
Component Type	Conscia - Universal API Connector

• Click the Submit button

• Right-click the Storyblok Product Record Component and click Update.

• Fill in the form as follows:

Property	Get value from	Value
Connection		Storyblok Webservice Connection
Method		GET
Query Parameters #1, Parameter		filter_query[productID][like]
Query Parameters #1, Value	Context Field	Product SKU

• Click the Submit button

Query the Storyblok Product Record Component

Conscia's Storyblok sandbox has a content record for the product with SKU hobbit. We can query the Storyblok Product Record Component to retrieve the content record for the product with SKU hobbit as follows:

POST  {{dxEngineUrl}}/experience/components/_query
Content-Type: application/json
Authorization: Bearer {{token}}
X-Customer-Code: {{customerCode}}

{
  "componentCodes": ["storyblok-product-record"],
  "context": {
    "productSku": "hobbit"
  }
}

The response property, components.storyblok-product-record.response.stories[0].content, will contain the content record for the product with SKU hobbit.

Setup a Connection to Commercetools

In this section we will setup a Connection to Commercetools that will be used to retrieve the product record for the specified product SKU. We will use the Commercetools Connector.

• Navigate to the Settings --> Connections page in the DX Engine UI.
• Click the + button to create a new Connection.
• Fill in the Connection form as follows:

Property	Get value from	Value
Connection Code		commercetools-product-connection
Connection Name		Conscia Sandbox Connection for Commercetools
Connector		Commercetools
Project Key		omni-channel-experience
Hosting region		North America (Goolge Cloud, Iowa)
Client Id	Secret	conscia-sandbox-commercetools-client-id
Client Secret	Secret	conscia-sandbox-commercetools-client-secret

Setup a Component to retrieve the commercetools product record

• Navigate to Manage Experiences --> Experience Components
• Click the + button to create a new Component.
• Fill in the Component form as follows:

Property	Get value from	Value
Component Code		commercetools-product-record
Component Name		Commercetools Product Record
Component Type		Commercetools - Static Product List

• Click the Submit button
• Right-click the Commercetools Product Record Component and click Update.
• Fill in the form as follows:

Property	Get value from	Value
Connection		Conscia Sandbox Connection for Commercetools
Enter the IETF language code		en-uS
Product Keys	JS Expression	[contextField('productSku')]

note

We used the a JS Expresion to provide the Product Keys property with the value of the productSku Context Field within an array (since the Commercetools Connector expects an array of product SKUs).

• Click the Submit button

Query the Commercetools Product Record Component

Conscia's Commercetools sandbox has a product record with the SKU, hobbit. We can query the Commercetools Product Record Component to retrieve the product record for the product with SKU hobbit as follows:

POST  {{dxEngineUrl}}/experience/components/_query
Content-Type: application/json
Authorization: Bearer {{token}}
X-Customer-Code: {{customerCode}}

{
  "componentCodes": ["commercetools-product-record"],
  "context": {
    "productSku": "hobbit"
  }
}

Merging the Storyblok and Commercetools Records to create the data model required for the product detail page

To fetch both records in a single request, we specify both component codes in the request as follows:

POST  {{dxEngineUrl}}/experience/components/_query
Content-Type: application/json
Authorization: Bearer {{token}}
X-Customer-Code: {{customerCode}}

{
  "componentCodes": ["commercetools-product-record", "storyblok-product-record"],
  "context": {
    "productSku": "hobbit"
  }
}

The component response looks like this:

{
  "storyblok-product-record": {
    "@extras": {
      "rule": {
        "metadata": [],
        "attributes": {}
      }
    },
    "status": "VALID",
    "response": {
      "stories": [
        {
          "name": "Tolkien ",
          "created_at": "2023-06-29T15:59:37.120Z",
          "published_at": "2023-06-29T16:03:12.802Z",
          "id": 174100,
          "uuid": "ff638562-a2a5-4f5f-b1ea-28930582fc3f",
          "content": {
            "url": "",
            "_uid": "d9673bfc-b842-4ea0-8a73-58fcd254ff68",
            "body": "https://chat.openai.com",
            "image": {
              "id": null,
              "alt": null,
              "name": "",
              "focus": null,
              "title": null,
              "source": null,
              "filename": "",
              "copyright": null,
              "fieldtype": "asset",
              "meta_data": {}
            },
            "title": "J.R.R. Tolkien: A Literary Legend and Father of Modern Fantasy",
            "component": "Articles",
            "description": "Explore the captivating life and extraordinary imagination of J.R.R. Tolkien, the brilliant author behind the beloved Middle-earth tales and the creator of a genre that continues to inspire generations of readers and writers alike.",
            "productID": "hobbit"
          },
          "slug": "tolkien",
          "full_slug": "tolkien",
          "sort_by_date": null,
          "position": -50,
          "tag_list": [],
          "is_startpage": false,
          "parent_id": null,
          "meta_data": null,
          "group_id": "795c0364-edca-4792-bd34-ada6a85961a5",
          "first_published_at": "2023-06-29T16:03:12.802Z",
          "release_id": null,
          "lang": "default",
          "path": null,
          "alternates": [],
          "default_full_slug": null,
          "translated_slugs": null
        }
      ],
      "cv": 1688054630,
      "rels": [],
      "links": []
    }
  },
  "commercetools-product-record": {
    "@extras": {
      "rule": {
        "metadata": [],
        "attributes": {}
      }
    },
    "status": "VALID",
    "response": [
      {
        "id": "13ea40eb-1b9a-438d-8824-6347f96862e0",
        "version": 3,
        "productType": {
          "typeId": "product-type",
          "id": "6eb9f35c-d8ed-4120-8c10-49de4900a056"
        },
        "name": {
          "en-US": "The Hobbit"
        },
        "description": {
          "en-US": "Best selling fantasy fiction"
        },
        "categories": [],
        "categoryOrderHints": {},
        "slug": {
          "en-US": "the-hobbit"
        },
        "metaTitle": {
          "de-DE": "",
          "en-US": ""
        },
        "metaDescription": {
          "de-DE": "",
          "en-US": ""
        },
        "variants": [],
        "masterVariant": {
          "attributes": [],
          "assets": [],
          "images": [],
          "prices": [],
          "key": "123",
          "sku": "123",
          "id": 1
        },
        "searchKeywords": {},
        "hasStagedChanges": false,
        "published": true,
        "key": "hobbit",
        "createdAt": "2022-09-29T23:34:54.800Z",
        "lastModifiedAt": "2022-09-29T23:44:57.762Z"
      }
    ]
  }
}

Setup a Component to merge the Storyblok and Commercetools Records

• Navigate to Manage Experiences --> Experience Components
• Click the + button to create a new Component.
• Fill in the Component form as follows:

Property	Get value from	Value
Component Code		full-product-record
Component Name		Full Product Record from Storyblok and Commercetools
Component Type		Conscia - Object Mapper

• Click the Submit button
• Right-click the Full Product Record from Storyblok and Commercetools Component and click Update.
• Create two Object Maps by clicking + Add another item twice.

Editing the first Object Map

• Fill in the form as follows:

Property	Get value from	Value
Source data	Component Response	Storyblok Product Record
Expression Type		jmespath

• Edit Schema by clicking Edit Object
• Add expressions, contentTitle and contentDescription under Object Root

Editing the second Object Map

• Fill in the form as follows:

Property	Get value from	Value
Source data	Component Response	Commercetools Product Record
Expression Type		jmespath

• Edit Schema by clicking Edit Object
• Add expressions, productName and productSlug under Object Root

The final form should look like this:

• Click the Submit button

Query the Full Product Record

To fetch the merged records:

POST  {{dxEngineUrl}}/experience/components/_query
Content-Type: application/json
Authorization: Bearer {{token}}
X-Customer-Code: {{customerCode}}

{
  "componentCodes": ["full-product-record"],
  "context": {
    "productSku": "hobbit"
  }
}

The response will look like this:

{
  "contentTitle": "J.R.R. Tolkien: A Literary Legend and Father of Modern Fantasy",
  "contentDescription": "Explore the captivating life and extraordinary imagination of J.R.R. Tolkien, the brilliant author behind the beloved Middle-earth tales and the creator of a genre that continues to inspire generations of readers and writers alike.",
  "productName": "The Hobbit",
  "productSlug": "the-hobbit"
}

info

Since the full-product-record Component references both the Storyblok and Commercetools Components, DX Engine knows to run them both first (and in parallel) before running the full-product-record Component.