Skip to main content

Data Transformation Script

You can use Data Transformation Script Orchestration Components to embed arbitrary logic in Conscia DX Engine orchestration flows that would be cumbersome or impossible to implement in other types of Components. A Data Transformation Script obtains its data from a JavaScript Expression, a Component Response, or a Context Field, and generates a response like any other component. To avoid the potential for rogue scripts to compromise the DX Engine, Data Transformation Scripts run in isolated JavaScript Virtual Machines (VMs - "V8 isolates") separate from core system processes.

Data Transformation Script JavaScript Capabilities

Data Transformation Script JavaScript VMs do not have access to DX Engine functions such as componentResponse() or external helper libraries as other JS Expression fields in DX Engine do. Instead of using these functions in the Script itself, use the Data to Modify field to pass data to the Data Transformation Script. In this field you can access DX Engine functions and helper libraries and declare the data required for the Script.

To support re-use of JavaScript isolates and hence improve performance, use var or const to declare JavaScript functions in Data Transformation Scripts.

Data Transformation Script JavaScript Virtual Machines

While the DX Engine uses a Least Recently Used (LRU) cache for isolates and compiled scripts, spawning a JavaScript virtual machine involves a minor performance impact as the DX engine follows this process:

  1. Create a V8 isolate.
  2. Compile the Data Transformation Script into the isolate.
  3. Copy the current context into the data variable passed into the isolate.
  4. Execute the script in the isolate.

Conscia allocates up to 8MB of memory to each Data Transformation Script VM and will terminate a VM that does not complete within two seconds. Most scripts should complete within a few milliseconds. If you need to implement processes that run for longer, consider implementing the logic as a service rather than a Component and implement a Universal API Component to invoke that service, potentially asynchronously.

Creating a Data Transformation Script Orchestration Component

To create a Data Transformation Script Orchestration Component, in the DX Engine user interface:

  1. In the top navigation, click Manage Experiences, and then click Components. The Manage Components page appears.
  2. Click Add Component. The Create Component wizard appears.
  3. For Component Code, enter an identifier for the Component.
  4. For Component Name, enter a name for the Component.
  5. Optionally, enter a Component Description.
  6. Select the No Rules checkbox. In general, Experience Rules do not apply to Data Transformation Script Components.
  7. If subsequent components do not depend on the response of the Data Transformation Script, and its output is not needed in the orchestration response, and you don’t want the orchestration flow to block while the Data Transformation Script runs, then select the Is Asynchronous checkbox.
  8. For Component Type, select Conscia – Data Transformation Script.
  9. Click Submit.

To configure the Data Transformation Script Orchestration Component, in the Manage Components page:

  1. Click the Edit button next to the Component. The Edit Component wizard appears.
  2. For Data to modify, select the source of data for the component. This populates the variable named data that the DX Engine passes to the Data Transformation Script:
    • JS Expression: Enter a JavaScript expression.
    • Component Response: Select a component.
    • Context Field: Specify the identifier of a Context Field.
  3. For Script, enter JavaScript to invoke. The DX Engine will use the final line of this script as the response of the Component.
  4. Click Submit.

Filtering Example​

For example, an organization would like to personalize their website navigation based on a visitor's interests. Each navigation element has a title, a category, and a URL. The response of an Orchestration Component with id nav-elements determines all navigation elements without filtering and responds in the following format.

{
  "elements":
  [
    {
      "navTitle": "Home",
      "url": "/"
    },
    {
      "navTitle": "About Us",
      "category": "about",
      "url": "/about"
    },
    {
      "navTitle": "Jobs",
      "category": "about",
      "url": "/about/jobs"
    },
    {
      "navTitle": "Engineering Jobs",
      "category": "about",
      "url": "/about/jobs/engineering"
    },
    {
      "navTitle": "Services",
      "category": "services",
      "url": "/services"
    },
    {
      "navTitle": "Products",
      "category": "products",
      "url": "/products"
    }
  ]
}

If the visitor is not interested in a category, then navigation elements associated with that category should not appear in the site navigation.

For this example, the client building the navigation sends the visitor's categories of interest to the orchestration flow in Context under the categories key.

{
  "categories": [ "products", "about" ];
}

A Filter Navigation Digital Transformation Script Orchestration Component filters the navigation.

The Data to modify for the Filter Navigation Digital Transformation Component is set to the following JS Expression:

_.assign({},{
  "nav": componentResponse('nav-elements'),
  "categories": contextField('categories')
})

This registers a dependency on the nav-elements Component and passes its response to the Data Transformation Script as data.nav, and passes the categories from Context as data.categories.

The Script for the Filter Navigation Digital Transformation Component is set to the following:

data.nav.elements.filter(check => data.categories.includes(check.category), data.nav.elements);

This returns a new array that includes only the elements in data.nav.elements for which the category is in the data.categories array.

Restructuring Example

The following Script creates a tree structure based on the URL paths of the navigation elements. Note the use of const in the addChildNodes function definition.

// add child navigation nodes to a node (recursive)
//   node = nav element (start with root, where node.url === "/")
//   elements = array of all nav elements
const addChildNodes = (node, elements) => {
  // add all nodes that have the node's URL as their parent
  node.children = elements.filter(element => element.parent === node.url);

  // if the node has children, iterate (recursive)
  if (node.children.length != 0) {
    for (let i = 0; i < node.children.length; ++i) {
      addChildNodes(node.children[i], elements);
      // delete the "parent" key to reduce the JSON payload
      delete node.children[i].parent;
    }
  } else {
    // delete the "children" key to reduce the JSON payload
    delete node.children;
  }
}

// find the element that represents the home page
let root = data.nav.elements.find((item) => item.url === '/');

// if a root node exists
if (root) {
  // filter the elements to those in the specified categories
  // (or all elements, if no categories were specified)
  const elements = data.categories
    ? data.nav.elements.filter(check => data.categories.includes(check.category), data.categories)
    : data.nav.elements;

  // add a temporary property to each node to identify its parent
  for (let i = 0; i < elements.length; ++i) {
    const index = elements[i].url.lastIndexOf('/');

    if (index > 0) {
      elements[i].parent = elements[i].url.substring(0, index);
    } else if (elements[i].url != "/") {
      elements[i].parent = "/";
    }
  }

  // recurse from the root down
  addChildNodes(root, elements);
}

// return the entire tree as the response of this component
root;

Here is the result of invoking this code with the Context and sample data provided previously.

{
  "navTitle": "Home",
  "url": "/",
  "children": [
    {
      "navTitle": "About Us",
      "category": "about",
      "url": "/about",
      "children": [
        {
          "navTitle": "Jobs",
          "category": "about",
          "url": "/about/jobs",
          "children": [
            {
              "navTitle": "Engineering Jobs",
              "category": "about",
              "url": "/about/jobs/engineering"
            }
          ]
        }
      ]
    },
    {
      "navTitle": "Products",
      "category": "products",
      "url": "/products"
    }
  ]
}

References​