Batch Data Operations with a Sub Component

This straightforward recipe demonstrates a core Conscia behaviour - interacting with an API endpoint, and executing a task per-record against the results. In this example, we will be reading and writing from an Airtable, an innovative spreadsheet-type collaboration tool. We will read from a Table and for each unserved row, perform a unit of work on that input in Conscia, then write the output to another Table.

Airtable accounts are free, and no premium capabilities are required for this recipe.

The architectural approach demonstrated here is valuable for iterative extract, transform and load from a web service; catching up to a queue asynchronously; and many more common applications.

Mapping Out DX Engine Elements

This recipe, as it stands, is not designed to supply a digital experience in real-time. Instead, it showcases Conscia's usefulness as an intermediary data management suite, building modular low-code functions to streamline operations and maximize business user workflows.

No context is required as input. The output is the result of each POST operation back to Airtable, to validate the success or failure of these efforts.

DX Engine Configuration Details

The topics in this section explain how to implement the elements involved in this recipe.

Secrets

Airtable Secret

First, visit the Airtable Builder Hub and create a personal access token with, at a minimum, data.records:read and data.records:write permissions.

Then, to create a Secret in Conscia used to store the Airtable token, in the DX Engine UI:

• Navigate to the Secrets page (Settings --> Secrets).
• Click the + Add Secret button.
• Enter the following and click Submit:

Field	Value
Secret Code	airtable
Secret Name	Airtable
Secret Value	Enter your Airtable token.

Connections

Configure and Connect to Airtable

Visit airtable.com, hit +Create, and start a new Base. Then create the following two Tables:

Table Name	Simple Fields
Input	Equation_ID - Single line text, primary field Equation - Single line text
Output	Calculation_Date - Date, primary field, Include time Solution - Single line text

Then we can create a relationship between the two tables. This can be originated from either table (creating a 1:1 link from Output to Input creates the reciprocal Field in Input for Output). Regardless of the first field we add, we need the following two fields present each Table:

Table Name	Relationship Fields
Input	Calculation_Date - Link to Output Solution (from Output) - Lookup, source Calculation_Date, Output field Solution
Output	Equation_ID - Link to Input Equation (from Input) - Lookup, source Equation_ID, Output field Equation

Now we have two linked Tables. Rows in Input can be created with an ID and an equation each; the recipe will execute the calculations, and create rows in Output that link to the Input and solve the equation.

The final requirement is to instantiate Airtable in Conscia. You'll need to record the Base ID (the first segment of the URL, it starts with app) and the Table's ID (the next segment of the URL, it starts with viw).

• Navigate to the Connections page (Settings --> Connections).
• Click the + Add Connection button.
• Enter the following and click Submit:

Field	Value
Connection Code	airtable
Connection Name	Airtable
Connector	Universal API Connector
Base URL	Get value from: Literal https://api.airtable.com/v0/ + the Base ID recorded above + /
Base Headers	Header: Authorization Value: Get value from: JS Expression "Bearer " + secret('airtable')
Base Headers	Header: content-type Value: Get value from: Literal application/json

Components

Component to Retrieve Equations from Airtable

• Navigate to the Experience Components page (Manage Experiences --> Components).
• Click the + Add Component button.
• Enter the following and click Submit.

Field	Value
Component Code	get-airtable-equations
Component Name	GET Equations from Airtable
No Rules	Checked
Component Type	Conscia - Universal API Connector
Connection	Airtable
Webservice Path	Get value from: Literal the Table ID recorded above
Method	GET
Query Parameters	Parameter: filterByFormula Get value from: Literal - NOT({{Calculation_Date}})
Response Transform	response.records.length == 0 ? null : response.records

Component to Evaluate Equations

A variety of techniques and approaches would serve; we'll use a Data Transformation Script to execute a unit of work on each row retrieved.

• Navigate to the Experience Components page (Manage Experiences --> Components).
• Click the + Add Component button.
• Enter the following and click Submit.

Field	Value
Component Code	calculator
Component Name	Calculator
No Rules	Checked
Component Type	Conscia - Data Transformation Script
Data to modify	Get value from: JS Expression (contextField('airtableRecord').fields.Equation)
Script	var resultString; try { const result = eval(data); // Evaluate the equation resultString = result.toString(); // Convert the result to a string and return... } catch (error) { resultString = "Error: Invalid equation: " + error.toString(); } const ret = resultString;

Component to Store Solutions in Airtable

• Navigate to the Experience Components page (Manage Experiences --> Components).
• Click the + Add Component button.
• Enter the following and click Submit.

Field	Value
Component Code	post-to-airtable
Component Name	POST to Airtable
No Rules	Checked
Component Type	Conscia - Universal API Connector
Connection	Airtable
Webservice Path	Get value from: Literal Output
Method	POST
Body	{ "records": [ { "fields": { "Calculation_Date": DateTime.now().toUTC(), "Equation_ID": [contextField('airtableRecord').id], "Solution": componentResponse('calculator') } } ] }

Sub Component Execution

Now, in order to complete the recipe, the Airtable GET Component must query Airtable and, for each record which passes the filter criteria, execute both the Calculator and the POST to Airtable Component. As the Calculator is a required input to POST to Airtable, we instantiate the POST to Airtable as a Sub Component of Airtable GET, making it a Parent Component:

• Navigate to the Experience Components page (Manage Experiences --> Components).
• Click the Airtable GET Component.
• Navigate to the Sub Components tab.
• Enter the following and click Submit.

Field	Value
Sub Components	Add one entry.
Property Name	llm_runs
Component Codes	Add one entry: post-to-airtable
Context Field for Sub Component	Add one entry: Context Field: airtableRecord Expression: response

As response is an array of Table rows, the Sub Component will be invoked response.length times and each execution of post-to-airtable will receive one entry as airtableRecord.

After populating a few sample equations, successful invocation of the Airtable GET Component will produce a result like the following in Airtable:

References

• Airtable API Reference
• Common-sense critique of using eval()