Batch Microservice Workflows

Conscia Recipes are designed to be intentionally open-ended, providing a robust starting point for building custom workflows. However, just as Components can come togther to form more complex Flows or populate bespoke experiences, Recipes can be built using others as a foundation.

For this Recipe, we will merge Batch Data Operations with a Sub Component and Polling with a Microservice. Instead of running a simple calculator function in the Sub Component flow (foreach), we will be communicating with an AI Large Language Model (LLM) using a microservice.

We are taking an existing model for executing work, an existing tool or function, and marrying one into the other to serve a more complex use case - reusing more than half of the previous work, and creating only a handful of new Components. This is a perfect example of why implementing projects with Conscia reduces implementation time by 75%, since each thoughtfully-implemented 'atom' can be used in a variety of 'molecules'! It's the DRYest way to compose digital experiences and orchestrate API-first systems.

Prework

This Recipe presumes that you have completed both of the above Recipes in a single Environment. If you have not, please do so now; don't worry about instantiating the Calculator Component.

Mapping Out DX Engine Elements

This recipe, as it stands, is not designed to supply a digital experience in real-time. Instead, it showcases Conscia's usefulness as an intermediary data management suite, building modular low-code functions to streamline operations and maximize business user workflows.

No context is required as input. The output is the result of each POST operation back to Airtable, to show the work being performed by an AI Large Language Model.

Airtable Configuration Details

Create Airtable

Visit airtable.com, hit +Create, and start a new Base. Then create the following four Tables:

Table Name	Simple Fields
Commands to Send to LLM	call_id - Autonumber, primary field initial_message - Single line text
Responses from LLM	llm_called_date - Date, primary field, Include time response - Long text
Assistants	assistant_name - Single line text, primary field description - Long text
Connections	Name - Single line text, primary field

Establish Airtable Relationships

Now we can create relationships between the tables.

Command-Response Relationship

This can be originated from either table (creating a 1:1 link creates a reciprocal Field). Regardless of the first field we add, we need the following fields present each Table:

Table Name	Relationship Fields
Commands to Send to LLM	llm_called_date - Link to Responses from LLM response - Lookup, source llm_called_date, looking up response
Responses from LLM	call_id - Link to Commands to Send to LLM initial_message - Lookup, source call_id, looking up initial_message

Command-Assistant Relationship

This can be originated from either table (creating a 1:1 link creates a reciprocal Field). Regardless of the first field we add, we need the following fields present each Table:

Table Name	Relationship Fields
Commands to Send to LLM	assistant_name - Link to Assistants assistant_id - Lookup, source assistant_name, looking up assistant_id
Assistants	Commands to Send to LLM - Link to Commands to Send to LLM

Command-Connections Relationship

This can be originated from either table (creating a 1:1 link creates a reciprocal Field). Regardless of the first field we add, we need the following fields present each Table:

Table Name	Relationship Fields
Commands to Send to LLM	relayConnectionID - Link to Connections relayConnection - Lookup, source relayConnectionID, looking up Name
Connections	Commands to Send to LLM - Link to Commands to Send to LLM

Now we have linked Tables. Each row in Commands to Send to LLM has an Assistant and Connection specified and an initial message; the recipe will run those requests to the LLM, and create rows in Responses from LLM that link to Commands to Send to LLM and show the output from the specified Assistant.

Populate Initial Rows

Create a row in Assistants for the Assistant you created in Polling with a Microservice, Succinct Jimmy, or any other Assistant you are using. Create a row in Connections for the Connection you created in Polling with a Microservice, relay-openai.

DX Engine Configuration Details

The topics in this section explain how to implement the elements involved in this recipe.

Context Fields

We need to create a Context Field for keeping track of an individual Airtable record.

1. In the top navigation, click Settings, and then click Context Fields. The Manage Context Fields page appears.
2. Click Add Context Field. The Create Context Field wizard appears.
3. For Context Field Name, enter airtableRecord.
4. For Data Type, select String.
5. Click Submit.

Secrets

As previous recipes.

Connections

Airtable Connection

We must instantiate our new Airtable in Conscia. You'll need to retrieve the Base ID (the first segment of the URL, it starts with app) and the Table's ID (the next segment of the URL, it starts with viw).

• Navigate to the Connections page (Settings --> Connections).
• Duplicate the airtable Connection, Edit it to modify it, or click the + Add Connection button.
• Change or enter the following and click Submit:

Field	Value
Connection Code	relay-airtable
Connection Name	Relay Airtable
Connector	Universal API Connector
Base URL	Get value from: Literal https://api.airtable.com/v0/ + the Base ID recorded above + /
Base Headers	Header: Authorization Value: Get value from: JS Expression "Bearer " + secret('airtable')
Base Headers	Header: content-type Value: Get value from: Literal application/json

Components

We will continue to use Relay: Rules and Relay: Rules Transform unmodified.

Component to Retrieve LLM Commands from Airtable

This Component replaces get-airtable-equations.

• Navigate to the Experience Components page (Manage Experiences --> Components).
• On the get-airtable-equations row, click the Duplicate button.
• Modify the following and click Submit.

Field	Value
Component Code	relay-get-llm-commands-airtable
Component Name	Relay: GET LLM commands (Airtable)
Webservice Path	Get value from: Literal the Table ID retained above
Query Parameters	Parameter: filterByFormula Get value from: Literal - NOT({llm_called_date})

Component to Convert Airtable Records to Relay Inputs

In each Sub Component run, we will have an AirtableRecord Context Field provided that needs to be broken down into Context Fields for consumption by the Relay Components we're already using.

• Navigate to the Experience Components page (Manage Experiences --> Components).
• Click the + Add Component button.
• Enter the following and click Submit.

Field	Value
Component Code	airtable-record-to-relay
Component Name	Relay: Airtable Record to Relay
No Rules	Checked
Component Type	Conscia - Property Mapper
Component Description	Converts an Airtable record into Relay context fields.
Default Source	Get value from: Context Field airtableRecord
Default Expression Type	JavaScript

• Enter the following Property Mappings:

Target property	Source expression
assistantID	data.fields.assistant_id? data.fields.assistant_id[0] : null
relayConnection	data.fields.relayConnection[0]
messageContent	data.fields.initial_message
callID	data.id

• Finally, establish these mappings as Context Fields:

Context Field	Expression
assistantID	response.assistantID
relayConnection	response.relayConnection
messageContent	response.messageContent
callID	response.callID

Component to Run Relay as a Sub Component

This Component replaces calculator as the work performer in each Sub Component run. Start with relay to reduce data entry. While we still ingest relay-rules-transform, we need to layer in additional data from the Airtable record.

• Navigate to the Experience Components page (Manage Experiences --> Components).
• On the relay row, click the Duplicate button.
• Enter or modify the following and click Submit.

Field	Value
Component Code	relay-sub
Component Name	Relay: Run Relay as a Sub Component
Description	Run Relay N times, as part of a Sub Component flow.
Body	({ ...componentResponse('relay-rules-transform'), "apiBody1": { "assistant_id": contextField('assistantID'), "thread": { "messages": [{ "role": "user", "content": contextField('messageContent') }] } } })

Component to Store Solutions in Airtable

• Navigate to the Experience Components page (Manage Experiences --> Components).
• Click the + Add Component button.
• Enter the following and click Submit.

Field	Value
Component Code	post-to-airtable
Component Name	POST to Airtable
No Rules	Checked
Component Type	Conscia - Universal API Connector
Connection	Airtable
Webservice Path	Get value from: Literal Output
Method	POST
Body	{ "records": [ { "fields": { "Calculation_Date": DateTime.now().toUTC(), "Equation_ID": [contextField('airtableRecord').id], "Solution": componentResponse('calculator') } } ] }

Sub Component Execution

As before, the Relay: GET LLM commands (Airtable) Component must query Airtable and, for each record which passes the filter criteria, execute the Relay: POST to Airtable Component, which will call the others as dependencies. Instantiate Relay: POST to Airtable as a Sub Component of Relay: GET LLM commands (Airtable), making it a Parent Component:

• Navigate to the Experience Components page (Manage Experiences --> Components).
• Click the Relay: GET LLM commands (Airtable) Component.
• Navigate to the Sub Components tab.
• Enter the following and click Submit.

Field	Value
Sub Components	Add one entry.
Property Name	llm_runs
Component Codes	Add one entry: relay-post-to-airtable
Context Field for Sub Component	Add one entry: Context Field: airtableRecord Expression: response

As response is an array of Table rows, the Sub Component will be invoked response.length times and each execution of relay-post-to-airtable will receive one entry as airtableRecord.

After populating a few sample prompts, successful invocation of the Relay: GET LLM commands (Airtable) Component will produce a result like the following in Airtable:

References

• Airtable API Reference
• OpenAI API Reference
• Axios
• Express
• Cloud Run quickstarts