Environment Management using Github Actions

This recipe provides simple building blocks for coordinating Conscia environments using version control.

More information on the underlying mechanics is available on the Environments page.

Work Required in Conscia

This recipe requires very little work on the Conscia side. To follow the recipe exactly, create two Environments on one Conscia instance named 'preview' and 'production'. It's more likely, however, that you will have these Conscia Environments on two separate servers, or three environments - or more! This recipe can accommodate; simply update the variables in the Github Actions appropriately.

We will, of course, need a Conscia API key to store in Github.

Work Required in Github

On Github, we will create our Actions, Scripts, Variables and Secrets in one repository. This repo can be used to manage other code as well, for instance this could be part of a broader orchestration codebase that deployed microservices originate from.

We have the following workflows and scripts created and available in Github.

Establish Secrets and Variables

In Github, go to /settings/personal-access-tokens/new and create a token with, at a minimum, contents: write and pull-requests: write scopes.

In your Github Repository, go to /[repo-name]/settings/secrets/actions and add the following two Secrets:

Name	Details
CONSCIA_TOKEN	Your DX Engine API token. If traceability of commits is a concern, the recipe can be modified to use per-user tokens.
GH_TOKEN	Your Github API token, as created above, for creating Github commits using Github Actions.

In your Github Repository, go to /[repo-name]/settings/variables/actions and add the following Variable:

Name	Details
CUSTOMER_CODE	The name of your Conscia instance.

Create Workflows

For this Recipe, we are providing three main Workflows:

• Get-preview-env.yml, which fetches a Conscia Environment config and commits it to your repository;
• Put-preview-env.yml, which imports a previously-fetched Conscia Environment into your instance; and
• Promote-preview-to-prod.yml, which fetches from preview and commits, pushes it to production, then fetches from production and commits to ensure your repository is fully up-to-date.

If you have more environments, environments with different names, et cetera then these Workflows can be duplicated with changes to environment variables and filenames.

Create get-preview-env.yml

In order, this Action:

• Checks out the repository.
• Stores the current date.
• Activates the get-env.sh script to retrieve a Conscia configuration from preview.
• Creates a Pull Request on this Repository to store that configuration.

Create the following file at .github/workflows/get-preview-env.yml:

name: Get Conscia Preview Environment

on:
  workflow_dispatch

env:
  ENVIRONMENT_CODE: 'preview'
  SRC_CONSCIA_SERVER: 'engine-staging'

jobs:
  get-config:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Get current date
        run: echo "DATE=$(date +'%Y-%m-%d')" >> $GITHUB_ENV

      - name: GET DX Engine Config from an Environment
        run: .github/scripts/get-env.sh
        env:
          ENVIRONMENT_CODE: ${{ env.ENVIRONMENT_CODE }}
          CUSTOMER_CODE: ${{ vars.CUSTOMER_CODE }}
          CONSCIA_TOKEN: ${{ secrets.CONSCIA_TOKEN }}
          CONSCIA_SERVER: ${{ env.SRC_CONSCIA_SERVER }}
          OUTPUT_FILE: ${{ env.ENVIRONMENT_CODE }}-env.json

      - name: Create Pull Request
        uses: peter-evans/create-pull-request@v6.1.0
        with:
          file-path: ${{ env.ENVIRONMENT_CODE }}-env.json
          branch: update-${{ env.ENVIRONMENT_CODE }}
          branch-suffix: timestamp
          title: Import of Conscia configuration from ${{ env.ENVIRONMENT_CODE }} on ${{ env.DATE }}
          body: The latest environment configuration was retrieved from ${{ env.SRC_CONSCIA_SERVER}}/${{ env.ENVIRONMENT_CODE }} and committed to this repository.
          commit-message: Retrieved updated environment from ${{ env.ENVIRONMENT_CODE }} on ${{ env.DATE }}.
          token: ${{ secrets.GH_TOKEN }}

Create put-preview-env.yml

In order, this Action:

• Checks out the repository.
• Activates the put-env.sh script, pushing the preview config to the preview environment.

Create the following file at .github/workflows/put-preview-env.yml:

name: Put Conscia Preview Environment

on:
  workflow_dispatch

env:
  ENVIRONMENT_CODE: 'preview'
  DEST_CONSCIA_SERVER: 'engine-staging'

jobs:
  put-config:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: PUT DX Engine Config back into an Environment
        run: .github/scripts/put-env.sh
        env:
          ENVIRONMENT_CODE: ${{ env.ENVIRONMENT_CODE }}
          CUSTOMER_CODE: ${{ vars.CUSTOMER_CODE }}
          CONSCIA_TOKEN: ${{ secrets.CONSCIA_TOKEN }}
          CONSCIA_SERVER: ${{ env.DEST_CONSCIA_SERVER }}
          INPUT_FILE: ${{ env.ENVIRONMENT_CODE }}-env.json

Create promote-preview-to-prod.yml

In order, this Action:

• Runs get-config on the preview environment, creates a PR, and stores the branch created.
• Runs put-config on the production environment using the preview config.
• Runs get-config on the production envrionment and creates a PR.

We get-config production because it typically has different Secrets and Environment Variables; we want to ensure that the exact snapshot of "Production as it stands today" includes the newly promoted Components, Templates, et cetera but also accurately mirrors the full state so we have a backup we can rollback to.

Create the following file at .github/workflows/promote-preview-to-prod.yml:

name: Promote Conscia Preview Environment to Production

on:
  workflow_dispatch

env:
  SRC_ENVIRONMENT_CODE: 'preview'
  SRC_CONSCIA_SERVER: 'engine-staging'

  DEST_ENVIRONMENT_CODE: 'production'
  DEST_CONSCIA_SERVER: 'engine-staging'

jobs:
  get-config-preview:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Get current date
        run: echo "DATE=$(date +'%Y-%m-%d')" >> $GITHUB_ENV

      - name: GET DX Engine Config from Preview Environment
        run: .github/scripts/get-env.sh
        env:
          ENVIRONMENT_CODE: ${{ env.SRC_ENVIRONMENT_CODE }}
          CUSTOMER_CODE: ${{ vars.CUSTOMER_CODE }}
          CONSCIA_TOKEN: ${{ secrets.CONSCIA_TOKEN }}
          CONSCIA_SERVER: ${{ env.SRC_CONSCIA_SERVER }}
          OUTPUT_FILE: ${{ env.SRC_ENVIRONMENT_CODE }}-env.json

      - name: Create Pull Request for Preview
        uses: peter-evans/create-pull-request@v6.1.0
        id: create_PR
        with:
          file-path: ${{ env.SRC_ENVIRONMENT_CODE }}-env.json
          branch: update-${{ env.SRC_ENVIRONMENT_CODE }}
          branch-suffix: timestamp
          title: Import of Conscia configuration from ${{ env.SRC_ENVIRONMENT_CODE }} on ${{ env.DATE }}
          body: The latest environment configuration was retrieved from ${{ env.SRC_CONSCIA_SERVER}}/${{ env.SRC_ENVIRONMENT_CODE }} and committed to this repository.
          commit-message: Retrieved updated environment from ${{ env.SRC_ENVIRONMENT_CODE }} on ${{ env.DATE }}.
          token: ${{ secrets.GH_TOKEN }}
    outputs:
      pull-request-branch: ${{ steps.create_PR.outputs.pull-request-branch }}

  put-config-production:
    runs-on: ubuntu-latest
    needs: get-config-preview
    steps:
      - name: Checkout code
        uses: actions/checkout@v4.1.7
        with:
          ref: ${{ needs.get-config-preview.outputs.pull-request-branch }}

      - name: PUT Preview DX Engine Config into Production
        run: .github/scripts/put-env.sh
        env:
          ENVIRONMENT_CODE: ${{ env.DEST_ENVIRONMENT_CODE }}
          CUSTOMER_CODE: ${{ vars.CUSTOMER_CODE }}
          CONSCIA_TOKEN: ${{ secrets.CONSCIA_TOKEN }}
          CONSCIA_SERVER: ${{ env.DEST_CONSCIA_SERVER }}
          INPUT_FILE: ${{ env.SRC_ENVIRONMENT_CODE }}-env.json

  commit-config-production:
    runs-on: ubuntu-latest
    needs: [get-config-preview, put-config-production]
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Get current date
        run: echo "DATE=$(date +'%Y-%m-%d')" >> $GITHUB_ENV

      - name: GET DX Engine Config from Production Environment
        run: .github/scripts/get-env.sh
        env:
          ENVIRONMENT_CODE: ${{ env.DEST_ENVIRONMENT_CODE }}
          CUSTOMER_CODE: ${{ vars.CUSTOMER_CODE }}
          CONSCIA_TOKEN: ${{ secrets.CONSCIA_TOKEN }}
          CONSCIA_SERVER: ${{ env.DEST_CONSCIA_SERVER }}
          OUTPUT_FILE: ${{ env.DEST_ENVIRONMENT_CODE }}-env.json

      - name: Create Pull Request for Production
        uses: peter-evans/create-pull-request@v6.1.0
        with:
          file-path: ${{ env.DEST_ENVIRONMENT_CODE }}-env.json
          branch: update-${{ env.DEST_ENVIRONMENT_CODE }}
          branch-suffix: timestamp
          title: Import of Conscia configuration from ${{ env.DEST_ENVIRONMENT_CODE }} on ${{ env.DATE }}
          body: The latest environment configuration was retrieved from ${{ env.DEST_CONSCIA_SERVER}}/${{ env.DEST_ENVIRONMENT_CODE }} and committed to this repository.
          commit-message: Retrieved updated environment from ${{ env.DEST_ENVIRONMENT_CODE }} on ${{ env.DATE }}.
          token: ${{ secrets.GH_TOKEN }}

Create Scripts

Two generic, all-purpose Scripts are used in a variety of ways by the Actions configured above: A "GET" for environments, and a "PUT" for environments.

Create get-env.sh

Create the following file at .github/scripts/get-env.sh:

#!/bin/bash

# Set the API endpoint URL
API_URL="https://$CONSCIA_SERVER.conscia.io/api/raw-engine-config"

# For it to be useful as a reimport, we will need to wrap the API results in an engineConfig property.
echo '{"engineConfig": ' > raw.json

# Call the API endpoint using curl and append the response to a file
curl -s -X GET $API_URL \
    -H "Content-Type: application/json" \
    -H "x-customer-code: $CUSTOMER_CODE" \
    -H "x-environment-code: $ENVIRONMENT_CODE" \
    -H "Authorization: Bearer $CONSCIA_TOKEN" \
    >> raw.json

# Complete the wrap.
echo '}' >> raw.json

# Finally, pretty-print the JSON and store it on multiple lines in the actual OUTPUT_FILE, and remove raw.json.
cat raw.json | json_pp > $OUTPUT_FILE
rm raw.json

# Confirm that the response has been written to the file
echo "Response written to $OUTPUT_FILE"

Create put-env.sh

Create the following file at .github/scripts/put-env.sh:

#!/bin/bash

# Set the API endpoint URL
API_URL="https://$CONSCIA_SERVER.conscia.io/api/engine-config?preserveEnvironmentVariables=true&preserveSecrets=true"

# Call the API endpoint using curl and write the response to a file
curl -s -X PUT $API_URL \
    -H "Content-Type: application/json" \
    -H "x-customer-code: $CUSTOMER_CODE" \
    -H "x-environment-code: $ENVIRONMENT_CODE" \
    -H "Authorization: Bearer $CONSCIA_TOKEN" \
    --data-binary "@$INPUT_FILE"

# Confirm that the response has been written to the file
echo "Response written to Conscia from $INPUT_FILE"

Note that this script uses both nondescructive flags, as detailed on the Environments page.

note

If you run into a "Permission denied" error when running the workflows, you will need to upgrade permissions on each of the shell scripts:

git update-index --chmod=+x <your_script>.sh

What's next?

Creating an overwrite-env.sh script and a restore-preview.yml Action which calls it is left as an exercise for the reader.

References

• Github Actions documentation
• DX Engine Environments
• The create-pull-request Github Action we leveraged to streamline the Actions