Skip to content

CICD-Sync-Deploy

This workflow automatically synchronizes n8n workflows from a GitHub repository to a live production n8n instance. It reads workflow JSON files from the main branch, compares them with existing workflows, and deploys updates or creates new workflows as needed while respecting an allowlist configuration.

Purpose

No business context provided yet — add a context.md to enrich this documentation.

This workflow serves as the deployment component of a CI/CD pipeline for n8n workflows. It ensures that workflow definitions stored in version control are automatically synchronized to the production environment, enabling infrastructure-as-code practices for workflow management.

How It Works

  1. Initialization: The workflow starts either manually or via hourly schedule, loading production environment configuration including API keys, repository details, and credential mappings.

  2. Allowlist Loading: Reads the deployment allowlist from cicd/sync-config.md in the GitHub repository to determine which workflows are permitted for deployment.

  3. Discovery: Lists all JSON files in the workflows/ directory of the GitHub repository and fetches the current list of workflows from the live n8n instance.

  4. Planning: Creates a deployment plan by matching GitHub workflow files with live workflows, checking allowlist permissions, and determining the required action for each workflow.

  5. Processing: Processes each workflow file individually:

    • Downloads the workflow JSON from GitHub
    • Parses and validates the content
    • Routes based on status (new deployment, update needed, blocked, or skipped)

  6. Comparison: For existing workflows, strips volatile fields and credentials, then compares normalized JSON to detect configuration drift.

  7. Deployment: Calls specialized sub-workflows to handle first-time deployments or updates while preserving live credential configurations.

  8. Summary: Aggregates results showing counts of synchronized, deployed, skipped, and blocked workflows.

Mermaid Diagram

graph TD
    A[Manual Trigger] --> C[Globals PROD]
    B[Hourly Sync] --> C
    C --> D[Read Allowlist sync-config.md]
    D --> E[Parse Allowlist]
    E --> F[GitHub List workflows/]
    F --> G[List Live n8n Workflows]
    G --> H[Build Plan]
    H --> I[OneAtATime]
    I --> J[GitHub File:Get]
    J --> K[Decode + Match]
    K --> L[Route]
    L --> M[GET Live Workflow]
    L --> N[Call: First Deploy]
    L --> O[NoOp skipped]
    L --> P[NoOp blocked]
    M --> Q[Compare strip credentials]
    Q --> R[Switch by sync_status]
    R --> S[NoOp in_sync]
    R --> T[Call: Redeploy Ritual]
    S --> I
    T --> I
    N --> I
    O --> I
    P --> I
    I --> U[Aggregate Summary]

Trigger

  • Manual Trigger: Can be started manually for immediate synchronization
  • Schedule Trigger: Runs automatically every hour (cron: 0 * * * *)

Nodes Used

Node Type Purpose
Manual Trigger Allows manual execution of the sync process
Schedule Trigger Runs hourly automated synchronization
Set (Globals PROD) Configures environment variables and settings
GitHub (List/Get) Retrieves workflow files and allowlist from repository
HTTP Request Communicates with n8n API to fetch live workflows
Code Processes data, compares workflows, and builds deployment plans
Switch Routes workflows based on their sync status
Split In Batches Processes workflows one at a time
Execute Workflow Calls sub-workflows for deployment operations
No Operation Handles skipped or blocked workflows

External Services & Credentials Required

GitHub API

  • Credential Name: cicd-github
  • Type: GitHub Personal Access Token
  • Permissions: Contents (Read/Write)
  • Purpose: Access repository workflow files and configuration

n8n API

  • Authentication: API Key (configured in Globals)
  • Endpoint: Production n8n instance
  • Purpose: List, create, and update workflows

Environment Variables

Configured in the "Globals (PROD)" node:

  • env_name: Environment identifier (prod)
  • env_url: Production n8n instance URL
  • api_key: n8n API key for authentication
  • repo_owner: GitHub repository owner
  • repo_name: Repository name containing workflows
  • repo_branch: Branch to sync from (main)
  • path_prefix: Directory containing workflow files (workflows/)
  • credential_lookup: JSON mapping of credential names to IDs
  • allowlist: Fallback list of permitted workflows

Data Flow

Input

  • GitHub repository containing workflow JSON files
  • Live n8n instance with existing workflows
  • Allowlist configuration from cicd/sync-config.md

Processing

  • Workflow comparison using normalized JSON (credentials stripped)
  • Credential ID mapping for cross-environment deployment
  • Allowlist filtering for security

Output

  • Synchronized workflows in production n8n
  • Deployment summary with counts by status
  • Error logs for blocked or failed deployments

Error Handling

  • Parse Errors: Workflows with invalid JSON are marked as "blocked"
  • API Failures: HTTP requests include retry logic (2-3 attempts)
  • Credential Issues: Missing credential mappings block deployment
  • Allowlist Violations: Unauthorized workflows are skipped
  • Sub-workflow Errors: Deployment failures are logged but don't stop the batch

Known Limitations

  • Credential mappings must be manually maintained in the lookup table
  • Large workflow batches may hit API rate limits
  • No rollback mechanism for failed deployments
  • Allowlist changes require repository access
  • CICD-Redeploy-Ritual (S1BQfSMseWbNcBna): Handles updates to existing workflows
  • CICD-First-Deploy (LI84TqALXx3VYLs3): Manages initial deployment of new workflows
  • CICD-Cron-Backup: Complementary workflow that backs up live workflows to repository

Setup Instructions

  1. Import Workflow: Import the JSON into your n8n instance

  2. Create GitHub Credential:

    • Create a new credential of type "GitHub API"
    • Name it cicd-github
    • Generate a Personal Access Token with Contents (Read/Write) permissions
    • Configure the credential in all GitHub nodes

  3. Configure Environment:

    • Update the "Globals (PROD)" node with your environment details
    • Set the correct n8n API URL and generate an API key
    • Update repository owner/name and branch as needed

  4. Set Up Credential Mapping:

    • Map credential names to their production IDs in credential_lookup
    • Ensure all workflow credentials have corresponding entries

  5. Configure Allowlist:

    • Create cicd/sync-config.md in your repository
    • Add a table listing permitted workflow names
    • Or use ["*"] in Globals for unrestricted access

  6. Deploy Sub-workflows:

    • Import and configure CICD-Redeploy-Ritual and CICD-First-Deploy
    • Update the Execute Workflow nodes with correct workflow IDs

  7. Test:

    • Run manually first to verify configuration
    • Check logs for any credential or permission issues
    • Activate the hourly schedule once confirmed working