CICD-Sync-Deploy

A continuous integration/continuous deployment workflow that synchronizes n8n workflows from a GitHub repository to a live n8n instance. This workflow pulls workflow JSON files from the main branch and enforces them onto the production environment, ensuring the live system stays in sync with the source code repository.

Purpose

No business context provided yet — add a context.md to enrich this documentation.

This workflow serves as the deployment pipeline for n8n workflow management, automatically keeping production workflows synchronized with their source code versions stored in GitHub. It operates as the reverse of a backup system, pushing changes from the repository to the live environment rather than pulling them.

How It Works

1. Trigger Activation: The workflow starts either manually or via an hourly scheduled trigger
2. Environment Setup: Global configuration is loaded, including API keys, repository details, and deployment settings
3. Repository Scan: Lists all JSON files in the workflows/ directory of the specified GitHub repository
4. Live System Query: Retrieves the current list of workflows from the target n8n instance
5. Planning Phase: Builds a deployment plan by matching repository files with live workflows
6. File Processing: For each workflow file, the system:
   • Downloads the JSON content from GitHub
   • Parses and validates the workflow definition
   • Checks against the allowlist to determine if deployment is permitted
7. Routing Decision: Based on the workflow state, routes to one of four paths:
   • Compare: For existing workflows, fetches live version and compares
   • First Deploy: For new workflows not present in the live system
   • Skip: For workflows not in the allowlist
   • Block: For workflows with parsing errors
8. Sync Analysis: Strips credentials and volatile fields, then compares normalized JSON to detect drift
9. Deployment Action: Executes appropriate deployment ritual (redeploy for drift, first deploy for new workflows)
10. Summary Generation: Aggregates results across all processed workflows

Workflow Diagram

graph TD
    A[Manual Trigger] --> C[Globals PROD]
    B[Hourly Sync] --> C
    C --> D[GitHub List workflows/]
    D --> E[List Live n8n Workflows]
    E --> F[Build Plan]
    F --> G[OneAtATime]
    G --> H[Aggregate Summary]
    G --> I[GitHub File:Get]
    I --> J[Decode + Match]
    J --> K[Route]
    K --> L[GET Live Workflow]
    K --> M[Call: First Deploy]
    K --> N[NoOp skipped]
    K --> O[NoOp blocked]
    L --> P[Compare strip credentials]
    P --> Q[Switch by sync_status]
    Q --> R[NoOp in_sync]
    Q --> S[Call: Redeploy Ritual]
    M --> G
    N --> G
    O --> G
    R --> G
    S --> G

Trigger

• Manual Trigger: Can be started manually for on-demand deployments
• Schedule Trigger: Runs automatically every hour (cron: 0 * * * *) to maintain continuous synchronization

Nodes Used

Node Type	Purpose
Manual Trigger	Allows manual workflow execution
Schedule Trigger	Automated hourly execution
Set (Globals)	Configures environment variables and settings
GitHub	Lists repository files and retrieves workflow content
HTTP Request	Communicates with n8n API for workflow management
Code	Custom logic for comparison, routing, and data processing
Split in Batches	Processes workflows one at a time
Switch	Routes workflows based on their deployment status
Execute Workflow	Calls sub-workflows for deployment operations
NoOp	Placeholder for workflows that require no action

External Services & Credentials Required

GitHub Integration

• Credential Type: GitHub API Token
• Required Permissions: Contents read access to the target repository
• Usage: Repository file listing and content retrieval

n8n API Access

• Credential Type: API Key
• Required Permissions: Workflow read/write access
• Usage: Querying existing workflows and deploying updates

Sub-workflow Dependencies

• CICD-Redeploy-Ritual (ID: S1BQfSMseWbNcBna): Handles redeployment of existing workflows
• CICD-First-Deploy (ID: LI84TqALXx3VYLs3): Manages initial deployment of new workflows

Environment Variables

The workflow uses these key configuration variables:

• env_name: Environment identifier (prod)
• env_url: Target n8n instance URL
• api_key: n8n API authentication key
• repo_owner: GitHub repository owner
• repo_name: GitHub repository name
• repo_branch: Source branch (main)
• path_prefix: Directory path for workflow files (workflows/)
• credential_lookup: JSON mapping of credential names to IDs
• allowlist: Array of workflow names permitted for deployment

Data Flow

Input: - GitHub repository containing workflow JSON files - Live n8n instance with existing workflows

Processing: - Workflow definitions are normalized by stripping credentials and volatile fields - Comparison logic determines sync status (in_sync, drift_redeploy, first_deploy, etc.) - Credential mapping ensures proper credential assignment during deployment

Output: - Synchronized workflows in the target n8n environment - Deployment summary with counts by status category - Error logs for blocked or failed deployments

Error Handling

• GitHub API Failures: Nodes continue with regular output on errors, allowing partial processing
• Parse Errors: Workflows with invalid JSON are routed to blocked status
• Deployment Failures: Sub-workflow calls continue on error to prevent cascade failures
• Retry Logic: HTTP requests include retry configuration (2-3 attempts with delays)

Known Limitations

• Allowlist Restriction: Currently limited to ["[Weekly Report] 9PM Timeout Cleanup"] - other workflows are skipped
• Single Environment: Only production lane is implemented; staging and development lanes are pending
• Credential Dependencies: Deployment can be blocked if credential name mapping is unresolvable
• Manual Credential Setup: Requires manual creation of GitHub credentials before activation

Related Workflows

• CICD-Redeploy-Ritual: Handles redeployment of existing workflows with drift
• CICD-First-Deploy: Manages initial deployment of new workflows
• CICD-Cron-Backup: Complementary workflow that backs up live workflows to repository

Setup Instructions

1. Import Workflow: Import the JSON definition into your n8n instance

2. Configure GitHub Credentials:

   • Create a GitHub API credential named "cicd-github"
   • Ensure the token has Contents read/write permissions
   • Apply this credential to both GitHub nodes

3. Update Global Configuration:

   • Modify the "Globals (PROD)" node with your environment details
   • Set correct env_url and api_key for your target n8n instance
   • Update repo_owner and repo_name for your GitHub repository
   • Configure credential_lookup JSON with your credential mappings

4. Verify Sub-workflows:

   • Ensure CICD-Redeploy-Ritual and CICD-First-Deploy workflows exist
   • Update workflow IDs in the Execute Workflow nodes if necessary

5. Test Deployment:

   • Start with manual trigger to verify configuration
   • Check allowlist settings match your intended workflows
   • Monitor execution logs for any credential or API issues

6. Activate Automation:

   • Enable the workflow to activate hourly synchronization
   • Monitor initial runs to ensure stable operation