Skip to content

WhatsApp Voice Note Processing

This workflow processes WhatsApp voice messages by transcribing them to text and providing intelligent responses through AI assistants. It handles both small and large audio files with automatic chunking for optimal transcription quality, then routes the transcribed content to specialized AI agents for contextual responses.

Purpose

No business context provided yet — add a context.md to enrich this documentation.

How It Works

  1. Webhook Reception: Receives WhatsApp voice message webhooks from Twilio
  2. Message Analysis: Extracts metadata and determines if the message contains audio
  3. Audio Download: Downloads the voice note file from Twilio's media URL
  4. File Size Analysis: Analyzes audio file size and determines processing strategy:
    • Small files (< 2MB): Direct transcription
    • Medium files (2-10MB): Direct transcription
    • Large files (10-25MB): Chunked transcription with 1MB segments
    • Too large files (> 25MB): Error handling
  5. Transcription Processing:
    • Uses Google Gemini 2.0 Flash for direct transcription
    • For large files, splits into chunks, transcribes each, then merges results
  6. AI Response Generation: Routes transcribed text to specialized AI assistants:
    • SkillUp Support: For SkillUp Bootcamp-related queries
    • E!BA Support: For E!BA Academy program queries
  7. Response Delivery: Sends AI-generated response back via WhatsApp

Workflow Diagram

graph TD
    A[Webhook4 - WhatsApp Message] --> B[Message Type Detection]
    B -->|Voice Note| C[Extract Metadata]
    B -->|Text| D[Direct to AI Agent]
    C --> E[Download Audio File]
    E --> F[Analyze File Size]
    F -->|Small/Medium| G[Direct Transcription - Gemini]
    F -->|Large| H[Chunk Audio Files]
    F -->|Too Large| I[Error Response]
    H --> J[Transcribe Each Chunk]
    J --> K[Merge Transcriptions]
    G --> L[AI Agent Processing]
    K --> L
    D --> L
    L --> M[Text Cleanup]
    M --> N[Send WhatsApp Response]

Trigger

Webhook: Listens for POST requests from Twilio WhatsApp webhooks at the endpoint /33898c22-f5f0-45f0-9983-3fd19c2daebb

Nodes Used

Node Type Purpose
Webhook Receives WhatsApp messages from Twilio
Code (JavaScript) Extracts metadata, analyzes file sizes, processes chunks
HTTP Request Downloads audio files from Twilio media URLs
Switch Routes processing based on file size and message type
Google Gemini Transcribes audio files to text
AI Agent Provides intelligent responses using OpenAI models
Vector Store (PGVector) Retrieves relevant information from knowledge bases
Twilio Sends WhatsApp responses back to users
Write Binary File Temporarily stores audio chunks for processing
Function Item Collects and processes multiple transcription results

External Services & Credentials Required

Required Credentials:

  • Twilio API: For receiving webhooks and sending WhatsApp messages
  • Google Gemini API: For audio transcription services
  • OpenAI API: For AI chat models and embeddings
  • PostgreSQL: For vector storage and retrieval
  • Cohere API: For result reranking in vector searches

HTTP Basic Auth:

  • Twilio Media Access: For downloading audio files from Twilio's media URLs

Environment Variables

No specific environment variables are documented in the workflow configuration.

Data Flow

Input:

  • WhatsApp voice messages via Twilio webhook
  • Text messages via Twilio webhook
  • Audio files in OGG format

Processing:

  • Audio file metadata extraction
  • File size analysis and chunking strategy
  • Audio transcription to text
  • Vector-based knowledge retrieval
  • AI-powered response generation

Output:

  • WhatsApp text responses
  • Transcribed audio content
  • Error messages for failed processing

Error Handling

The workflow includes several error handling mechanisms:

  1. File Size Limits: Files over 25MB are rejected with error messages
  2. Download Failures: HTTP request errors are handled with retry logic
  3. Transcription Errors: Failed transcriptions trigger fallback error responses
  4. Chunking Failures: Individual chunk failures are logged and processed
  5. AI Agent Errors: Failed AI responses trigger generic error messages

Known Limitations

Based on the workflow structure: - Maximum audio file size of 25MB - Chunking limited to 1MB segments for large files - Dependency on external services (Twilio, Google, OpenAI) for core functionality - No offline processing capability - Limited to WhatsApp as the messaging platform

No related workflows are explicitly referenced in the current configuration.

Setup Instructions

  1. Import Workflow: Import the JSON configuration into your n8n instance

  2. Configure Credentials:

    • Set up Twilio API credentials for webhook reception and message sending
    • Configure Google Gemini API access for transcription
    • Set up OpenAI API credentials for AI agents
    • Configure PostgreSQL connection for vector storage
    • Add Cohere API credentials for reranking

  3. Set Up Webhook:

    • Configure Twilio WhatsApp webhook to point to your n8n webhook URL
    • Ensure webhook path matches: /33898c22-f5f0-45f0-9983-3fd19c2daebb

  4. Prepare Vector Stores:

    • Set up PostgreSQL with PGVector extension
    • Create tables for SkillUp and E!BA knowledge bases
    • Upload relevant documents for AI context

  5. Test Configuration:

    • Send a test voice message to verify transcription
    • Send a test text message to verify AI responses
    • Verify error handling with oversized files

  6. Activate Workflow: Enable the workflow to start processing WhatsApp messages