Skip to content

refactor: improve chatbot integrations #1508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
May 27, 2025

Conversation

gomessguii
Copy link
Contributor

@gomessguii gomessguii commented May 27, 2025

Refactored chatbot integrations to better leverage base classes and improve code reuse.

Summary by Sourcery

Refactor chatbot integrations to consolidate common logic in BaseChatbotService for session management, message sending, audio transcription and media handling, remove duplication across dify, n8n, openai, flowise, evoai and typebot modules, clean up DTOs and controllers to use base class implementations, update validation schemas, and introduce Flowise enablement toggle in configuration.

Bug Fixes:

  • Validate session presence before sending messages in N8nService
  • Check for duplicate OpenAI API keys and credential names in controller to avoid conflicts
  • Guard against empty messages by verifying content before invoking sendMessageWhatsApp

Enhancements:

  • Centralize audio transcription and image handling logic in BaseChatbotService to reduce per-service duplication
  • Remove custom sendMessageWhatsApp and createNewSession overrides in individual service classes
  • Add null checks and error logging for missing endpoints or sessions to prevent runtime failures
  • Standardize injection of OpenaiService dependency across all chatbot services
  • Extend environment config and introduce Flowise integration toggle

Chores:

  • Simplify controllers to rely on base class methods and apply only integration-specific validation
  • Clean up and trim DTO classes by removing redundant integration fields
  • Update JSON schemas for n8n and flowise with new properties and renamed fallback IDs
  • Remove obsolete telemetry calls scattered across services

This commit refactors the TypebotController and TypebotService to streamline the processTypebot method, aligning it with the base class pattern. It reduces complexity by consolidating parameters and improving readability. Additionally, it updates the TypebotDto and TypebotSettingDto to remove unused properties, enhancing code clarity and maintainability.
… management

This commit refactors the OpenAIController and OpenAIService to improve credential handling and error management. Key changes include:
- Added checks to prevent duplicate API keys and names during bot creation.
- Updated the getModels method to accept an optional credential ID for more flexible credential usage.
- Enhanced error handling with specific BadRequestException messages for better clarity.
- Removed unused methods and streamlined the speech-to-text functionality to utilize instance-specific settings.

These improvements enhance the maintainability and usability of the OpenAI integration.
This commit enhances the N8n integration by refining session management and validation logic. Key changes include:
- Added error handling for session creation failures in the BaseChatbotService.
- Removed unused methods and properties in N8nService and N8nDto to streamline the codebase.
- Updated N8n schema to enforce required fields and improve validation checks.
- Simplified message processing logic to utilize base class methods, enhancing maintainability.

These improvements contribute to a more robust and efficient N8n integration.
…alidation

This commit refines the Flowise integration by enhancing configuration management and validation logic. Key changes include:
- Reordered parameters in the FlowiseService constructor for consistency.
- Updated FlowiseController to utilize the configService for integration enablement checks.
- Simplified FlowiseDto and FlowiseSettingDto by removing unused properties.
- Enhanced validation logic in flowise.schema.ts to include new fields.
- Improved error handling in the createBot method to prevent duplicate entries.

These updates contribute to a more robust and maintainable Flowise integration.
…age handling

This commit refines the Evoai integration by updating the service and controller logic for better functionality and maintainability. Key changes include:
- Added the `openaiService` as a parameter in the EvoaiService constructor for improved dependency management.
- Enhanced the createBot method in EvoaiController to include EvoAI-specific validation and duplicate checks.
- Updated EvoaiDto and EvoaiSettingDto to remove unnecessary comments and add a fallback property.
- Refined the message processing logic in EvoaiService to handle audio messages more effectively and improve logging clarity.
- Adjusted the schema for Evoai settings to rename `evoaiIdFallback` to `botIdFallback` for better clarity.

These updates contribute to a more robust and maintainable Evoai integration.
…ge processing

This commit refines the Dify integration by updating the controller and service logic for better functionality and maintainability. Key changes include:
- Added Dify-specific validation in the createBot method to prevent duplicate entries.
- Simplified comments for clarity and removed unused methods in DifyController.
- Enhanced message processing in DifyService to handle audio messages more effectively and improve error handling.
- Updated DifyDto and DifySettingDto to streamline properties and improve clarity.

These updates contribute to a more robust and maintainable Dify integration.
Copy link
Contributor

sourcery-ai bot commented May 27, 2025

Reviewer's Guide

This PR refactors each chatbot integration to leverage the shared BaseChatbotService and consolidates repetitive logic—especially around audio transcription, message sending, session creation, controllers, DTOs, and validation—into unified implementations, improving code reuse and reducing boilerplate.

Sequence Diagram for Audio Message Processing via DifyService and OpenaiService

sequenceDiagram
    actor User
    participant ChatPlatform
    participant DifyService
    participant OpenaiService
    participant DifyAPI

    User->>ChatPlatform: Sends audio message
    ChatPlatform->>DifyService: Forwards audio message (msg, instance)
    DifyService->>OpenaiService: speechToText(msg, instance)
    OpenaiService->>OpenaiService: Download/process audio buffer
    OpenaiService->>DifyAPI: Request transcription (OpenAI Whisper)
    DifyAPI-->>OpenaiService: Return transcription
    OpenaiService-->>DifyService: Return transcription text
    DifyService->>DifyService: processedContent = "[audio] " + transcription
    DifyService->>DifyAPI: POST /chat-messages (query: processedContent)
    DifyAPI-->>DifyService: Bot response
    DifyService->>ChatPlatform: Send reply message
    ChatPlatform-->>User: Delivers reply
Loading

Sequence Diagram for Message Processing and Session Handling

sequenceDiagram
    actor User
    participant ChatPlatform
    participant ChatbotController
    participant BaseChatbotService
    participant SpecificBotService
    participant PrismaRepository
    participant ExternalBotAPI

    User->>ChatPlatform: Sends message
    ChatPlatform->>ChatbotController: routeMessage(instance, ...)
    ChatbotController->>BaseChatbotService: process(instance, remoteJid, bot, session, settings, content, ...)
    BaseChatbotService->>BaseChatbotService: Check/Get session
    alt No existing session
        BaseChatbotService->>BaseChatbotService: createNewSession(instanceInfo, sessionData, botType)
        BaseChatbotService->>PrismaRepository: Save new session
        PrismaRepository-->>BaseChatbotService: New session created
    end
    BaseChatbotService->>SpecificBotService: sendMessageToBot(instance, session, settings, bot, ...)
    SpecificBotService->>SpecificBotService: Prepare payload (e.g., transcribe audio if needed)
    SpecificBotService->>ExternalBotAPI: Send processed message
    ExternalBotAPI-->>SpecificBotService: Receive bot response
    SpecificBotService->>BaseChatbotService: sendMessageWhatsApp(instance, remoteJid, responseText, settings)
    BaseChatbotService->>ChatPlatform: Send reply to user
    ChatPlatform-->>User: Deliver reply
Loading

Entity Relationship Diagram for N8N Schema Updates

erDiagram
    n8nSchema {
        boolean enabled "Required"
        string description
        string webhookUrl "Required"
        string basicAuthUser
        string basicAuthPassword "Renamed from basicAuthPass"
        string triggerType "New, Required, Enum: all, keyword, none, advanced"
        string triggerOperator "New, Enum: equals, contains, ..."
        string triggerValue "New"
        integer expire "New"
        string keywordFinish "New"
        integer delayMessage "New"
        string unknownMessage "New"
        boolean listeningFromMe "New"
        boolean stopBotFromMe "New"
        boolean keepOpen "New"
        integer debounceTime "New"
        array ignoreJids "New"
        boolean splitMessages "New"
        integer timePerChar "New"
    }

    n8nSettingSchema {
        integer expire "Required"
        string keywordFinish "Required"
        integer delayMessage "Required"
        string unknownMessage "Required"
        boolean listeningFromMe "Required"
        boolean stopBotFromMe "Required"
        boolean keepOpen "Required"
        integer debounceTime "Required"
        array ignoreJids "Required"
        string botIdFallback "Renamed from n8nIdFallback"
        boolean splitMessages "Required"
        integer timePerChar "Required"
    }

    n8nStatusSchema {
        string remoteJid "Required"
        string status "Required, Enum: opened, closed, paused, delete (order changed)"
    }

    n8nIgnoreJidSchema {
        string remoteJid "Required"
        string action "Required, Enum: add, remove"
    }
    note "Shows key field changes and additions to N8N JSON schemas based on the diff."
Loading

Class Diagram for Chatbot Service Refactoring

classDiagram
    class BaseChatbotService {
        +String serviceName
        +ConfigService configService
        +PrismaRepository prismaRepository
        +WAMonitoringService waMonitor
        +process(instance, remoteJid, bot, session, settings, content, pushName, msg)
        +createNewSession(instanceInfo, sessionData, botType): IntegrationSession
        +sendMessageWhatsApp(instance, remoteJid, message, settings)
        +isAudioMessage(content): boolean
        +isImageMessage(content): boolean
        #abstract getBotType(): string
        #abstract sendMessageToBot(instance, session, settings, bot, remoteJid, pushName, content, msg): Promise~void~
    }
    class DifyService {
        +OpenaiService openaiService
        +constructor(waMonitor, prisma, config, openai)
        +getBotType(): string
        +sendMessageToBot(instance, session, settings, dify, remoteJid, pushName, content, msg): Promise~void~
        -createNewSession(instance, data) "Removed"
    }
    class N8nService {
        +OpenaiService openaiService
        +constructor(waMonitor, prisma, config, openai)
        +getBotType(): string
        +sendMessageToBot(instance, session, settings, n8n, remoteJid, pushName, content, msg): Promise~void~
        -createBot() "Removed"
        -findBots() "Removed"
        -fetchBot() "Removed"
        -updateBot() "Removed"
        -deleteBot() "Removed"
        -createNewSession() "Removed"
        -sendMessageWhatsApp() "Removed"
    }
    class OpenaiService {
        +constructor(waMonitor, prisma, config)
        +getBotType(): string
        +speechToText(msg, instance): Promise~string~ "Refactored"
        +sendMessageToBot(instance, session, settings, bot, remoteJid, pushName, content, msg): Promise~void~
        -createNewSession() "Removed"
    }
    class FlowiseService {
        +OpenaiService openaiService
        +constructor(waMonitor, prisma, config, openai) "Updated signature"
        +getBotType(): string
        +processBot(instance, remoteJid, bot, session, settings, content, pushName, msg) "Added"
        +sendMessageToBot(instance, session, settings, bot, remoteJid, pushName, content, msg): Promise~void~
    }
    class EvoaiService {
        +OpenaiService openaiService
        +constructor(waMonitor, prisma, config, openai) "Updated signature"
        +getBotType(): string
        +sendMessageToBot(instance, session, settings, evoai, remoteJid, pushName, content, msg): Promise~void~
        -process() "Removed"
        -createNewSession() "Removed"
    }
    BaseChatbotService <|-- DifyService
    BaseChatbotService <|-- N8nService
    BaseChatbotService <|-- OpenaiService
    BaseChatbotService <|-- FlowiseService
    BaseChatbotService <|-- EvoaiService
Loading

File-Level Changes

Change Details Files
Centralize audio transcription and request preprocessing
  • Remove per-service Whisper transcription code blocks
  • Replace with unified openaiService.speechToText(msg, instance) calls
  • Add endpoint and session null checks before processing
  • Use processedContent instead of raw content in payloads
src/api/integrations/chatbot/dify/services/dify.service.ts
src/api/integrations/chatbot/n8n/services/n8n.service.ts
src/api/integrations/chatbot/flowise/services/flowise.service.ts
src/api/integrations/chatbot/evoai/services/evoai.service.ts
src/api/integrations/chatbot/typebot/services/typebot.service.ts
Consolidate WhatsApp message sending via base method
  • Remove custom sendMessageWhatsApp in each service
  • Guard sendMessageWhatsApp calls with message presence checks
  • Rely on BaseChatbotService.sendMessageWhatsApp instead
src/api/integrations/chatbot/dify/services/dify.service.ts
src/api/integrations/chatbot/n8n/services/n8n.service.ts
src/api/integrations/chatbot/flowise/services/flowise.service.ts
src/api/integrations/chatbot/evoai/services/evoai.service.ts
Simplify controllers to use BaseChatbotController patterns
  • Remove redundant find/fetch/update/delete implementations
  • Override createBot only for integration-specific validation
  • Update processBot to async await and call service.process
src/api/integrations/chatbot/dify/controllers/dify.controller.ts
src/api/integrations/chatbot/n8n/controllers/n8n.controller.ts
src/api/integrations/chatbot/evoai/controllers/evoai.controller.ts
src/api/integrations/chatbot/flowise/controllers/flowise.controller.ts
src/api/integrations/chatbot/typebot/controllers/typebot.controller.ts
Refactor OpenaiService and controller
  • Simplify speechToText signature to (msg, instance)
  • Remove internal helpers and use repo for settings/creds
  • Adjust getModels to accept optional openaiCredsId
  • Use BadRequestException for validation errors
src/api/integrations/chatbot/openai/services/openai.service.ts
src/api/integrations/chatbot/openai/controllers/openai.controller.ts
Improve dependency injection and module wiring
  • Reorder constructor arguments for FlowiseService and EvoaiService
  • Update server.module to pass correct parameters
src/api/server.module.ts
Refine DTOs and JSON schema validation
  • Remove unused DTO fields and advanced properties
  • Simplify BaseChatbotDto extensions
  • Add isNotEmpty helper and tighten required fields
  • Rename/standardize schema property names
src/api/integrations/chatbot/flowise/dto/flowise.dto.ts
src/api/integrations/chatbot/n8n/dto/n8n.dto.ts
src/api/integrations/chatbot/typebot/dto/typebot.dto.ts
src/api/integrations/chatbot/dify/dto/dify.dto.ts
src/api/integrations/chatbot/evoai/dto/evoai.dto.ts
src/api/integrations/chatbot/flowise/validate/flowise.schema.ts
src/api/integrations/chatbot/n8n/validate/n8n.schema.ts
src/api/integrations/chatbot/evoai/validate/evoai.schema.ts
Enhance session handling in BaseChatbotService
  • Check createNewSession result before assigning session
  • Log errors on session creation failure
src/api/integrations/chatbot/base-chatbot.service.ts
Extend configuration for Flowise integration
  • Add Flowise type to Env and ConfigService
  • Expose FLOWISE_ENABLED env var
src/config/env.config.ts

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

…tbot controllers

This commit refines the Flowise and Typebot integrations by simplifying the integration enablement checks in their respective controllers. Key changes include:
- Consolidated the integration checks in the createBot method of FlowiseController and startBot method of TypebotController for improved readability.
- Removed unnecessary line breaks to enhance code clarity.

These updates contribute to a cleaner and more maintainable codebase for chatbot integrations.
This commit updates the asset references in the index.html file to point to new CSS and JS files. Key changes include:
- Replaced the old JavaScript file `index-mxi8bQ4k.js` with `index-D-oOjDYe.js`.
- Updated the CSS file reference from `index-DNOCacL_.css` to the new `index-CXH2BdD4.css`.
- Removed the old CSS and JS files to clean up the codebase.

These updates ensure that the application uses the latest styles and scripts, contributing to improved performance and maintainability.
…ot services

- Add YAML file loader to tsup.config.ts to fix build compilation errors
- Fix OpenAI speechToText method signature across all chatbot services
- Correct DifyService constructor parameter order in server.module.ts and channel.service.ts
- Add missing OpenAI service dependency injection to EvoaiService
- Standardize audio transcription logic in FlowiseService to match N8N implementation
- Fix speechToText calls in WhatsApp Baileys and Evolution channel services
- Ensure consistent error handling and parameter passing for audio processing

This resolves the "Cannot read properties of undefined" errors and ensures
all chatbot integrations (OpenAI, N8N, Flowise, EvoAI, Dify, EvolutionBot)
properly handle audio message transcription using OpenAI Whisper.
This commit refines the EvolutionBot integration by reordering constructor parameters for consistency and removing unused properties from the EvolutionBotDto and EvolutionBotSettingDto classes. Key changes include:
- Adjusted the parameter order in the EvolutionBotService constructor for improved clarity.
- Streamlined the EvolutionBotDto and EvolutionBotSettingDto by eliminating unnecessary fields.

These updates enhance the maintainability and readability of the EvolutionBot integration.
@DavidsonGomes DavidsonGomes merged commit 3500fbe into EvolutionAPI:develop May 27, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants