Overview

Relevant source files

TDengine is a distributed time-series database designed for high-performance data ingestion, storage, and real-time analytics. This document provides an overview of the system architecture, core components, and their interactions within the codebase.

The system supports SQL queries, schemaless data ingestion, stream processing, and distributed cluster management. For specific implementation details of individual components, see the architecture section (2) and subsystem-specific pages (3 through 10).

System Purpose and Capabilities

TDengine serves as a complete time-series data platform with the following primary functions:

High-Performance Data Ingestion: Supports both SQL and schemaless protocols (InfluxDB, OpenTSDB, JSON)
Distributed Storage: Automatic data partitioning and replication across cluster nodes
Real-Time Analytics: SQL-based queries with time-window functions and aggregations
Stream Processing: Continuous query processing and materialized aggregations
Message Queuing: Built-in TMQ (TDengine Message Queue) for data streaming

High-Level Architecture

Sources: source/libs/parser/src/parTranslater.c1-100 source/libs/executor/src/scanoperator.c1-50 source/dnode/vnode/src/tq/tq.c1-50 include/common/tmsg.h1-100 source/common/src/tglobal.c1-50

Core Component Architecture

The system implements a layered architecture where each component has specific responsibilities:

Sources: source/libs/planner/src/planOptimizer.c1-50 source/libs/executor/src/executil.c1-50 source/dnode/vnode/src/tsdb/tsdbCommit.c1-50 source/client/src/clientMain.c1-50 source/dnode/mnode/impl/src/mndDnode.c1-50

Data Flow Architecture

TDengine processes data through multiple pathways depending on the operation type:

Sources: source/client/src/clientImpl.c291-340 source/libs/parser/src/parTranslater.c421-500 source/dnode/vnode/src/tq/tq.c380-464 source/dnode/vnode/src/tsdb/tsdbRead2.c1-100

Key System Components

Management Node (MNode)

The management node handles cluster-wide metadata and coordination:

Database Management: Schema definitions, table metadata in mndDb.c and mndStb.c
Node Registration: Data node lifecycle management in mndDnode.c
Transaction Coordination: Distributed transaction management in mndTrans.c
Cluster State: Maintains global cluster state and configurations

Virtual Node (VNode)

Virtual nodes handle data storage and query processing:

Query Execution: SQL query processing through executor.c and operator implementations
Storage Engine: Time-series data storage via TSDB components (tsdb.h, tsdbCache.c)
Data Ingestion: Write path processing and WAL management
Local Metadata: Table and column metadata for assigned data partitions

Stream Node (SNode)

Stream nodes provide real-time data processing capabilities:

Topic Queue Management: Message streaming through tq.c implementation
Stream Processing: Continuous query execution and state management
Consumer Management: TMQ subscriber coordination and message delivery

Error Handling System

The system implements comprehensive error management:

Error Codes: Standardized error definitions in taoserror.h with over 500 error types
Error Context: Thread-local error state management in terror.c
Error Propagation: Consistent error handling across all system components

Sources: source/dnode/mnode/impl/inc/mndDef.h1-100 source/dnode/vnode/src/inc/tsdb.h1-200 include/util/taoserror.h1-100 source/util/src/terror.c24-50

Configuration and Global State

TDengine maintains extensive configuration management through a centralized system:

Global Configuration: System-wide parameters managed in tglobal.c
Runtime Parameters: Dynamic configuration updates and cluster coordination
Storage Configuration: Data directory, compression, and retention settings
Network Configuration: RPC transport, endpoint management, and security settings

The configuration system supports over 200 parameters covering performance tuning, resource allocation, security, and operational settings.

Sources: source/common/src/tglobal.c36-100 source/common/src/tglobal.c400-500