Skip to content

Commit e8e63ae

Browse files
bhoustonbhouston
committed
docs: Add message compaction to docs website
- Added message-compaction.md to packages/docs/docs/usage - Updated usage index to include message compaction - Added compactHistory tool to the tools table
1 parent 6276bc0 commit e8e63ae

File tree

2 files changed

+113
-0
lines changed

2 files changed

+113
-0
lines changed

‎packages/docs/docs/usage/index.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,9 +147,11 @@ MyCoder has access to a variety of tools that enable it to perform complex tasks
147147
| **sessionMessage** | Performs actions in an active browser | Navigating websites, extracting information |
148148
| **agentStart** | Starts a sub-agent and returns immediately | Creating asynchronous specialized agents for parallel tasks |
149149
| **agentMessage** | Interacts with a running sub-agent | Checking status, providing guidance, or terminating sub-agents |
150+
| **compactHistory** | Summarizes older messages to reduce token usage | Managing context window for long-running agents |
150151

151152
For more detailed information about specific features, check the following pages:
152153

153154
- [Configuration Options](./configuration)
154155
- [GitHub Mode](./github-mode)
155156
- [Performance Profiling](./performance-profiling)
157+
- [Message Compaction](./message-compaction)

‎packages/docs/docs/usage/message-compaction.md

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
---
2+
sidebar_position: 8
3+
---
4+
5+
# Message Compaction
6+
7+
When agents run for extended periods, they accumulate a large history of messages that eventually fills up the LLM's context window, causing errors when the token limit is exceeded. The message compaction feature helps prevent this by providing agents with awareness of their token usage and tools to manage their context window.
8+
9+
## How It Works
10+
11+
### Token Usage Tracking
12+
13+
MyCoder's LLM abstraction tracks and returns:
14+
- Total tokens used in the current completion request
15+
- Maximum allowed tokens for the model/provider
16+
17+
This information is used to monitor context window usage and trigger appropriate actions.
18+
19+
### Status Updates
20+
21+
Agents receive status updates with information about:
22+
- Current token usage and percentage of the maximum
23+
- Cost so far
24+
- Active sub-agents and their status
25+
- Active shell processes and their status
26+
- Active browser sessions and their status
27+
28+
Status updates are sent:
29+
1. Every 5 agent interactions (periodic updates)
30+
2. Whenever token usage exceeds 50% of the maximum (threshold-based updates)
31+
32+
Example status update:
33+
```
34+
--- STATUS UPDATE ---
35+
Token Usage: 45,235/100,000 (45%)
36+
Cost So Far: $0.23
37+
38+
Active Sub-Agents: 2
39+
- sa_12345: Analyzing project structure and dependencies
40+
- sa_67890: Implementing unit tests for compactHistory tool
41+
42+
Active Shell Processes: 3
43+
- sh_abcde: npm test
44+
- sh_fghij: npm run watch
45+
- sh_klmno: git status
46+
47+
Active Browser Sessions: 1
48+
- bs_12345: https://www.typescriptlang.org/docs/handbook/utility-types.html
49+
50+
Your token usage is high (45%). It is recommended to use the 'compactHistory' tool now to reduce context size.
51+
--- END STATUS ---
52+
```
53+
54+
### Message Compaction Tool
55+
56+
The `compactHistory` tool allows agents to compact their message history by summarizing older messages while preserving recent context. This tool:
57+
58+
1. Takes a parameter for how many recent messages to preserve unchanged
59+
2. Summarizes all older messages into a single, concise summary
60+
3. Replaces the original messages with the summary and preserved messages
61+
4. Reports on the reduction in context size
62+
63+
## Usage
64+
65+
Agents are instructed to monitor their token usage through status updates and use the `compactHistory` tool when token usage approaches 50% of the maximum:
66+
67+
```javascript
68+
// Example of agent using the compactHistory tool
69+
{
70+
name: "compactHistory",
71+
preserveRecentMessages: 10,
72+
customPrompt: "Focus on summarizing our key decisions and current tasks."
73+
}
74+
```
75+
76+
### Parameters
77+
78+
The `compactHistory` tool accepts the following parameters:
79+
80+
| Parameter | Type | Description | Default |
81+
|-----------|------|-------------|---------|
82+
| `preserveRecentMessages` | number | Number of recent messages to preserve unchanged | 10 |
83+
| `customPrompt` | string (optional) | Custom prompt for the summarization | Default compaction prompt |
84+
85+
## Benefits
86+
87+
- Prevents context window overflow errors
88+
- Maintains important context for agent operation
89+
- Enables longer-running agent sessions
90+
- Makes the system more robust for complex tasks
91+
- Gives agents self-awareness of resource usage
92+
93+
## Model Token Limits
94+
95+
MyCoder includes token limits for various models:
96+
97+
### Anthropic Models
98+
- claude-3-opus-20240229: 200,000 tokens
99+
- claude-3-sonnet-20240229: 200,000 tokens
100+
- claude-3-haiku-20240307: 200,000 tokens
101+
- claude-2.1: 100,000 tokens
102+
103+
### OpenAI Models
104+
- gpt-4o: 128,000 tokens
105+
- gpt-4-turbo: 128,000 tokens
106+
- gpt-3.5-turbo: 16,385 tokens
107+
108+
### Ollama Models
109+
- llama2: 4,096 tokens
110+
- mistral: 8,192 tokens
111+
- mixtral: 32,768 tokens

0 commit comments

Comments
 (0)