Skip to content

Commit ab8375f

Browse files
zuobiao-zhouzuobiao-zhou
authored
Add ShenYu benchmark test report (apache#888)
* add ShenYu benchmark test report * add ShenYu benchmark test report
1 parent 4b575db commit ab8375f

27 files changed

+424
-0
lines changed

‎docs/benchmark-test/_category_.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"label": "Benchmark Test",
3+
"position": 8
4+
}

‎docs/benchmark-test/benchmark-test.md

Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
---
2+
sidebar_position: 8
3+
title: ShenYu Benchmark Test Report
4+
keywords: ["test", "benchmark-test"]
5+
description: ShenYu Benchmark Test Report
6+
---
7+
8+
### Hardware
9+
10+
Back-end Mock Service Server:
11+
12+
- CPU: 4 cores and 8 threads Intel Cascade Lake @ 3.0GHz
13+
- RAM: 16G
14+
15+
Gateway node server:
16+
17+
- CPU: 4 cores and 8 threads Intel Cascade Lake @ 3.0GHz
18+
- RAM: 16G
19+
20+
The test tool takes up few resources and is installed on the gateway node server.
21+
22+
### ShenYu Version
23+
24+
- ShenYu Admin: 2.6.0
25+
- ShenYu Bootstrap: 2.6.0
26+
27+
### Test Tool
28+
29+
wrk-4.2.0
30+
31+
### Test Case Description
32+
33+
- Use the Mock service to simulate an interface with an average response time of 20ms and about 2k response messages
34+
- Each test lasts 3 minutes
35+
- The HTTP request side is tested with `NettyClient` and `WebClient` respectively
36+
- JVM Configuration：
37+
- -Xmx 4g
38+
- -Xms 4g
39+
- -Xmn 1g
40+
- -Xss 512k
41+
- -XX: +DisableExplicitGC
42+
- -XX: LargePageSizeInBytes=128m
43+
- shenyu-bootstrap 配置：
44+
45+
```yml
46+
matchCache:
47+
selector:
48+
selectorEnabled: false
49+
initialCapacity: 10000 # initial capacity in cache
50+
maximumSize: 10000 # max size in cache
51+
rule:
52+
initialCapacity: 10000 # initial capacity in cache
53+
maximumSize: 65536 # max size in cache
54+
trie:
55+
enabled: true
56+
childrenSize: 10000
57+
pathVariableSize: 1000
58+
pathRuleCacheSize: 1000
59+
matchMode: antPathMatch
60+
```
61+
62+
- Netty 配置
63+
64+
```yml
65+
netty:
66+
http:
67+
# set to false, user can custom the netty tcp server config.
68+
webServerFactoryEnabled: true
69+
selectCount: 1
70+
workerCount: 8
71+
accessLog: false
72+
serverSocketChannel:
73+
soRcvBuf: 87380
74+
soBackLog: 128
75+
soReuseAddr: false
76+
connectTimeoutMillis: 10000
77+
writeBufferHighWaterMark: 65536
78+
writeBufferLowWaterMark: 32768
79+
writeSpinCount: 16
80+
autoRead: false
81+
allocType: "pooled"
82+
messageSizeEstimator: 8
83+
singleEventExecutorPerGroup: true
84+
socketChannel:
85+
soKeepAlive: false
86+
soReuseAddr: false
87+
soLinger: -1
88+
tcpNoDelay: true
89+
soRcvBuf: 87380
90+
soSndBuf: 16384
91+
ipTos: 0
92+
allowHalfClosure: false
93+
connectTimeoutMillis: 10000
94+
writeBufferHighWaterMark: 65536
95+
writeBufferLowWaterMark: 32768
96+
writeSpinCount: 16
97+
autoRead: false
98+
allocType: "pooled"
99+
messageSizeEstimator: 8
100+
singleEventExecutorPerGroup: true
101+
```
102+
103+
- HttpClient 配置
104+
105+
```yml
106+
httpclient:
107+
strategy: webClient # netty
108+
connectTimeout: 45000
109+
responseTimeout: 3000
110+
readerIdleTime: 3000
111+
writerIdleTime: 3000
112+
allIdleTime: 3000
113+
readTimeout: 3000
114+
writeTimeout: 3000
115+
wiretap: false
116+
keepAlive: false
117+
maxInMemorySize: 1 #1mb
118+
pool:
119+
type: ELASTIC
120+
name: proxy
121+
maxConnections: 16
122+
acquireTimeout: 45000
123+
maxIdleTime: 3000
124+
```
125+
126+
### Benchmark Test Results
127+
128+
- `Direct` access to back-end services under 600 concurrency
129+
130+
| **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **Avg response time (ms)** | **Max response time (ms)** |
131+
|:---------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:----------------:|:----------------:|
132+
| 28998.20 | 19.81 | 23.78 | 28.26 | 41.24 | 20.92 | 402.90 |
133+
134+
- Access to back-end services via `NettyClient` under 600, 800, 1000 concurrency
135+
136+
| | **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **Avg response time (ms)** | **Max response time (ms)** |
137+
|:-----------:|:--------:|:--------------------:|:--------------------:|:--------------------:|:--------------------:|:--------------:|:--------------:|
138+
| 600 concurrency | 20472.95 | 19.37 | 25.36 | 32.89 | 69.92 | 22.09 | 1043.33 |
139+
| 800 concurrency | 20703.55 | 23.57 | 31.32 | 40.11 | 77.28 | 26.11 | 576.47 |
140+
| 1000 concurrency | 20979.91 | 29.21 | 37.86 | 47.23 | 80.91 | 31.20 | 860.55 |
141+
| 1200 concurrency | 21129.88 | 32.45 | 42.40 | 52.68 | 96.10 | 35.06 | 1070 |
142+
143+
- Access to back-end services via `HttpClient` under 600, 800, 1000 concurrency
144+
145+
| | **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **Avg response time (ms)** | **Max response time (ms)** |
146+
|:-----------:|:--------:|:--------------------:|:--------------------:|:--------------------:|:--------------------:|:--------------:|:--------------:|
147+
| 600 concurrency | 18640.47 | 15.77 | 24.77 | 38.26 | 80.31 | 20.32 | 852.06 |
148+
| 800 concurrency | 18723.44 | 18.12 | 28.69 | 44.96 | 95.3 | 23.52 | 765.26 |
149+
| 1000 concurrency | 18928.99 | 19.99 | 31.42 | 49.09 | 108.84 | 25.93 | 1040 |
150+
| 1200 concurrency | 18965.37 | 22.10 | 34.62 | 54.48 | 122.31 | 28.66 | 1075 |
151+
152+
### Screenshot of Benchmark Test Result
153+
154+
- `Direct` access to back-end services
155+
156+
<img src="/img/shenyu/benchmark-test/origin/1.png" width="100%" height="100%" />
157+
158+
- Access to back-end services via `NettyClient`
159+
- 600 concurrency
160+
161+
<img src="/img/shenyu/benchmark-test/netty/600-currency/8-thread/1-netty-cache-selector.png" width="100%" height="100%" />
162+
<img src="/img/shenyu/benchmark-test/netty/600-currency/8-thread/2-netty-cache-selector.png" width="100%" height="100%" />
163+
<img src="/img/shenyu/benchmark-test/netty/600-currency/8-thread/3-netty-cache-selector.png" width="100%" height="100%" />
164+
165+
- 800 concurrency
166+
167+
<img src="/img/shenyu/benchmark-test/netty/800-currency/8-thread/1-netty-cache-selector.jpg" width="100%" height="100%" />
168+
<img src="/img/shenyu/benchmark-test/netty/800-currency/8-thread/2-netty-cache-selector.jpg" width="100%" height="100%" />
169+
<img src="/img/shenyu/benchmark-test/netty/800-currency/8-thread/3-netty-cache-selector.jpg" width="100%" height="100%" />
170+
171+
- 1000 concurrency
172+
173+
<img src="/img/shenyu/benchmark-test/netty/1000-currency/8-thread/1-netty-cache-selector.jpg" width="100%" height="100%" />
174+
<img src="/img/shenyu/benchmark-test/netty/1000-currency/8-thread/2-netty-cache-selector.jpg" width="100%" height="100%" />
175+
<img src="/img/shenyu/benchmark-test/netty/1000-currency/8-thread/3-netty-cache-selector.jpg" width="100%" height="100%" />
176+
177+
- 1200 concurrency
178+
179+
<img src="/img/shenyu/benchmark-test/netty/1200-currency/8-thread/1-netty-cache-selector.jpg" width="100%" height="100%" />
180+
<img src="/img/shenyu/benchmark-test/netty/1200-currency/8-thread/2-netty-cache-selector.jpg" width="100%" height="100%" />
181+
<img src="/img/shenyu/benchmark-test/netty/1200-currency/8-thread/3-netty-cache-selector.jpg" width="100%" height="100%" />
182+
183+
- Access to back-end services via `HttpClient`
184+
- 600 concurrency
185+
186+
<img src="/img/shenyu/benchmark-test/http/600-currency/8-thread/1-netty-cache-selector.jpg" width="100%" height="100%" />
187+
<img src="/img/shenyu/benchmark-test/http/600-currency/8-thread/2-netty-cache-selector.jpg" width="100%" height="100%" />
188+
<img src="/img/shenyu/benchmark-test/http/600-currency/8-thread/3-netty-cache-selector.jpg" width="100%" height="100%" />
189+
190+
- 800 concurrency
191+
192+
<img src="/img/shenyu/benchmark-test/http/800-currency/8-thread/1-netty-cache-selector.jpg" width="100%" height="100%" />
193+
<img src="/img/shenyu/benchmark-test/http/800-currency/8-thread/2-netty-cache-selector.jpg" width="100%" height="100%" />
194+
195+
- 1000 concurrency
196+
197+
<img src="/img/shenyu/benchmark-test/http/1000-currency/8-thread/1-netty-cache-selector.jpg" width="100%" height="100%" />
198+
<img src="/img/shenyu/benchmark-test/http/1000-currency/8-thread/2-netty-cache-selector.jpg" width="100%" height="100%" />
199+
200+
- 1200 concurrency
201+
202+
<img src="/img/shenyu/benchmark-test/http/1200-currency/8-thread/1-netty-cache-selector.jpg" width="100%" height="100%" />
203+
<img src="/img/shenyu/benchmark-test/http/1200-currency/8-thread/2-netty-cache-selector.jpg" width="100%" height="100%" />
204+

‎i18n/zh/docusaurus-plugin-content-docs/current.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,5 +70,12 @@
7070
"sidebar.tutorialSidebar.category.SPI": {
7171
"message": "SPI",
7272
"description": "The label for category SPI in sidebar tutorialSidebar"
73+
},
74+
"sidebar.tutorialSidebar.category.Benchmark Test": {
75+
76+
"message": "基准测试",
77+
78+
"description": "ShenYu Benchmark Test"
79+
7380
}
7481
}

‎i18n/zh/docusaurus-plugin-content-docs/current/benchmark-test/_category_.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"label": "基准测试",
3+
"position": 8
4+
}

0 commit comments

Comments
 (0)