Skip to content

Commit e6324b8

Browse files
moremindmoremind
authored
[type:refactor] refactor benchmark test (apache#892)
* [type:refactor] refactor benchmark test * [type:refactor] refactor benchmark test * [type:refactor] refactor benchmark test * [type:refactor] refactor benchmark test * [type:refactor] refactor benchmark test * [type:refactor] refactor benchmark test
1 parent 086d722 commit e6324b8

File tree

2 files changed

+261
-160
lines changed

2 files changed

+261
-160
lines changed

‎docs/benchmark-test/benchmark-test.md

Lines changed: 133 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -34,109 +34,160 @@ wrk-4.2.0
3434

3535
- Use the Mock service to simulate an interface with an average response time of 20ms and about 2k response messages
3636
- Each test lasts 3 minutes
37+
- JDK version: OpenJdk-1.8.0
3738
- The HTTP request side is tested with `NettyClient` and `WebClient` respectively
39+
- Log level is set to `WARN`
40+
- Apache ShenYu Bootstrap deploy mode: standalone
41+
- Apache ShenYu Admin deploy in `Back-end Mock Service Server`
3842

3943
### JVM Configuration
4044

41-
- -Xmx 4g
42-
- -Xms 4g
43-
- -Xmn 1g
44-
- -Xss 512k
45-
- -XX: +DisableExplicitGC
46-
- -XX: LargePageSizeInBytes=128m
45+
```
46+
-Xmx 4g
47+
-Xms 4g
48+
-Xmn 1g
49+
-Xss 512k
50+
-XX: +DisableExplicitGC
51+
-XX: LargePageSizeInBytes=128m
52+
```
4753

4854
### Public Configuration
4955

5056
```yml
5157
matchCache:
52-
selector:
58+
selector:
5359
selectorEnabled: false
5460
initialCapacity: 10000 # initial capacity in cache
5561
maximumSize: 10000 # max size in cache
56-
rule:
62+
rule:
5763
initialCapacity: 10000 # initial capacity in cache
5864
maximumSize: 65536 # max size in cache
5965
trie:
60-
enabled: true
61-
childrenSize: 10000
62-
pathVariableSize: 1000
63-
pathRuleCacheSize: 1000
64-
matchMode: antPathMatch
66+
enabled: true
67+
childrenSize: 10000
68+
pathVariableSize: 1000
69+
pathRuleCacheSize: 1000
70+
matchMode: antPathMatch
6571
```
6672
67-
### Netty Configuration
68-
6973
```yml
7074
netty:
71-
http:
72-
# set to false, user can custom the netty tcp server config.
73-
webServerFactoryEnabled: true
74-
selectCount: 1
75-
workerCount: 8
76-
accessLog: false
77-
serverSocketChannel:
78-
soRcvBuf: 87380
79-
soBackLog: 128
80-
soReuseAddr: false
81-
connectTimeoutMillis: 10000
82-
writeBufferHighWaterMark: 65536
83-
writeBufferLowWaterMark: 32768
84-
writeSpinCount: 16
85-
autoRead: false
86-
allocType: "pooled"
87-
messageSizeEstimator: 8
88-
singleEventExecutorPerGroup: true
89-
socketChannel:
90-
soKeepAlive: false
91-
soReuseAddr: false
92-
soLinger: -1
93-
tcpNoDelay: true
94-
soRcvBuf: 87380
95-
soSndBuf: 16384
96-
ipTos: 0
97-
allowHalfClosure: false
98-
connectTimeoutMillis: 10000
99-
writeBufferHighWaterMark: 65536
100-
writeBufferLowWaterMark: 32768
101-
writeSpinCount: 16
102-
autoRead: false
103-
allocType: "pooled"
104-
messageSizeEstimator: 8
105-
singleEventExecutorPerGroup: true
75+
http:
76+
# set to false, user can custom the netty tcp server config.
77+
webServerFactoryEnabled: true
78+
selectCount: 1
79+
workerCount: 8
80+
accessLog: false
81+
serverSocketChannel:
82+
soRcvBuf: 87380
83+
soBackLog: 128
84+
soReuseAddr: false
85+
connectTimeoutMillis: 10000
86+
writeBufferHighWaterMark: 65536
87+
writeBufferLowWaterMark: 32768
88+
writeSpinCount: 16
89+
autoRead: false
90+
allocType: "pooled"
91+
messageSizeEstimator: 8
92+
singleEventExecutorPerGroup: true
93+
socketChannel:
94+
soKeepAlive: false
95+
soReuseAddr: false
96+
soLinger: -1
97+
tcpNoDelay: true
98+
soRcvBuf: 87380
99+
soSndBuf: 16384
100+
ipTos: 0
101+
allowHalfClosure: false
102+
connectTimeoutMillis: 10000
103+
writeBufferHighWaterMark: 65536
104+
writeBufferLowWaterMark: 32768
105+
writeSpinCount: 16
106+
autoRead: false
107+
allocType: "pooled"
108+
messageSizeEstimator: 8
109+
singleEventExecutorPerGroup: true
106110
```
107111
108112
### WebClient Configuration
109113
110114
```yml
111115
httpclient:
112-
strategy: webClient # netty
113-
connectTimeout: 45000 # 45000
114-
responseTimeout: 3000 # 3000
115-
readerIdleTime: 3000 # 3000
116-
writerIdleTime: 3000 # 3000
117-
allIdleTime: 3000 # 3000
118-
readTimeout: 3000 # 3000
119-
writeTimeout: 3000 # 3000
120-
wiretap: false # false
121-
keepAlive: false # false
122-
maxInMemorySize: 1 # 1
123-
pool:
124-
type: ELASTIC # ELASTIC
125-
name: proxy # proxy
126-
maxConnections: 16 # 16
127-
acquireTimeout: 45000 # 45000
128-
maxIdleTime: 3000 # 3000
116+
strategy: webClient # netty
117+
connectTimeout: 45000 # 45000
118+
responseTimeout: 3000 # 3000
119+
readerIdleTime: 3000 # 3000
120+
writerIdleTime: 3000 # 3000
121+
allIdleTime: 3000 # 3000
122+
readTimeout: 3000 # 3000
123+
writeTimeout: 3000 # 3000
124+
wiretap: false # false
125+
keepAlive: false # false
126+
maxInMemorySize: 1 # 1
127+
pool:
128+
type: ELASTIC # ELASTIC
129+
name: proxy # proxy
130+
maxConnections: 16 # 16
131+
acquireTimeout: 45000 # 45000
132+
maxIdleTime: 3000 # 3000
133+
```
134+
135+
### Netty Client Configuration
136+
137+
```yml
138+
httpclient:
139+
strategy: netty # netty
140+
connectTimeout: 45000 # 45000
141+
responseTimeout: 3000 # 3000
142+
readerIdleTime: 3000 # 3000
143+
writerIdleTime: 3000 # 3000
144+
allIdleTime: 3000 # 3000
145+
readTimeout: 3000 # 3000
146+
writeTimeout: 3000 # 3000
147+
wiretap: false # false
148+
keepAlive: false # false
149+
maxInMemorySize: 1 # 1
150+
pool:
151+
type: ELASTIC # ELASTIC
152+
name: proxy # proxy
153+
maxConnections: 16 # 16
154+
acquireTimeout: 45000 # 45000
155+
maxIdleTime: 3000 # 3000
129156
```
130157
131158
## Benchmark Test Results
132159
160+
* Direct Access to Back-end Test Result
161+
162+
| **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **平均响应时间(ms)** | **最大响应时间(ms)** |
163+
|:---------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:----------------:|:----------------:|
164+
| 28998.20 | 19.81 | 23.78 | 28.26 | 41.24 | 20.92 | 402.90 |
165+
166+
* netty
167+
168+
| currency | QPS | 50% latency (ms) | 75% latency (ms) | 90% latency (ms) | 99% latency (ms) | 平均响应时间(ms) | 最大响应时间(ms) |
169+
|:-------------:|:----------:|:------------------:|:------------------:|:------------------:|:------------------:|:------------:|:------------:|
170+
| 600 currency | 20472.95 | 19.37 | 25.36 | 32.89 | 69.92 | 22.09 | 1043.33 |
171+
| 800 currency | 20703.55 | 23.57 | 31.32 | 40.11 | 77.28 | 26.11 | 576.47 |
172+
| 1000 currency | 20979.91 | 29.21 | 37.86 | 47.23 | 80.91 | 31.20 | 860.55 |
173+
| 1200 currency | 21129.88 | 32.45 | 42.40 | 52.68 | 96.10 | 35.06 | 1070 |
174+
175+
* webClient
176+
177+
| currency | QPS | 50% latency (ms) | 75% latency (ms) | 90% latency (ms) | 99% latency (ms) | 平均响应时间(ms) | 最大响应时间(ms) |
178+
|:--------:|:----------:|:------------------:|:------------------:|:------------------:|:------------------:|:------------:|:------------:|
179+
| 600 currency | 18640.47 | 15.77 | 24.77 | 38.26 | 80.31 | 20.32 | 852.06 |
180+
| 800 currency | 18723.44 | 18.12 | 28.69 | 44.96 | 95.3 | 23.52 | 765.26 |
181+
| 1000 currency | 18928.99 | 19.99 | 31.42 | 49.09 | 108.84 | 25.93 | 1040 |
182+
| 1200 currency | 18965.37 | 22.10 | 34.62 | 54.48 | 122.31 | 28.66 | 1075 |
183+
133184
### Direct Access to Back-end Test Result
134185
135186
#### Test Result
136187
137-
| **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **Avg response time (ms)** | **Max response time (ms)** |
138-
|:---------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:----------------:|:----------------:|
139-
| 28998.20 | 19.81 | 23.78 | 28.26 | 41.24 | 20.92 | 402.90 |
188+
| **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **Avg response time (ms)** | **Max response time (ms)** |
189+
|:---------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:----------------:|:----------------:|
190+
| 28998.20 | 19.81 | 23.78 | 28.26 | 41.24 | 20.92 | 402.90 |
140191
141192
#### Screenshot of Test Result
142193
@@ -146,12 +197,12 @@ httpclient:
146197
147198
##### Test Result
148199
149-
| | **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **Avg response time (ms)** | **Max response time (ms)** |
150-
|:-----------:|:--------:|:--------------------:|:--------------------:|:--------------------:|:--------------------:|:--------------:|:--------------:|
151-
| 600 concurrency | 20472.95 | 19.37 | 25.36 | 32.89 | 69.92 | 22.09 | 1043.33 |
152-
| 800 concurrency | 20703.55 | 23.57 | 31.32 | 40.11 | 77.28 | 26.11 | 576.47 |
153-
| 1000 concurrency | 20979.91 | 29.21 | 37.86 | 47.23 | 80.91 | 31.20 | 860.55 |
154-
| 1200 concurrency | 21129.88 | 32.45 | 42.40 | 52.68 | 96.10 | 35.06 | 1070 |
200+
| currency | **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **Avg response time (ms)** | **Max response time (ms)** |
201+
|:-----------:|:--------:|:--------------------:|:--------------------:|:--------------------:|:--------------------:|:--------------:|:--------------:|
202+
| 600 concurrency | 20472.95 | 19.37 | 25.36 | 32.89 | 69.92 | 22.09 | 1043.33 |
203+
| 800 concurrency | 20703.55 | 23.57 | 31.32 | 40.11 | 77.28 | 26.11 | 576.47 |
204+
| 1000 concurrency | 20979.91 | 29.21 | 37.86 | 47.23 | 80.91 | 31.20 | 860.55 |
205+
| 1200 concurrency | 21129.88 | 32.45 | 42.40 | 52.68 | 96.10 | 35.06 | 1070 |
155206
156207
#### Screenshot of Test Result
157208
@@ -183,12 +234,12 @@ httpclient:
183234
184235
#### Test Result
185236
186-
| | **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **Avg response time (ms)** | **Max response time (ms)** |
187-
|:-----------:|:--------:|:--------------------:|:--------------------:|:--------------------:|:--------------------:|:--------------:|:--------------:|
188-
| 600 concurrency | 18640.47 | 15.77 | 24.77 | 38.26 | 80.31 | 20.32 | 852.06 |
189-
| 800 concurrency | 18723.44 | 18.12 | 28.69 | 44.96 | 95.3 | 23.52 | 765.26 |
190-
| 1000 concurrency | 18928.99 | 19.99 | 31.42 | 49.09 | 108.84 | 25.93 | 1040 |
191-
| 1200 concurrency | 18965.37 | 22.10 | 34.62 | 54.48 | 122.31 | 28.66 | 1075 |
237+
| currency | **QPS** | **50% latency (ms)** | **75% latency (ms)** | **90% latency (ms)** | **99% latency (ms)** | **Avg response time (ms)** | **Max response time (ms)** |
238+
|:-----------:|:--------:|:--------------------:|:--------------------:|:--------------------:|:--------------------:|:--------------:|:--------------:|
239+
| 600 concurrency | 18640.47 | 15.77 | 24.77 | 38.26 | 80.31 | 20.32 | 852.06 |
240+
| 800 concurrency | 18723.44 | 18.12 | 28.69 | 44.96 | 95.3 | 23.52 | 765.26 |
241+
| 1000 concurrency | 18928.99 | 19.99 | 31.42 | 49.09 | 108.84 | 25.93 | 1040 |
242+
| 1200 concurrency | 18965.37 | 22.10 | 34.62 | 54.48 | 122.31 | 28.66 | 1075 |
192243
193244
#### Screenshot of Test Result
194245

0 commit comments

Comments
 (0)