Skip to content

Commit d9a3ea6

Browse files
zkl94zkl94
committed
new README.md
1 parent 3a54f59 commit d9a3ea6

File tree

6 files changed

+389
-138
lines changed

6 files changed

+389
-138
lines changed

‎README.md

Lines changed: 241 additions & 138 deletions
Original file line numberDiff line numberDiff line change
@@ -1,140 +1,243 @@
1-
shadowsocks
2-
===========
1+
# shadowsocks源码分析
2+
3+
## 项目结构：
4+
###### 1、asyndns.py 用于处理dns请求
5+
###### 2、common.py
6+
###### 3、daemon.py，提供daemon运行机制
7+
###### 4、encrypt，处理shadowsocks协议的加密解密
8+
###### 5、eventloop，事件循环，使用select、poll、epoll、kequeue实现IO复用，作者讲三种底层实现包装成一个类Eventloop
9+
###### 6、local，在本地运行的程序
10+
###### 7、lru_cache.py，作者实现的一个基于LRU的缓存
11+
###### 8、server.py，在远程运行的程序
12+
###### 9、tcprelay，实现tcp的转达，用在远程端中使远程和dest连接
13+
###### 10、udprelay，实现udp的转达，用于local端处理local和客户端的socks5协议通信，用于local端和远程端shadowsocks协议的通信；用于远程端与local端shadowsocks协议的通信，用于远程端和dest端的通信
14+
###### 11、utils.py
15+
16+
> 代码质量相当的高，感觉都能达到重用的级别。而且由于作者设计的思想是，一个配置文件，同一段程序，在本地和远程通用，所以其中的代码，常常能够达到一个函数，在本地和服务器有不同的功能这样的效果。
17+
18+
===============================================================
19+
## 核心：eventloop.py，udprelay.py，tcprelay.py，asyndns.py
20+
eventloop使用select、epoll、kqueue等IO复用实现异步处理。优先级为epoll\>kqueue\>select。Eventloop将三种复用机制的add，remove，poll，add_handler，remve_handler接口统一起来，程序员只需要使用这些函数即可，不需要处理底层细节。
21+
22+
后三个文件分别实现用来处理udp的请求，tcp的请求，dns的查询请求，并且将三种请求的处理包装成handler。对于tcp，udp的handler，它们bind到特定的端口，并且将socket交给eventloop，并且将自己的处理函数加到eventloop的handlers；对于dns的handler，它接受来自udp handler和tcp handler的dns查询请求，并且向远程dns服务器发出udp请求；
23+
24+
当eventloop监测到socket的数据，程序就将所有监测到的socket和事件交给所有handler去处理，每个handler通过socket和事件判断自己是否要处理该事件，并进行相对的处理：
25+
##### 当local收到udprelay handler绑定的端口的事件，说明客户端发来请求，local对SOCKS5协议的内容进行处理之后经过加密转发给远程；
26+
27+
<pre>
28+
+----+------+------+----------+----------+----------+
29+
|RSV | FRAG | ATYP | DST.ADDR | DST.PORT | DATA |
30+
+----+------+------+----------+----------+----------+
31+
| 2 | 1 | 1 | Variable | 2 | Variable |
32+
+----+------+------+----------+----------+----------+
33+
</pre>
34+
35+
trim-\>
36+
<pre>
37+
+------+----------+----------+----------+
38+
| ATYP | DST.ADDR | DST.PORT | DATA |
39+
+------+----------+----------+----------+
40+
| 1 | Variable | 2 | Variable |
41+
+------+----------+----------+----------+
42+
</pre>
43+
44+
-\>encrypt
45+
<pre>
46+
+-------+--------------+
47+
| IV | PAYLOAD |
48+
+-------+--------------+
49+
| Fixed | Variable |
50+
+-------+--------------+
51+
</pre>
52+
53+
54+
##### 当local新建的socket收到连接请求时，说明远程向local发送结果，此时对信息进行解密，并且对shadowsocks协议进行适当加工，发回给客户端
55+
56+
<pre>
57+
+-------+--------------+
58+
| IV | PAYLOAD |
59+
+-------+--------------+
60+
| Fixed | Variable |
61+
+-------+--------------+
62+
</pre>
63+
64+
-\>decrypt
65+
66+
<pre>
67+
+------+----------+----------+----------+
68+
| ATYP | DST.ADDR | DST.PORT | DATA |
69+
+------+----------+----------+----------+
70+
| 1 | Variable | 2 | Variable |
71+
+------+----------+----------+----------+
72+
</pre>
73+
74+
-\>add
75+
76+
<pre>
77+
+----+------+------+----------+----------+----------+
78+
|RSV | FRAG | ATYP | DST.ADDR | DST.PORT | DATA |
79+
+----+------+------+----------+----------+----------+
80+
| 2 | 1 | 1 | Variable | 2 | Variable |
81+
+----+------+------+----------+----------+----------+
82+
</pre>
83+
84+
##### 当远程端收到udp handler绑定的端口的事件，说明local端发来请求，远程端对信息进行解密并根据dest服务器/端口的协议类型对其发出tcp连接或者udp连接；
85+
86+
<pre>
87+
+-------+--------------+
88+
| IV | PAYLOAD |
89+
+-------+--------------+
90+
| Fixed | Variable |
91+
+-------+--------------+
92+
</pre>
93+
94+
-\>decrypt
95+
96+
<pre>
97+
+------+----------+----------+----------+
98+
| ATYP | DST.ADDR | DST.PORT | DATA |
99+
+------+----------+----------+----------+
100+
| 1 | Variable | 2 | Variable |
101+
+------+----------+----------+----------+
102+
</pre>
103+
104+
-\>trim
105+
106+
<pre>
107+
+----------+
108+
| DATA |
109+
+----------+
110+
| Variable |
111+
+----------+
112+
</pre>
113+
114+
-\>getaddrinfo-\>tcp/udp
115+
-\>send to dest server via tcp/udp
116+
117+
118+
##### 当远程新建的socket收到连接请求时，说明dest服务器向远程端发出响应，远程端对其进行加密，并且转发给local端
119+
120+
<pre>
121+
+----------+
122+
| DATA |
123+
+----------+
124+
| Variable |
125+
+----------+
126+
</pre>
127+
128+
-\>add
129+
130+
<pre>
131+
+------+----------+----------+----------+
132+
| ATYP | DST.ADDR | DST.PORT | DATA |
133+
+------+----------+----------+----------+
134+
| 1 | Variable | 2 | Variable |
135+
+------+----------+----------+----------+
136+
</pre>
137+
138+
-\>encrypt
139+
140+
<pre>
141+
+-------+--------------+
142+
| IV | PAYLOAD |
143+
+-------+--------------+
144+
| Fixed | Variable |
145+
+-------+--------------+
146+
</pre>
147+
148+
-\>send to local
149+
150+
在handler函数里面的基本逻辑就是：
151+
<pre>
152+
if sock == self._server_socket:
153+
self._handle_server()
154+
elif sock and (fd in self._sockets):
155+
self._handle_client(sock)
156+
</pre>
157+
158+
协议解析和构建用的struct.pack()和struct.unpack()
159+
160+
===============================================================
161+
##### asyndns.py实现的是一个DNS服务器，封装得相当的好
162+
1.1、读取/etc/hosts和/etc/resolv.conf文件，如果没有设置，就设置dns服务器为8.8.8.8和8.8.4.4
163+
1.2、收到tcp handler和udp handler的dns请求之后，建立socket并且向远程服务器发送请求，并把（hostname：callback）加入_hostname_to_cb
164+
1.3、收到响应之后触发callback _hostname_to_cb[hostname](#)
165+
166+
###### 作者全程用二进制构建dns报文，非常值得学习
167+
168+
<pre>
169+
# 请求
170+
# 1 1 1 1 1 1
171+
# 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
172+
# +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
173+
# | ID |
174+
# +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
175+
# |QR| Opcode |AA|TC|RD|RA| Z | RCODE |
176+
# +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
177+
# | QDCOUNT |
178+
# +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
179+
# | ANCOUNT |
180+
# +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
181+
# | NSCOUNT |
182+
# +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
183+
# | ARCOUNT |
184+
# +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
185+
</pre>
186+
187+
响应：
188+
<pre>
189+
1 1 1 1 1 1
190+
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
191+
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
192+
| |
193+
/ /
194+
/ NAME /
195+
| |
196+
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
197+
| TYPE |
198+
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
199+
| CLASS |
200+
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
201+
| TTL |
202+
| |
203+
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
204+
| RDLENGTH |
205+
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--|
206+
/ RDATA /
207+
/ /
208+
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
209+
</pre>
210+
211+
===============================================================
212+
##### lru_cache.py实现的是一个缓存
213+
214+
<pre>
215+
self._store =
216+
self._time_to_keys = collections.defaultdict(list)
217+
self._keys_to_last_time =
218+
self._last_visits = collections.deque()
219+
</pre>
220+
221+
222+
###### 1、先找访问时间_last_visits中超出timeout的所有键
223+
###### 2、然后去找_time_to_keys，找出所有可能过期的键
224+
###### 3、因为最早访问时间访问过的键之后可能又访问了，所以要_keys_to_last_time
225+
###### 4、找出那些没被访问过的，然后删除
226+
227+
===============================================================
228+
##### 学到的其他东西：
229+
###### 1、__future__
230+
###### 2、json.loads(f.read().decode('utf8'),object_hook=_decode_dict)
231+
###### 3、python内置的logging也可作大规模使用
232+
###### 4、把我理解层面阔伸到协议层面，学到怎么构建一个协议（协议的设计还要学习）
233+
###### 5、网络编程和信息安全息息相关
234+
###### 6、这个网络编程的学习路线挺不错的：爬虫-\>翻墙软件。不知道下一步怎么加深
235+
236+
一些问题：
237+
###### 1、如何做到线程安全？
238+
###### 2、大量对变量是否存在的检查是为了什么？
239+
###### 3、FSM的思想怎么应用到网络编程？
240+
###### 4、防火墙到底是怎么工作的？（其实这个问题我自己觉得问的挺逗的。。）
241+
###### 5、linux的内核异步IO怎么调用（操作系统）
3242

4-
[![PyPI version]][PyPI]
5-
[![Build Status]][Travis CI]
6-
[![Coverage Status]][Coverage]
7243

8-
A fast tunnel proxy that helps you bypass firewalls.
9-
10-
[中文说明][Chinese Readme]
11-
12-
Install
13-
-------
14-
15-
You'll have a client on your local side, and setup a server on a
16-
remote server.
17-
18-
### Client
19-
20-
* [Windows] / [OS X]
21-
* [Android] / [iOS]
22-
* [OpenWRT]
23-
24-
### Server
25-
26-
#### Debian / Ubuntu:
27-
28-
apt-get install python-pip
29-
pip install shadowsocks
30-
31-
Or simply `apt-get install shadowsocks` if you have [Debian sid] in your
32-
source list.
33-
34-
#### CentOS:
35-
36-
yum install python-setuptools
37-
easy_install pip
38-
pip install shadowsocks
39-
40-
#### Windows:
41-
42-
Download [OpenSSL for Windows] and install. Then install shadowsocks via
43-
easy_install and pip as Linux. If you don't know how to use them, you can
44-
directly download [the package], and use `python shadowsocks/server.py`
45-
instead of `ssserver` command below.
46-
47-
Configuration
48-
-------------
49-
50-
On your server create a config file `/etc/shadowsocks.json`.
51-
Example:
52-
53-
{
54-
"server":"my_server_ip",
55-
"server_port":8388,
56-
"local_address": "127.0.0.1",
57-
"local_port":1080,
58-
"password":"mypassword",
59-
"timeout":300,
60-
"method":"aes-256-cfb",
61-
"fast_open": false
62-
}
63-
64-
Explanation of the fields:
65-
66-
| Name | Explanation |
67-
| ------------- | ----------------------------------------------- |
68-
| server | the address your server listens |
69-
| server_port | server port |
70-
| local_address | the address your local listens |
71-
| local_port | local port |
72-
| password | password used for encryption |
73-
| timeout | in seconds |
74-
| method | default: "aes-256-cfb", see [Encryption] |
75-
| fast_open | use [TCP_FASTOPEN], true / false |
76-
| workers | number of workers, available on Unix/Linux |
77-
78-
On your server:
79-
80-
To run in the foreground:
81-
82-
ssserver -c /etc/shadowsocks.json
83-
84-
To run in the background:
85-
86-
ssserver -c /etc/shadowsocks.json -d start
87-
ssserver -c /etc/shadowsocks.json -d stop
88-
89-
On your client machine, use the same configuration as your server. Check the
90-
README of your client for more information.
91-
92-
Command Line Options
93-
--------------------
94-
95-
Check the options via `-h`.You can use args to override settings from
96-
`config.json`.
97-
98-
sslocal -s server_name -p server_port -l local_port -k password -m bf-cfb
99-
ssserver -p server_port -k password -m bf-cfb --workers 2
100-
ssserver -c /etc/shadowsocks/config.json -d start --pid-file=/tmp/shadowsocks.pid
101-
ssserver -c /etc/shadowsocks/config.json -d stop --pid-file=/tmp/shadowsocks.pid
102-
103-
Documentation
104-
-------------
105-
106-
You can find all the documentation in the wiki:
107-
https://github.com/clowwindy/shadowsocks/wiki
108-
109-
License
110-
-------
111-
MIT
112-
113-
Bugs and Issues
114-
----------------
115-
116-
* [Troubleshooting]
117-
* [Issue Tracker]
118-
* [Mailing list]
119-
120-
121-
[Android]: https://github.com/clowwindy/shadowsocks/wiki/Ports-and-Clients#android
122-
[Build Status]: https://img.shields.io/travis/clowwindy/shadowsocks/master.svg?style=flat
123-
[Chinese Readme]: https://github.com/clowwindy/shadowsocks/wiki/Shadowsocks-%E4%BD%BF%E7%94%A8%E8%AF%B4%E6%98%8E
124-
[Coverage Status]: http://192.81.132.184/result/shadowsocks
125-
[Coverage]: http://192.81.132.184/job/Shadowsocks/ws/htmlcov/index.html
126-
[Debian sid]: https://packages.debian.org/unstable/python/shadowsocks
127-
[the package]: https://pypi.python.org/pypi/shadowsocks
128-
[Encryption]: https://github.com/clowwindy/shadowsocks/wiki/Encryption
129-
[iOS]: https://github.com/shadowsocks/shadowsocks-iOS/wiki/Help
130-
[Issue Tracker]: https://github.com/clowwindy/shadowsocks/issues?state=open
131-
[Mailing list]: http://groups.google.com/group/shadowsocks
132-
[OpenSSL for Windows]: http://slproweb.com/products/Win32OpenSSL.html
133-
[OpenWRT]: https://github.com/clowwindy/shadowsocks/wiki/Ports-and-Clients#openwrt
134-
[OS X]: https://github.com/shadowsocks/shadowsocks-iOS/wiki/Shadowsocks-for-OSX-Help
135-
[PyPI]: https://pypi.python.org/pypi/shadowsocks
136-
[PyPI version]: https://img.shields.io/pypi/v/shadowsocks.svg?style=flat
137-
[TCP_FASTOPEN]: https://github.com/clowwindy/shadowsocks/wiki/TCP-Fast-Open
138-
[Travis CI]: https://travis-ci.org/clowwindy/shadowsocks
139-
[Troubleshooting]: https://github.com/clowwindy/shadowsocks/wiki/Troubleshooting
140-
[Windows]: https://github.com/clowwindy/shadowsocks/wiki/Ports-and-Clients#windows

0 commit comments

Comments
 (0)