Skip to content

Commit afcee0d

Browse files
oremanjoremanj
authored
Merge pull request oremanj#74 from oremanj/updates
Modernize, add tests, allow Packet to outlive the callback it's passed to
2 parents ec2ae29 + c2d7ce8 commit afcee0d

12 files changed

+5094
-1938
lines changed

‎.github/workflows/ci.yml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
pull_request:
8+
9+
jobs:
10+
Ubuntu:
11+
name: 'Ubuntu (${{ matrix.python }})'
12+
timeout-minutes: 10
13+
runs-on: 'ubuntu-latest'
14+
strategy:
15+
fail-fast: false
16+
matrix:
17+
python:
18+
- '3.6'
19+
- '3.7'
20+
- '3.8'
21+
- '3.9'
22+
- '3.10'
23+
steps:
24+
- name: Checkout
25+
uses: actions/checkout@v2
26+
- name: Setup python
27+
uses: actions/setup-python@v2
28+
with:
29+
python-version: ${{ fromJSON(format('["{0}", "{1}"]', format('{0}.0-alpha - {0}.X', matrix.python), matrix.python))[startsWith(matrix.python, 'pypy')] }}
30+
- name: Run tests
31+
run: ./ci.sh
32+
env:
33+
# Should match 'name:' up above
34+
JOB_NAME: 'Ubuntu (${{ matrix.python }})'

‎CHANGES.txt

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,13 @@
1-
v0.8.1 30 Jan 2017
1+
v0.9.0, unreleased
2+
Improve usability when Packet objects are retained past the callback
3+
Add Packet.retain() to save the packet contents in such cases
4+
Eliminate warnings during build on py3
5+
Add CI and basic test suite
6+
Raise a warning, not an error, if we don't get the bufsize we want
7+
Don't allow bind() more than once on the same NetfilterQueue, since
8+
that would leak the old queue handle
9+
10+
v0.8.1, 30 Jan 2017
211
Fix bug #25- crashing when used in OUTPUT or POSTROUTING chains
312

413
v0.8, 15 Dec 2016

‎README.rst

Lines changed: 67 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,10 @@ NetfilterQueue
33
==============
44

55
NetfilterQueue provides access to packets matched by an iptables rule in
6-
Linux. Packets so matched can be accepted, dropped, altered, or given a mark.
6+
Linux. Packets so matched can be accepted, dropped, altered, reordered,
7+
or given a mark.
78

8-
Libnetfilter_queue (the netfilter library, not this module) is part of the
9+
libnetfilter_queue (the netfilter library, not this module) is part of the
910
`Netfilter project <http://netfilter.org/projects/libnetfilter_queue/>`_.
1011

1112
Example
@@ -15,18 +16,18 @@ The following script prints a short description of each packet before accepting
1516
it. ::
1617

1718
from netfilterqueue import NetfilterQueue
18-
19+
1920
def print_and_accept(pkt):
2021
print(pkt)
2122
pkt.accept()
22-
23+
2324
nfqueue = NetfilterQueue()
2425
nfqueue.bind(1, print_and_accept)
2526
try:
2627
nfqueue.run()
2728
except KeyboardInterrupt:
2829
print('')
29-
30+
3031
nfqueue.unbind()
3132

3233
You can also make your own socket so that it can be used with gevent, for example. ::
@@ -56,7 +57,7 @@ To send packets destined for your LAN to the script, type something like::
5657
Installation
5758
============
5859

59-
NetfilterQueue is a C extention module that links against libnetfilter_queue.
60+
NetfilterQueue is a C extention module that links against libnetfilter_queue.
6061
Before installing, ensure you have:
6162

6263
1. A C compiler
@@ -81,9 +82,9 @@ From source
8182

8283
To install from source::
8384

84-
git clone git@github.com:kti/python-netfilterqueue.git
85+
git clone https://github.com/oremanj/python-netfilterqueue
8586
cd python-netfilterqueue
86-
python setup.py install
87+
pip install .
8788

8889
If Cython is installed, Distutils will use it to regenerate the .c source from the .pyx. It will then compile the .c into a .so.
8990

@@ -104,9 +105,12 @@ NetfilterQueue objects
104105
A NetfilterQueue object represents a single queue. Configure your queue with
105106
a call to ``bind``, then start receiving packets with a call to ``run``.
106107

107-
``QueueHandler.bind(queue_num, callback[, max_len[, mode[, range, [sock_len]]]])``
108-
Create and bind to the queue. ``queue_num`` must match the number in your
109-
iptables rule. ``callback`` is a function or method that takes one
108+
``QueueHandler.bind(queue_num, callback[, max_len[, mode[, range[, sock_len]]]])``
109+
Create and bind to the queue. ``queue_num`` uniquely identifies this
110+
queue for the kernel. It must match the ``--queue-num`` in your iptables
111+
rule, but there is no ordering requirement: it's fine to either ``bind()``
112+
first or set up the iptables rule first.
113+
``callback`` is a function or method that takes one
110114
argument, a Packet object (see below). ``max_len`` sets the largest number
111115
of packets that can be in the queue; new packets are dropped if the size of
112116
the queue reaches this number. ``mode`` determines how much of the packet
@@ -119,17 +123,21 @@ a call to ``bind``, then start receiving packets with a call to ``run``.
119123
Remove the queue. Packets matched by your iptables rule will be dropped.
120124

121125
``QueueHandler.get_fd()``
122-
Get the file descriptor of the queue handler.
126+
Get the file descriptor of the socket used to receive queued
127+
packets and send verdicts. If you're using an async event loop,
128+
you can poll this FD for readability and call ``run(False)`` every
129+
time data appears on it.
123130

124131
``QueueHandler.run([block])``
125-
Send packets to your callback. By default, this method blocks. Set
126-
block=False to let your thread continue. You can get the file descriptor
127-
of the socket with the ``get_fd`` method.
132+
Send packets to your callback. By default, this method blocks, running
133+
until an exception is raised (such as by Ctrl+C). Set
134+
block=False to process the pending messages without waiting for more.
135+
You can get the file descriptor of the socket with the ``get_fd`` method.
128136

129137
``QueueHandler.run_socket(socket)``
130138
Send packets to your callback, but use the supplied socket instead of
131139
recv, so that, for example, gevent can monkeypatch it. You can make a
132-
socket with ``socket.fromfd(nfqueue.get_fd(), socket.AF_UNIX, socket.SOCK_STREAM)``
140+
socket with ``socket.fromfd(nfqueue.get_fd(), socket.AF_NETLINK, socket.SOCK_RAW)``
133141
and optionally make it non-blocking with ``socket.setblocking(False)``.
134142

135143
Packet objects
@@ -138,55 +146,78 @@ Packet objects
138146
Objects of this type are passed to your callback.
139147

140148
``Packet.get_payload()``
141-
Return the packet's payload as a string (Python 2) or bytes (Python 3).
149+
Return the packet's payload as a bytes object. The returned value
150+
starts with the IP header. You must call ``retain()`` if you want
151+
to be able to ``get_payload()`` after your callback has returned.
142152

143153
``Packet.set_payload(payload)``
144-
Set the packet payload. ``payload`` is a bytes.
154+
Set the packet payload. Call this before ``accept()`` if you want to
155+
change the contents of the packet before allowing it to be released.
156+
Don't forget to update the transport-layer checksum (or clear it,
157+
if you're using UDP), or else the recipient is likely to drop the
158+
packet. If you're changing the length of the packet, you'll also need
159+
to update the IP length, IP header checksum, and probably some
160+
transport-level fields (such as UDP length for UDP).
145161

146162
``Packet.get_payload_len()``
147163
Return the size of the payload.
148164

149165
``Packet.set_mark(mark)``
150-
Give the packet a kernel mark. ``mark`` is a 32-bit number.
166+
Give the packet a kernel mark, which can be used in future iptables
167+
rules. ``mark`` is a 32-bit number.
151168

152169
``Packet.get_mark()``
153-
Get the mark already on the packet.
170+
Get the mark already on the packet (either the one you set using
171+
``set_mark()``, or the one it arrived with if you haven't called
172+
``set_mark()``).
154173

155174
``Packet.get_hw()``
156175
Return the hardware address as a Python string.
157176

177+
``Packet.retain()``
178+
Allocate a copy of the packet payload for use after the callback
179+
has returned. ``get_payload()`` will raise an exception at that
180+
point if you didn't call ``retain()``.
181+
158182
``Packet.accept()``
159-
Accept the packet.
183+
Accept the packet. You can reorder packets by accepting them
184+
in a different order than the order in which they were passed
185+
to your callback.
160186

161187
``Packet.drop()``
162188
Drop the packet.
163-
189+
164190
``Packet.repeat()``
165-
Iterate the same cycle once more.
166-
191+
Restart processing of this packet from the beginning of its
192+
Netfilter hook (iptables chain, roughly). Any changes made
193+
using ``set_payload()`` or ``set_mark()`` are preserved; in the
194+
absence of such changes, the packet will probably come right
195+
back to the same queue.
196+
167197
Callback objects
168198
----------------
169199

170-
Your callback can be function or a method and must accept one argument, a
171-
Packet object. You must call either Packet.accept() or Packet.drop() before
172-
returning.
173-
174-
``callback(packet)`` or ``callback(self, packet)``
175-
Handle a single packet from the queue. You must call either
176-
``packet.accept()`` or ``packet.drop()``.
200+
Your callback can be any one-argument callable and will be invoked with
201+
a ``Packet`` object as argument. You must call ``retain()`` within the
202+
callback if you want to be able to ``get_payload()`` after the callback
203+
has returned. You can hang onto ``Packet`` objects and resolve them later,
204+
but note that packets continue to count against the queue size limit
205+
until they've been given a verdict (accept, drop, or repeat). Also, the
206+
kernel stores the enqueued packets in a linked list, so keeping lots of packets
207+
outstanding is likely to adversely impact performance.
177208

178209
Usage
179210
=====
180211

181212
To send packets to the queue::
182213

183214
iptables -I <table or chain> <match specification> -j NFQUEUE --queue-num <queue number>
184-
215+
185216
For example::
186217

187218
iptables -I INPUT -d 192.168.0.0/24 -j NFQUEUE --queue-num 1
188-
189-
The only special part of the rule is the target. Rules can have any match and
219+
220+
The only special part of the rule is the target. Rules can have any match and
190221
can be added to any table or chain.
191222

192223
Valid queue numbers are integers from 0 to 65,535 inclusive.
@@ -228,7 +259,7 @@ Limitations
228259
* Omits methods for getting information about the interface a packet has
229260
arrived on or is leaving on
230261
* Probably other stuff is omitted too
231-
262+
232263
Source
233264
======
234265

@@ -237,7 +268,7 @@ https://github.com/kti/python-netfilterqueue
237268
License
238269
=======
239270

240-
Copyright (c) 2011, Kerkhoff Technologies, Inc.
271+
Copyright (c) 2011, Kerkhoff Technologies, Inc, and contributors.
241272

242273
`MIT licensed <https://github.com/kti/python-netfilterqueue/blob/master/LICENSE.txt>`_
243274

‎ci.sh

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#!/bin/bash
2+
3+
set -ex -o pipefail
4+
5+
pip install -U pip setuptools wheel
6+
sudo apt-get install libnetfilter-queue-dev
7+
python setup.py sdist --formats=zip
8+
pip install dist/*.zip
9+
pip install -r test-requirements.txt
10+
11+
cd tests
12+
pytest -W error -ra -v .

0 commit comments

Comments
 (0)