Skip to content

Commit 2225833

Browse files
authored
fix: fix an indent error
1 parent 71a80a8 commit 2225833

File tree

1 file changed

+6
-6
lines changed

1 file changed

+6
-6
lines changed

source/c02/p18_tokenizing_text.rst

+6-6
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
.. code-block:: python
2323
2424
tokens = [('NAME', 'foo'), ('EQ','='), ('NUM', '23'), ('PLUS','+'),
25-
('NUM', '42'), ('TIMES', '*'), ('NUM', '10')]
25+
('NUM', '42'), ('TIMES', '*'), ('NUM', '10')]
2626
2727
为了执行这样的切分,第一步就是像下面这样利用命名捕获组的正则表达式来定义所有可能的令牌,包括空格:
2828

@@ -76,10 +76,10 @@
7676
.. code-block:: python
7777
7878
def generate_tokens(pat, text):
79-
Token = namedtuple('Token', ['type', 'value'])
80-
scanner = pat.scanner(text)
81-
for m in iter(scanner.match, None):
82-
yield Token(m.lastgroup, m.group())
79+
Token = namedtuple('Token', ['type', 'value'])
80+
scanner = pat.scanner(text)
81+
for m in iter(scanner.match, None):
82+
yield Token(m.lastgroup, m.group())
8383
8484
# Example use
8585
for tok in generate_tokens(master_pat, 'foo = 42'):
@@ -97,7 +97,7 @@
9797
.. code-block:: python
9898
9999
tokens = (tok for tok in generate_tokens(master_pat, text)
100-
if tok.type != 'WS')
100+
if tok.type != 'WS')
101101
for tok in tokens:
102102
print(tok)
103103

0 commit comments

Comments
 (0)