Google Git
Sign in
chromium / external / github.com / python / cpython / refs/tags/3.3 / . / Lib / re.py
blob: a46ecc84f22fcac7de6f7c8f123cd7e443d8bca0 [file] [log] [blame]
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]1#
2# Secret Labs' Regular Expression Engine
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]3#
4# re-compatible interface for the sre matching engine
5#
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]6# Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]7#
Fredrik Lundh29c4ba92000-08-01 18:20:07[diff] [blame]8# This version of the SRE library can be redistributed under CNRI's
9# Python 1.6 license. For any other use, please contact Secret Labs
10# AB (info@pythonware.com).
11#
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]12# Portions of this engine have been developed in cooperation with
Fredrik Lundh29c4ba92000-08-01 18:20:07[diff] [blame]13# CNRI. Hewlett-Packard provided funding for 1.6 integration and
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]14# other compatibility work.
15#
16
Fred Drake9f5b8222001-09-04 19:20:06[diff] [blame]17r"""Support for regular expressions (RE).
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]18
19This module provides regular expression matching operations similar to
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]20those found in Perl. It supports both 8-bit and Unicode strings; both
21the pattern and the strings being processed can contain null bytes and
22characters outside the US ASCII range.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]23
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]24Regular expressions can contain both special and ordinary characters.
25Most ordinary characters, like "A", "a", or "0", are the simplest
26regular expressions; they simply match themselves. You can
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]27concatenate ordinary characters, so last matches the string 'last'.
28
29The special characters are:
30 "." Matches any character except a newline.
31 "^" Matches the start of the string.
Georg Brandl86def6c2008-01-21 20:36:10[diff] [blame]32 "$" Matches the end of the string or just before the newline at
33 the end of the string.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]34 "*" Matches 0 or more (greedy) repetitions of the preceding RE.
35 Greedy means that it will match as many repetitions as possible.
36 "+" Matches 1 or more (greedy) repetitions of the preceding RE.
37 "?" Matches 0 or 1 (greedy) of the preceding RE.
38 *?,+?,?? Non-greedy versions of the previous three special characters.
39 {m,n} Matches from m to n repetitions of the preceding RE.
40 {m,n}? Non-greedy version of the above.
Christian Heimesaf98da12008-01-27 15:18:18[diff] [blame]41 "\\" Either escapes special characters or signals a special sequence.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]42 [] Indicates a set of characters.
43 A "^" as the first character indicates a complementing set.
44 "|" A|B, creates an RE that will match either A or B.
45 (...) Matches the RE inside the parentheses.
46 The contents can be retrieved or matched later in the string.
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]47 (?aiLmsux) Set the A, I, L, M, S, U, or X flag for the RE (see below).
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]48 (?:...) Non-grouping version of regular parentheses.
49 (?P<name>...) The substring matched by the group is accessible by name.
50 (?P=name) Matches the text matched earlier by the group named name.
51 (?#...) A comment; ignored.
52 (?=...) Matches if ... matches next, but doesn't consume the string.
53 (?!...) Matches if ... doesn't match next.
Christian Heimesaf98da12008-01-27 15:18:18[diff] [blame]54 (?<=...) Matches if preceded by ... (must be fixed length).
55 (?<!...) Matches if not preceded by ... (must be fixed length).
56 (?(id/name)yes|no) Matches yes pattern if the group with id/name matched,
57 the (optional) no pattern otherwise.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]58
59The special sequences consist of "\\" and a character from the list
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]60below. If the ordinary character is not on the list, then the
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]61resulting RE will match the second character.
Fred Drake9f5b8222001-09-04 19:20:06[diff] [blame]62 \number Matches the contents of the group of the same number.
63 \A Matches only at the start of the string.
64 \Z Matches only at the end of the string.
65 \b Matches the empty string, but only at the start or end of a word.
66 \B Matches the empty string, but not at the start or end of a word.
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]67 \d Matches any decimal digit; equivalent to the set [0-9] in
68 bytes patterns or string patterns with the ASCII flag.
69 In string patterns without the ASCII flag, it will match the whole
70 range of Unicode digits.
71 \D Matches any non-digit character; equivalent to [^\d].
Ezio Melottie9900922012-04-29 05:23:25[diff] [blame]72 \s Matches any whitespace character; equivalent to [ \t\n\r\f\v] in
73 bytes patterns or string patterns with the ASCII flag.
74 In string patterns without the ASCII flag, it will match the whole
75 range of Unicode whitespace characters.
Ezio Melotti38992832012-04-29 10:35:55[diff] [blame]76 \S Matches any non-whitespace character; equivalent to [^\s].
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]77 \w Matches any alphanumeric character; equivalent to [a-zA-Z0-9_]
78 in bytes patterns or string patterns with the ASCII flag.
79 In string patterns without the ASCII flag, it will match the
80 range of Unicode alphanumeric characters (letters plus digits
81 plus underscore).
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]82 With LOCALE, it will match the set [0-9_] plus characters defined
83 as letters for the current locale.
Fred Drake9f5b8222001-09-04 19:20:06[diff] [blame]84 \W Matches the complement of \w.
85 \\ Matches a literal backslash.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]86
87This module exports the following functions:
88 match Match a regular expression pattern to the beginning of a string.
89 search Search a string for the presence of a pattern.
90 sub Substitute occurrences of a pattern found in a string.
91 subn Same as sub, but also return the number of substitutions made.
92 split Split a string by the occurrences of a pattern.
93 findall Find all occurrences of a pattern in a string.
Christian Heimesaf98da12008-01-27 15:18:18[diff] [blame]94 finditer Return an iterator yielding a match object for each match.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]95 compile Compile a pattern into a RegexObject.
96 purge Clear the regular expression cache.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]97 escape Backslash all non-alphanumerics in a string.
98
99Some of the functions in this module takes flags as optional parameters:
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]100 A ASCII For string patterns, make \w, \W, \b, \B, \d, \D
101 match the corresponding ASCII character categories
102 (rather than the whole Unicode categories, which is the
103 default).
104 For bytes patterns, this flag is the only available
105 behaviour and needn't be specified.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]106 I IGNORECASE Perform case-insensitive matching.
107 L LOCALE Make \w, \W, \b, \B, dependent on the current locale.
Georg Brandl86def6c2008-01-21 20:36:10[diff] [blame]108 M MULTILINE "^" matches the beginning of lines (after a newline)
109 as well as the string.
110 "$" matches the end of lines (before a newline) as well
111 as the end of the string.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]112 S DOTALL "." matches any character at all, including the newline.
113 X VERBOSE Ignore whitespace and comments for nicer looking RE's.
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]114 U UNICODE For compatibility only. Ignored for string patterns (it
115 is the default), and forbidden for bytes patterns.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]116
117This module also defines an exception 'error'.
118
119"""
Fredrik Lundh21009b92001-09-18 18:47:09[diff] [blame]120
Fredrik Lundh703ce812001-10-24 22:16:30[diff] [blame]121import sys
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]122import sre_compile
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]123import sre_parse
Antoine Pitrou3060c452010-08-13 16:27:38[diff] [blame]124import functools
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]125
Fredrik Lundhf2989b22001-02-18 12:05:16[diff] [blame]126# public symbols
127__all__ = [ "match", "search", "sub", "subn", "split", "findall",
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]128 "compile", "purge", "template", "escape", "A", "I", "L", "M", "S", "X",
129 "U", "ASCII", "IGNORECASE", "LOCALE", "MULTILINE", "DOTALL", "VERBOSE",
Fredrik Lundhf2989b22001-02-18 12:05:16[diff] [blame]130 "UNICODE", "error" ]
131
Fredrik Lundhbec95b92001-10-21 16:47:57[diff] [blame]132__version__ = "2.2.1"
Fredrik Lundhb25e1ad2001-03-22 15:50:10[diff] [blame]133
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]134# flags
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]135A = ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]136I = IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
137L = LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]138U = UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]139M = MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
140S = DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
141X = VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]142
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]143# sre extensions (experimental, don't rely on these)
144T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
145DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]146
147# sre exception
Fredrik Lundhbe2211e2000-06-29 16:57:40[diff] [blame]148error = sre_compile.error
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]149
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]150# --------------------------------------------------------------------
151# public interface
152
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]153def match(pattern, string, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]154 """Try to apply the pattern at the start of the string, returning
155 a match object, or None if no match was found."""
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]156 return _compile(pattern, flags).match(string)
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]157
158def search(pattern, string, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]159 """Scan through string looking for a match to the pattern, returning
160 a match object, or None if no match was found."""
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]161 return _compile(pattern, flags).search(string)
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]162
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]163def sub(pattern, repl, string, count=0, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]164 """Return the string obtained by replacing the leftmost
165 non-overlapping occurrences of the pattern in string by the
Andrew M. Kuchling43ab0cd2004-08-07 17:41:54[diff] [blame]166 replacement repl. repl can be either a string or a callable;
Benjamin Peterson75edad02009-01-01 15:05:06[diff] [blame]167 if a string, backslash escapes in it are processed. If it is
168 a callable, it's passed the match object and must return
Andrew M. Kuchling43ab0cd2004-08-07 17:41:54[diff] [blame]169 a replacement string to be used."""
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]170 return _compile(pattern, flags).sub(repl, string, count)
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]171
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]172def subn(pattern, repl, string, count=0, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]173 """Return a 2-tuple containing (new_string, number).
174 new_string is the string obtained by replacing the leftmost
175 non-overlapping occurrences of the pattern in the source
176 string by the replacement repl. number is the number of
Andrew M. Kuchling43ab0cd2004-08-07 17:41:54[diff] [blame]177 substitutions that were made. repl can be either a string or a
Benjamin Peterson75edad02009-01-01 15:05:06[diff] [blame]178 callable; if a string, backslash escapes in it are processed.
179 If it is a callable, it's passed the match object and must
Andrew M. Kuchling43ab0cd2004-08-07 17:41:54[diff] [blame]180 return a replacement string to be used."""
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]181 return _compile(pattern, flags).subn(repl, string, count)
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]182
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]183def split(pattern, string, maxsplit=0, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]184 """Split the source string by the occurrences of the pattern,
Georg Brandle144c742012-03-10 08:26:53[diff] [blame]185 returning a list containing the resulting substrings. If
186 capturing parentheses are used in pattern, then the text of all
187 groups in the pattern are also returned as part of the resulting
188 list. If maxsplit is nonzero, at most maxsplit splits occur,
189 and the remainder of the string is returned as the final element
190 of the list."""
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]191 return _compile(pattern, flags).split(string, maxsplit)
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]192
Raymond Hettinger596ba4d2004-09-24 03:41:05[diff] [blame]193def findall(pattern, string, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]194 """Return a list of all non-overlapping matches in the string.
195
Georg Brandle144c742012-03-10 08:26:53[diff] [blame]196 If one or more capturing groups are present in the pattern, return
197 a list of groups; this will be a list of tuples if the pattern
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]198 has more than one group.
199
200 Empty matches are included in the result."""
Raymond Hettinger596ba4d2004-09-24 03:41:05[diff] [blame]201 return _compile(pattern, flags).findall(string)
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]202
Fredrik Lundh703ce812001-10-24 22:16:30[diff] [blame]203if sys.hexversion >= 0x02020000:
Guido van Rossum0d976552002-10-14 12:22:17[diff] [blame]204 __all__.append("finditer")
Raymond Hettinger596ba4d2004-09-24 03:41:05[diff] [blame]205 def finditer(pattern, string, flags=0):
Fredrik Lundhb7747e22001-10-28 20:15:40[diff] [blame]206 """Return an iterator over all non-overlapping matches in the
207 string. For each match, the iterator returns a match object.
Fredrik Lundh703ce812001-10-24 22:16:30[diff] [blame]208
209 Empty matches are included in the result."""
Raymond Hettinger596ba4d2004-09-24 03:41:05[diff] [blame]210 return _compile(pattern, flags).finditer(string)
Fredrik Lundh703ce812001-10-24 22:16:30[diff] [blame]211
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]212def compile(pattern, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]213 "Compile a regular expression pattern, returning a pattern object."
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]214 return _compile(pattern, flags)
215
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]216def purge():
Antoine Pitrou3060c452010-08-13 16:27:38[diff] [blame]217 "Clear the regular expression caches"
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]218 _cache.clear()
219 _cache_repl.clear()
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]220
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]221def template(pattern, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]222 "Compile a template pattern, returning a pattern object"
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]223 return _compile(pattern, flags|T)
224
Guido van Rossum698280d2008-09-10 17:44:35[diff] [blame]225_alphanum_str = frozenset(
Ezio Melotti88fdeb42011-04-10 09:59:16[diff] [blame]226 "_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890")
Guido van Rossum698280d2008-09-10 17:44:35[diff] [blame]227_alphanum_bytes = frozenset(
Ezio Melotti88fdeb42011-04-10 09:59:16[diff] [blame]228 b"_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890")
Raymond Hettingerebb7f302005-09-12 22:50:37[diff] [blame]229
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]230def escape(pattern):
Ezio Melotti88fdeb42011-04-10 09:59:16[diff] [blame]231 """
232 Escape all the characters in pattern except ASCII letters, numbers and '_'.
233 """
Guido van Rossum698280d2008-09-10 17:44:35[diff] [blame]234 if isinstance(pattern, str):
235 alphanum = _alphanum_str
236 s = list(pattern)
Ezio Melottiebbf1e62011-03-25 12:19:30[diff] [blame]237 for i, c in enumerate(pattern):
Guido van Rossum698280d2008-09-10 17:44:35[diff] [blame]238 if c not in alphanum:
239 if c == "\000":
240 s[i] = "\\000"
241 else:
242 s[i] = "\\" + c
243 return "".join(s)
244 else:
245 alphanum = _alphanum_bytes
246 s = []
247 esc = ord(b"\\")
248 for c in pattern:
249 if c in alphanum:
250 s.append(c)
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]251 else:
Guido van Rossum698280d2008-09-10 17:44:35[diff] [blame]252 if c == 0:
253 s.extend(b"\\000")
254 else:
255 s.append(esc)
256 s.append(c)
257 return bytes(s)
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]258
259# --------------------------------------------------------------------
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]260# internals
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]261
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]262_cache = {}
263_cache_repl = {}
264
Fredrik Lundh397a6542001-10-18 19:30:16[diff] [blame]265_pattern_type = type(sre_compile.compile("", 0))
266
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]267_MAXCACHE = 512
Raymond Hettingerd9e8cc62010-08-14 21:17:48[diff] [blame]268def _compile(pattern, flags):
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]269 # internal: compile pattern
Antoine Pitroud2cc7432014-02-03 19:59:59[diff] [blame]270 bypass_cache = flags & DEBUG
271 if not bypass_cache:
272 try:
273 return _cache[type(pattern), pattern, flags]
274 except KeyError:
275 pass
Just van Rossum12723ba2003-07-02 20:03:04[diff] [blame]276 if isinstance(pattern, _pattern_type):
Christian Heimes5fb7c2a2007-12-24 08:52:31[diff] [blame]277 if flags:
Guido van Rossum698280d2008-09-10 17:44:35[diff] [blame]278 raise ValueError(
279 "Cannot process flags argument with a compiled pattern")
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]280 return pattern
Just van Rossum74902502003-07-02 21:37:16[diff] [blame]281 if not sre_compile.isstring(pattern):
Collin Winterce36ad82007-08-30 01:19:48[diff] [blame]282 raise TypeError("first argument must be string or compiled pattern")
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]283 p = sre_compile.compile(pattern, flags)
Antoine Pitroud2cc7432014-02-03 19:59:59[diff] [blame]284 if not bypass_cache:
285 if len(_cache) >= _MAXCACHE:
286 _cache.clear()
287 _cache[type(pattern), pattern, flags] = p
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]288 return p
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]289
Raymond Hettingerd9e8cc62010-08-14 21:17:48[diff] [blame]290def _compile_repl(repl, pattern):
Fredrik Lundhb25e1ad2001-03-22 15:50:10[diff] [blame]291 # internal: compile replacement pattern
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]292 try:
293 return _cache_repl[repl, pattern]
294 except KeyError:
295 pass
296 p = sre_parse.parse_template(repl, pattern)
297 if len(_cache_repl) >= _MAXCACHE:
298 _cache_repl.clear()
299 _cache_repl[repl, pattern] = p
300 return p
Fredrik Lundhb25e1ad2001-03-22 15:50:10[diff] [blame]301
Fredrik Lundh5644b7f2000-09-21 17:03:25[diff] [blame]302def _expand(pattern, match, template):
303 # internal: match.expand implementation hook
304 template = sre_parse.parse_template(template, pattern)
305 return sre_parse.expand_template(template, match)
306
Fredrik Lundhbec95b92001-10-21 16:47:57[diff] [blame]307def _subx(pattern, template):
308 # internal: pattern.sub/subn implementation helper
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]309 template = _compile_repl(template, pattern)
310 if not template[0] and len(template[1]) == 1:
Fredrik Lundhbec95b92001-10-21 16:47:57[diff] [blame]311 # literal replacement
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]312 return template[1][0]
313 def filter(match, template=template):
314 return sre_parse.expand_template(template, match)
315 return filter
Fredrik Lundh0640e112000-06-30 13:55:15[diff] [blame]316
317# register myself for pickling
318
Alexandre Vassalottif7fa63d2008-05-11 08:55:36[diff] [blame]319import copyreg
Fredrik Lundh0640e112000-06-30 13:55:15[diff] [blame]320
321def _pickle(p):
322 return _compile, (p.pattern, p.flags)
323
Alexandre Vassalottif7fa63d2008-05-11 08:55:36[diff] [blame]324copyreg.pickle(_pattern_type, _pickle, _compile)
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]325
326# --------------------------------------------------------------------
327# experimental stuff (see python-dev discussions for details)
328
329class Scanner:
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]330 def __init__(self, lexicon, flags=0):
Fredrik Lundh29c4ba92000-08-01 18:20:07[diff] [blame]331 from sre_constants import BRANCH, SUBPATTERN
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]332 self.lexicon = lexicon
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]333 # combine phrases into a compound pattern
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]334 p = []
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]335 s = sre_parse.Pattern()
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]336 s.flags = flags
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]337 for phrase, action in lexicon:
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]338 p.append(sre_parse.SubPattern(s, [
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]339 (SUBPATTERN, (len(p)+1, sre_parse.parse(phrase, flags))),
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]340 ]))
Christian Heimes2202f872008-02-06 14:31:34[diff] [blame]341 s.groups = len(p)+1
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]342 p = sre_parse.SubPattern(s, [(BRANCH, (None, p))])
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]343 self.scanner = sre_compile.compile(p)
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]344 def scan(self, string):
345 result = []
346 append = result.append
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]347 match = self.scanner.scanner(string).match
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]348 i = 0
349 while 1:
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]350 m = match()
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]351 if not m:
352 break
353 j = m.end()
354 if i == j:
355 break
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]356 action = self.lexicon[m.lastindex-1][1]
Florent Xicluna5d1155c2011-10-28 12:45:05[diff] [blame]357 if callable(action):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]358 self.match = m
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]359 action = action(self, m.group())
360 if action is not None:
361 append(action)
362 i = j
363 return result, string[i:]