Google Git
Sign in
chromium / external / github.com / python / cpython / refs/tags/3.7 / . / Lib / re.py
blob: 4798043ffcc5d1287eddad8a2384905d31648ec9 [file] [log] [blame]
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]1#
2# Secret Labs' Regular Expression Engine
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]3#
4# re-compatible interface for the sre matching engine
5#
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]6# Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]7#
Fredrik Lundh29c4ba92000-08-01 18:20:07[diff] [blame]8# This version of the SRE library can be redistributed under CNRI's
9# Python 1.6 license. For any other use, please contact Secret Labs
10# AB (info@pythonware.com).
11#
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]12# Portions of this engine have been developed in cooperation with
Fredrik Lundh29c4ba92000-08-01 18:20:07[diff] [blame]13# CNRI. Hewlett-Packard provided funding for 1.6 integration and
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]14# other compatibility work.
15#
16
Fred Drake9f5b8222001-09-04 19:20:06[diff] [blame]17r"""Support for regular expressions (RE).
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]18
19This module provides regular expression matching operations similar to
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]20those found in Perl. It supports both 8-bit and Unicode strings; both
21the pattern and the strings being processed can contain null bytes and
22characters outside the US ASCII range.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]23
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]24Regular expressions can contain both special and ordinary characters.
25Most ordinary characters, like "A", "a", or "0", are the simplest
26regular expressions; they simply match themselves. You can
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]27concatenate ordinary characters, so last matches the string 'last'.
28
29The special characters are:
30 "." Matches any character except a newline.
31 "^" Matches the start of the string.
Georg Brandl86def6c2008-01-21 20:36:10[diff] [blame]32 "$" Matches the end of the string or just before the newline at
33 the end of the string.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]34 "*" Matches 0 or more (greedy) repetitions of the preceding RE.
35 Greedy means that it will match as many repetitions as possible.
36 "+" Matches 1 or more (greedy) repetitions of the preceding RE.
37 "?" Matches 0 or 1 (greedy) of the preceding RE.
38 *?,+?,?? Non-greedy versions of the previous three special characters.
39 {m,n} Matches from m to n repetitions of the preceding RE.
40 {m,n}? Non-greedy version of the above.
Christian Heimesaf98da12008-01-27 15:18:18[diff] [blame]41 "\\" Either escapes special characters or signals a special sequence.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]42 [] Indicates a set of characters.
43 A "^" as the first character indicates a complementing set.
44 "|" A|B, creates an RE that will match either A or B.
45 (...) Matches the RE inside the parentheses.
46 The contents can be retrieved or matched later in the string.
Miss Islington (bot)0dad7482020-03-25 19:03:34[diff] [blame]47 (?aiLmsux) The letters set the corresponding flags defined below.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]48 (?:...) Non-grouping version of regular parentheses.
49 (?P<name>...) The substring matched by the group is accessible by name.
50 (?P=name) Matches the text matched earlier by the group named name.
51 (?#...) A comment; ignored.
52 (?=...) Matches if ... matches next, but doesn't consume the string.
53 (?!...) Matches if ... doesn't match next.
Christian Heimesaf98da12008-01-27 15:18:18[diff] [blame]54 (?<=...) Matches if preceded by ... (must be fixed length).
55 (?<!...) Matches if not preceded by ... (must be fixed length).
56 (?(id/name)yes|no) Matches yes pattern if the group with id/name matched,
57 the (optional) no pattern otherwise.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]58
59The special sequences consist of "\\" and a character from the list
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]60below. If the ordinary character is not on the list, then the
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]61resulting RE will match the second character.
Fred Drake9f5b8222001-09-04 19:20:06[diff] [blame]62 \number Matches the contents of the group of the same number.
63 \A Matches only at the start of the string.
64 \Z Matches only at the end of the string.
65 \b Matches the empty string, but only at the start or end of a word.
66 \B Matches the empty string, but not at the start or end of a word.
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]67 \d Matches any decimal digit; equivalent to the set [0-9] in
68 bytes patterns or string patterns with the ASCII flag.
69 In string patterns without the ASCII flag, it will match the whole
70 range of Unicode digits.
71 \D Matches any non-digit character; equivalent to [^\d].
Ezio Melottie9900922012-04-29 05:23:25[diff] [blame]72 \s Matches any whitespace character; equivalent to [ \t\n\r\f\v] in
73 bytes patterns or string patterns with the ASCII flag.
74 In string patterns without the ASCII flag, it will match the whole
75 range of Unicode whitespace characters.
Ezio Melotti38992832012-04-29 10:35:55[diff] [blame]76 \S Matches any non-whitespace character; equivalent to [^\s].
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]77 \w Matches any alphanumeric character; equivalent to [a-zA-Z0-9_]
78 in bytes patterns or string patterns with the ASCII flag.
79 In string patterns without the ASCII flag, it will match the
80 range of Unicode alphanumeric characters (letters plus digits
81 plus underscore).
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]82 With LOCALE, it will match the set [0-9_] plus characters defined
83 as letters for the current locale.
Fred Drake9f5b8222001-09-04 19:20:06[diff] [blame]84 \W Matches the complement of \w.
85 \\ Matches a literal backslash.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]86
87This module exports the following functions:
Serhiy Storchaka32eddc12013-11-23 21:20:30[diff] [blame]88 match Match a regular expression pattern to the beginning of a string.
89 fullmatch Match a regular expression pattern to all of a string.
90 search Search a string for the presence of a pattern.
91 sub Substitute occurrences of a pattern found in a string.
92 subn Same as sub, but also return the number of substitutions made.
93 split Split a string by the occurrences of a pattern.
94 findall Find all occurrences of a pattern in a string.
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]95 finditer Return an iterator yielding a Match object for each match.
96 compile Compile a pattern into a Pattern object.
Serhiy Storchaka32eddc12013-11-23 21:20:30[diff] [blame]97 purge Clear the regular expression cache.
98 escape Backslash all non-alphanumerics in a string.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]99
Miss Islington (bot)0dad7482020-03-25 19:03:34[diff] [blame]100Each function other than purge and escape can take an optional 'flags' argument
101consisting of one or more of the following module constants, joined by "|".
102A, L, and U are mutually exclusive.
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]103 A ASCII For string patterns, make \w, \W, \b, \B, \d, \D
104 match the corresponding ASCII character categories
105 (rather than the whole Unicode categories, which is the
106 default).
107 For bytes patterns, this flag is the only available
108 behaviour and needn't be specified.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]109 I IGNORECASE Perform case-insensitive matching.
110 L LOCALE Make \w, \W, \b, \B, dependent on the current locale.
Georg Brandl86def6c2008-01-21 20:36:10[diff] [blame]111 M MULTILINE "^" matches the beginning of lines (after a newline)
112 as well as the string.
113 "$" matches the end of lines (before a newline) as well
114 as the end of the string.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]115 S DOTALL "." matches any character at all, including the newline.
116 X VERBOSE Ignore whitespace and comments for nicer looking RE's.
Antoine Pitroufd036452008-08-19 17:56:33[diff] [blame]117 U UNICODE For compatibility only. Ignored for string patterns (it
118 is the default), and forbidden for bytes patterns.
Fred Drakeb8f22742001-09-04 19:10:20[diff] [blame]119
120This module also defines an exception 'error'.
121
122"""
Fredrik Lundh21009b92001-09-18 18:47:09[diff] [blame]123
Victor Stinnerfb6a6592016-11-14 11:35:55[diff] [blame]124import enum
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]125import sre_compile
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]126import sre_parse
Raymond Hettinger5820f3a2016-09-19 03:17:21[diff] [blame]127import functools
Serhiy Storchakacf7b1ca2014-10-30 23:34:45[diff] [blame]128try:
129 import _locale
130except ImportError:
131 _locale = None
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]132
Serhiy Storchaka114454e2017-09-26 16:47:36[diff] [blame]133
Fredrik Lundhf2989b22001-02-18 12:05:16[diff] [blame]134# public symbols
Serhiy Storchakabe4de522014-09-28 12:36:18[diff] [blame]135__all__ = [
136 "match", "fullmatch", "search", "sub", "subn", "split",
137 "findall", "finditer", "compile", "purge", "template", "escape",
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]138 "error", "Pattern", "Match", "A", "I", "L", "M", "S", "X", "U",
Serhiy Storchakabe4de522014-09-28 12:36:18[diff] [blame]139 "ASCII", "IGNORECASE", "LOCALE", "MULTILINE", "DOTALL", "VERBOSE",
140 "UNICODE",
141]
Fredrik Lundhf2989b22001-02-18 12:05:16[diff] [blame]142
Fredrik Lundhbec95b92001-10-21 16:47:57[diff] [blame]143__version__ = "2.2.1"
Fredrik Lundhb25e1ad2001-03-22 15:50:10[diff] [blame]144
Victor Stinnerfb6a6592016-11-14 11:35:55[diff] [blame]145class RegexFlag(enum.IntFlag):
146 ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
147 IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
148 LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
149 UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
150 MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
151 DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
152 VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
153 A = ASCII
154 I = IGNORECASE
155 L = LOCALE
156 U = UNICODE
157 M = MULTILINE
158 S = DOTALL
159 X = VERBOSE
160 # sre extensions (experimental, don't rely on these)
161 TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
162 T = TEMPLATE
163 DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
164globals().update(RegexFlag.__members__)
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]165
166# sre exception
Fredrik Lundhbe2211e2000-06-29 16:57:40[diff] [blame]167error = sre_compile.error
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]168
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]169# --------------------------------------------------------------------
170# public interface
171
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]172def match(pattern, string, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]173 """Try to apply the pattern at the start of the string, returning
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]174 a Match object, or None if no match was found."""
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]175 return _compile(pattern, flags).match(string)
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]176
Serhiy Storchaka32eddc12013-11-23 21:20:30[diff] [blame]177def fullmatch(pattern, string, flags=0):
178 """Try to apply the pattern to all of the string, returning
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]179 a Match object, or None if no match was found."""
Serhiy Storchaka32eddc12013-11-23 21:20:30[diff] [blame]180 return _compile(pattern, flags).fullmatch(string)
181
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]182def search(pattern, string, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]183 """Scan through string looking for a match to the pattern, returning
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]184 a Match object, or None if no match was found."""
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]185 return _compile(pattern, flags).search(string)
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]186
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]187def sub(pattern, repl, string, count=0, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]188 """Return the string obtained by replacing the leftmost
189 non-overlapping occurrences of the pattern in string by the
Andrew M. Kuchling43ab0cd2004-08-07 17:41:54[diff] [blame]190 replacement repl. repl can be either a string or a callable;
Benjamin Peterson75edad02009-01-01 15:05:06[diff] [blame]191 if a string, backslash escapes in it are processed. If it is
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]192 a callable, it's passed the Match object and must return
Andrew M. Kuchling43ab0cd2004-08-07 17:41:54[diff] [blame]193 a replacement string to be used."""
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]194 return _compile(pattern, flags).sub(repl, string, count)
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]195
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]196def subn(pattern, repl, string, count=0, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]197 """Return a 2-tuple containing (new_string, number).
198 new_string is the string obtained by replacing the leftmost
199 non-overlapping occurrences of the pattern in the source
200 string by the replacement repl. number is the number of
Andrew M. Kuchling43ab0cd2004-08-07 17:41:54[diff] [blame]201 substitutions that were made. repl can be either a string or a
Benjamin Peterson75edad02009-01-01 15:05:06[diff] [blame]202 callable; if a string, backslash escapes in it are processed.
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]203 If it is a callable, it's passed the Match object and must
Andrew M. Kuchling43ab0cd2004-08-07 17:41:54[diff] [blame]204 return a replacement string to be used."""
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]205 return _compile(pattern, flags).subn(repl, string, count)
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]206
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]207def split(pattern, string, maxsplit=0, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]208 """Split the source string by the occurrences of the pattern,
Georg Brandle144c742012-03-10 08:26:53[diff] [blame]209 returning a list containing the resulting substrings. If
210 capturing parentheses are used in pattern, then the text of all
211 groups in the pattern are also returned as part of the resulting
212 list. If maxsplit is nonzero, at most maxsplit splits occur,
213 and the remainder of the string is returned as the final element
214 of the list."""
Gregory P. Smithccc5ae72009-03-02 05:21:55[diff] [blame]215 return _compile(pattern, flags).split(string, maxsplit)
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]216
Raymond Hettinger596ba4d2004-09-24 03:41:05[diff] [blame]217def findall(pattern, string, flags=0):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]218 """Return a list of all non-overlapping matches in the string.
219
Georg Brandle144c742012-03-10 08:26:53[diff] [blame]220 If one or more capturing groups are present in the pattern, return
221 a list of groups; this will be a list of tuples if the pattern
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]222 has more than one group.
223
224 Empty matches are included in the result."""
Raymond Hettinger596ba4d2004-09-24 03:41:05[diff] [blame]225 return _compile(pattern, flags).findall(string)
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]226
Serhiy Storchakabe4de522014-09-28 12:36:18[diff] [blame]227def finditer(pattern, string, flags=0):
228 """Return an iterator over all non-overlapping matches in the
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]229 string. For each match, the iterator returns a Match object.
Fredrik Lundh703ce812001-10-24 22:16:30[diff] [blame]230
Serhiy Storchakabe4de522014-09-28 12:36:18[diff] [blame]231 Empty matches are included in the result."""
232 return _compile(pattern, flags).finditer(string)
Fredrik Lundh703ce812001-10-24 22:16:30[diff] [blame]233
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]234def compile(pattern, flags=0):
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]235 "Compile a regular expression pattern, returning a Pattern object."
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]236 return _compile(pattern, flags)
237
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]238def purge():
Antoine Pitrou3060c452010-08-13 16:27:38[diff] [blame]239 "Clear the regular expression caches"
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]240 _cache.clear()
Raymond Hettinger5820f3a2016-09-19 03:17:21[diff] [blame]241 _compile_repl.cache_clear()
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]242
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]243def template(pattern, flags=0):
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]244 "Compile a template pattern, returning a Pattern object"
Fredrik Lundh436c3d582000-06-29 08:58:44[diff] [blame]245 return _compile(pattern, flags|T)
246
Serhiy Storchaka59083002017-04-13 18:06:43[diff] [blame]247# SPECIAL_CHARS
248# closing ')', '}' and ']'
249# '-' (a range in character set)
Serhiy Storchaka05cb7282017-11-16 10:38:26[diff] [blame]250# '&', '~', (extended character set operations)
Serhiy Storchaka59083002017-04-13 18:06:43[diff] [blame]251# '#' (comment) and WHITESPACE (ignored) in verbose mode
Serhiy Storchaka05cb7282017-11-16 10:38:26[diff] [blame]252_special_chars_map = {i: '\\' + chr(i) for i in b'()[]{}?*+-|^$\\.&~# \t\n\r\v\f'}
Raymond Hettingerebb7f302005-09-12 22:50:37[diff] [blame]253
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]254def escape(pattern):
Ezio Melotti88fdeb42011-04-10 09:59:16[diff] [blame]255 """
Serhiy Storchaka59083002017-04-13 18:06:43[diff] [blame]256 Escape special characters in a string.
Ezio Melotti88fdeb42011-04-10 09:59:16[diff] [blame]257 """
Guido van Rossum698280d2008-09-10 17:44:35[diff] [blame]258 if isinstance(pattern, str):
Serhiy Storchaka59083002017-04-13 18:06:43[diff] [blame]259 return pattern.translate(_special_chars_map)
Guido van Rossum698280d2008-09-10 17:44:35[diff] [blame]260 else:
Serhiy Storchaka59083002017-04-13 18:06:43[diff] [blame]261 pattern = str(pattern, 'latin1')
262 return pattern.translate(_special_chars_map).encode('latin1')
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]263
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]264Pattern = type(sre_compile.compile('', 0))
265Match = type(sre_compile.compile('', 0).match(''))
266
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]267# --------------------------------------------------------------------
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]268# internals
Guido van Rossum7627c0d2000-03-31 14:58:54[diff] [blame]269
Miss Islington (bot)39441fc2018-03-11 07:01:58[diff] [blame]270_cache = {} # ordered!
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]271
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]272_MAXCACHE = 512
Raymond Hettingerd9e8cc62010-08-14 21:17:48[diff] [blame]273def _compile(pattern, flags):
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]274 # internal: compile pattern
INADA Naokic1c47c12017-10-05 08:19:26[diff] [blame]275 if isinstance(flags, RegexFlag):
276 flags = flags.value
Serhiy Storchaka4d75a012014-09-29 15:13:02[diff] [blame]277 try:
Serhiy Storchaka898ff032017-05-05 05:53:40[diff] [blame]278 return _cache[type(pattern), pattern, flags]
Serhiy Storchaka4d75a012014-09-29 15:13:02[diff] [blame]279 except KeyError:
280 pass
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]281 if isinstance(pattern, Pattern):
Christian Heimes5fb7c2a2007-12-24 08:52:31[diff] [blame]282 if flags:
Guido van Rossum698280d2008-09-10 17:44:35[diff] [blame]283 raise ValueError(
Serhiy Storchaka632a77e2015-03-25 19:03:47[diff] [blame]284 "cannot process flags argument with a compiled pattern")
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]285 return pattern
Just van Rossum74902502003-07-02 21:37:16[diff] [blame]286 if not sre_compile.isstring(pattern):
Collin Winterce36ad82007-08-30 01:19:48[diff] [blame]287 raise TypeError("first argument must be string or compiled pattern")
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]288 p = sre_compile.compile(pattern, flags)
Serhiy Storchaka4d75a012014-09-29 15:13:02[diff] [blame]289 if not (flags & DEBUG):
Antoine Pitroud2cc7432014-02-03 19:59:59[diff] [blame]290 if len(_cache) >= _MAXCACHE:
Miss Islington (bot)39441fc2018-03-11 07:01:58[diff] [blame]291 # Drop the oldest item
Serhiy Storchaka114454e2017-09-26 16:47:36[diff] [blame]292 try:
Miss Islington (bot)39441fc2018-03-11 07:01:58[diff] [blame]293 del _cache[next(iter(_cache))]
294 except (StopIteration, RuntimeError, KeyError):
Serhiy Storchaka114454e2017-09-26 16:47:36[diff] [blame]295 pass
Serhiy Storchaka898ff032017-05-05 05:53:40[diff] [blame]296 _cache[type(pattern), pattern, flags] = p
Serhiy Storchaka0f606a62013-03-16 20:52:09[diff] [blame]297 return p
Jeremy Hyltonb1aa1952000-06-01 17:39:12[diff] [blame]298
Raymond Hettinger5820f3a2016-09-19 03:17:21[diff] [blame]299@functools.lru_cache(_MAXCACHE)
Raymond Hettingerd9e8cc62010-08-14 21:17:48[diff] [blame]300def _compile_repl(repl, pattern):
Fredrik Lundhb25e1ad2001-03-22 15:50:10[diff] [blame]301 # internal: compile replacement pattern
Raymond Hettinger5820f3a2016-09-19 03:17:21[diff] [blame]302 return sre_parse.parse_template(repl, pattern)
Fredrik Lundhb25e1ad2001-03-22 15:50:10[diff] [blame]303
Fredrik Lundh5644b7f2000-09-21 17:03:25[diff] [blame]304def _expand(pattern, match, template):
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]305 # internal: Match.expand implementation hook
Fredrik Lundh5644b7f2000-09-21 17:03:25[diff] [blame]306 template = sre_parse.parse_template(template, pattern)
307 return sre_parse.expand_template(template, match)
308
Fredrik Lundhbec95b92001-10-21 16:47:57[diff] [blame]309def _subx(pattern, template):
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]310 # internal: Pattern.sub/subn implementation helper
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]311 template = _compile_repl(template, pattern)
312 if not template[0] and len(template[1]) == 1:
Fredrik Lundhbec95b92001-10-21 16:47:57[diff] [blame]313 # literal replacement
Fredrik Lundhdac58492001-10-21 21:48:30[diff] [blame]314 return template[1][0]
315 def filter(match, template=template):
316 return sre_parse.expand_template(template, match)
317 return filter
Fredrik Lundh0640e112000-06-30 13:55:15[diff] [blame]318
319# register myself for pickling
320
Alexandre Vassalottif7fa63d2008-05-11 08:55:36[diff] [blame]321import copyreg
Fredrik Lundh0640e112000-06-30 13:55:15[diff] [blame]322
323def _pickle(p):
324 return _compile, (p.pattern, p.flags)
325
Serhiy Storchaka0b5e61d2017-10-04 17:09:49[diff] [blame]326copyreg.pickle(Pattern, _pickle, _compile)
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]327
328# --------------------------------------------------------------------
329# experimental stuff (see python-dev discussions for details)
330
331class Scanner:
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]332 def __init__(self, lexicon, flags=0):
Fredrik Lundh29c4ba92000-08-01 18:20:07[diff] [blame]333 from sre_constants import BRANCH, SUBPATTERN
INADA Naokic1c47c12017-10-05 08:19:26[diff] [blame]334 if isinstance(flags, RegexFlag):
335 flags = flags.value
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]336 self.lexicon = lexicon
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]337 # combine phrases into a compound pattern
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]338 p = []
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]339 s = sre_parse.Pattern()
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]340 s.flags = flags
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]341 for phrase, action in lexicon:
Serhiy Storchaka4eea62f2015-02-21 08:07:35[diff] [blame]342 gid = s.opengroup()
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]343 p.append(sre_parse.SubPattern(s, [
Serhiy Storchakabe9a4e52016-09-09 21:57:55[diff] [blame]344 (SUBPATTERN, (gid, 0, 0, sre_parse.parse(phrase, flags))),
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]345 ]))
Serhiy Storchaka4eea62f2015-02-21 08:07:35[diff] [blame]346 s.closegroup(gid, p[-1])
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]347 p = sre_parse.SubPattern(s, [(BRANCH, (None, p))])
Fredrik Lundh8a3ebf82000-07-23 21:46:17[diff] [blame]348 self.scanner = sre_compile.compile(p)
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]349 def scan(self, string):
350 result = []
351 append = result.append
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]352 match = self.scanner.scanner(string).match
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]353 i = 0
Serhiy Storchakaab140882014-11-11 19:13:28[diff] [blame]354 while True:
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]355 m = match()
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]356 if not m:
357 break
358 j = m.end()
359 if i == j:
360 break
Fredrik Lundh1296a8d2001-10-21 18:04:11[diff] [blame]361 action = self.lexicon[m.lastindex-1][1]
Florent Xicluna5d1155c2011-10-28 12:45:05[diff] [blame]362 if callable(action):
Fredrik Lundh770617b2001-01-14 15:06:11[diff] [blame]363 self.match = m
Fredrik Lundh7cafe4d2000-07-02 17:33:27[diff] [blame]364 action = action(self, m.group())
365 if action is not None:
366 append(action)
367 i = j
368 return result, string[i:]