Skip to content

bpo-43292: Fix file leak in ET.iterparse() when not exhausted #31696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 7, 2022
Merged

bpo-43292: Fix file leak in ET.iterparse() when not exhausted #31696

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1495ccd
bpo-43292: Fix file leak in iterparse when not exhausted
jacobtylerwalls Mar 5, 2022
b29dba7
Update ElementTree.py
serhiy-storchaka Mar 6, 2022
f532022
Update Lib/test/test_xml_etree.py
serhiy-storchaka Mar 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Lib/test/test_xml_etree.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -658,6 +658,14 @@ def test_iterparse(self):
'junk after document element: line 1, column 12')
del cm, it

# Not exhausting the iterator still closes the resource (bpo-43292)
with warnings_helper.check_no_resource_warning(self):
it = iterparse(TESTFN)
del it

with self.assertRaises(FileNotFoundError):
iterparse("nonexistent")

def test_writefile(self):
elem = ET.Element("tag")
elem.text = "text"
Expand Down
16 changes: 9 additions & 7 deletions Lib/xml/etree/ElementTree.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1244,8 +1244,14 @@ def iterparse(source, events=None, parser=None):
# Use the internal, undocumented _parser argument for now; When the
# parser argument of iterparse is removed, this can be killed.
pullparser = XMLPullParser(events=events, _parser=parser)
def iterator():

def iterator(source):
close_source = False
try:
if not hasattr(source, "read"):
source = open(source, "rb")
close_source = True
yield None
while True:
yield from pullparser.read_events()
# load event buffer
Expand All @@ -1261,16 +1267,12 @@ def iterator():
source.close()

class IterParseIterator(collections.abc.Iterator):
__next__ = iterator().__next__
__next__ = iterator(source).__next__
it = IterParseIterator()
it.root = None
del iterator, IterParseIterator

close_source = False
if not hasattr(source, "read"):
source = open(source, "rb")
close_source = True

next(it)
return it


Expand Down
1 change: 1 addition & 0 deletions Misc/ACKS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1879,6 +1879,7 @@ Wojtek Walczak
Charles Waldman
Richard Walker
Larry Wall
Jacob Walls
Kevin Walzer
Rodrigo Steinmuller Wanderley
Dingyuan Wang
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2022-03-05-09-43-53.bpo-25707.gTlclP.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fixed a file leak in :func:`xml.etree.ElementTree.iterparse` when the
iterator is not exhausted. Patch by Jacob Walls.