Skip to content

bpo-11416: handle multiple .netrc entries per host #17823

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions Doc/library/netrc.rst
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ the Unix :program:`ftp` program and other FTP clients.
:func:`os.path.expanduser` is used to find the location of the
:file:`.netrc` file when *file* is not passed as argument.

.. versionchanged:: 3.9 Handles multiple entries for the same host.


.. exception:: NetrcParseError

Expand All @@ -56,18 +58,22 @@ netrc Objects
A :class:`~netrc.netrc` instance has the following methods:


.. method:: netrc.authenticators(host)
.. method:: netrc.authenticators(host, [login])

Return a 3-tuple ``(login, account, password)`` of authenticators for *host*.
Return a 3-tuple ``(login, account, password)`` of authenticators for *host*
and *login*.
If the netrc file did not contain an entry for the given host, return the tuple
associated with the 'default' entry. If neither matching host nor default entry
is available, return ``None``.
If optional parameter *login* is not provided, it returns the first
authenticators for the matching host. For further information see
:class:`~netrc.netrc._use_first`.


.. method:: netrc.__repr__()

Dump the class data as a string in the format of a netrc file. (This discards
comments and may reorder the entries.)
comments.)

Instances of :class:`~netrc.netrc` have public instance variables:

Expand All @@ -76,12 +82,33 @@ Instances of :class:`~netrc.netrc` have public instance variables:

Dictionary mapping host names to ``(login, account, password)`` tuples. The
'default' entry, if any, is represented as a pseudo-host by that name.
This dictionary only contains a single entry per host. For further information
see :class:`~netrc.netrc._use_first`.


.. attribute:: netrc.macros

Dictionary mapping macro names to string lists.

Configuration of class :class:`~netrc.netrc` behavior:


.. attribute:: netrc._use_first

Controls the order of machine entries for the same host. If *True*,
:class:`~netrc.netrc.authenticators` will return the first entry for a host,
when called without providing *login*. Also :class:`~netrc.netrc.hosts` will
contain the first entry for this machine. This would be inline with the common
netrc implementation of other Unix tools.
For backward compatibility the default value is *False*, i.e. the last item
is used when no *login* is provided or :class:`~netrc.netrc.hosts` is looked up.
Libraries shall not change this value (hence it is not a constructor
parameter). The intention is that the end-user can configure the target Python
installation (e.g. via sitecustomize) for consistent behavior.

.. versionchanged:: 3.9
Added to control entry order for hosts with multiple login.

.. note::

Passwords are limited to a subset of the ASCII character set. All ASCII
Expand Down
39 changes: 27 additions & 12 deletions Lib/netrc.py
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,15 @@ def __str__(self):


class netrc:
_use_first = False

def __init__(self, file=None):
self._use_first = self._use_first
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't use self._use_first = False

default_netrc = file is None
if file is None:
file = os.path.join(os.path.expanduser("~"), ".netrc")
self.hosts = {}
self.entries = []
self.macros = {}
with open(file) as fp:
self._parse(file, fp, default_netrc)
Expand All @@ -45,10 +49,13 @@ def _parse(self, file, fp, default_netrc):
continue
elif tt == 'machine':
entryname = lexer.get_token()
default_entry = False
elif tt == 'default':
entryname = 'default'
default_entry = True
elif tt == 'macdef': # Just skip to end of macdefs
entryname = lexer.get_token()
default_entry = False
self.macros[entryname] = []
lexer.whitespace = ' \t'
while 1:
Expand All @@ -65,13 +72,16 @@ def _parse(self, file, fp, default_netrc):
# We're looking at start of an entry for a named machine or default.
login = ''
account = password = None
self.hosts[entryname] = {}
while 1:
tt = lexer.get_token()
if (tt.startswith('#') or
tt in {'', 'machine', 'default', 'macdef'}):
if password:
self.hosts[entryname] = (login, account, password)
entry = (login, account, password)
if entryname not in self.hosts or not self._use_first:
self.hosts[entryname] = entry
entryhost = None if default_entry else entryname
self.entries.append((entryhost, *entry))
lexer.push_token(tt)
break
else:
Expand Down Expand Up @@ -110,21 +120,26 @@ def _parse(self, file, fp, default_netrc):
raise NetrcParseError("bad follower token %r" % tt,
file, lexer.lineno)

def authenticators(self, host):
def authenticators(self, host, login=None):
"""Return a (user, account, password) tuple for given host."""
if host in self.hosts:
return self.hosts[host]
elif 'default' in self.hosts:
return self.hosts['default']
else:
return None
direction = 1 if self._use_first else -1
default_auth = None
for entry in self.entries[::direction]:
if login is None or entry[1] == login:
if entry[0] == host:
return entry[1:]
elif entry[0] is None and default_auth is None:
default_auth = entry[1:]
return default_auth

def __repr__(self):
"""Dump the class data in the format of a .netrc file."""
rep = ""
for host in self.hosts.keys():
attrs = self.hosts[host]
rep += f"machine {host}\n\tlogin {attrs[0]}\n"
for entry in self.entries:
host = entry[0]
attrs = entry[1:]
machine = "default" if host is None else f"machine {host}"
rep += f"{machine}\n\tlogin {attrs[0]}\n"
if attrs[1]:
rep += f"\taccount {attrs[1]}\n"
rep += f"\tpassword {attrs[2]}\n"
Expand Down
86 changes: 85 additions & 1 deletion Lib/test/test_netrc.py
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
import netrc, os, unittest, sys, tempfile, textwrap
import netrc, os, unittest, sys, tempfile, textwrap, contextlib, re
from test import support

@contextlib.contextmanager
def set_use_first(value):
original_value = netrc.netrc._use_first
try:
netrc.netrc._use_first = value
yield
finally:
netrc.netrc._use_first = original_value


class NetrcTestCase(unittest.TestCase):

Expand Down Expand Up @@ -59,6 +68,19 @@ def test_password_with_internal_hash(self):
machine host.domain.com login log password pa#ss account acct
""", 'pa#ss')

def test_password_for_multiple_entries_default_behavior(self):
self._test_passwords("""\
machine host.domain.com login log password pass_1 account acct
machine host.domain.com login log password pass_2 account acct
""", 'pass_2')

def test_password_for_multiple_entries_use_first(self):
with set_use_first(True):
self._test_passwords("""\
machine host.domain.com login log password pass_1 account acct
machine host.domain.com login log password pass_2 account acct
""", 'pass_1')

def _test_comment(self, nrc, passwd='pass'):
nrc = self.make_nrc(nrc)
self.assertEqual(nrc.hosts['foo.domain.com'], ('bar', None, passwd))
Expand Down Expand Up @@ -103,6 +125,68 @@ def test_comment_at_end_of_machine_line_pass_has_hash(self):
machine bar.domain.com login foo password pass
""", '#pass')

def _test_authenticators(self, nrc, login, passwd):
nrc = self.make_nrc(nrc)
self.assertEqual(
nrc.authenticators('host.domain.com', login),
(login or 'log', 'acct', passwd)
)

def test_authenticators(self):
self._test_authenticators("""\
machine host.domain.com login log password pass account acct
""", 'log', 'pass')

def test_authenticators_multiple_entries_default_behavior(self):
self._test_authenticators("""\
machine host.domain.com login log password pass_1 account acct
machine host.domain.com login log password pass_2 account acct
""", None, 'pass_2')

def test_authenticators_multiple_entries_use_first(self):
with set_use_first(True):
self._test_authenticators("""\
machine host.domain.com login log password pass_1 account acct
machine host.domain.com login log password pass_2 account acct
""", 'log', 'pass_1')

def test_authenticators_multiple_entries_select_user(self):
with set_use_first(True):
self._test_authenticators("""\
machine host.domain.com login log_1 password pass_1 account acct
machine host.domain.com login log_2 password pass_2 account acct
""", 'log_2', 'pass_2')

def test_authenticators_multiple_entries_select_default_user(self):
with set_use_first(True):
self._test_authenticators("""\
machine host.domain.com login log_1 password pass_1 account acct
machine host.domain.com login log_2 password pass_2 account acct
default login log_3 password pass_3 account acct
""", 'log_3', 'pass_3')

def _test_repr_simplified(self, nrc):
# assumes order: login, account, password
# comments are not handled
_nrc = self.make_nrc(nrc)
self.assertEqual(
re.sub(r'\s+', ' ', repr(_nrc)).strip(),
re.sub(r'\s+', ' ', nrc).strip(),
)

def test_repr(self):
self._test_repr_simplified("""\
machine host.domain.com login log account acct password pass
""")

def test_repr_multiple_entries(self):
self.maxDiff = None
self._test_repr_simplified("""\
machine host.domain.com login log_1 account acct password pass
machine host.domain.com login log_2 account acct password pass
default login log_3 account acct password pass_3
""")


@unittest.skipUnless(os.name == 'posix', 'POSIX only test')
def test_security(self):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Class :class:`~netrc.netrc` supports multiple entries for the same machine, lookup by login name and configurable lookup order.