-
-
Notifications
You must be signed in to change notification settings - Fork 31.8k
gh-107432: Rework the style of Doc/howto/functional.rst
#107449
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
eecc635
9bdb87c
49ca6dc
8e8ce6b
a403c17
a67d2a8
d999392
2c87aa0
e6aa077
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -1,34 +1,24 @@ | ||||||
******************************** | ||||||
Functional Programming HOWTO | ||||||
Functional Programming in Python | ||||||
******************************** | ||||||
|
||||||
:Author: A. M. Kuchling | ||||||
:Release: 0.32 | ||||||
|
||||||
In this document, we'll take a tour of Python's features suitable for | ||||||
implementing programs in a functional style. After an introduction to the | ||||||
concepts of functional programming, we'll look at language features such as | ||||||
:term:`iterator`\s and :term:`generator`\s and relevant library modules such as | ||||||
:mod:`itertools` and :mod:`functools`. | ||||||
|
||||||
|
||||||
Introduction | ||||||
============ | ||||||
|
||||||
This section explains the basic concept of functional programming; if | ||||||
you're just interested in learning about Python language features, | ||||||
skip to the next section on :ref:`functional-howto-iterators`. | ||||||
The basics of functional programming | ||||||
==================================== | ||||||
|
||||||
Programming languages support decomposing problems in several different ways: | ||||||
|
||||||
* Most programming languages are **procedural**: programs are lists of | ||||||
* Many programming languages are **procedural**: programs are lists of | ||||||
instructions that tell the computer what to do with the program's input. C, | ||||||
Pascal, and even Unix shells are procedural languages. | ||||||
|
||||||
* In **declarative** languages, you write a specification that describes the | ||||||
problem to be solved, and the language implementation figures out how to | ||||||
perform the computation efficiently. SQL is the declarative language you're | ||||||
most likely to be familiar with; a SQL query describes the data set you want | ||||||
perform the computation efficiently. SQL is an example of a declarative | ||||||
language; a SQL query describes the data set you want | ||||||
to retrieve, and the SQL engine decides whether to scan tables or use indexes, | ||||||
which subclauses should be performed first, etc. | ||||||
|
||||||
|
@@ -57,7 +47,7 @@ functional, for example. | |||||
|
||||||
In a functional program, input flows through a set of functions. Each function | ||||||
operates on its input and produces some output. Functional style discourages | ||||||
functions with side effects that modify internal state or make other changes | ||||||
functions with *side effects* that modify internal state or make other changes | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This document mostly uses strong-emphasis to indicate a term being defined. To make it consistent:
Suggested change
|
||||||
that aren't visible in the function's return value. Functions that have no side | ||||||
effects at all are called **purely functional**. Avoiding side effects means | ||||||
not using data structures that get updated as a program runs; every function's | ||||||
|
@@ -177,7 +167,7 @@ a few functions specialized for the current task. | |||||
Iterators | ||||||
========= | ||||||
|
||||||
I'll start by looking at a Python language feature that's an important | ||||||
Let's start by considering a Python language feature that's an important | ||||||
foundation for writing functional-style programs: iterators. | ||||||
|
||||||
An iterator is an object representing a stream of data; this object returns the | ||||||
|
@@ -259,7 +249,6 @@ consume all of the iterator's output, and if you need to do something different | |||||
with the same stream, you'll have to create a new iterator. | ||||||
|
||||||
|
||||||
|
||||||
Data Types That Support Iterators | ||||||
--------------------------------- | ||||||
|
||||||
|
@@ -326,7 +315,6 @@ elements:: | |||||
13 | ||||||
|
||||||
|
||||||
|
||||||
Generator expressions and list comprehensions | ||||||
============================================= | ||||||
|
||||||
|
@@ -440,7 +428,7 @@ Generators are a special class of functions that simplify the task of writing | |||||
iterators. Regular functions compute a value and return it, but generators | ||||||
return an iterator that returns a stream of values. | ||||||
|
||||||
You're doubtless familiar with how regular function calls work in Python or C. | ||||||
You may be familiar with how regular function calls work in Python or C. | ||||||
When you call a function, it gets a private namespace where its local variables | ||||||
are created. When the function reaches a ``return`` statement, the local | ||||||
variables are destroyed and the value is returned to the caller. A later call | ||||||
|
@@ -528,19 +516,16 @@ Passing values into a generator | |||||
------------------------------- | ||||||
|
||||||
In Python 2.4 and earlier, generators only produced output. Once a generator's | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we could drop the versions here and just describe the state of the language in Python 3.x. Maybe drop the first paragraph with "there was no elegant way..." entirely, and the new first paragraph would be something like:
|
||||||
code was invoked to create an iterator, there was no way to pass any new | ||||||
information into the function when its execution is resumed. You could hack | ||||||
together this ability by making the generator look at a global variable or by | ||||||
passing in some mutable object that callers then modify, but these approaches | ||||||
are messy. | ||||||
code was invoked to create an iterator, there was no elegant way to pass any new | ||||||
information into the function when its execution is resumed. | ||||||
|
||||||
In Python 2.5 there's a simple way to pass values into a generator. | ||||||
:keyword:`yield` became an expression, returning a value that can be assigned to | ||||||
a variable or otherwise operated on:: | ||||||
In Python 2.5, :keyword:`yield` became an expression, returning a value that | ||||||
can be assigned to a variable or otherwise operated on, providing a simple | ||||||
way to pass values into a generator:: | ||||||
|
||||||
val = (yield i) | ||||||
|
||||||
I recommend that you **always** put parentheses around a ``yield`` expression | ||||||
It's recommended that you **always** put parentheses around a ``yield`` expression | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The current wording is more concise. It's also more precise -- with your wording, I'm left wondering who exactly is recommending this? Lots of things in this article are just one person's opinion, and never claim to be anything else, but with your rewording here, it makes it sound like it's the official pronouncement from on high about the proper programming style for Python. Removing the usage of the first person throughout the article really muddies the water here about the purpose of this article, in my opinion. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If they are just one person's opinion, they don't belong in the documentation. The documentation's contract with the user is exactly that: that it is the authoritative pronouncement from on-high about programming in Python. We would never allow the code to be so permissive and lax, to represent merely the way someone feels it's good to do it, and we should hold documentation to the same standards. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Says who? Again, that's not how the CPython project has traditionally thought of HOWTO documents, and it's not what the Personally, I think it's great that we have a space in the CPython documentation for more subjective, personal reflections on programming practices. I'd much prefer we say "it's not a how-to" and move it to some other directory than try to turn the document into something it's not. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I understood - I believe - that there is consensus on adopting Diátaxis for Python's documentation. There have been discussions, I ran a couple of workshops with the documentation team, and so on. It's a big project, and it's going to take a long time. My goal is to help set new standards of quality in Python's documentation. It is going to change a lot, and set new expectations. There isn't any other way around this.
Yes, that is precisely what I described as the goal in the original commit for this pull request. These are steps along the way. But even explanation-type documentation, which is what this particular document is, needs to have authority. Otherwise it is simply not documentation. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I've liked a lot of what I've seen from the Diataxis work on the CPython docs -- I think it's resulted in some very worthwhile improvements to the docs, and I'm excited to see it continue! We need to consider each change individually, however. There may be broad consensus about using Diataxis as a framework for the CPython docs overall, but I don't really feel like I've seen any kind of consensus-building about what to do with these older HOWTO documents specifically. Endorsement of a general principle/framework is a pretty different thing to endorsing a specific application of that framework that arguably involves redefining the purpose of an entire directory. I understand that the pace of the change here might be frustrating for you, and I'm sorry for that, but careful review of each change really is necessary, in my opinion.
(Super minor nitpick, but there is no "documentation team". There's a docs community -- a group of people who are interested and enthusiastic about improving the CPython docs. But docs contributions come from a wide variety of people, many of whom aren't part of that community, and there are lots of core developers interested in documentation who aren't part of that community. I think it would be a mistake to assume that the people in that community are necessarily representative of the Python community as a whole 🙂) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I don't understand -- if the goal is to move this document out of the |
||||||
when you're doing something with the returned value, as in the above example. | ||||||
The parentheses aren't always necessary, but it's easier to always add them | ||||||
instead of having to remember when they're needed. | ||||||
|
@@ -608,8 +593,8 @@ generators: | |||||
will also be called by Python's garbage collector when the generator is | ||||||
garbage-collected. | ||||||
|
||||||
If you need to run cleanup code when a :exc:`GeneratorExit` occurs, I suggest | ||||||
using a ``try: ... finally:`` suite instead of catching :exc:`GeneratorExit`. | ||||||
If you need to run cleanup code when a :exc:`GeneratorExit` occurs, it's recommended | ||||||
to use a ``try: ... finally:`` suite instead of catching :exc:`GeneratorExit`. | ||||||
Comment on lines
+596
to
+597
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I prefer the current wording, which is more concise, and more upfront about the fact that this is one person's opinion. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I doubt I had a personal opinion about catching vs. try...finally, & vaguely think I got the suggestion from somewhere. Could it have been from python-dev discussion when the feature was being added? Or maybe it was just general concern about the complexity of dealing with the GeneratorExit; do you need to re-raise it once your cleanup is done? Maybe this should say "it's simpler to use try...finally" instead of 'recommended'? (And do 3.x async features change any of this?) |
||||||
|
||||||
The cumulative effect of these changes is to turn generators from one-way | ||||||
producers of information into both producers and consumers. | ||||||
|
@@ -639,7 +624,7 @@ features of generator expressions: | |||||
>>> [upper(s) for s in ['sentence', 'fragment']] | ||||||
['SENTENCE', 'FRAGMENT'] | ||||||
|
||||||
You can of course achieve the same effect with a list comprehension. | ||||||
(You can achieve the same effect with a list comprehension.) | ||||||
|
||||||
:func:`filter(predicate, iter) <filter>` returns an iterator over all the | ||||||
sequence elements that meet a certain condition, and is similarly duplicated by | ||||||
|
@@ -1131,20 +1116,21 @@ usual way:: | |||||
def print_assign(name, value): | ||||||
return name + '=' + str(value) | ||||||
|
||||||
Which alternative is preferable? That's a style question; my usual course is to | ||||||
avoid using ``lambda``. | ||||||
Which alternative is preferable? That's mostly a question of style. | ||||||
Comment on lines
-1134
to
+1119
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems to me to be an unnecessary edit |
||||||
|
||||||
One reason for my preference is that ``lambda`` is quite limited in the | ||||||
You may wish to avoid using ``lambda``, as there are limits to the | ||||||
Comment on lines
-1137
to
+1121
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't find this necessary |
||||||
functions it can define. The result has to be computable as a single | ||||||
expression, which means you can't have multiway ``if... elif... else`` | ||||||
comparisons or ``try... except`` statements. If you try to do too much in a | ||||||
``lambda`` statement, you'll end up with an overly complicated expression that's | ||||||
hard to read. Quick, what's the following code doing? :: | ||||||
hard to read. | ||||||
|
||||||
Consider:: | ||||||
Comment on lines
-1142
to
+1128
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The use of a rhetorical question aimed at the reader has the effect of shaking up the pace of the prose by suddenly throwing in a different kind of sentence construction. I think it's an enjoyable effect, and don't think it needs to be removed There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, most of the changes above to remove 'I suggest' or 'I think' are fine with me, but here the sentence is trying to ask a somewhat punchy question. |
||||||
|
||||||
import functools | ||||||
total = functools.reduce(lambda a, b: (0, a[1] + b[1]), items)[1] | ||||||
|
||||||
You can figure it out, but it takes time to disentangle the expression to figure | ||||||
It takes some mental effort to disentangle the expression to figure | ||||||
out what's going on. Using a short nested ``def`` statements makes things a | ||||||
little bit better:: | ||||||
|
||||||
|
@@ -1154,7 +1140,7 @@ little bit better:: | |||||
|
||||||
total = functools.reduce(combine, items)[1] | ||||||
|
||||||
But it would be best of all if I had simply used a ``for`` loop:: | ||||||
But best of all would have been to use a ``for`` loop:: | ||||||
|
||||||
total = 0 | ||||||
for a, b in items: | ||||||
|
@@ -1166,19 +1152,6 @@ Or the :func:`sum` built-in and a generator expression:: | |||||
|
||||||
Many uses of :func:`functools.reduce` are clearer when written as ``for`` loops. | ||||||
|
||||||
Fredrik Lundh once suggested the following set of rules for refactoring uses of | ||||||
``lambda``: | ||||||
|
||||||
1. Write a lambda function. | ||||||
2. Write a comment explaining what the heck that lambda does. | ||||||
3. Study the comment for a while, and think of a name that captures the essence | ||||||
of the comment. | ||||||
4. Convert the lambda to a def statement, using that name. | ||||||
5. Remove the comment. | ||||||
|
||||||
I really like these rules, but you're free to disagree | ||||||
about whether this lambda-free style is better. | ||||||
|
||||||
Comment on lines
-1169
to
-1181
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't understand why this section needs to be deleted. It seems to me to be a useful way of thinking about the problem. It's true that it's written in a slightly more informal style, and a slightly more humorous style, than a lot of the rest of Python's documentation. But Python's I disagree with retrofitting a new, stricter definition of what it means for a document to be a HOWTO guide onto Python's There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A key problem with this page of documentation is that it is not a how-to guide. Whatever else needs to be done with it, it's in the wrong place. If a it does something other than guide the user through a task or problem, it's literally not a how-to guide. Names do matter. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
It is a guide that explains how to apply the functional programming paradigm to Python code There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Some of these HOWTOs, like this one and the Unicode one, were intros/justifications at the time for newly introduced language features, but now could use editing to just describe the current feature set, with less emphasis on their newness. I was also hoping that this would encourage others to write HOWTOs on particular topics such as secure Python programming, but that didn't work and sites like Stack Overflow largely took over that function. They could be called something like 'Learning paths' or 'topic guides', but no better term ever became clear. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. BTW, here's a more detailed reference for Fredrik Lundh's comment, dated April 1 2001, if you wanted to link it. Should it describe Fredrik somehow, as "long-time Python contributor"? Should it specify "the late"? |
||||||
|
||||||
Revision History and Acknowledgements | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it necessary to keep this "Revision History and Acknowledgements" section? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the history can be removed, Acknowledgements could be moved to the end (after References) or removed. The references section should also probably have the 'General' subhead removed (I can't add in-line comments there). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed; we can drop it. We could also drop the version number at the top of the file. |
||||||
===================================== | ||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why remove this paragraph? It seems useful for readers to have a summary at the beginning of what the article's going to cover.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A user's eyes will skip over this. They will just bounce off the new reader, for whom they make little connection. They will not add value for the reader who is returning to the page. The links don't take the reader to the topics in the page, but take them away from it, so they don't serve as useful signposts to what's in the page.
If anyone wants to see what's in the page, they can see that immediately from the page's table of contents, where they will find the topics in the context and order of other topics, and can see their place in the hierarchy of concepts.
In general, summaries of what something is going to discuss (and summaries of what has been discussed) add nothing of value to writing. (A summary of conclusions is a quite different matter.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's useful to mention
functools
anditertools
early-on as concrete examples of standard-library modules that allow you to use the functional programming style in Python, especially since the opening section is so abstract. I wouldn't mind it if you changed the links so that they pointed to the passages later on in this document where it explores theitertools
andfunctools
modules.