diff --git a/.gitignore b/.gitignore deleted file mode 100644 index ada944a53..000000000 --- a/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -.idea -.Rhistory diff --git a/README.md b/README.md index 46ecd60e8..001c16282 100644 --- a/README.md +++ b/README.md @@ -1,204 +1,238 @@ Comprehensive Python Cheatsheet =============================== -[Download text file](https://raw.githubusercontent.com/gto76/python-cheatsheet/master/README.md), [Fork me on GitHub](https://github.com/gto76/python-cheatsheet) or [Check out FAQ](https://github.com/gto76/python-cheatsheet/wiki/Frequently-Asked-Questions). +[Download text file](https://raw.githubusercontent.com/gto76/python-cheatsheet/main/README.md), [Fork me on GitHub](https://github.com/gto76/python-cheatsheet) or [Check out FAQ](https://github.com/gto76/python-cheatsheet/wiki/Frequently-Asked-Questions). ![Monty Python](web/image_888.jpeg) +Contents +-------- +**   ** **1. Collections:** ** ** **[`List`](#list)**__,__ **[`Dictionary`](#dictionary)**__,__ **[`Set`](#set)**__,__ **[`Tuple`](#tuple)**__,__ **[`Range`](#range)**__,__ **[`Enumerate`](#enumerate)**__,__ **[`Iterator`](#iterator)**__,__ **[`Generator`](#generator)**__.__ +**   ** **2. Types:** **          ** **[`Type`](#type)**__,__ **[`String`](#string)**__,__ **[`Regular_Exp`](#regex)**__,__ **[`Format`](#format)**__,__ **[`Numbers`](#numbers-1)**__,__ **[`Combinatorics`](#combinatorics)**__,__ **[`Datetime`](#datetime)**__.__ +**   ** **3. Syntax:** **         ** **[`Function`](#function)**__,__ **[`Inline`](#inline)**__,__ **[`Import`](#imports)**__,__ **[`Decorator`](#decorator)**__,__ **[`Class`](#class)**__,__ **[`Duck_Type`](#duck-types)**__,__ **[`Enum`](#enum)**__,__ **[`Except`](#exceptions)**__.__ +**   ** **4. System:** **        ** **[`Exit`](#exit)**__,__ **[`Print`](#print)**__,__ **[`Input`](#input)**__,__ **[`Command_Line_Arguments`](#command-line-arguments)**__,__ **[`Open`](#open)**__,__ **[`Path`](#paths)**__,__ **[`OS_Commands`](#os-commands)**__.__ +**   ** **5. Data:** **             ** **[`JSON`](#json)**__,__ **[`Pickle`](#pickle)**__,__ **[`CSV`](#csv)**__,__ **[`SQLite`](#sqlite)**__,__ **[`Bytes`](#bytes)**__,__ **[`Struct`](#struct)**__,__ **[`Array`](#array)**__,__ **[`Memory_View`](#memory-view)**__,__ **[`Deque`](#deque)**__.__ +**   ** **6. Advanced:** **   ** **[`Operator`](#operator)**__,__ **[`Match_Stmt`](#match-statement)**__,__ **[`Logging`](#logging)**__,__ **[`Introspection`](#introspection)**__,__ **[`Threading`](#threading)**__,__ **[`Coroutines`](#coroutines)**__.__ +**   ** **7. Libraries:** **      ** **[`Progress_Bar`](#progress-bar)**__,__ **[`Plot`](#plot)**__,__ **[`Table`](#table)**__,__ **[`Console_App`](#console-app)**__,__ **[`GUI`](#gui-app)**__,__ **[`Scraping`](#scraping)**__,__ **[`Web`](#web-app)**__,__ **[`Profile`](#profiling)**__.__ +**   ** **8. Multimedia:** **  ** **[`NumPy`](#numpy)**__,__ **[`Image`](#image)**__,__ **[`Animation`](#animation)**__,__ **[`Audio`](#audio)**__,__ **[`Synthesizer`](#synthesizer)**__,__ **[`Pygame`](#pygame)**__,__ **[`Pandas`](#pandas)**__,__ **[`Plotly`](#plotly)**__.__ + + Main ---- ```python -if __name__ == '__main__': - main() +if __name__ == '__main__': # Skips next line if file was imported. + main() # Runs `def main(): ...` function. ``` List ---- ```python - = [from_inclusive : to_exclusive : step_size] + = [, , ...] # Creates a list object. Also list(). +``` + +```python + = [index] # First index is 0. Last -1. Allows assignments. + = [] # Also [from_inclusive : to_exclusive : ±step]. ``` ```python -.append() # Or: += [] -.extend() # Or: += +.append() # Appends element to the end. Also += []. +.extend() # Appends elements to the end. Also += . ``` ```python -.sort() -.reverse() - = sorted() - = reversed() +.sort() # Sorts the elements in ascending order. +.reverse() # Reverses the order of list's elements. + = sorted() # Returns a new list with sorted elements. + = reversed() # Returns reversed iterator of elements. +``` + +```python + = max() # Returns largest element. Also min(, ...). + = sum() # Returns sum of elements. Also math.prod(). ``` ```python -sum_of_elements = sum() elementwise_sum = [sum(pair) for pair in zip(list_a, list_b)] sorted_by_second = sorted(, key=lambda el: el[1]) sorted_by_both = sorted(, key=lambda el: (el[1], el[0])) flatter_list = list(itertools.chain.from_iterable()) -product_of_elems = functools.reduce(lambda out, x: out * x, ) -list_of_chars = list() ``` +* **For details about sort(), sorted(), min() and max() see [Sortable](#sortable).** +* **Module [operator](#operator) has function itemgetter() that can replace listed [lambdas](#lambda).** +* **This text uses the term collection instead of iterable. For rationale see [Collection](#collection).** ```python -index = .index() # Returns first index of item. -.insert(index, ) # Inserts item at index and moves the rest to the right. - = .pop([index]) # Removes and returns item at index or from the end. -.remove() # Removes first occurrence of item or raises ValueError. -.clear() # Removes all items. + = len() # Returns number of items. Also works on dict, set and string. + = .count() # Returns number of occurrences. Also `if in : ...`. + = .index() # Returns index of the first occurrence or raises ValueError. + = .pop() # Removes and returns item from the end or at index if passed. +.insert(, ) # Inserts item at passed index and moves the rest to the right. +.remove() # Removes first occurrence of the item or raises ValueError. +.clear() # Removes all list's items. Also works on dictionary and set. ``` Dictionary ---------- ```python - = .keys() - = .values() - = .items() + = {key_1: val_1, key_2: val_2, ...} # Use `[key]` to get or set the value. +``` + +```python + = .keys() # Collection of keys that reflects changes. + = .values() # Collection of values that reflects changes. + = .items() # Coll. of key-value tuples that reflects chgs. ``` ```python -value = .get(key, default=None) # Returns default if key does not exist. -value = .setdefault(key, default=None) # Same, but also adds default to dict. - = collections.defaultdict() # Creates a dict with default value of type. - = collections.defaultdict(lambda: 1) # Creates a dict with default value 1. +value = .get(key, default=None) # Returns default argument if key is missing. +value = .setdefault(key, default=None) # Returns and writes default if key is missing. + = collections.defaultdict() # Returns a dict with default value `()`. + = collections.defaultdict(lambda: 1) # Returns a dict with default value 1. ``` ```python -.update() # Or: dict_a = {**dict_a, **dict_b}. = dict() # Creates a dict from coll. of key-value pairs. = dict(zip(keys, values)) # Creates a dict from two collections. = dict.fromkeys(keys [, value]) # Creates a dict from collection of keys. ``` ```python -value = .pop(key) # Removes item from dictionary. -{k: v for k, v in .items() if k in keys} # Filters dictionary by keys. +.update() # Adds items. Replaces ones with matching keys. +value = .pop(key) # Removes item or raises KeyError if missing. +{k for k, v in .items() if v == value} # Returns set of keys that point to the value. +{k: v for k, v in .items() if k in keys} # Filters the dictionary by specified keys. ``` ### Counter ```python >>> from collections import Counter ->>> colors = ['red', 'blue', 'yellow', 'blue', 'red', 'blue'] ->>> counter = Counter(colors) -Counter({'blue': 3, 'red': 2, 'yellow': 1}) ->>> counter.most_common()[0] -('blue', 3) +>>> counter = Counter(['blue', 'blue', 'blue', 'red', 'red']) +>>> counter['yellow'] += 1 +>>> print(counter.most_common()) +[('blue', 3), ('red', 2), ('yellow', 1)] ``` Set --- ```python - = set() + = {, , ...} # Use `set()` for empty set. ``` ```python -.add() # Or: |= {} -.update() # Or: |= +.add() # Or: |= {} +.update( [, ...]) # Or: |= ``` ```python - = .union() # Or: | - = .intersection() # Or: & - = .difference() # Or: - - = .symmetric_difference() # Or: ^ - = .issubset() # Or: <= - = .issuperset() # Or: >= + = .union() # Or: | + = .intersection() # Or: & + = .difference() # Or: - + = .symmetric_difference() # Or: ^ + = .issubset() # Or: <= + = .issuperset() # Or: >= ``` ```python -.remove() # Throws error. -.discard() # Doesn't throw error. + = .pop() # Raises KeyError if empty. +.remove() # Raises KeyError if missing. +.discard() # Doesn't raise an error. ``` -### Frozenset -#### Is hashable, meaning it can be used as a key in a dictionary or as an element in a set. +### Frozen Set +* **Is immutable and hashable.** +* **That means it can be used as a key in a dictionary or as an element in a set.** ```python = frozenset() ``` -Range +Tuple ----- +**Tuple is an immutable and hashable list.** ```python - = range(to_exclusive) - = range(from_inclusive, to_exclusive) - = range(from_inclusive, to_exclusive, ±step_size) + = () # Empty tuple. + = (,) # Or: , + = (, [, ...]) # Or: , [, ...] ``` +### Named Tuple +**Tuple's subclass with named elements.** ```python -from_inclusive = .start -to_exclusive = .stop +>>> from collections import namedtuple +>>> Point = namedtuple('Point', 'x y') +>>> p = Point(1, y=2) +>>> print(p) +Point(x=1, y=2) +>>> p.x, p[1] +(1, 2) ``` -Enumerate ---------- +Range +----- +**Immutable and hashable sequence of integers.** ```python -for i, el in enumerate( [, i_start]): - ... + = range(stop) # I.e. range(to_exclusive). + = range(start, stop) # I.e. range(from_inclusive, to_exclusive). + = range(start, stop, ±step) # I.e. range(from_inclusive, to_exclusive, ±step). ``` +```python +>>> [i for i in range(3)] +[0, 1, 2] +``` -Named Tuple ------------ -* **Tuple is an immutable and hashable list.** -* **Named tuple is its subclass with named elements.** +Enumerate +--------- ```python ->>> from collections import namedtuple ->>> Point = namedtuple('Point', 'x y') ->>> p = Point(1, y=2) -Point(x=1, y=2) ->>> p[0] -1 ->>> p.x -1 ->>> getattr(p, 'y') -2 ->>> p._fields # Or: Point._fields -('x', 'y') +for i, el in enumerate(, start=0): # Returns next element and its index on each pass. + ... ``` Iterator -------- -**In this cheatsheet `''` can also mean an iterator.** +**Potentially endless stream of elements.** ```python -from itertools import count, repeat, cycle, chain, islice + = iter() # `iter()` returns unmodified iterator. + = iter(, to_exclusive) # A sequence of return values until 'to_exclusive'. + = next( [, default]) # Raises StopIteration or returns 'default' on end. + = list() # Returns a list of iterator's remaining elements. ``` +### Itertools ```python - = iter() - = iter(, to_exclusive) # Sequence of return values until 'to_exclusive'. - = next( [, default]) # Raises StopIteration or returns 'default' on end. +import itertools as it ``` ```python - = count(start=0, step=1) # Returns incremented value endlessly. - = repeat( [, times]) # Returns element endlessly or 'times' times. - = cycle() # Repeats the sequence indefinitely. + = it.count(start=0, step=1) # Returns updated value endlessly. Accepts floats. + = it.repeat( [, times]) # Returns element endlessly or 'times' times. + = it.cycle() # Repeats the passed sequence of elements endlessly. ``` ```python - = chain(, , ...) # Empties collections in order. - = chain.from_iterable() # Empties collections inside a collection in order. + = it.chain(, [, ...]) # Empties collections in order (only figuratively). + = it.chain.from_iterable() # Empties collections inside a collection in order. ``` ```python - = islice(, to_exclusive) - = islice(, from_inclusive, to_exclusive) - = islice(, from_inclusive, to_exclusive, step_size) + = it.islice(, to_exclusive) # Only returns first 'to_exclusive' elements. + = it.islice(, from_inc, …) # `to_exclusive, +step_size`. Indices can be None. ``` Generator --------- -**Convenient way to implement the iterator protocol.** +* **Any function that contains a yield statement returns a generator.** +* **Generators and iterators are interchangeable.** ```python def count(start, step): @@ -216,329 +250,456 @@ def count(start, step): Type ---- +* **Everything is an object.** +* **Every object has a type.** +* **Type and class are synonymous.** + +```python + = type() # Or: .__class__ + = isinstance(, ) # Or: issubclass(type(), ) +``` + +```python +>>> type('a'), 'a'.__class__, str +(, , ) +``` + +#### Some types do not have built-in names, so they must be imported: ```python - = type() # / / ... +from types import FunctionType, MethodType, LambdaType, GeneratorType, ModuleType ``` +### Abstract Base Classes +**Each abstract base class specifies a set of virtual subclasses. These classes are then recognized by isinstance() and issubclass() as subclasses of the ABC, although they are really not. ABC can also manually decide whether or not a specific class is its virtual subclass, usually based on which methods the class has implemented. For instance, Iterable ABC looks for method iter(), while Collection ABC looks for iter(), contains() and len().** + ```python -from numbers import Integral, Rational, Real, Complex, Number - = isinstance(, Number) +>>> from collections.abc import Iterable, Collection, Sequence +>>> isinstance([1, 2, 3], Iterable) +True +``` + +```text ++------------------+------------+------------+------------+ +| | Iterable | Collection | Sequence | ++------------------+------------+------------+------------+ +| list, range, str | yes | yes | yes | +| dict, set | yes | yes | | +| iter | yes | | | ++------------------+------------+------------+------------+ ``` ```python - = callable() +>>> from numbers import Number, Complex, Real, Rational, Integral +>>> isinstance(123, Number) +True +``` + +```text ++--------------------+----------+----------+----------+----------+----------+ +| | Number | Complex | Real | Rational | Integral | ++--------------------+----------+----------+----------+----------+----------+ +| int | yes | yes | yes | yes | yes | +| fractions.Fraction | yes | yes | yes | yes | | +| float | yes | yes | yes | | | +| complex | yes | yes | | | | +| decimal.Decimal | yes | | | | | ++--------------------+----------+----------+----------+----------+----------+ ``` String ------ +**Immutable sequence of characters.** + ```python = .strip() # Strips all whitespace characters from both ends. - = .strip('') # Strips all passed characters from both ends. + = .strip('') # Strips passed characters. Also lstrip/rstrip(). ``` ```python - = .split() # Splits on any whitespace character. - = .split(sep=None, maxsplit=-1) # Splits on 'sep' str at most 'maxsplit' times. - = .join() # Joins elements using string as separator. + = .split() # Splits on one or more whitespace characters. + = .split(sep=None, maxsplit=-1) # Splits on 'sep' string at most 'maxsplit' times. + = .splitlines(keepends=False) # On [\n\r\f\v\x1c-\x1e\x85\u2028\u2029] and \r\n. + = .join() # Joins elements by using string as a separator. ``` ```python - = .replace(old, new [, count]) # Replaces 'old' with 'new' at most 'count' times. + = in # Checks if string contains the substring. = .startswith() # Pass tuple of strings for multiple options. - = .endswith() # Pass tuple of strings for multiple options. - = .index() # Returns start index of first match. + = .find() # Returns start index of the first match or -1. ``` ```python - = .isnumeric() # True if str contains only numeric characters. - = textwrap.wrap(, width) # Nicely breaks string into lines. + = .lower() # Lowers the case. Also upper/capitalize/title(). + = .casefold() # Same, but converts ẞ/ß to ss, Σ/ς to σ, etc. + = .replace(old, new [, count]) # Replaces 'old' with 'new' at most 'count' times. + = .translate() # Use `str.maketrans()` to generate table. ``` -### Char ```python - = chr() # Converts int to unicode char. - = ord() # Converts unicode char to int. + = chr() # Converts passed integer to Unicode character. + = ord() # Converts passed Unicode character to integer. ``` +* **Use `'unicodedata.normalize("NFC", )'` on strings like `'Motörhead'` before comparing them to other strings, because `'ö'` can be stored as one or two characters.** +* **`'NFC'` converts such characters to a single character, while `'NFD'` converts them to two.** +### Property Methods ```python ->>> ord('0'), ord('9') -(48, 57) ->>> ord('A'), ord('Z') -(65, 90) ->>> ord('a'), ord('z') -(97, 122) + = .isdecimal() # Checks for [0-9]. Also [०-९] and [٠-٩]. + = .isdigit() # Checks for [²³¹…] and isdecimal(). + = .isnumeric() # Checks for [¼½¾…], [零〇一…] and isdigit(). + = .isalnum() # Checks for [a-zA-Z…] and isnumeric(). + = .isprintable() # Checks for [ !#$%…] and isalnum(). + = .isspace() # Checks for [ \t\n\r\f\v\x1c-\x1f\x85…]. ``` Regex ----- +**Functions for regular expression matching.** + ```python import re - = re.sub(, new, text, count=0) # Substitutes all occurrences. - = re.findall(, text) # Returns all occurrences. - = re.split(, text, maxsplit=0) # Use brackets in regex to keep the matches. - = re.search(, text) # Searches for first occurrence of pattern. - = re.match(, text) # Searches only at the beginning of the text. - = re.finditer(, text) # Returns all occurrences as match objects. + = re.sub(r'', new, text, count=0) # Substitutes all occurrences with 'new'. + = re.findall(r'', text) # Returns all occurrences of the pattern. + = re.split(r'', text, maxsplit=0) # Add brackets around regex to keep matches. + = re.search(r'', text) # First occurrence of the pattern or None. + = re.match(r'', text) # Searches only at the beginning of the text. + = re.finditer(r'', text) # Returns all occurrences as Match objects. ``` -* **Parameter `'flags=re.IGNORECASE'` can be used with all functions.** -* **Parameter `'flags=re.DOTALL'` makes dot also accept newline.** -* **Use `r'\1'` or `'\\1'` for backreference.** -* **Use `'?'` to make an operator non-greedy.** +* **Raw string literals do not interpret escape sequences, thus enabling us to use regex-specific escape sequences that cause SyntaxWarning in normal string literals (since 3.12).** +* **Argument 'new' of re.sub() can be a function that accepts Match object and returns a str.** +* **Argument `'flags=re.IGNORECASE'` can be used with all functions that are listed above.** +* **Argument `'flags=re.MULTILINE'` makes `'^'` and `'$'` match the start/end of each line.** +* **Argument `'flags=re.DOTALL'` makes `'.'` also accept the `'\n'` (besides all other chars).** +* **`'re.compile()'` returns a Pattern object with methods sub(), findall(), etc.** ### Match Object ```python - = .group() # Whole match. - = .group(1) # Part in first bracket. - = .groups() # All bracketed parts. - = .start() # Start index of a match. - = .end() # Exclusive end index of a match. + = .group() # Returns the whole match. Also group(0). + = .group(1) # Returns part inside the first brackets. + = .groups() # Returns all bracketed parts as strings. + = .start() # Returns start index of the whole match. + = .end() # Returns its exclusive end index. ``` ### Special Sequences -**Expressions below hold true for strings that contain only ASCII characters. Use capital letters for negation.** ```python -'\d' == '[0-9]' # Digit -'\s' == '[ \t\n\r\f\v]' # Whitespace -'\w' == '[a-zA-Z0-9_]' # Alphanumeric +'\d' == '[0-9]' # Also [०-९…]. Matches a decimal character. +'\w' == '[a-zA-Z0-9_]' # Also [ª²³…]. Matches an alphanumeric or _. +'\s' == '[ \t\n\r\f\v]' # Also [\x1c-\x1f…]. Matches a whitespace. ``` +* **By default, decimal characters and alphanumerics from all alphabets are matched unless `'flags=re.ASCII'` is used. It restricts special sequence matches to the first 128 Unicode characters and also prevents `'\s'` from accepting `'\x1c'`, `'\x1d'`, `'\x1e'` and `'\x1f'` (non-printable characters that divide text into files, tables, rows and fields, respectively).** +* **Use a capital letter for negation (all non-ASCII characters will be matched when used in combination with ASCII flag).** Format ------ -```python - = f'{}, {}' - = '{}, {}'.format(, ) +```perl + = f'{}, {}' # Curly braces can also contain expressions. + = '{}, {}'.format(, ) # Same as '{0}, {a}'.format(, a=). + = '%s, %s' % (, ) # Redundant and inferior C-style formatting. ``` +### Example ```python >>> Person = collections.namedtuple('Person', 'name height') >>> person = Person('Jean-Luc', 187) ->>> f'{person.height}' -'187' ->>> '{p.height}'.format(p=person) -'187' +>>> f'{person.name} is {person.height / 100} meters tall.' +'Jean-Luc is 1.87 meters tall.' ``` ### General Options ```python -{:<10} # ' ' -{:>10} # ' ' -{:^10} # ' ' -{:->10} # '------' -{:>0} # '' +{:<10} # ' ' +{:^10} # ' ' +{:>10} # ' ' +{:.<10} # '......' +{:0} # '' ``` +* **Objects are rendered by calling the `'format(, "")'` function.** +* **Options inside curly braces can be generated dynamically: `f'{:{}[…]}'`.** +* **Adding `'='` to the expression prepends it to the output: `f'{1+1=}'` returns `'1+1=2'`.** +* **Adding `'!r'` to the expression converts object to string by calling its [repr()](#class) method.** -### String Options -**`'!r'` calls object's repr() method, instead of format(), to get a string.** +### Strings ```python -{'abcde'!r:<10} # "'abcde' " -{'abcde':.3} # 'abc' -{'abcde':10.3} # 'abc ' +{'abcde':10} # 'abcde ' +{'abcde':10.3} # 'abc ' +{'abcde':.3} # 'abc' +{'abcde'!r:10} # "'abcde' " ``` -### Float Options +### Numbers ```python -{1.23456:.3f} # '1.235' -{1.23456:10.3f} # ' 1.235' -{1.23456:10.3e} # ' 1.235e+00' -{1.23456:10.3%} # ' 123.456%' +{123456:10} # ' 123456' +{123456:10,} # ' 123,456' +{123456:10_} # ' 123_456' +{123456:+10} # ' +123456' +{123456:=+10} # '+ 123456' +{123456: } # ' 123456' +{-123456: } # '-123456' ``` -### Int Options +### Floats ```python -{ 123456:10,} # ' 123,456' -{ 123456:10_} # ' 123_456' -{ 123456:+10} # ' +123456' -{-123456:=10} # '- 123456' -{ 123456: } # ' 123456' -{-123456: } # '-123456' +{1.23456:10.3} # ' 1.23' +{1.23456:10.3f} # ' 1.235' +{1.23456:10.3e} # ' 1.235e+00' +{1.23456:10.3%} # ' 123.456%' +``` + +#### Comparison of presentation types: +```text ++--------------+----------------+----------------+----------------+----------------+ +| | {} | {:f} | {:e} | {:%} | ++--------------+----------------+----------------+----------------+----------------+ +| 0.000056789 | '5.6789e-05' | '0.000057' | '5.678900e-05' | '0.005679%' | +| 0.00056789 | '0.00056789' | '0.000568' | '5.678900e-04' | '0.056789%' | +| 0.0056789 | '0.0056789' | '0.005679' | '5.678900e-03' | '0.567890%' | +| 0.056789 | '0.056789' | '0.056789' | '5.678900e-02' | '5.678900%' | +| 0.56789 | '0.56789' | '0.567890' | '5.678900e-01' | '56.789000%' | +| 5.6789 | '5.6789' | '5.678900' | '5.678900e+00' | '567.890000%' | +| 56.789 | '56.789' | '56.789000' | '5.678900e+01' | '5678.900000%' | ++--------------+----------------+----------------+----------------+----------------+ +``` + +```text ++--------------+----------------+----------------+----------------+----------------+ +| | {:.2} | {:.2f} | {:.2e} | {:.2%} | ++--------------+----------------+----------------+----------------+----------------+ +| 0.000056789 | '5.7e-05' | '0.00' | '5.68e-05' | '0.01%' | +| 0.00056789 | '0.00057' | '0.00' | '5.68e-04' | '0.06%' | +| 0.0056789 | '0.0057' | '0.01' | '5.68e-03' | '0.57%' | +| 0.056789 | '0.057' | '0.06' | '5.68e-02' | '5.68%' | +| 0.56789 | '0.57' | '0.57' | '5.68e-01' | '56.79%' | +| 5.6789 | '5.7' | '5.68' | '5.68e+00' | '567.89%' | +| 56.789 | '5.7e+01' | '56.79' | '5.68e+01' | '5678.90%' | ++--------------+----------------+----------------+----------------+----------------+ ``` +* **`'{:g}'` is `'{:.6}'` with stripped zeros, exponent starting at `'1e+06'`.** +* **When both rounding up and rounding down are possible, the one that returns result with even last digit is chosen. That makes `'{6.5:.0f}'` a `'6'` and `'{7.5:.0f}'` an `'8'`.** +* **This rule only effects numbers that can be represented exactly by a float (`.5`, `.25`, …).** +### Ints ```python -{90:c} # 'Z' -{90:X} # '5A' -{3:08b} # '00000011' +{90:c} # 'Z'. Unicode character with value 90. +{90:b} # '1011010'. Binary representation of the int. +{90:X} # '5A'. Hexadecimal with upper-case letters. ``` Numbers ------- -### Basic Functions ```python - = pow(, ) # Or: ** - = abs() - = round() - = round(, ±ndigits) + = int() # Whole number of any size. Truncates floats. + = float() # 64-bit decimal number. Also e±. + = complex(real=0, imag=0) # A complex number. Also ` ± j`. + = fractions.Fraction(, ) # E.g. `Fraction(1, 2) / 3 == Fraction(1, 6)`. + = decimal.Decimal() # E.g. `Decimal((1, (2, 3), 4)) == -230_000`. +``` +* **`'int()'` and `'float()'` raise ValueError if passed string is malformed.** +* **Decimal objects store numbers exactly, unlike most floats where `'1.1 + 2.2 != 3.3'`.** +* **Floats can be compared with: `'math.isclose(, , rel_tol=1e-09)'`.** +* **Precision of decimal operations is set with: `'decimal.getcontext().prec = '`.** +* **Bools can be used anywhere ints can, because bool is a subclass of int: `'True + 1 == 2'`.** + +### Built-in Functions +```python + = pow(, ) # E.g. `pow(2, 3) == 2 ** 3 == 8`. + = abs() # E.g. `abs(complex(3, 4)) == 5`. + = round( [, ±ndigits]) # E.g. `round(123, -1) == 120`. + = min() # Also max(, [, ...]). + = sum() # Also math.prod(). ``` ### Math ```python -from math import e, pi -from math import cos, acos, sin, asin, tan, atan, degrees, radians -from math import log, log10, log2 -from math import inf, nan, isinf, isnan +from math import floor, ceil, trunc # They convert floats into integers. +from math import pi, inf, nan, isnan # `inf * 0` and `nan + 1` return nan. +from math import sqrt, factorial # `sqrt(-1)` will raise ValueError. +from math import sin, cos, tan # Also: asin, acos, degrees, radians. +from math import log, log10, log2 # Log accepts base as second argument. ``` ### Statistics ```python -from statistics import mean, median, variance, pvariance, pstdev +from statistics import mean, median, mode # Mode returns the most common item. +from statistics import variance, stdev # Also: pvariance, pstdev, quantiles. ``` ### Random ```python -from random import random, randint, choice, shuffle - = random() - = randint(from_inclusive, to_inclusive) - = choice() -shuffle() +from random import random, randint, uniform # Also: gauss, choice, shuffle, seed. ``` +```python + = random() # Returns a float inside [0, 1). + = randint/uniform(a, b) # Returns an int/float inside [a, b]. + = gauss(mean, stdev) # Also triangular(low, high, mode). + = choice() # Keeps it intact. Also sample(p, n). +shuffle() # Works on all mutable sequences. +``` -Combinatorics -------------- -* **Every function returns an iterator.** -* **If you want to print the iterator, you need to pass it to the list() function!** - +### Hexadecimal Numbers ```python -from itertools import product, combinations, combinations_with_replacement, permutations + = 0x # E.g. `0xFF == 255`. Also 0b. + = int('±', 16) # Also int('±0x/±0b', 0). + = hex() # Returns '[-]0x'. Also bin(). ``` +### Bitwise Operators ```python ->>> product([0, 1], repeat=3) -[(0, 0, 0), (0, 0, 1), (0, 1, 0), (0, 1, 1), - (1, 0, 0), (1, 0, 1), (1, 1, 0), (1, 1, 1)] + = & # E.g. `0b1100 & 0b1010 == 0b1000`. + = | # E.g. `0b1100 | 0b1010 == 0b1110`. + = ^ # E.g. `0b1100 ^ 0b1010 == 0b0110`. + = << n_bits # E.g. `0b1111 << 4 == 0b11110000`. + = ~ # E.g. `~0b1 == -0b10 == -(0b1+1)`. ``` + +Combinatorics +------------- ```python ->>> product('ab', '12') -[('a', '1'), ('a', '2'), - ('b', '1'), ('b', '2')] +import itertools as it ``` ```python ->>> combinations('abc', 2) -[('a', 'b'), ('a', 'c'), ('b', 'c')] +>>> list(it.product('abc', repeat=2)) # a b c +[('a', 'a'), ('a', 'b'), ('a', 'c'), # a x x x + ('b', 'a'), ('b', 'b'), ('b', 'c'), # b x x x + ('c', 'a'), ('c', 'b'), ('c', 'c')] # c x x x ``` ```python ->>> combinations_with_replacement('abc', 2) -[('a', 'a'), ('a', 'b'), ('a', 'c'), - ('b', 'b'), ('b', 'c'), - ('c', 'c')] +>>> list(it.permutations('abc', 2)) # a b c +[('a', 'b'), ('a', 'c'), # a . x x + ('b', 'a'), ('b', 'c'), # b x . x + ('c', 'a'), ('c', 'b')] # c x x . ``` ```python ->>> permutations('abc', 2) -[('a', 'b'), ('a', 'c'), - ('b', 'a'), ('b', 'c'), - ('c', 'a'), ('c', 'b')] +>>> list(it.combinations('abc', 2)) # a b c +[('a', 'b'), ('a', 'c'), # a . x x + ('b', 'c') # b . . x +] # c . . . ``` Datetime -------- -* **Module 'datetime' provides 'date' ``, 'time' ``, 'datetime' `
` and 'timedelta' `
` classes. All are immutable and hashable.** -* **Time and datetime can be 'aware' ``, meaning they have defined timezone, or 'naive' ``, meaning they don't.** -* **If object is naive it is presumed to be in system's timezone.** +**Provides 'date', 'time', 'datetime' and 'timedelta' classes. All are immutable and hashable.** ```python -from datetime import date, time, datetime, timedelta -from dateutil.tz import UTC, tzlocal, gettz +# $ pip3 install python-dateutil +from datetime import date, time, datetime, timedelta, timezone +import zoneinfo, dateutil.tz ``` -### Constructors ```python - = date(year, month, day) - = time(hour=0, minute=0, second=0, microsecond=0, tzinfo=None, fold=0) -
= datetime(year, month, day, hour=0, minute=0, second=0, ...) -
 = timedelta(days=0, seconds=0, microseconds=0, milliseconds=0, - minutes=0, hours=0, weeks=0) + = date(year, month, day) # Only accepts valid dates from 1 to 9999 AD. + = time(hour=0, minute=0, second=0) # Also: `microsecond=0, tzinfo=None, fold=0`. +
= datetime(year, month, day, hour=0) # Also: `minute=0, second=0, microsecond=0, …`. +
 = timedelta(weeks=0, days=0, hours=0) # Also: `minutes=0, seconds=0, microseconds=0`. ``` -* **Use `'.weekday()'` to get the day of the week (Mon == 0).** -* **`'fold=1'` means second pass in case of time jumping back for one hour.** +* **Times and datetimes that have defined timezone are called aware and ones that don't, naive. If time or datetime object is naive, it is presumed to be in the system's timezone!** +* **`'fold=1'` means the second pass in case of time jumping back (usually for one hour).** +* **Timedelta normalizes arguments to ±days, seconds (< 86 400) and microseconds (< 1M). Its str() method returns `'[±D, ]H:MM:SS[.…]'` and total_seconds() a float of all seconds.** +* **Use `'.weekday()'` to get the day of the week as an integer, with Monday being 0.** ### Now ```python - = D/DT.today() # Current local date or naive datetime. - = DT.utcnow() # Naive datetime from current UTC time. - = DT.now() # Aware datetime from current tz time. + = D/DT.today() # Current local date or naive DT. Also DT.now(). + = DT.now() # Aware DT from current time in passed timezone. ``` +* **To extract time use `'.time()'`, `'.time()'` or `'.timetz()'`.** -### Timezone +### Timezones ```python - = UTC # UTC timezone. - = tzlocal() # Local timezone. - = gettz('/') # Timezone from 'Continent/City_Name' str. -``` - -```python - =
.astimezone() # Datetime, converted to passed timezone. - = .replace(tzinfo=) # Unconverted object with new timezone. + = timezone.utc # Coordinated universal time (UK without DST). + = timezone() # Timezone with fixed offset from universal time. + = dateutil.tz.tzlocal() # Local timezone with dynamic offset from UTC. + = zoneinfo.ZoneInfo('') # 'Continent/City_Name' zone with dynamic offset. + =
.astimezone([]) # Converts DT to the passed or local fixed zone. + = .replace(tzinfo=) # Changes object's timezone without conversion. ``` +* **Timezones returned by tzlocal(), ZoneInfo() and implicit local timezone of naive objects have offsets that vary through time due to DST and historical changes of the base offset.** +* **To get ZoneInfo() to work on Windows run `'> pip3 install tzdata'`.** ### Encode ```python - = D/T/DT.fromisoformat('') # Object from ISO string. -
= DT.strptime(, '') # Datetime from str, according to format. - = D/DT.fromordinal() # D/DTn from days since Christ. - = DT.fromtimestamp(, ) # DTa from seconds since Epoch in tz time. + = D/T/DT.fromisoformat() # Object from ISO string. Raises ValueError. +
= DT.strptime(, '') # Datetime from custom string. See Format. + = D/DT.fromordinal() # D/DT from days since the Gregorian NYE 1. + = DT.fromtimestamp() # Local naive DT from seconds since the Epoch. + = DT.fromtimestamp(, ) # Aware datetime from seconds since the Epoch. ``` -* **ISO strings come in following forms: `'YYYY-MM-DD'`, `'HH:MM:SS.ffffff[±]'`, or both separated by `'T'`.** -* **On Unix systems Epoch is `'1970-01-01 00:00 UTC'`, `'1970-01-01 01:00 CET'`, ...** +* **ISO strings come in following forms: `'YYYY-MM-DD'`, `'HH:MM:SS.mmmuuu[±HH:MM]'`, or both separated by an arbitrary character. All parts following the hours are optional.** +* **Python uses the Unix Epoch: `'1970-01-01 00:00 UTC'`, `'1970-01-01 01:00 CET'`, ...** ### Decode ```python - = .isoformat() # ISO string representation. - = .strftime('') # Custom string representation. - = .toordinal() # Days since Christ, ignoring time and tz. - =
.timestamp() # Seconds since Epoch in local time or tz. + = .isoformat(sep='T') # Also `timespec='auto/hours/minutes/seconds…'`. + = .strftime('') # Custom string representation of the object. + = .toordinal() # Days since NYE 1. Ignores DT's time and zone. + = .timestamp() # Seconds since the Epoch, from local naive DT. + = .timestamp() # Seconds since the Epoch, from aware datetime. ``` ### Format ```python ->>> dt = datetime.strptime('2015-05-14 23:39:00.00 +0200', '%Y-%m-%d %H:%M:%S.%f %z') ->>> dt.strftime("%A %dth %B '%y, %I:%M%p %Z") -"Thursday 14th May '15, 11:39PM UTC+02:00" +>>> dt = datetime.strptime('2025-08-14 23:39:00.00 +0200', '%Y-%m-%d %H:%M:%S.%f %z') +>>> dt.strftime("%dth of %B '%y (%a), %I:%M %p %Z") +"14th of August '25 (Thu), 11:39 PM UTC+02:00" ``` +* **`'%z'` accepts `'±HH[:]MM'` and returns `'±HHMM'` or empty string if object is naive.** +* **`'%Z'` accepts `'UTC/GMT'` and local timezone's code and returns timezone's name, `'UTC[±HH:MM]'` if timezone is nameless, or an empty string if object is naive.** -#### Rest of the codes: -* **`'a'` - Weekday, abbreviated name.** -* **`'b'` - Month, abbreviated name.** +### Arithmetics +```python + = > # Ignores time jumps (fold attribute). Also ==. + = > # Ignores time jumps if they share tzinfo object. +
 = - # Ignores jumps. Convert to UTC for actual delta. + = - # Ignores jumps if they share the tzinfo object. + = ± # Returned datetime can fall into missing hour. + = * # Also ` = abs()`, ` = ± `. + = / # Also `(, ) = divmod(, )`. +``` -Arguments ---------- -### Inside Function Call +Function +-------- +**Independent block of code that returns a value when called.** ```python -() # f(0, 0) -() # f(x=0, y=0) -(, ) # f(0, y=0) +def (): ... # E.g. `def func(x, y): ...`. +def (): ... # E.g. `def func(x=0, y=0): ...`. +def (, ): ... # E.g. `def func(x, y=0): ...`. ``` +* **Function returns None if it doesn't encounter `'return '` statement.** +* **Run `'global '` inside the function before assigning to global variable.** +* **Default values are evaluated when function is first encountered in the scope. Any mutation of a mutable default value will persist between invocations!** + +### Function Call -### Inside Function Definition ```python -def f(): # def f(x, y) -def f(): # def f(x=0, y=0) -def f(, ): # def f(x, y=0) + = () # E.g. `func(0, 0)`. + = () # E.g. `func(x=0, y=0)`. + = (, ) # E.g. `func(0, y=0)`. ``` Splat Operator -------------- -### Inside Function Call **Splat expands a collection into positional arguments, while splatty-splat expands a dictionary into keyword arguments.** ```python -args = (1, 2) -kwargs = {'x': 3, 'y': 4, 'z': 5} +args, kwargs = (1, 2), {'z': 3} func(*args, **kwargs) ``` #### Is the same as: ```python -func(1, 2, x=3, y=4, z=5) +func(1, 2, z=3) ``` ### Inside Function Definition @@ -553,36 +714,27 @@ def add(*a): 6 ``` -#### Legal argument combinations: -```python -def f(*args): # f(1, 2, 3) -def f(x, *args): # f(1, 2, 3) -def f(*args, z): # f(1, 2, z=3) -def f(x, *args, z): # f(1, 2, z=3) -``` - -```python -def f(**kwargs): # f(x=1, y=2, z=3) -def f(x, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) -``` - -```python -def f(*args, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) | f(1, 2, z=3) | f(1, 2, 3) -def f(x, *args, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) | f(1, 2, z=3) | f(1, 2, 3) -def f(*args, y, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) -def f(x, *args, z, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) | f(1, 2, z=3) +#### Allowed compositions of arguments and the ways they can be called: +```text ++---------------------------+--------------+--------------+----------------+ +| | func(1, 2) | func(1, y=2) | func(x=1, y=2) | ++---------------------------+--------------+--------------+----------------+ +| func(x, *args, **kwargs): | yes | yes | yes | +| func(*args, y, **kwargs): | | yes | yes | +| func(*, x, **kwargs): | | | yes | ++---------------------------+--------------+--------------+----------------+ ``` ### Other Uses ```python - = [* [, ...]] - = {* [, ...]} - = (*, [...]) - = {** [, ...]} + = [* [, ...]] # Or: list() [+ ...] + = (*, [...]) # Or: tuple() [+ ...] + = {* [, ...]} # Or: set() [| ...] + = {** [, ...]} # Or: | ... ``` ```python -head, *body, tail = +head, *body, tail = # Head or tail can be omitted. ``` @@ -590,79 +742,97 @@ Inline ------ ### Lambda ```python - = lambda: - = lambda , : + = lambda: # A single statement function. + = lambda , : # Also allows default arguments. ``` -### Comprehension +### Comprehensions ```python - = [i+1 for i in range(10)] # [1, 2, ..., 10] - = {i for i in range(10) if i > 5} # {6, 7, 8, 9} - = (i+5 for i in range(10)) # (5, 6, ..., 14) - = {i: i*2 for i in range(10)} # {0: 0, 1: 2, ..., 9: 18} + = [i+1 for i in range(10)] # Returns [1, 2, ..., 10]. + = (i for i in range(10) if i > 5) # Returns iter([6, 7, 8, 9]). + = {i+5 for i in range(10)} # Returns {5, 6, ..., 14}. + = {i: i*2 for i in range(10)} # Returns {0: 0, 1: 2, ..., 9: 18}. ``` ```python -out = [i+j for i in range(10) for j in range(10)] +>>> [l+r for l in 'abc' for r in 'abc'] # Inner loop is on the right side. +['aa', 'ab', 'ac', ..., 'cc'] ``` -#### Is the same as: +### Map, Filter, Reduce ```python -out = [] -for i in range(10): - for j in range(10): - out.append(i+j) +from functools import reduce ``` -### Map, Filter, Reduce ```python -from functools import reduce - = map(lambda x: x + 1, range(10)) # (1, 2, ..., 10) - = filter(lambda x: x > 5, range(10)) # (6, 7, 8, 9) - = reduce(lambda out, x: out + x, range(10)) # 45 + = map(lambda x: x + 1, range(10)) # Returns iter([1, 2, ..., 10]). + = filter(lambda x: x > 5, range(10)) # Returns iter([6, 7, 8, 9]). + = reduce(lambda out, x: out + x, range(10)) # Returns 45. ``` ### Any, All ```python - = any() # False if empty. - = all(el[1] for el in ) # True if empty. + = any() # Is `bool()` True for any el? + = all() # True for all? Also True if empty. ``` -### If - Else +### Conditional Expression ```python - if else + = if else # Only one expression is evaluated. ``` ```python ->>> [a if a else 'zero' for a in (0, 1, 0, 3)] -['zero', 1, 'zero', 3] +>>> [i if i else 'zero' for i in (0, 1, 2, 3)] # `any([0, '', [], None]) == False` +['zero', 1, 2, 3] ``` -### Namedtuple, Enum, Class +### And, Or ```python -from collections import namedtuple -Point = namedtuple('Point', 'x y') -point = Point(0, 0) + = and [and ...] # Returns first false or last object. + = or [or ...] # Returns first true or last object. +``` + +### Walrus Operator +```python +>>> [i for ch in '0123' if (i := int(ch)) > 0] # Assigns to variable mid-sentence. +[1, 2, 3] ``` +### Named Tuple, Enum, Dataclass ```python +from collections import namedtuple +Point = namedtuple('Point', 'x y') # Creates tuple's subclass. +point = Point(0, 0) # Returns its instance. + from enum import Enum -Direction = Enum('Direction', 'n e s w') -Cutlery = Enum('Cutlery', {'fork': 1, 'knife': 2, 'spoon': 3}) +Direction = Enum('Direction', 'N E S W') # Creates Enum's subclass. +direction = Direction.N # Returns its member. + +from dataclasses import make_dataclass +Player = make_dataclass('Player', ['loc', 'dir']) # Creates a class. +player = Player(point, direction) # Returns its instance. ``` + +Imports +------- +**Mechanism that makes code in one file available to another file.** + ```python -# Warning: Objects will share the objects that are initialized in the dictionary! -Creature = type('Creature', (), {'p': Point(0, 0), 'd': Direction.n}) -creature = Creature() +import # Imports a built-in or '.py'. +import # Imports a built-in or '/__init__.py'. +import . # Imports a built-in or '/.py'. ``` +* **Package is a collection of modules, but it can also define its own functions, classes, etc.** +* **On a filesystem this corresponds to a directory of Python files with an optional init script.** +* **Running `'import '` does not automatically provide access to the package's modules unless they are explicitly imported in the `'/__init__.py'` script.** +* **Directory of the file that is passed to python command serves as a root of local imports.** +* **For relative imports use `'from .[…][[.…]] import '`.** Closure ------- -**We have a closure in Python when:** -* **A nested function references a value of its enclosing function and then** -* **the enclosing function returns the nested function.** +**We have/get a closure in Python when a nested function references a value of its enclosing function and then the enclosing function returns its nested function.** ```python def get_multiplier(a): @@ -676,25 +846,25 @@ def get_multiplier(a): >>> multiply_by_3(10) 30 ``` - -* **If multiple nested functions within enclosing function reference the same value, that value gets shared.** -* **To dynamically access function's first free variable use `'.__closure__[0].cell_contents'`.** +* **Any value that is referenced from within multiple nested functions gets shared.** ### Partial ```python from functools import partial - = partial( [, , , ...]) + = partial( [, [, ...]]) ``` ```python ->>> import operator as op ->>> multiply_by_3 = partial(op.mul, 3) +>>> def multiply(a, b): +... return a * b +>>> multiply_by_3 = partial(multiply, 3) >>> multiply_by_3(10) 30 ``` +* **Partial is also useful in cases when a function needs to be passed as an argument because it enables us to set its arguments beforehand (`'collections.defaultdict()'`, `'iter(, to_exc)'` and `'dataclasses.field(default_factory=)'`).** -### Nonlocal -**If variable is being assigned to anywhere in the scope, it is regarded as a local variable, unless it is declared as a 'global' or 'nonlocal'.** +### Non-Local +**If variable is being assigned to anywhere in the scope, it is regarded as a local variable, unless it is declared as a 'global' or a 'nonlocal'.** ```python def get_counter(): @@ -715,7 +885,7 @@ def get_counter(): Decorator --------- -**A decorator takes a function, adds some functionality and returns it.** +**A decorator takes a function, adds some functionality and returns it. It can be any [callable](#callable), but is usually implemented as a function that returns a [closure](#closure).** ```python @decorator_name @@ -724,7 +894,7 @@ def function_that_gets_passed_to_decorator(): ``` ### Debugger Example -**Decorator that prints function's name every time it gets called.** +**Decorator that prints function's name every time the function is called.** ```python from functools import wraps @@ -740,21 +910,20 @@ def debug(func): def add(x, y): return x + y ``` -* **Wraps is a helper decorator that copies metadata of function add() to function out().** -* **Without it `'add.__name__'` would return `'out'`.** +* **Wraps is a helper decorator that copies the metadata of the passed function (func) to the function it is wrapping (out). Without it, `'add.__name__'` would return `'out'`.** -### LRU Cache +### Cache **Decorator that caches function's return values. All function's arguments must be hashable.** ```python -from functools import lru_cache +from functools import cache -@lru_cache(maxsize=None) +@cache def fib(n): return n if n < 2 else fib(n-2) + fib(n-1) ``` - -* **Recursion depth is limited to 1000 by default. To increase it use `'sys.setrecursionlimit()'`.** +* **Potential problem with cache is that it can grow indefinitely. To clear stored values run `'.cache_clear()'`, or use `'@lru_cache(maxsize=)'` decorator instead.** +* **CPython interpreter limits recursion depth to 3000 by default. To increase it run `'sys.setrecursionlimit()'`.** ### Parametrized Decorator **A decorator that accepts arguments and returns a normal decorator that accepts a function.** @@ -775,53 +944,158 @@ def debug(print_result=False): def add(x, y): return x + y ``` +* **Using only `'@debug'` to decorate the add() function would not work here, because debug would then receive the add() function as a 'print_result' argument. Decorators can however manually check if the argument they received is a function and act accordingly.** Class ----- +**A template for creating user-defined objects.** + ```python -class : +class MyClass: def __init__(self, a): self.a = a + def __str__(self): + return str(self.a) def __repr__(self): class_name = self.__class__.__name__ return f'{class_name}({self.a!r})' - def __str__(self): - return str(self.a) @classmethod def get_class_name(cls): return cls.__name__ ``` -### Constructor Overloading ```python -class : - def __init__(self, a=None): - self.a = a +>>> obj = MyClass(1) +>>> obj.a, str(obj), repr(obj) +(1, '1', 'MyClass(1)') +``` +* **Methods whose names start and end with two underscores are called special methods. They are executed when object is passed to a built-in function or used as an operand, for example, `'print(a)'` calls `'a.__str__()'` and `'a + b'` calls `'a.__add__(b)'`.** +* **Methods decorated with `'@staticmethod'` receive neither 'self' nor 'cls' argument.** +* **Return value of str() special method should be readable and of repr() unambiguous. If only repr() is defined, it will also be used for str().** + +#### Expressions that call the str() method: +```python +print() +f'{}' +logging.warning() +csv.writer().writerow([]) ``` -### Inheritance +#### Expressions that call the repr() method: +```python +print/str/repr([]) +print/str/repr({: }) +f'{!r}' +Z = make_dataclass('Z', ['a']); print/str/repr(Z()) +``` + +### Subclass +* **Inheritance is a mechanism that enables a class to extend some other class (that is, subclass to extend its parent), and by doing so inherit all of its methods and attributes.** +* **Subclass can then add its own methods and attributes or override inherited ones by reusing their names.** + ```python class Person: - def __init__(self, name, age): + def __init__(self, name): self.name = name - self.age = age + def __repr__(self): + return f'Person({self.name!r})' + def __lt__(self, other): + return self.name < other.name class Employee(Person): - def __init__(self, name, age, staff_num): - super().__init__(name, age) + def __init__(self, name, staff_num): + super().__init__(name) self.staff_num = staff_num + def __repr__(self): + return f'Employee({self.name!r}, {self.staff_num})' +``` + +```python +>>> people = {Person('Ann'), Employee('Bob', 0)} +>>> sorted(people) +[Person('Ann'), Employee('Bob', 0)] +``` + +### Type Annotations +* **They add type hints to variables, arguments and functions (`'def f() -> :'`).** +* **Hints are used by type checkers like [mypy](https://pypi.org/project/mypy/), data validation libraries such as [Pydantic](https://pypi.org/project/pydantic/) and lately also by [Cython](https://pypi.org/project/Cython/) compiler. However, they are not enforced by CPython interpreter.** +```python +from collections import abc + +: [| ...] [= ] +: list/set/abc.Iterable/abc.Sequence[] [= ] +: tuple/dict[, ...] [= ] +``` + +### Dataclass +**Decorator that uses class variables to generate init(), repr() and eq() special methods.** +```python +from dataclasses import dataclass, field, make_dataclass + +@dataclass(order=False, frozen=False) +class : + : + : = + : list/dict/set = field(default_factory=list/dict/set) +``` +* **Objects can be made [sortable](#sortable) with `'order=True'` and immutable with `'frozen=True'`.** +* **For object to be [hashable](#hashable), all attributes must be hashable and 'frozen' must be True.** +* **Function field() is needed because `': list = []'` would make a list that is shared among all instances. Its 'default_factory' argument accepts any [callable](#callable) object.** +* **For attributes of arbitrary type use `'typing.Any'`.** + +```python +P = make_dataclass('P', ['x', 'y']) +P = make_dataclass('P', [('x', float), ('y', float)]) +P = make_dataclass('P', [('x', float, 0), ('y', float, 0)]) +``` + +### Property +**Pythonic way of implementing getters and setters.** +```python +class Person: + @property + def name(self): + return ' '.join(self._name) + + @name.setter + def name(self, value): + self._name = value.split() +``` + +```python +>>> person = Person() +>>> person.name = '\t Guido van Rossum \n' +>>> person.name +'Guido van Rossum' +``` + +### Slots +**Mechanism that restricts objects to attributes listed in 'slots'.** + +```python +class MyClassWithSlots: + __slots__ = ['a'] + def __init__(self): + self.a = 1 ``` +### Copy ```python ->>> Employee.mro() -[, , ] +from copy import copy, deepcopy + = copy/deepcopy() ``` + +Duck Types +---------- +**A duck type is an implicit type that prescribes a set of special methods. Any object that has those methods defined is considered a member of that duck type.** + ### Comparable -* **If eq() method is not overridden, it returns `'id(self) == id(other)'`, which is the same as `'self is other'`.** -* **That means all objects compare not equal by default.** +* **If eq() method is not overridden, it returns `'id(self) == id(other)'`, which is the same as `'self is other'`. That means all user-defined objects compare not equal by default (because id() returns object's memory address that is guaranteed to be unique).** +* **Only the left side object has eq() method called, unless it returns NotImplemented, in which case the right object is consulted. Result is False if both return NotImplemented.** +* **Ne() automatically works on any object that has eq() defined.** ```python class MyComparable: @@ -830,48 +1104,83 @@ class MyComparable: def __eq__(self, other): if isinstance(other, type(self)): return self.a == other.a - return False + return NotImplemented ``` ### Hashable -* **Hashable object needs both hash() and eq() methods and its hash value should never change.** -* **Hashable objects that compare equal must have the same hash value, meaning default hash() that returns `'id(self)'` will not do.** -* **That is why Python automatically makes classes unhashable if you only implement eq().** +* **Hashable object needs both hash() and eq() methods and its hash value must not change.** +* **Hashable objects that compare equal must have the same hash value, meaning default hash() that returns `'id(self)'` will not do. That is why Python automatically makes classes unhashable if you only implement eq().** ```python class MyHashable: def __init__(self, a): - self.__a = copy.deepcopy(a) + self._a = a @property def a(self): - return self.__a + return self._a def __eq__(self, other): if isinstance(other, type(self)): return self.a == other.a - return False + return NotImplemented def __hash__(self): return hash(self.a) ``` -### Sequence -* **Methods do not depend on each other, so they can be skipped if not needed.** -* **Any object with defined getitem() is considered iterable, even if it lacks iter().** +### Sortable +* **With 'total_ordering' decorator, you only need to provide eq() and one of lt(), gt(), le() or ge() special methods (used by <, >, <=, >=) and the rest will be automatically generated.** +* **Functions sorted() and min() only require lt() method, while max() only requires gt(). However, it is best to define them all so that confusion doesn't arise in other contexts.** +* **When two lists, strings or dataclasses are compared, their values get compared in order until a pair of unequal values is found. The comparison of this two values is then returned. The shorter sequence is considered smaller in case of all values being equal.** +* **To sort collection of strings in proper alphabetical order pass `'key=locale.strxfrm'` to sorted() after running `'locale.setlocale(locale.LC_COLLATE, "en_US.UTF-8")'`.** + ```python -class MySequence: +from functools import total_ordering + +@total_ordering +class MySortable: def __init__(self, a): self.a = a - def __len__(self): - return len(self.a) - def __getitem__(self, i): - return self.a[i] - def __setitem__(self, i, value): - self.a[i] = value + def __eq__(self, other): + if isinstance(other, type(self)): + return self.a == other.a + return NotImplemented + def __lt__(self, other): + if isinstance(other, type(self)): + return self.a < other.a + return NotImplemented +``` + +### Iterator +* **Any object that has methods next() and iter() is an iterator.** +* **Next() should return next item or raise StopIteration exception.** +* **Iter() should return an iterator of remaining items, i.e. 'self'.** +* **Any object that has iter() method can be used in a for loop.** +```python +class Counter: + def __init__(self): + self.i = 0 + def __next__(self): + self.i += 1 + return self.i def __iter__(self): - for el in self.a: - yield el + return self +``` + +```python +>>> counter = Counter() +>>> next(counter), next(counter), next(counter) +(1, 2, 3) ``` +#### Python has many different iterator objects: +* **Sequence iterators returned by the [iter()](#iterator) function, such as list\_iterator, etc.** +* **Objects returned by the [itertools](#itertools) module, such as count, repeat and cycle.** +* **Generators returned by the [generator functions](#generator) and [generator expressions](#comprehensions).** +* **File objects returned by the [open()](#open) function, etc.** + ### Callable +* **All functions and classes have a call() method that is executed when they are called.** +* **Use `'callable()'` or `'isinstance(, collections.abc.Callable)'` to check if object is callable. You can also call the object and check if it raised TypeError.** +* **When this cheatsheet uses `''` as an argument, it means `''`.** ```python class Counter: def __init__(self): @@ -887,15 +1196,20 @@ class Counter: (1, 2, 3) ``` -### Withable +### Context Manager +* **With statements only work on objects that have enter() and exit() special methods.** +* **Enter() should lock the resources and optionally return an object (file, lock, etc.).** +* **Exit() should release the resources (for example close a file, release a lock, etc.).** +* **Any exception that happens inside the with block is passed to the exit() method.** +* **The exit() method can suppress the exception by returning a true value.** ```python -class MyOpen(): +class MyOpen: def __init__(self, filename): self.filename = filename def __enter__(self): self.file = open(self.filename) return self.file - def __exit__(self, *args): + def __exit__(self, exc_type, exception, traceback): self.file.close() ``` @@ -907,191 +1221,386 @@ class MyOpen(): Hello World! ``` -### Copy + +Iterable Duck Types +------------------- +### Iterable +* **Only required method is iter(). It should return an iterator of object's items.** +* **Contains() automatically works on any object that has iter() defined.** ```python -from copy import copy, deepcopy - = copy() - = deepcopy() +class MyIterable: + def __init__(self, a): + self.a = a + def __iter__(self): + return iter(self.a) + def __contains__(self, el): + return el in self.a +``` + +```python +>>> obj = MyIterable([1, 2, 3]) +>>> [el for el in obj] +[1, 2, 3] +>>> 1 in obj +True ``` +### Collection +* **Only required methods are iter() and len(). Len() should return the number of items.** +* **This cheatsheet actually means `''` when it uses the `''`.** +* **I chose not to use the name 'iterable' because it sounds scarier and more vague than 'collection'. The main drawback of this decision is that the reader could think a certain function doesn't accept iterators when it does, since iterators are the only built-in objects that are iterable but are not collections.** +```python +class MyCollection: + def __init__(self, a): + self.a = a + def __iter__(self): + return iter(self.a) + def __contains__(self, el): + return el in self.a + def __len__(self): + return len(self.a) +``` + +### Sequence +* **Only required methods are getitem() and len(). Getitem() should return the item at the passed index or raise IndexError (it may also support negative indices and/or slices).** +* **Iter() and contains() automatically work on any object that has getitem() defined.** +* **Reversed() automatically works on any object that has getitem() and len() defined. It returns reversed iterator of object's items.** +```python +class MySequence: + def __init__(self, a): + self.a = a + def __iter__(self): + return iter(self.a) + def __contains__(self, el): + return el in self.a + def __len__(self): + return len(self.a) + def __getitem__(self, i): + return self.a[i] + def __reversed__(self): + return reversed(self.a) +``` + +#### Discrepancies between glossary definitions and abstract base classes: +* **Python's glossary defines iterable as any object with special methods iter() and/or getitem() and sequence as any object with getitem() and len(). It doesn't define collection.** +* **Passing ABC Iterable to isinstance() or issubclass() only checks whether object/class has special method iter(), while ABC Collection checks for iter(), contains() and len().** + +### ABC Sequence +* **It's a richer interface than the basic sequence that also requires just getitem() and len().** +* **Extending it generates iter(), contains(), reversed(), index() and count() special methods.** +* **Unlike `'abc.Iterable'` and `'abc.Collection'`, it is not a duck type. That is why `'issubclass(MySequence, abc.Sequence)'` would return False even if MySequence had all the methods defined. It however recognizes list, tuple, range, str, bytes, bytearray, array, memoryview and deque, since they are registered as Sequence's virtual subclasses.** +```python +from collections import abc + +class MyAbcSequence(abc.Sequence): + def __init__(self, a): + self.a = a + def __len__(self): + return len(self.a) + def __getitem__(self, i): + return self.a[i] +``` + +#### Table of required and automatically available special methods: +```text ++------------+------------+------------+------------+--------------+ +| | Iterable | Collection | Sequence | abc.Sequence | ++------------+------------+------------+------------+--------------+ +| iter() | REQ | REQ | Yes | Yes | +| contains() | Yes | Yes | Yes | Yes | +| len() | | REQ | REQ | REQ | +| getitem() | | | REQ | REQ | +| reversed() | | | Yes | Yes | +| index() | | | | Yes | +| count() | | | | Yes | ++------------+------------+------------+------------+--------------+ +``` +* **Method iter() is required for `'isinstance(, abc.Iterable)'` to return True, however any object with getitem() will work with any code expecting an iterable.** +* **MutableSequence, Set, MutableSet, Mapping and MutableMapping ABCs are also extendable. Use `'.__abstractmethods__'` to get names of required methods.** + Enum ---- +**Class of named constants called members.** + ```python from enum import Enum, auto +``` +```python class (Enum): - = - = , - = auto() + = auto() # Increment of the last numeric value or 1. + = # Values don't have to be hashable or unique. + = , # Values can be collections. This is a tuple. +``` +* **Methods receive the member they were called on as the 'self' argument.** +* **Accessing a member named after a reserved keyword causes SyntaxError.** - @classmethod - def get_member_names(cls): - return [a.name for a in cls.__members__.values()] +```python + = . # Returns a member. Raises AttributeError. + = [''] # Returns a member. Raises KeyError. + = () # Returns a member. Raises ValueError. + = .name # Returns the member's name. + = .value # Returns the member's value. ``` ```python - = . - = [''] - = () -name = .name -value = .value + = list() # Returns a list of enum's members. + = ._member_names_ # Returns a list of member names. + = [m.value for m in ] # Returns a list of member values. ``` ```python -list_of_members = list() -member_names = [a.name for a in ] -member_values = [a.value for a in ] -random_member = random.choice(list()) + = type() # Returns an enum. Also .__class__. + = itertools.cycle() # Returns an endless iterator of members. + = random.choice(list()) # Randomly selects one of the members. ``` ### Inline ```python -Cutlery = Enum('Cutlery', ['fork', 'knife', 'spoon']) -Cutlery = Enum('Cutlery', 'fork knife spoon') -Cutlery = Enum('Cutlery', {'fork': 1, 'knife': 2, 'spoon': 3}) +Cutlery = Enum('Cutlery', 'FORK KNIFE SPOON') +Cutlery = Enum('Cutlery', ['FORK', 'KNIFE', 'SPOON']) +Cutlery = Enum('Cutlery', {'FORK': 1, 'KNIFE': 2, 'SPOON': 3}) ``` -#### Functions can not be values, so they must be wrapped: +#### User-defined functions cannot be values, so they must be wrapped: ```python from functools import partial LogicOp = Enum('LogicOp', {'AND': partial(lambda l, r: l and r), - 'OR' : partial(lambda l, r: l or r)}) + 'OR': partial(lambda l, r: l or r)}) ``` Exceptions ---------- ```python -while True: - try: - x = int(input('Please enter a number: ')) - except ValueError: - print('Oops! That was no valid number. Try again...') - else: - print('Thank you.') - break +try: + +except : + ``` -### Raising Exception +### Complex Example ```python -raise ValueError('A very specific message!') +try: + +except : + +except : + +else: + +finally: + ``` +* **Code inside the `'else'` block will only be executed if `'try'` block had no exceptions.** +* **Code inside the `'finally'` block will always be executed (unless a signal is received).** +* **All variables that are initialized in executed blocks are also visible in all subsequent blocks, as well as outside the try statement (only the function block delimits scope).** +* **To catch signals use `'signal.signal(signal_number, handler_function)'`.** -### Finally +### Catching Exceptions ```python ->>> try: -... raise KeyboardInterrupt -... finally: -... print('Goodbye, world!') -Goodbye, world! -Traceback (most recent call last): - File "", line 2, in -KeyboardInterrupt +except : ... +except as : ... +except (, [...]): ... +except (, [...]) as : ... ``` +* **Also catches subclasses, e.g. `'IndexError'` is caught by `'except LookupError:'`.** +* **Use `'traceback.print_exc()'` to print the full error message to standard error stream.** +* **Use `'print()'` to print just the cause of the exception (that is, its arguments).** +* **Use `'logging.exception()'` to log the passed message, followed by the full error message of the caught exception. For details about setting up the logger see [Logging](#logging).** +* **Use `'sys.exc_info()'` to get exception type, object, and traceback of caught exception.** - -Print ------ +### Raising Exceptions ```python -print(, ..., sep=' ', end='\n', file=sys.stdout, flush=False) +raise +raise () +raise ( [, ...]) ``` -* **Use `'file=sys.stderr'` for errors.** - -### Pretty Print +#### Re-raising caught exception: ```python ->>> from pprint import pprint ->>> pprint(dir()) -['__annotations__', - '__builtins__', - '__doc__', ...] +except [as ]: + ... + raise ``` +### Exception Object +```python +arguments = .args +exc_type = .__class__ +filename = .__traceback__.tb_frame.f_code.co_filename +func_name = .__traceback__.tb_frame.f_code.co_name +line_str = linecache.getline(filename, .__traceback__.tb_lineno) +trace_str = ''.join(traceback.format_tb(.__traceback__)) +error_msg = ''.join(traceback.format_exception(type(), , .__traceback__)) +``` -Input ------ -* **Reads a line from user input or pipe if present.** -* **Trailing newline gets stripped.** -* **Prompt string is printed to the standard output before reading input.** +### Built-in Exceptions +```text +BaseException + +-- SystemExit # Raised by the sys.exit() function (see #Exit for details). + +-- KeyboardInterrupt # Raised when the user hits the interrupt key (control-c). + +-- Exception # User-defined exceptions should be derived from this class. + +-- ArithmeticError # Base class for arithmetic errors such as ZeroDivisionError. + +-- AssertionError # Raised by `assert ` if expression returns false value. + +-- AttributeError # Raised when object doesn't have requested attribute/method. + +-- EOFError # Raised by input() when it hits an end-of-file condition. + +-- LookupError # Base class for errors when a collection can't find an item. + | +-- IndexError # Raised when index of a sequence (list/str) is out of range. + | +-- KeyError # Raised when a dictionary key or set element is missing. + +-- MemoryError # Out of memory. May be too late to start deleting variables. + +-- NameError # Raised when nonexistent name (variable/func/class) is used. + | +-- UnboundLocalError # Raised when local name is used before it's being defined. + +-- OSError # Errors such as FileExistsError, TimeoutError (see #Open). + | +-- ConnectionError # Errors such as BrokenPipeError and ConnectionAbortedError. + +-- RuntimeError # Raised by errors that don't fall into other categories. + | +-- NotImplementedEr… # Can be raised by abstract methods or by unfinished code. + | +-- RecursionError # Raised if max recursion depth is exceeded (3k by default). + +-- StopIteration # Raised when exhausted (empty) iterator is passed to next(). + +-- TypeError # When an argument of the wrong type is passed to function. + +-- ValueError # When argument has the right type but inappropriate value. +``` + +#### Collections and their exceptions: +```text ++-----------+------------+------------+------------+ +| | List | Set | Dict | ++-----------+------------+------------+------------+ +| getitem() | IndexError | | KeyError | +| pop() | IndexError | KeyError | KeyError | +| remove() | ValueError | KeyError | | +| index() | ValueError | | | ++-----------+------------+------------+------------+ +``` +#### Useful built-in exceptions: ```python - = input(prompt=None) +raise TypeError('Argument is of the wrong type!') +raise ValueError('Argument has the right type but an inappropriate value!') +raise RuntimeError('I am too lazy to define my own exception!') ``` -#### Prints lines until EOF: +### User-defined Exceptions ```python -while True: - try: - print(input()) - except EOFError: - break +class MyError(Exception): pass +class MyInputError(MyError): pass ``` -Command Line Arguments ----------------------- +Exit +---- +**Exits the interpreter by raising SystemExit exception.** ```python import sys -script_name = sys.argv[0] -arguments = sys.argv[1:] +sys.exit() # Exits with exit code 0 (success). +sys.exit() # Exits with the passed exit code. +sys.exit() # Prints to stderr and exits with 1. ``` -### Argparse -```python + +Print +----- +```python +print(, ..., sep=' ', end='\n', file=sys.stdout, flush=False) +``` +* **Use `'file=sys.stderr'` for messages about errors so they can be processed separately.** +* **Stdout and stderr streams hold output in a buffer until they receive a string containing '\n' or '\r', buffer reaches 4096 characters, `'flush=True'` is used, or program exits.** + +### Pretty Print +```python +from pprint import pprint +pprint(, width=80, depth=None, compact=False, sort_dicts=True) +``` +* **Each item is printed on its own line if collection exceeds 'width' characters.** +* **Nested collections that are 'depth' levels deep get printed as '...'.** + + +Input +----- +```python + = input() +``` +* **Reads a line from the user input or pipe if present (trailing newline gets stripped).** +* **If argument is passed, it gets printed to the standard output before input is read.** +* **EOFError is raised if user hits EOF (ctrl-d/ctrl-z⏎) or if stream is already exhausted.** + + +Command Line Arguments +---------------------- +```python +import sys +scripts_path = sys.argv[0] +arguments = sys.argv[1:] +``` + +### Argument Parser +```python from argparse import ArgumentParser, FileType -p = ArgumentParser(description=) -p.add_argument('-', '--', action='store_true') # Flag -p.add_argument('-', '--', type=) # Option -p.add_argument('', type=, nargs=1) # Argument -p.add_argument('', type=, nargs='+') # Arguments -args = p.parse_args() -value = args. +p = ArgumentParser(description=) # Returns a parser object. +p.add_argument('-', '--', action='store_true') # Flag (defaults to False). +p.add_argument('-', '--', type=) # Option (defaults to None). +p.add_argument('', type=, nargs=1) # Mandatory first argument. +p.add_argument('', type=, nargs='+') # Mandatory remaining args. +p.add_argument('', type=, nargs='?/*') # Optional argument/s. +args = p.parse_args() # Exits on parsing error. + = args. # Returns `()`. ``` -* **Use `'help='` for argument description.** -* **Use `'type=FileType()'` for files.** +* **Use `'help='` to set argument description that will be displayed in help message.** +* **Use `'default='` to override None as option's or optional argument's default value.** +* **Use `'type=FileType()'` for files. It accepts 'encoding', but 'newline' is None.** Open ---- -**Opens a file and returns a corresponding file object.** +**Opens a file and returns the corresponding file object.** ```python - = open('', mode='r', encoding=None) + = open(, mode='r', encoding=None, newline=None) ``` +* **`'encoding=None'` means that the default encoding is used, which is platform dependent. Best practice is to use `'encoding="utf-8"'` until it becomes the default (Python 3.15).** +* **`'newline=None'` means all different end of line combinations are converted to '\n' on read, while on write all '\n' characters are converted to system's default line separator.** +* **`'newline=""'` means no conversions take place, but input is still broken into chunks by readline() and readlines() on every '\n', '\r' and '\r\n'.** ### Modes -* **`'r'` - Read (default).** -* **`'w'` - Write (truncate).** +* **`'r'` - Read. Used by default.** +* **`'w'` - Write. Deletes existing contents.** * **`'x'` - Write or fail if the file already exists.** -* **`'a'` - Append.** -* **`'w+'` - Read and write (truncate).** +* **`'a'` - Append. Creates new file if it doesn't exist.** +* **`'w+'` - Read and write. Deletes existing contents.** * **`'r+'` - Read and write from the start.** * **`'a+'` - Read and write from the end.** -* **`'t'` - Text mode (default).** -* **`'b'` - Binary mode.** +* **`'b'` - Binary mode (`'rb'`, `'wb'`, `'xb'`, …).** + +### Exceptions +* **`'FileNotFoundError'` can be raised when reading with `'r'` or `'r+'`.** +* **`'FileExistsError'` can be raised when writing with `'x'`.** +* **`'IsADirectoryError'` and `'PermissionError'` can be raised by any.** +* **`'OSError'` is the parent class of all listed exceptions.** -### File +### File Object ```python -.seek(0) # Moves to the start of the file. +.seek(0) # Moves current position to the start of file. .seek(offset) # Moves 'offset' chars/bytes from the start. -.seek(offset, ) # Anchor: 0 start, 1 current pos., 2 end. +.seek(0, 2) # Moves current position to the end of file. +.seek(±offset, origin) # Origin: 0 start, 1 current position, 2 end. ``` ```python - = .read(size=-1) # Reads 'size' chars/bytes or until EOF. - = .readline() # Returns a line. - = .readlines() # Returns a list of lines. - = next() # Returns a line using buffer. Do not mix. + = .read(size=-1) # Reads 'size' chars/bytes or until the EOF. + = .readline() # Returns a line or empty string/bytes on EOF. + = .readlines() # Returns remaining lines. Also list(). + = next() # Returns a line using a read-ahead buffer. ``` ```python -.write() # Writes a string or bytes object. -.writelines() # Writes a list of strings or bytes objects. -.flush() # Flushes write buffer. +.write() # Writes a str or bytes object to write buffer. +.writelines() # Writes a coll. of strings or bytes objects. +.flush() # Flushes write buffer. Runs every 4096/8192 B. +.close() # Closes the file after flushing write buffer. ``` -* **Methods do not add or strip trailing newlines.** +* **Methods do not add or strip trailing newlines, not even writelines().** ### Read Text from File ```python @@ -1108,134 +1617,183 @@ def write_to_file(filename, text): ``` -Path ----- +Paths +----- ```python -from os import path, listdir - = path.exists('') - = path.isfile('') - = path.isdir('') - = listdir('') +import os, glob +from pathlib import Path ``` ```python ->>> from glob import glob ->>> glob('../*.gif') -['1.gif', 'card.gif'] + = os.getcwd() # Returns working dir. Starts as shell's $PWD. + = os.path.join(, ...) # Uses os.sep to join strings or Path objects. + = os.path.realpath() # Resolves symlinks and calls path.abspath(). ``` -### Pathlib ```python -from pathlib import Path -cwd = Path() - = Path('' [, '', , ...]) - = / '' / '' + = os.path.basename() # Returns final component of the path. + = os.path.dirname() # Returns path without the final component. + = os.path.splitext() # Splits on last period of the final component. ``` ```python - = .exists() - = .is_file() - = .is_dir() - = .iterdir() + = os.listdir(path='.') # Returns all filenames located at the path. + = glob.glob('') # Returns paths matching the wildcard pattern. ``` ```python - = .glob('') + = os.path.exists() # Checks if path exists. Also .exists(). + = os.path.isfile() # Also .is_file() and .is_file(). + = os.path.isdir() # Also .is_dir() and .is_dir(). ``` ```python - = str() # Returns path as a string. - = .parts # Returns all components as strings. - = .resolve() # Returns absolute Path without symlinks. + = os.stat() # File's status. Also .stat(). + = .st_mtime/st_size/… # Returns modification time, size in bytes, etc. ``` +### DirEntry +**Unlike listdir(), scandir() returns DirEntry objects that cache isfile, isdir, and on Windows also stat information, thus significantly increasing the performance of code that requires it.** + ```python - = .name # Final component. - = .stem # Final component without extension. - = .suffix # Final component's extension. - = .parent # Path without final component. + = os.scandir(path='.') # Returns DirEntry objects located at the path. + = .path # Is absolute if 'path' argument was absolute. + = .name # Returns path's final component as a string. + = open() # Opens the file and returns its file object. ``` +### Path Object +```python + = Path( [, ...]) # Accepts strings, Paths, and DirEntry objects. + = / [/ ...] # First or second path must be a Path object. + = .resolve() # Returns absolute path with resolved symlinks. +``` -Command Execution ------------------ ```python -import os - = os.popen().read() + = Path() # Returns current working dir. Also Path('.'). + = Path.cwd() # Returns absolute CWD. Also Path().resolve(). + = Path.home() # Returns user's home directory (absolute). + = Path(__file__).resolve() # Returns module's path if CWD wasn't changed. ``` -### Subprocess ```python ->>> import subprocess ->>> a = subprocess.run(['ls', '-a'], stdout=subprocess.PIPE) ->>> a.stdout -b'.\n..\nfile1.txt\nfile2.txt\n' ->>> a.returncode -0 + = .parent # Returns Path without the final component. + = .name # Returns path's final component as a string. + = .suffix # Returns name's last extension, e.g. '.gz'. + = .stem # Returns name without the last extension. + = .parts # Returns all path's components as strings. ``` +```python + = .iterdir() # Returns directory contents as Path objects. + = .glob('') # Returns Paths matching the wildcard pattern. +``` -CSV ---- ```python -import csv + = str() # Returns path as string. Also .as_uri(). + = open() # Also .read_text/write_bytes(). ``` -### Read Rows from CSV File + +OS Commands +----------- ```python -def read_csv_file(filename): - with open(filename, encoding='utf-8') as file: - return csv.reader(file, delimiter=';') +import os, shutil, subprocess ``` -### Write Rows to CSV File ```python -def write_to_csv_file(filename, rows): - with open(filename, 'w', encoding='utf-8') as file: - writer = csv.writer(file, delimiter=';') - writer.writerows(rows) +os.chdir() # Changes the current working directory (CWD). +os.mkdir(, mode=0o777) # Creates a directory. Permissions are in octal. +os.makedirs(, mode=0o777) # Creates all path's dirs. Also `exist_ok=False`. +``` + +```python +shutil.copy(from, to) # Copies the file. 'to' can exist or be a dir. +shutil.copy2(from, to) # Also copies creation and modification time. +shutil.copytree(from, to) # Copies the directory. 'to' must not exist. +``` + +```python +os.rename(from, to) # Renames or moves the file or directory 'from'. +os.replace(from, to) # Same, but overwrites file 'to' even on Windows. +shutil.move(from, to) # Rename() that moves into 'to' if it's a dir. +``` + +```python +os.remove() # Deletes the file. Or `pip3 install send2trash`. +os.rmdir() # Deletes the empty directory or raises OSError. +shutil.rmtree() # Deletes the directory and all of its contents. +``` +* **Paths can be either strings, Path objects, or DirEntry objects.** +* **Functions report OS related errors by raising OSError or one of its [subclasses](#exceptions-1).** + +### Shell Commands +```python + = os.popen('') # Executes commands in sh/cmd. Returns combined stdout. + = .read(size=-1) # Reads 'size' chars or until EOF. Also readline/s(). + = .close() # Returns None if last command exited with returncode 0. +``` + +#### Sends "1 + 1" to the basic calculator and captures its output: +```python +>>> subprocess.run('bc', input='1 + 1\n', capture_output=True, text=True) +CompletedProcess(args='bc', returncode=0, stdout='2\n', stderr='') +``` + +#### Sends test.in to the basic calculator running in standard mode and saves its output to test.out: +```python +>>> from shlex import split +>>> os.popen('echo 1 + 1 > test.in') +>>> subprocess.run(split('bc -s'), stdin=open('test.in'), stdout=open('test.out', 'w')) +CompletedProcess(args=['bc', '-s'], returncode=0) +>>> open('test.out').read() +'2\n' ``` JSON ---- +**Text file format for storing collections of strings and numbers.** + ```python import json - = json.dumps(, ensure_ascii=True, indent=None) - = json.loads() + = json.dumps() # Converts collection to JSON string. + = json.loads() # Converts JSON string to collection. ``` -### Read Object from JSON File +### Read Collection from JSON File ```python def read_json_file(filename): with open(filename, encoding='utf-8') as file: return json.load(file) ``` -### Write Object to JSON File +### Write Collection to JSON File ```python -def write_to_json_file(filename, an_object): +def write_to_json_file(filename, collection): with open(filename, 'w', encoding='utf-8') as file: - json.dump(an_object, file, ensure_ascii=False, indent=2) + json.dump(collection, file, ensure_ascii=False, indent=2) ``` Pickle ------ +**Binary file format for storing Python objects.** + ```python import pickle - = pickle.dumps() - = pickle.loads() + = pickle.dumps() # Converts object to bytes object. + = pickle.loads() # Converts bytes object to object. ``` -### Read Object from File +### Read Object from Pickle File ```python def read_pickle_file(filename): with open(filename, 'rb') as file: return pickle.load(file) ``` -### Write Object to File +### Write Object to Pickle File ```python def write_to_pickle_file(filename, an_object): with open(filename, 'wb') as file: @@ -1243,54 +1801,173 @@ def write_to_pickle_file(filename, an_object): ``` +CSV +--- +**Text file format for storing spreadsheets.** + +```python +import csv +``` + +```python + = open(, newline='') # Opens the CSV (text) file for reading. + = csv.reader() # Also: `dialect='excel', delimiter=','`. + = next() # Returns next row as a list of strings. + = list() # Returns a list of all remaining rows. +``` +* **Without the `'newline=""'` argument, every '\r\n' sequence that is embedded inside a quoted field will get converted to '\n'! For details about newline argument see [Open](#open).** +* **To print the spreadsheet to the console use [Tabulate](#table) or PrettyTable library.** +* **For XML and binary Excel files (xlsx, xlsm and xlsb) use [Pandas](#fileformats) library.** +* **Reader accepts any iterator (or collection) of strings, not just files.** + +### Write +```python + = open(, 'w', newline='') # Opens the CSV (text) file for writing. + = csv.writer() # Also: `dialect='excel', delimiter=','`. +.writerow() # Encodes each object using `str()`. +.writerows() # Appends multiple rows to the file. +``` +* **If file is opened without the `'newline=""'` argument, '\r' will be added in front of every '\n' on platforms that use '\r\n' line endings (i.e., newlines may get doubled on Windows)!** +* **Open existing file with `'mode="a"'` to append to it or `'mode="w"'` to overwrite it.** + +### Parameters +* **`'dialect'` - Master parameter that sets the default values. String or a 'csv.Dialect' object.** +* **`'delimiter'` - A one-character string that separates fields (comma, tab, semicolon, etc.).** +* **`'lineterminator'` - How writer terminates rows. Reader looks for '\n', '\r' and '\r\n'.** +* **`'quotechar'` - Character for quoting fields containing delimiters, quotechars, '\n' or '\r'.** +* **`'escapechar'` - Character for escaping quotechars (not needed if doublequote is True).** +* **`'doublequote'` - Whether quotechars inside fields are/get doubled instead of escaped.** +* **`'quoting'` - 0: As necessary, 1: All, 2: All but numbers which are read as floats, 3: None.** +* **`'skipinitialspace'` - Is space character at the start of the field stripped by the reader.** + +### Dialects +```text ++------------------+--------------+--------------+--------------+ +| | excel | excel-tab | unix | ++------------------+--------------+--------------+--------------+ +| delimiter | ',' | '\t' | ',' | +| lineterminator | '\r\n' | '\r\n' | '\n' | +| quotechar | '"' | '"' | '"' | +| escapechar | None | None | None | +| doublequote | True | True | True | +| quoting | 0 | 0 | 1 | +| skipinitialspace | False | False | False | ++------------------+--------------+--------------+--------------+ +``` + +### Read Rows from CSV File +```python +def read_csv_file(filename, **csv_params): + with open(filename, encoding='utf-8', newline='') as file: + return list(csv.reader(file, **csv_params)) +``` + +### Write Rows to CSV File +```python +def write_to_csv_file(filename, rows, mode='w', **csv_params): + with open(filename, mode, encoding='utf-8', newline='') as file: + writer = csv.writer(file, **csv_params) + writer.writerows(rows) +``` + + SQLite ------ +**A server-less database engine that stores each database into its own file.** + ```python import sqlite3 -db = sqlite3.connect('') -... -db.close() + = sqlite3.connect() # Opens existing or new file. Also ':memory:'. +.close() # Closes connection. Discards uncommitted data. ``` ### Read ```python -cursor = db.execute('') -if cursor: - = cursor.fetchone() # First row. - = cursor.fetchall() # Remaining rows. + = .execute('') # Can raise a subclass of sqlite3.Error. + = .fetchone() # Returns the next row. Also next(). + = .fetchall() # Returns remaining rows. Also list(). ``` ### Write ```python -db.execute('') -db.commit() +.execute('') # Can raise a subclass of sqlite3.Error. +.commit() # Saves all changes since the last commit. +.rollback() # Discards all changes since the last commit. +``` + +#### Or: +```python +with : # Exits the block with commit() or rollback(), + .execute('') # depending on whether any exception occurred. +``` + +### Placeholders +```python +.execute('', ) # Replaces every question mark with an item. +.execute('', ) # Replaces every : with a matching value. +.executemany('', ) # Runs execute() once for each collection. +``` +* **Passed values can be of type str, int, float, bytes, None, or bool (stored as 1 or 0).** +* **SQLite does not restrict columns to any type unless table is declared as strict.** + +### Example +**Values are not actually saved in this example because `'conn.commit()'` is omitted!** +```python +>>> conn = sqlite3.connect('test.db') +>>> conn.execute('CREATE TABLE person (name TEXT, height INTEGER) STRICT') +>>> conn.execute('INSERT INTO person VALUES (?, ?)', ('Jean-Luc', 187)) +>>> conn.execute('SELECT rowid, * FROM person').fetchall() +[(1, 'Jean-Luc', 187)] +``` + +### SQLAlchemy +**Library for interacting with various DB systems via SQL, method chaining, or ORM.** +```python +# $ pip3 install sqlalchemy +from sqlalchemy import create_engine, text + = create_engine('') # Url: 'dialect://user:password@host/dbname'. + = .connect() # Creates a connection. Also .close(). + = .execute(text(''), …) # ``. Replaces every : with value. +with .begin(): ... # Exits the block with commit or rollback. +``` + +```text ++-----------------+--------------+----------------------------------+ +| Dialect | pip3 install | Dependencies | ++-----------------+--------------+----------------------------------+ +| mysql | mysqlclient | www.pypi.org/project/mysqlclient | +| postgresql | psycopg2 | www.pypi.org/project/psycopg2 | +| mssql | pyodbc | www.pypi.org/project/pyodbc | +| oracle+oracledb | oracledb | www.pypi.org/project/oracledb | ++-----------------+--------------+----------------------------------+ ``` Bytes ----- -**Bytes object is an immutable sequence of single bytes. Mutable version is called 'bytearray'.** +**A bytes object is an immutable sequence of single bytes. Mutable version is called bytearray.** ```python - = b'' - = [] - = [] - = list() - = b''.join() + = b'' # Only accepts ASCII characters and \x00-\xff. + = [index] # Returns an integer in range from 0 to 255. + = [] # Returns bytes even if it has only one element. + = .join() # Joins elements by using bytes as a separator. ``` ### Encode ```python - = .encode(encoding='utf-8') - = .to_bytes(, byteorder='big|little', signed=False) - = bytes.fromhex('') + = bytes() # Integers must be in range from 0 to 255. + = bytes(, 'utf-8') # Encodes the string. Also .encode(). + = bytes.fromhex('') # Hex pairs can be separated by whitespaces. + = .to_bytes(n_bytes, …) # `byteorder='big/little', signed=False`. ``` ### Decode ```python - = .decode(encoding='utf-8') - = int.from_bytes(, byteorder='big|little', signed=False) - = .hex() + = list() # Returns integers in range from 0 to 255. + = str(, 'utf-8') # Returns a string. Also .decode(). + = .hex() # Returns hex pairs. Accepts `sep=`. + = int.from_bytes(, …) # `byteorder='big/little', signed=False`. ``` ### Read Bytes from File @@ -1310,305 +1987,402 @@ def write_bytes(filename, bytes_obj): Struct ------ -* **Module that performs conversions between Python values and a C struct, represented as a Python bytes object.** -* **Machine’s native type sizes and byte order are used by default.** +* **Module that performs conversions between a sequence of numbers and a bytes object.** +* **System’s type sizes, byte order, and alignment rules are used by default.** ```python -from struct import pack, unpack, iter_unpack, calcsize - = pack('', [, , ...]) - = unpack('', ) - = iter_unpack('', ) +from struct import pack, unpack + + = pack('', [, ...]) # Packs numbers according to format string. + = unpack('', ) # Use iter_unpack() to get iterator of tuples. ``` -### Example ```python >>> pack('>hhl', 1, 2, 3) b'\x00\x01\x00\x02\x00\x00\x00\x03' >>> unpack('>hhl', b'\x00\x01\x00\x02\x00\x00\x00\x03') (1, 2, 3) ->>> calcsize('>hhl') -8 ``` ### Format -#### For standard sizes start format string with: -* **`'='` - native byte order** -* **`'<'` - little-endian** -* **`'>'` - big-endian** - -#### Use capital letter for unsigned type. Standard sizes are in brackets: -* **`'x'` - pad byte** -* **`'c'` - char (1)** -* **`'h'` - short (2)** -* **`'i'` - int (4)** -* **`'l'` - long (4)** -* **`'q'` - long long (8)** -* **`'f'` - float (4)** -* **`'d'` - double (8)** +#### For standard type sizes and manual alignment (padding) start format string with: +* **`'='` - System's byte order (usually little-endian).** +* **`'<'` - Little-endian (i.e. least significant byte first).** +* **`'>'` - Big-endian (also `'!'`).** + +#### Besides numbers, pack() and unpack() also support bytes objects as a part of the sequence: +* **`'c'` - A bytes object with a single element. For pad byte use `'x'`.** +* **`'s'` - A bytes object with n elements (not effected by byte order).** + +#### Integer types. Use a capital letter for unsigned type. Minimum and standard sizes are in brackets: +* **`'b'` - char (1/1)** +* **`'h'` - short (2/2)** +* **`'i'` - int (2/4)** +* **`'l'` - long (4/4)** +* **`'q'` - long long (8/8)** + +#### Floating point types (struct always uses standard sizes): +* **`'f'` - float (4/4)** +* **`'d'` - double (8/8)** Array ----- -**List that can hold only elements of predefined type. Available types are listed above.** +**List that can only hold numbers of a predefined type. Available types and their minimum sizes in bytes are listed above. Type sizes and byte order are always determined by the system, however bytes of each element can be reversed (by calling the byteswap() method).** ```python from array import array - = array('' [, ]) +``` + +```python + = array('', ) # Creates an array from collection of numbers. + = array('', ) # Writes passed bytes to the array's memory. + = array('', ) # Treats passed array as a sequence of numbers. +.fromfile(, n_items) # Appends file contents to the array's memory. +``` + +```python + = bytes() # Returns a copy of array's memory as bytes. +.write() # Writes array's memory to the binary file. ``` Memory View ----------- -**Used for accessing the internal data of an object that supports the buffer protocol.** +**A sequence object that points to the memory of another bytes-like object. Each element can reference a single or multiple consecutive bytes, depending on format. Order and number of elements can be changed with slicing.** + +```python + = memoryview() # Immutable if bytes is passed, else mutable. + = [index] # Returns ints/floats. Bytes if format is 'c'. + = [] # Returns memoryview with rearranged elements. + = .cast('') # Only works between B/b/c and other types. +.release() # Releases memory buffer of the base object. +``` + +```python + = bytes() # Returns a new bytes object. Also bytearray(). + = .join() # Joins memoryviews using bytes as a separator. + = array('', ) # Treats memoryview as a sequence of numbers. +.write() # Writes `bytes()` to the binary file. +``` ```python - = memoryview( / / ) -.release() + = list() # Returns a list of ints, floats or bytes. + = str(, 'utf-8') # Treats passed memoryview as a bytes object. + = .hex() # Returns hex pairs. Accepts `sep=`. ``` Deque ----- -**Thread-safe list with efficient appends and pops from either side. Pronounced "deck".** +**List with efficient appends and pops from either side.** ```python from collections import deque - = deque(, maxlen=None) ``` ```python -.appendleft() - = .popleft() + = deque() # Use `maxlen=` to set size limit. +.appendleft() # Opposite element is dropped if full. +.extendleft() # Prepends reversed coll. to the deque. +.rotate(n=1) # Last element becomes the first one. + = .popleft() # Raises IndexError if deque is empty. ``` + +Operator +-------- +**Module of functions that provide the functionality of operators. Functions are grouped by operator precedence, from least to most binding. Functions and operators in first, third and fifth line are also ordered by precedence within a group.** ```python -.extendleft() # Collection gets reversed. -.rotate(n=1) # Rotates elements to the right. +import operator as op ``` +```python + = op.not_() # or, and, not (or/and missing) + = op.eq/ne/lt/ge/is_/is_not/contains(, ) # ==, !=, <, >=, is, is not, in + = op.or_/xor/and_(, ) # |, ^, & (sorted by precedence) + = op.lshift/rshift(, ) # <<, >> (i.e. << n_bits) + = op.add/sub/mul/truediv/floordiv/mod(, ) # +, -, *, /, //, % (two groups) + = op.neg/invert() # -, ~ (negate and bitwise not) + = op.pow(, ) # ** (pow() accepts 3 arguments) + = op.itemgetter/attrgetter/methodcaller( [, ...]) # [index/key], .name, .name([…]) +``` -Threading ---------- ```python -from threading import Thread, RLock +elementwise_sum = map(op.add, list_a, list_b) +sorted_by_second = sorted(, key=op.itemgetter(1)) +sorted_by_both = sorted(, key=op.itemgetter(1, 0)) +first_element = op.methodcaller('pop', 0)() ``` +* **Most operators call the object's special method that is named after them (second object is passed as an argument), while logical operators call their own code that relies on bool().** +* **Comparisons can be chained: `'x < y < z'` gets converted to `'(x < y) and (y < z)`'.** + + +Match Statement +--------------- +**Executes the first block with matching pattern.** -### Thread ```python -thread = Thread(target=, args=(, )) -thread.start() -... -thread.join() +match : + case [if ]: + + ... ``` -### Lock +### Patterns ```python -lock = RLock() -lock.acquire() -... -lock.release() + = 1/'abc'/True/None/math.pi # Matches the literal or a dotted name. + = () # Matches any object of that type (or ABC). + = _ # Matches any object. Useful in last case. + = # Matches any object and binds it to name. + = as # Binds match to name. Also (). + = | [| ...] # Matches if any of listed patterns match. + = [, ...] # Matches a sequence. All items must match. + = {: , ...} # Matches a dict that has matching items. + = (=, ...) # Matches an object if attributes match. ``` +* **Sequence pattern can also be written as a tuple, either with or without the brackets.** +* **Use `'*'` and `'**'` in sequence/mapping patterns to bind remaining items.** +* **Sequence pattern must match all items of the collection, while mapping pattern does not.** +* **Patterns can be surrounded with brackets to override their precedence: `'|'` > `'as'` > `','`. For example, `'[1, 2]'` is matched by the `'case 1|2, 2|3 as x if x == 2:'` block.** +* **All names that are bound in the matching case, as well as variables initialized in its body, are visible after the match statement (only function block delimits scope).** +### Example +```python +>>> from pathlib import Path +>>> match Path('/home/gto/python-cheatsheet/README.md'): +... case Path( +... parts=['/', 'home', user, *_] +... ) as p if p.name.lower().startswith('readme') and p.is_file(): +... print(f'{p.name} is a readme file that belongs to user {user}.') +README.md is a readme file that belongs to user gto. +``` -Introspection -------------- -**Inspecting code at runtime.** -### Variables +Logging +------- +```python +import logging as log +``` + ```python - = dir() # Names of variables in current scope. - = locals() # Dict of local variables. Also vars(). - = globals() # Dict of global variables. +log.basicConfig(filename=, level='DEBUG') # Configures the root logger (see Setup). +log.debug/info/warning/error/critical() # Sends passed message to the root logger. + = log.getLogger(__name__) # Returns a logger named after the module. +.() # Sends the message. Same levels as above. +.exception() # Error() that appends caught exception. ``` -### Attributes +### Setup ```python - = vars() - = hasattr(, '') -value = getattr(, '') -setattr(, '', value) +log.basicConfig( + filename=None, # Logs to stderr or appends to file. + format='%(levelname)s:%(name)s:%(message)s', # Add '%(asctime)s' for local datetime. + level=log.WARNING, # Drops messages with lower priority. + handlers=[log.StreamHandler(sys.stderr)] # Uses FileHandler if filename is set. +) ``` -### Parameters ```python -from inspect import signature - = signature() -no_of_params = len(.parameters) -param_names = list(.parameters.keys()) + = log.Formatter('') # Formats messages according to format str. + = log.FileHandler(, mode='a') # Appends to file. Also `encoding=None`. +.setFormatter() # Only outputs bare messages by default. +.setLevel() # Prints or saves every message by default. +.addHandler() # Logger can have more than one handler. +.setLevel() # What is sent to its/ancestors' handlers. +.propagate = # Cuts off ancestors' handlers if False. ``` +* **Parent logger can be specified by naming the child logger `'.'`.** +* **If logger doesn't have a set level, it inherits it from the first ancestor that does.** +* **Formatter also accepts: pathname, filename, funcName, lineno, thread and process.** +* **RotatingFileHandler creates and deletes files based on 'maxBytes', 'backupCount' args.** +* **An object with `'filter()'` method (or the method itself) can be added to loggers and handlers via addFilter(). Message is dropped if filter() returns a false value.** +#### Creates a logger that writes all messages to a file and sends them to the root's handler that prints warnings or higher: +```python +>>> logger = log.getLogger('my_module') +>>> handler = log.FileHandler('test.log', encoding='utf-8') +>>> handler.setFormatter(log.Formatter('%(asctime)s %(levelname)s:%(name)s:%(message)s')) +>>> logger.addHandler(handler) +>>> logger.setLevel('DEBUG') +>>> log.basicConfig() +>>> log.root.handlers[0].setLevel('WARNING') +>>> logger.critical('Running out of disk space.') +CRITICAL:my_module:Running out of disk space. +>>> print(open('test.log').read()) +2023-02-07 23:21:01,430 CRITICAL:my_module:Running out of disk space. +``` -Metaprograming --------------- -**Code that generates code.** -### Type -**Type is the root class. If only passed the object it returns its type (class). Otherwise it creates a new class.** +Introspection +------------- +```python + = dir() # Local names of variables, functions, classes and modules. + = vars() # Dict of local names and their objects. Same as locals(). + = globals() # Dict of global names and their objects, e.g. __builtin__. +``` ```python - = type(, , ) + = dir() # Returns names of object's attributes (including methods). + = vars() # Returns dict of writable attributes. Also .__dict__. + = hasattr(, '') # Checks if object possesses attribute with passed name. +value = getattr(, '') # Returns the object's attribute or raises AttributeError. +setattr(, '', value) # Sets attribute. Only works on objects with __dict__ attr. +delattr(, '') # Deletes attribute from __dict__. Also `del .`. ``` ```python ->>> Z = type('Z', (), {'a': 'abcde', 'b': 12345}) ->>> z = Z() + = inspect.signature() # Returns Signature object of the passed function or class. + = .parameters # Returns dict of Parameters. Also .return_annotation. + = .kind # Returns ParameterKind member (Parameter.KEYWORD_ONLY, …). + = .annotation # Returns Parameter.empty if missing. Also .default. ``` -### Meta Class -**Class that creates class.** +Threading +--------- +**CPython interpreter can only run a single thread at a time. Using multiple threads won't result in a faster execution, unless at least one of the threads contains an I/O operation.** ```python -def my_meta_class(name, parents, attrs): - attrs['a'] = 'abcde' - return type(name, parents, attrs) +from threading import Thread, Lock, RLock, Semaphore, Event, Barrier +from concurrent.futures import ThreadPoolExecutor, as_completed ``` -#### Or: +### Thread ```python -class MyMetaClass(type): - def __new__(cls, name, parents, attrs): - attrs['a'] = 'abcde' - return type.__new__(cls, name, parents, attrs) + = Thread(target=) # Use `args=` to set the arguments. +.start() # Starts the thread. Also .is_alive(). +.join() # Waits for the thread to finish executing. ``` -* **New() is a class method that gets called before init(). If it returns an instance of its class, then that instance gets passed to init() as a 'self' argument.** -* **It receives the same arguments as init(), except for the first one that specifies the desired class of returned instance (**`'MyMetaClass'` **in our case).** -* **New() can also be called directly, usually from a new() method of a child class (**`def __new__(cls): return super().__new__(cls)`**), in which case init() is not called.** +* **Use `'kwargs='` to pass keyword arguments to the function.** +* **Use `'daemon=True'`, or the program won't be able to exit while the thread is alive.** -### Metaclass Attribute -**When class is created it checks if it has metaclass defined. If not, it recursively checks if any of his parents has it defined and eventually comes to type.** +### Lock +```python + = Lock/RLock() # RLock can only be released by acquirer thread. +.acquire() # Blocks (waits) until lock becomes available. +.release() # Releases the lock so it can be acquired again. +``` +#### Or: ```python -class MyClass(metaclass=MyMetaClass): - b = 12345 +with : # Enters the block by calling method acquire(). + ... # Exits it by calling release(), even on error. ``` +### Semaphore, Event, Barrier ```python ->>> MyClass.a, MyClass.b -('abcde', 12345) + = Semaphore(value=1) # Lock that can be acquired by 'value' threads. + = Event() # Method wait() blocks until set() is called. + = Barrier() # Wait() blocks until it's called int times. ``` +### Queue +```python + = queue.Queue(maxsize=0) # A first-in-first-out queue. It's thread safe. +.put() # Call blocks until queue stops being full. +.put_nowait() # Raises queue.Full exception if queue is full. + = .get() # Call blocks until queue stops being empty. + = .get_nowait() # Raises queue.Empty exception if it's empty. +``` -Operator --------- +### Thread Pool Executor ```python -from operator import add, sub, mul, truediv, floordiv, mod, pow, neg, abs -from operator import eq, ne, lt, le, gt, ge -from operator import not_, and_, or_ -from operator import itemgetter, attrgetter, methodcaller + = ThreadPoolExecutor(max_workers=None) # Also `with ThreadPoolExecutor() as : …`. + = .map(, , ...) # Multithreaded and non-lazy map(). Keeps order. + = .submit(, , ...) # Creates a thread and returns its Future obj. +.shutdown() # Waits for all threads to finish executing. ``` ```python -import operator as op -product_of_elems = functools.reduce(op.mul, ) -sorted_by_second = sorted(, key=op.itemgetter(1)) -sorted_by_both = sorted(, key=op.itemgetter(1, 0)) -LogicOp = enum.Enum('LogicOp', {'AND': op.and_, 'OR' : op.or_}) -last_el = op.methodcaller('pop')() + = .done() # Checks if the thread has finished executing. + = .result(timeout=None) # Raises TimeoutError after 'timeout' seconds. + = .cancel() # Cancels or returns False if running/finished. + = as_completed() # `next()` returns next completed Future. ``` +* **Map() and as\_completed() also accept 'timeout'. It causes futures.TimeoutError when next() is called/blocking. Map() times from original call and as_completed() from first call to next(). As\_completed() fails if next() is called too late, even if all threads are done.** +* **Exceptions that happen inside threads are raised when map iterator's next() or Future's result() are called. Future's exception() method returns an exception object or None.** +* **ProcessPoolExecutor provides true parallelism but: everything sent to/from workers must be [pickable](#pickle), queues must be sent using executor's 'initargs' and 'initializer' parameters, and executor should only be reachable via `'if __name__ == "__main__": ...'`.** -Eval ----- -### Basic +Coroutines +---------- +* **Coroutines have a lot in common with threads, but unlike threads, they only give up control when they call another coroutine and they don’t consume as much memory.** +* **Coroutine definition starts with `'async'` keyword and its call with `'await'`.** +* **Use `'asyncio.run()'` to start the first/main coroutine.** + ```python ->>> from ast import literal_eval ->>> literal_eval('1 + 2') -3 ->>> literal_eval('[1, 2, 3]') -[1, 2, 3] ->>> literal_eval('abs(1)') -ValueError: malformed node or string +import asyncio as aio ``` -### Using Abstract Syntax Trees ```python -import ast -from ast import Num, BinOp, UnaryOp -import operator as op - -LEGAL_OPERATORS = {ast.Add: op.add, # + - ast.Sub: op.sub, # - - ast.Mult: op.mul, # * - ast.Div: op.truediv, # / - ast.Pow: op.pow, # ** - ast.BitXor: op.xor, # ^ - ast.USub: op.neg} # - - -def evaluate(expression): - root = ast.parse(expression, mode='eval') - return eval_node(root.body) - -def eval_node(node): - node_type = type(node) - if node_type == Num: - return node.n - if node_type not in [BinOp, UnaryOp]: - raise TypeError(node) - operator_type = type(node.op) - if operator_type not in LEGAL_OPERATORS: - raise TypeError(f'Illegal operator {node.op}') - operator = LEGAL_OPERATORS[operator_type] - if node_type == BinOp: - left, right = eval_node(node.left), eval_node(node.right) - return operator(left, right) - elif node_type == UnaryOp: - operand = eval_node(node.operand) - return operator(operand) -``` - -```python ->>> evaluate('2 ^ 6') -4 ->>> evaluate('2 ** 6') -64 ->>> evaluate('1 + 2 * 3 ** (4 ^ 5) / (6 + -7)') --5.0 -``` - - -Coroutine ---------- -* **Similar to generator, but generator pulls data through the pipe with iteration, while coroutine pushes data into the pipeline with send().** -* **Coroutines provide more powerful data routing possibilities than iterators.** -* **If you build a collection of simple data processing components, you can glue them together into complex arrangements of pipes, branches, merging, etc.** - -### Helper Decorator -* **All coroutines must be "primed" by first calling next().** -* **Remembering to call next() is easy to forget.** -* **Solved by wrapping coroutines with a decorator:** + = () # Creates a coroutine by calling async def function. + = await # Starts the coroutine. Returns its result or None. + = aio.create_task() # Schedules it for execution. Always keep the task. + = await # Returns coroutine's result. Also .cancel(). +``` ```python -def coroutine(func): - def out(*args, **kwargs): - cr = func(*args, **kwargs) - next(cr) - return cr - return out + = aio.gather(, ...) # Schedules coros. Returns list of results on await. + = aio.wait(, return_when=…) # `'ALL/FIRST_COMPLETED'`. Returns (done, pending). + = aio.as_completed() # Iter of coros that return next result on await. ``` -### Pipeline Example +#### Runs a terminal game where you control an asterisk that must avoid numbers: ```python -def reader(target): - for i in range(10): - target.send(i) - target.close() +import asyncio, collections, curses, curses.textpad, enum, random + +P = collections.namedtuple('P', 'x y') # Position (x and y coordinates). +D = enum.Enum('D', 'n e s w') # Direction (north, east, etc.). +W, H = 15, 7 # Width and height of the field. + +def main(screen): + curses.curs_set(0) # Makes the cursor invisible. + screen.nodelay(True) # Makes getch() non-blocking. + asyncio.run(main_coroutine(screen)) # Starts running asyncio code. + +async def main_coroutine(screen): + moves = asyncio.Queue() + state = {'*': P(0, 0)} | {id_: P(W//2, H//2) for id_ in range(10)} + ai = [random_controller(id_, moves) for id_ in range(10)] + mvc = [controller(screen, moves), model(moves, state), view(state, screen)] + tasks = [asyncio.create_task(coro) for coro in ai + mvc] + await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED) -@coroutine -def adder(target): +async def random_controller(id_, moves): while True: - value = (yield) - target.send(value + 100) + d = random.choice(list(D)) + moves.put_nowait((id_, d)) + await asyncio.sleep(random.triangular(0.01, 0.65)) -@coroutine -def printer(): +async def controller(screen, moves): while True: - value = (yield) - print(value) + key_mappings = {258: D.s, 259: D.n, 260: D.w, 261: D.e} + if d := key_mappings.get(screen.getch()): + moves.put_nowait(('*', d)) + await asyncio.sleep(0.005) + +async def model(moves, state): + while state['*'] not in (state[id_] for id_ in range(10)): + id_, d = await moves.get() + deltas = {D.n: P(0, -1), D.e: P(1, 0), D.s: P(0, 1), D.w: P(-1, 0)} + state[id_] = P((state[id_].x + deltas[d].x) % W, (state[id_].y + deltas[d].y) % H) + +async def view(state, screen): + offset = P(curses.COLS//2 - W//2, curses.LINES//2 - H//2) + while True: + screen.erase() + curses.textpad.rectangle(screen, offset.y-1, offset.x-1, offset.y+H, offset.x+W) + for id_, p in state.items(): + screen.addstr(offset.y + (p.y - state['*'].y + H//2) % H, + offset.x + (p.x - state['*'].x + W//2) % W, str(id_)) + screen.refresh() + await asyncio.sleep(0.005) -reader(adder(printer())) # 100, 101, ..., 109 +if __name__ == '__main__': + curses.wrapper(main) ``` +
-

Libraries ========= @@ -1617,12 +2391,10 @@ Progress Bar ------------ ```python # $ pip3 install tqdm -from tqdm import tqdm -from time import sleep -for i in tqdm([1, 2, 3]): - sleep(0.2) -for i in tqdm(range(100)): - sleep(0.02) +>>> import tqdm, time +>>> for el in tqdm.tqdm([1, 2, 3], desc='Processing'): +... time.sleep(1) +Processing: 100%|████████████████████| 3/3 [00:03<00:00, 1.00s/it] ``` @@ -1630,223 +2402,250 @@ Plot ---- ```python # $ pip3 install matplotlib -from matplotlib import pyplot -pyplot.plot( [, , ...]) -pyplot.savefig(, transparent=True) -pyplot.show() +import matplotlib.pyplot as plt + +plt.plot/bar/scatter(x_data, y_data [, label=]) # Also plt.plot(y_data). +plt.legend() # Adds a legend of labels. +plt.title/xlabel/ylabel() # Adds title or axis label. +plt.show() # Also plt.savefig(). +plt.clf() # Clears the plot (figure). ``` Table ----- -#### Prints CSV file as ASCII table: +#### Prints a CSV spreadsheet to the console: ```python # $ pip3 install tabulate -from tabulate import tabulate -import csv -with open(, encoding='utf-8') as file: - lines = csv.reader(file, delimiter=';') - headers = [header.title() for header in next(lines)] - table = tabulate(lines, headers) - print(table) +import csv, tabulate +with open('test.csv', encoding='utf-8', newline='') as file: + rows = list(csv.reader(file)) +print(tabulate.tabulate(rows, headers='firstrow')) ``` -Curses ------- -```python -from curses import wrapper - -def main(): - wrapper(draw) - -def draw(screen): - screen.clear() - screen.addstr(0, 0, 'Press ESC to quit.') - while screen.getch() != 27: - pass +Console App +----------- +#### Runs a basic file explorer in the console: +```python +# $ pip3 install windows-curses +import curses, os +from curses import A_REVERSE, KEY_UP, KEY_DOWN, KEY_LEFT, KEY_RIGHT + +def main(screen): + ch, first, selected, paths = 0, 0, 0, os.listdir() + while ch != ord('q'): + height, width = screen.getmaxyx() + screen.erase() + for y, filename in enumerate(paths[first : first+height]): + color = A_REVERSE if filename == paths[selected] else 0 + screen.addnstr(y, 0, filename, width-1, color) + ch = screen.getch() + selected -= (ch == KEY_UP) and (selected > 0) + selected += (ch == KEY_DOWN) and (selected < len(paths)-1) + first -= (first > selected) + first += (first < selected-(height-1)) + if ch in [KEY_LEFT, KEY_RIGHT, ord('\n')]: + new_dir = '..' if ch == KEY_LEFT else paths[selected] + if os.path.isdir(new_dir): + os.chdir(new_dir) + first, selected, paths = 0, 0, os.listdir() -def get_border(screen): - from collections import namedtuple - P = namedtuple('P', 'x y') - height, width = screen.getmaxyx() - return P(width-1, height-1) +if __name__ == '__main__': + curses.wrapper(main) ``` -Logging +GUI App ------- -```python -# $ pip3 install loguru -from loguru import logger -``` +#### Runs a desktop app for converting weights from metric units into pounds: ```python -logger.add('debug_{time}.log', colorize=True) # Connects a log file. -logger.add('error_{time}.log', level='ERROR') # Another file for errors or higher. -logger.('A logging message') +# $ pip3 install PySimpleGUI +import PySimpleGUI as sg + +text_box = sg.Input(default_text='100', enable_events=True, key='QUANTITY') +dropdown = sg.InputCombo(['g', 'kg', 't'], 'kg', readonly=True, enable_events=True, k='UNIT') +label = sg.Text('100 kg is 220.462 lbs.', key='OUTPUT') +window = sg.Window('Weight Converter', [[text_box, dropdown], [label], [sg.Button('Close')]]) + +while True: + event, values = window.read() + if event in [sg.WIN_CLOSED, 'Close']: + break + try: + quantity = float(values['QUANTITY']) + except ValueError: + continue + unit = values['UNIT'] + lbs = quantity * {'g': 0.001, 'kg': 1, 't': 1000}[unit] / 0.45359237 + window['OUTPUT'].update(value=f'{quantity} {unit} is {lbs:g} lbs.') +window.close() ``` -* **Levels: `'debug'`, `'info'`, `'success'`, `'warning'`, `'error'`, `'critical'`.** -### Rotation -**Parameter that sets a condition when a new log file is created.** + +Scraping +-------- +#### Scrapes Python's URL and logo from its Wikipedia page: ```python -rotation=||| +# $ pip3 install requests beautifulsoup4 +import requests, bs4, os + +get = lambda url: requests.get(url, headers={'User-Agent': 'cpc-bot'}) +response = get('https://en.wikipedia.org/wiki/Python_(programming_language)') +document = bs4.BeautifulSoup(response.text, 'html.parser') +table = document.find('table', class_='infobox vevent') +python_url = table.find('th', text='Website').next_sibling.a['href'] +logo_url = table.find('img')['src'] +filename = os.path.basename(logo_url) +with open(filename, 'wb') as file: + file.write(get(f'https:{logo_url}').content) +print(f'URL: {python_url}, logo: file://{os.path.abspath(filename)}') ``` -* **`''` - Max file size in bytes.** -* **`''` - Max age of a file.** -* **`'