diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5d9f03aa1..fb3abe19b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -17,7 +17,7 @@ pip install --user sphinx Then navigate to the directory of the Makefile and ```make build``` or ```make html```. Sphinx will then generate the HTML in a folder called `_build/html/` -After navigating to this folder, you can then use Python's built in webserver to view your changes locally: +After navigating to this folder, you can then use Python's built in web server to view your changes locally: ``` bash python3 -m http.server diff --git a/Readme.rst b/Readme.rst index d7d26dfb1..f35c71225 100644 --- a/Readme.rst +++ b/Readme.rst @@ -3,6 +3,8 @@ Hitchhiker's Guide to Python **Python Best Practices Guidebook** +→ Read the free guide at: `docs.python-guide.org `_ + .. image:: https://farm1.staticflickr.com/628/33173824932_58add34581_k_d.jpg ----------- diff --git a/docs/404.rst b/docs/404.rst index 1cb26757c..f4fc9fcc1 100644 --- a/docs/404.rst +++ b/docs/404.rst @@ -1,5 +1,8 @@ +:orphan: + +################# 404 — Not Found -=============== +################# **Sorry, but we couldn't find the page you requested.** diff --git a/docs/Makefile b/docs/Makefile index 13201e251..963704aff 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -19,7 +19,7 @@ ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . -.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext +.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf latexpdfja text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" diff --git a/docs/_extra/_redirects b/docs/_extra/_redirects index 5faad715d..2cb986392 100644 --- a/docs/_extra/_redirects +++ b/docs/_extra/_redirects @@ -15,4 +15,4 @@ https://python-guide.org/* https://docs.python-guide.org/:s # Redirect Netlify version to primary domain https://python-guide.netlify.com/* https://docs.python-guide.org/:splat 301! -/guide-book https://realpython.com/asins/1491933178/ 302 +/guide-book https://www.amazon.com/Hitchhikers-Guide-Python-Practices-Development/dp/1491933178/ref=as_li_ss_il?ie=UTF8&linkCode=li2&tag=bookforkind-20&linkId=804806ebdacaf3b56567347f3afbdbca 302 diff --git a/docs/_extra/ads.txt b/docs/_extra/ads.txt new file mode 100644 index 000000000..52291a930 --- /dev/null +++ b/docs/_extra/ads.txt @@ -0,0 +1,340 @@ +33across.com, 0010b00002Mpn7AAAR, DIRECT, bbea06d9c4d2853c +33across.com, 0013300001kQj2HAAS, RESELLER, bbea06d9c4d2853c +33across.com, 0013300001qkdlwAAA, RESELLER +33across.com, 0013300001r0t9mAAA, RESELLER +33across.com, 0014000001aXjnGAAS, RESELLER, bbea06d9c4d2853c # 33 Across +33across.com,0013300001r0t9mAAA,reseller,bbea06d9c4d2853c +ad-generation.jp,12474,RESELLER,7f4ea9029ac04e53 +adform.com, 183, RESELLER +adform.com, 2708, DIRECT, 9f5210a2f0999e32 +adtech.com, 10947, DIRECT, e1a5b5b6e3255540 +adtech.com, 11095, RESELLER +adtech.com, 11119, RESELLER +adtech.com, 11208, RESELLER +adtech.com, 11341, DIRECT +adtech.com, 12094, RESELLER +adtech.com, 12094, RESELLER # 33 Across +adtech.com, 9904, RESELLER +adtech.com,12068,RESELLER,e1a5b5b6e3255540 +advangelists.com, 8d3bba7425e7c98c50f52ca1b52d3735, RESELLER, 60d26397ec060f98 +advangelists.com, 8d3bba7425e7c98c50f52ca1b52d3735, RESELLER, 60d26397ec060f98 # 33 Across +advertising.com, 11602, RESELLER +Advertising.com, 16736, RESELLER +advertising.com, 19623, RESELLER # AOL - One +advertising.com, 24410, RESELLER +advertising.com, 28409, DIRECT, e1a5b5b6e3255540 +advertising.com, 28509, DIRECT, e1a5b5b6e3255540 +advertising.com, 28605, RESELLER # RhythmOne +advertising.com, 7372, RESELLER +advertising.com, 8603, RESELLER # Taboola +amxrtb.com, 105199384, DIRECT +aol.com, 11119, DIRECT +aol.com, 53392, RESELLER # Taboola +aolcloud.net, 9904, RESELLER +appnexus.com, 1001, RESELLER, f5ab79cb980f11d1 +appnexus.com, 10239, RESELLER, f5ab79cb980f11d1 +appnexus.com, 10239, RESELLER, f5ab79cb980f11d1 # 33 Across +appnexus.com, 11786, RESELLER, f5ab79cb980f11d1 +appnexus.com, 11801, RESELLER +appnexus.com, 11924, RESELLER, f5ab79cb980f11d1 +appnexus.com, 12061, RESELLER +appnexus.com, 12263, RESELLER +appnexus.com, 12290, RESELLER, f5ab79cb980f11d1 +appnexus.com, 12366, RESELLER, f5ab79cb980f11d1 +appnexus.com, 1314, RESELLER +Appnexus.com, 1356, RESELLER, f5ab79cb980f11d1 +appnexus.com, 1360, RESELLER, f5ab79cb980f11d1 +appnexus.com, 1613, reseller +appnexus.com, 1908, RESELLER +appnexus.com, 1908, RESELLER, f5ab79cb980f11d1 +appnexus.com, 1908, RESELLER, f5ab79cb980f11d1 # DistrictM +appnexus.com, 1942, RESELLER, f5ab79cb980f11d1 +appnexus.com, 2530, RESELLER +appnexus.com, 2758, RESELLER, f5ab79cb980f11d1 +appnexus.com, 3135, RESELLER, f5ab79cb980f11d1 +appnexus.com, 3153, DIRECT +appnexus.com, 3153, RESELLER +appnexus.com, 3153, RESELLER, f5ab79cb980f11d1 +appnexus.com, 3326, reseller +appnexus.com, 4052, RESELLER +appnexus.com, 4052, RESELLER, f5ab79cb980f11d1 # Conversant +appnexus.com, 6849, RESELLER, f5ab79cb980f11d1 # RhythmOne +appnexus.com, 7118, RESELLER +appnexus.com, 7556, DIRECT, f5ab79cb980f11d1 +appnexus.com, 7597, RESELLER, f5ab79cb980f11d1 +appnexus.com, 7911, RESELLER #yieldmo +appnexus.com, 7944, RESELLER +appnexus.com, 8692, DIRECT, f5ab79cb980f11d1 +appnexus.com, 9316, RESELLER, f5ab79cb980f11d1 # AppNexus +appnexus.com, 9393, DIRECT +appnexus.com, 9393, RESELLER #Video #Display, f5ab79cb980f11d1 +appnexus.com,1001,reseller,f5ab79cb980f11d1 +appnexus.com,10239,reseller,f5ab79cb980f11d1 +appnexus.com,1356,reseller,f5ab79cb980f11d1 +appnexus.com,1908,RESELLER,f5ab79cb980f11d1 +appnexus.com,2758,reseller,f5ab79cb980f11d1 +appnexus.com,3663,RESELLER,f5ab79cb980f11d1 +appnexus.com,7597,reseller,f5ab79cb980f11d1 +appnexus.com,9316,reseller,f5ab79cb980f11d1 +aps.amazon.com,094e2c86-72d9-47d6-a647-d95ce39ad4c7,DIRECT +aps.amazon.com,2840f06c-5d89-4853-a03e-3bfa567dd33c,reseller +aps.amazon.com,48266a61-b3d9-4cb7-b172-553abc6a42a4,DIRECT +aralego.com, par-488A3E6BD8D997D0ED8B3BD34D8BA4B, RESELLER # ucFunnel +audienciad.com, 202922, DIRECT +bidtellect.com, 1407, RESELLER, 1c34aa2d85d45e93 +bidtellect.com,1407,reseller,1c34aa2d85d45e93 +brightcom.com, 20292, DIRECT +ccoxmt.com, 2000067997702, RESELLER +consumable.com, 2000908, DIRECT, aefcd3d2f45b5070 +contextweb.com, 558355, RESELLER +contextweb.com, 558511, RESELLER +contextweb.com, 560606, RESELLER +contextweb.com, 560606, RESELLER, 89ff185a4c4e857 +contextweb.com, 560606, RESELLER, 89ff185a4c4e857c +contextweb.com, 561118, RESELLER, 89ff185a4c4e857c #yieldmo +contextweb.com, 561998, RESELLER, 89ff185a4c4e857c +contextweb.com, 562350, RESELLER, 89ff185a4c4e857c +contextweb.com,558355,reseller +conversantmedia.com, 20923, RESELLER # Conversant +conversantmedia.com, 39882, DIRECT, 03113cd04947736d +conversantmedia.com, 40790, RESELLER, 03113cd04947736d +conversantmedia.com, 41812, DIRECT +coxmt.com, 2000067907202, RESELLER +criteo.com, 109412, DIRECT, 9fac4a4a87c2a44f +districtm.io, 100808, DIRECT +districtm.io, 100835, DIRECT +districtm.io, 100835, DIRECT, 3fd707be9c4527c3 +districtm.io, 101080, RESELLER +districtm.io, 101769, RESELLER, 3fd707be9c4527c3 # DistrictM +districtm.io,100962,RESELLER,3fd707be9c4527c3 +EMXDGT.com, 1133, DIRECT, 1e1d41537f7cad7f +emxdgt.com, 20, DIRECT, 1e1d41537f7cad7f +emxdgt.com, 326, RESELLER, 1e1d41537f7cad7f +emxdgt.com, 326, RESELLER, 1e1d41537f7cad7f # 33 Across +EMXDGT.com,1284,reseller,1e1d41537f7cad7f +engagebdr.com, 10417, RESELLER # EngageDBR +freewheel.tv, 146081, reseller +freewheel.tv, 19129, RESELLER +freewheel.tv, 19133, RESELLER +freewheel.tv, 33081, RESELLER +freewheel.tv, 33601, RESELLER +freewheel.tv, 799841, RESELLER # Taboola +freewheel.tv, 799921, RESELLER # Taboola +google.com, pub-1320774679920841, RESELLER, f08c47fec0942fa0 +google.com, pub-1409765517756851, reseller +google.com, pub-2290755540215120, RESELLER, f08c47fec0942fa0 +google.com, pub-2802445174821308, RESELLER, f08c47fec0942fa0 +google.com, pub-3848273848634341, RESELLER, f08c47fec0942fa0 +google.com, pub-4075894099602271, reseller +google.com, pub-4207323757133151, RESELLER, f08c47fec0942fa0 +google.com, pub-4641608711979091, DIRECT, f08c47fec0942fa0 +google.com, pub-5231479214411897, RESELLER, f08c47fec0942fa0 +Google.com, pub-5995202563537249, RESELLER, f08c47fec0942fa0 +google.com, pub-6314168058065736, RESELLER, f08c47fec0942fa0 +google.com, pub-8172268348509349, RESELLER, f08c47fec0942fa0 +google.com, pub-9557089510405422, RESELLER, f08c47fec0942fa0 +google.com, pub-9557089510405422, RESELLER, f08c47fec0942fa0 # 33 Across +google.com, pub-9685734445476814, RESELLER, f08c47fec0942fa0 +google.com, pub-9685734445476814, RESELLER, f08c47fec0942fa0 # DistrictM +google.com,pub-3848273848634341,reseller,f08c47fec0942fa0 +google.com,pub-9557089510405422,reseller,f08c47fec0942fa0 +gumgum.com, 11645, RESELLER, ffdef49475d318a9 +gumgum.com, 13174, DIRECT, ffdef49475d318a9 +gumgum.com, 13318, RESELLER, ffdef49475d318a9 +gumgum.com, 13318, RESELLER, ffdef49475d318a9 # 33 Across +gumgum.com, 13504, RESELLER, ffdef49475d318a9 +gumgum.com,13174,DIRECT,ffdef49475d318a9 +gumgum.com,14141,RESELLER,ffdef49475d318a9 +improvedigital.com, 1362, RESELLER +improvedigital.com, 1669, RESELLER # ImproveDigital +improvedigital.com, 185, RESELLER +indexexchange.com, 175407, RESELLER +indexexchange.com, 177754, RESELLER, 50b1c356f2c5c8fc +indexexchange.com, 182257, RESELLER, 50b1c356f2c5c8fc # RhythmOne +indexexchange.com, 183965, RESELLER, 50b1c356f2c5c8fc # AOL - One +indexexchange.com, 184914, DIRECT, 50b1c356f2c5c8fc +indexexchange.com, 186046, RESELLER +indexexchange.com, 186248, DIRECT, 50b1c356f2c5c8fc +indexexchange.com, 187196, DIRECT +indexexchange.com, 187454, DIRECT, 50b1c356f2c5c8fc +indexexchange.com, 189744, RESELLER +indexexchange.com, 189872, RESELLER +indexexchange.com, 191503, RESELLER, 50b1c356f2c5c8fc +indexexchange.com, 191740, RESELLER, 50b1c356f2c5c8fc # Index +indexexchange.com, 191923, RESELLER +indexexchange.com, 191973, RESELLER, 50b1c356f2c5c8fc +indexexchange.com, 193351, DIRECT +lijit.com, 217352, DIRECT, fafdf38b16bf6b2b +lijit.com, 217352-eb, DIRECT, fafdf38b16bf6b2b +lijit.com, 248396, DIRECT, fafdf38b16bf6b2b +lijit.com, 248396-eb, DIRECT, fafdf38b16bf6b2b +lijit.com, 260380, RESELLER, fafdf38b16bf6b2b +lijit.com, 270524, RESELLER, fafdf38b16bf6b2b +lijit.com,217352,DIRECT,fafdf38b16bf6b2b +lkqd.com, 470, RESELLER, 59c49fa9598a0117 +lkqd.net, 470, RESELLER, 59c49fa9598a0117 +Newormedia.com, 2169, DIRECT +newormedia.com, 4908, DIRECT +onetag.com, 572a470226457b8, RESELLER # OneTag +onetag.com, 664e107d9f2b748, RESELLER #yieldmo +onomagic.com, 202921, DIRECT +openx.com, 539824308, RESELLER, 6a698e2ec38604c6 +openx.com, 537120563, RESELLER, 6a698e2ec38604c6 +openx.com, 537120563, RESELLER, 6a698e2ec38604c6 # 33 Across +openx.com, 537120960, RESELLER +openx.com, 537127577, RESELLER, 6a698e2ec38604c6 +Openx.com, 537143344, RESELLER +openx.com, 537149485, RESELLER, 6a698e2ec38604c6 +openx.com, 537150004, DIRECT, 6a698e2ec38604c6 +openx.com, 537153209, RESELLER, 6a698e2ec38604c6 +openx.com, 538959099, RESELLER +openx.com, 538959099, RESELLER, 6a698e2ec38604c6 +openx.com, 539699341, DIRECT, 6a698e2ec38604c6 +openx.com, 539824308, RESELLER, 6a698e2ec38604c6 +openx.com, 540003333, RESELLER, 6a698e2ec38604c6 +openx.com, 540031703, RESELLER, 6a698e2ec38604c6 +openx.com, 540031703, RESELLER, 6a698e2ec38604c6 # Conversant +openx.com, 540258065, RESELLER, 6a698e2ec38604c6 +openx.com, 540274407, RESELLER, 6a698e2ec38604c6 +openx.com, 540337213, RESELLER, 6a698e2ec38604c6 +openx.com, 540401713, RESELLER, 6a698e2ec38604c6 # OpenX +openx.com, 541159484, RESELLER, 6a698e2ec38604c6 +openx.com, 542511596, RESELLER, 6a698e2ec38604c6 +openx.com, 83499, RESELLER +openx.com,537149485,reseller,6a698e2ec38604c6 +openx.com,540191398,RESELLER,6a698e2ec38604c6 +openx.com,540833447, RESELLER, 6a698e2ec38604c6 +outbrain.com, 01a755b08c8c22b15d46a8b753ab6955d4, DIRECT +outbrain.com, 01a755b08c8c22b15d46a8b753ab6955d4, RESELLER +outbrain.com,00254374f0c468f3b2732db17fd42cb6e5,reseller +pubmatic.com, 137711, RESELLER +pubmatic.com, 137711, RESELLER, 5d62403b186f2ace +pubmatic.com, 156084, RESELLER, 5d62403b186f2ace # AOL - One +pubmatic.com, 156212, RESELLER +pubmatic.com, 156212, RESELLER, 5d62403b186f2ace +pubmatic.com, 156307, RESELLER, 5d62403b186f2ace # Taboola +pubmatic.com, 156319, DIRECT, 5d62403b186f2ace +pubmatic.com, 156325, RESELLER, 5d62403b186f2ace # AOL - One +pubmatic.com, 156344, RESELLER, 5d62403b186f2ace # Pubmatic +pubmatic.com, 156423, RESELLER, 5d62403b186f2ace +pubmatic.com, 156423, RESELLER, 5d62403b186f2ace # 33 Across +pubmatic.com, 156458, RESELLER, 5d62403b186f2ace # AOL - One +pubmatic.com, 156557, RESELLER +pubmatic.com, 156595, RESELLER, 5d62403b186f2ace +pubmatic.com, 157367, DIRECT, 5d62403b186f2ace +pubmatic.com, 158100, RESELLER, 5d62403b186f2ace +pubmatic.com, 158355 , RESELLER, 5d62403b186f2ace +pubmatic.com, 158723, RESELLER, 5d62403b186f2ace +pubmatic.com, 159117, DIRECT, 5d62403b186f2ace +pubmatic.com, 159277, RESELLER, 5d62403b186f2ace # RhythmOne +pubmatic.com, 159330, RESELLER, 5d62403b186f2ace +pubmatic.com, 159477,RESELLER,5d62403b186f2ace +pubmatic.com, 160082 , RESELLER, 5d62403b186f2ace +pubmatic.com, 160131, RESELLER, 5d62403b186f2ace +pubmatic.com, 32987, RESELLER, 5d62403b186f2ace +pubmatic.com, 50758, RESELLER, 5d62403b186f2ace +pubmatic.com, 62483, RESELLER +pubmatic.com, 79136 , RESELLER, 5d62403b186f2ace +pubmatic.com,157150,RESELLER,5d62403b186f2ace +pubmatic.com,157897,reseller,5d62403b186f2ace +pubmatic.com,160006,RESELLER,5d62403b186f2ace +pubmatic.com,160096,RESELLER,5d62403b186f2ace +pubnative.net, 1007284, RESELLER, d641df8625486a7b #yieldmodisplay +pubnative.net, 1007285, RESELLER, d641df8625486a7b #yieldmonative +pubnative.net, 1007286, RESELLER, d641df8625486a7b #yieldmovideo +pubnx.com, 337-1, RESELLER, 8728b7e97e589da4 # Vertoz +revcontent.com, 76611, RESELLER +rhythmone.com, 1059622079, RESELLER +rhythmone.com, 1059622079, RESELLER, a670c89d4a324e47 +rhythmone.com, 1114124056, RESELLER, a670c89d4a324e47 +rhythmone.com, 1166984029, RESELLER, a670c89d4a324e47 # Taboola +rhythmone.com, 2241341073, RESELLER, a670c89d4a324e47 +rhythmone.com, 2310154583, DIRECT, a670c89d4a324e47 +rhythmone.com, 2439829435, RESELLER, a670c89d4a324e47 +rhythmone.com, 2439829435, RESELLER, a670c89d4a324e47 # 33 Across +rhythmone.com, 78519861, RESELLER +rhythmone.com, 905992537, RESELLER, a670c89d4a324e47 # RhythmOne +rhythmone.com,1654642120,RESELLER,a670c89d4a324e47 +rhythmone.com,2310154583,DIRECT,a670c89d4a324e47 +rhythmone.com,78519861,reseller,a670c89d4a324e47 +rtk.io, 819, DIRECT +rubiconproject.com, 13344, RESELLER, 0bfd66d529a55807 # Rubicon +rubiconproject.com, 15268, RESELLER, 0bfd66d529a55807 # RhythmOne +rubiconproject.com, 16414, RESELLER, 0bfd66d529a55807 +rubiconproject.com, 16414, RESELLER, 0bfd66d529a55807 # 33 Across +rubiconproject.com, 17070, RESELLER, 0bfd66d529a55807 #yieldmo +rubiconproject.com, 17632, DIRECT, 0bfd66d529a55807 +rubiconproject.com, 17790, RESELLER, 0bfd66d529a55807 +rubiconproject.com, 17792, RESELLER, 0bfd66d529a55807 +rubiconproject.com, 17822, DIRECT, 0bfd66d529a55807 +rubiconproject.com, 17822, RESELLER, 0bfd66d529a55807 +rubiconproject.com, 17960, RESELLER, 0bfd66d529a55807 +rubiconproject.com, 18222, RESELLER, 0bfd66d529a55807 # AOL - One +rubiconproject.com, 18694, RESELLER, 0bfd66d529a55807 +rubiconproject.com, 18890, DIRECT, 0bfd66d529a55807 +rubiconproject.com, 20130, RESELLER, 0bfd66d529a55807 +rubiconproject.com, 20416, RESELLER, 0bfd66d529a55807 +rubiconproject.com, 21310, RESELLER , 0bfd66d529a55807 +rubiconproject.com, 21642, RESELLER, 0bfd66d529a55807 +rubiconproject.com, 8861, reseller, 0bfd66d529a55807 +rubiconproject.com,18020,RESELLER,0bfd66d529a55807 +Sekindo.com, 20749, DIRECT, b6b21d256ef43532 +sharethrough.com, 3a0f657b, DIRECT, d53b998a7bd4ecd2 +sharethrough.com, d09156e5, RESELLER, d53b998a7bd4ecd2 +smaato.com, 1100033117, RESELLER +smaato.com, 1100047713, RESELLER, 07bcf65f187117b4 +smaato.com,1100044650,RESELLER,07bcf65f187117b4 +smartadserver.com, 3436, RESELLER +sonobi.com, 337f0e70cc, DIRECT +sonobi.com, 37dd19ad4a, RESELLER, d1a215d9eb5aee9e +sonobi.com, 6e5cfb5420, DIRECT, d1a215d9eb5aee9e +sonobi.com, e55fb5d7c2, DIRECT, d1a215d9eb5aee9e +sovrn.com, 217352, DIRECT, fafdf38b16bf6b2b +sovrn.com, 248396, DIRECT, fafdf38b16bf6b2b +sovrn.com, 260380, RESELLER, fafdf38b16bf6b2b +sovrn.com, 270524, RESELLER, fafdf38b16bf6b2b +sovrn.com,217352,DIRECT,fafdf38b16bf6b2b +sparcmedia.com, 310627, Direct +spotx.tv, 108933, RESELLER, 7842df1d2fe2db34 +spotx.tv, 147949, RESELLER, 7842df1d2fe2db34 +spotx.tv, 212457, RESELLER +spotx.tv, 228454, RESELLER, 7842df1d2fe2db34 +spotx.tv, 270977, DIRECT, 7842df1d2fe2db34 +spotx.tv, 285547, RESELLER, 7842df1d2fe2db34 # RhythmOne +spotx.tv, 71451, RESELLER, 7842df1d2fe2db34 # Taboola +spotx.tv, 84294, RESELLER, 7842df1d2fe2db34 +spotx.tv, 94794, RESELLER, 7842df1d2fe2db34 # SpotX +spotxchange.com, 108933, RESELLER, 7842df1d2fe2db34 +spotxchange.com, 147949, RESELLER, 7842df1d2fe2db34 +spotxchange.com, 212457, RESELLER +spotxchange.com, 228454, RESELLER, 7842df1d2fe2db34 +spotxchange.com, 270977, DIRECT, 7842df1d2fe2db34 +spotxchange.com, 285547, RESELLER, 7842df1d2fe2db34 # RhythmOne +spotxchange.com, 71451, RESELLER, 7842df1d2fe2db34 # Taboola +spotxchange.com, 84294, RESELLER, 7842df1d2fe2db34 +spotxchange.com, 94794, RESELLER, 7842df1d2fe2db34 # SpotX +springserve.com, 686, DIRECT, a24eb641fc82e93d +synacor.com, 82350, RESELLER, e108f11b2cdf7d5b +synacor.com,82151,reseller,e108f11b2cdf7d5b +teads.tv, 19014, DIRECT, 15a9c44f6d26cbe1 +telaria.com, mb9eo-oqsbf, RESELLER, 1a4e959a1b50034a +telaria.com, vtrdn-wjdav, DIRECT, 1a4e959a1b50034a +telaria.com, vtrdn-ysjam, DIRECT, 1a4e959a1b50034a +themediagrid.com, P5JONV, RESELLER, 35d5010d7789b49d # Media Grid (IPONWEB) +tremorhub.com, mb9eo-oqsbf, RESELLER, 1a4e959a1b50034a +tremorhub.com, vtrdn-wjdav, DIRECT, 1a4e959a1b50034a +tremorhub.com, vtrdn-ysjam, DIRECT, 1a4e959a1b50034a +tremorhub.com, z87wm, RESELLER, 1a4e959a1b50034a # Taboola +triplelift.com, 7205, DIRECT, 6c33edb13117fd86 +ucfunnel.com, par-488A3E6BD8D997D0ED8B3BD34D8BA4B, RESELLER # ucFunnel +vertamedia.com, 287605, DIRECT, 7de89dc7742b5b11 +vertamedia.com, 287605, RESELLER, 7de89dc7742b5b11 +video.unrulymedia.com, 2310154583, DIRECT +video.unrulymedia.com, 905992537, RESELLER, a670c89d4a324e47 # RhythmOne +yahoo.com, 55771, RESELLER, e1a5b5b6e3255540 +yahoo.com, 55104, DIRECT, e1a5b5b6e3255540 +yahoo.com, 55317, RESELLER # Verizon +yahoo.com, 57289, RESELLER, e1a5b5b6e3255540 +yahoo.com, 57695, DIRECT, e1a5b5b6e3255540 +yahoo.com,55029,RESELLER,e1a5b5b6e3255540 +yieldmo.com, 2417496099628458357, DIRECT diff --git a/docs/_extra/robots.txt b/docs/_extra/robots.txt new file mode 100644 index 000000000..eb0536286 --- /dev/null +++ b/docs/_extra/robots.txt @@ -0,0 +1,2 @@ +User-agent: * +Disallow: diff --git a/docs/_templates/layout.html b/docs/_templates/layout.html index 7f72a6be4..6a6a22b8a 100644 --- a/docs/_templates/layout.html +++ b/docs/_templates/layout.html @@ -40,6 +40,8 @@ + + {# Alabaster theme native GA integration is outdated (ga.js). #} {# Insert our own GA snippet instead. #} +

This opinionated guide exists to provide both novice and expert Python developers a best practice handbook to the installation, configuration, and usage of Python on a daily basis.

@@ -34,8 +56,8 @@

Contributors

Useful Links

@@ -44,8 +66,8 @@

Translations

  • English
  • French
  • Chinese
    • -
  • Japanese
    • +
  • Japanese
  • Korean
    • -
  • Filipino
    • -
  • Brazilian Portuguese
    • +
  • Filipino
    • +
  • Brazilian Portuguese
    • diff --git a/docs/_templates/sidebarlogo.html b/docs/_templates/sidebarlogo.html index 363663f1b..540b92a48 100644 --- a/docs/_templates/sidebarlogo.html +++ b/docs/_templates/sidebarlogo.html @@ -6,6 +6,28 @@

    + + + + +

    This opinionated guide exists to provide both novice and expert Python developers a best practice handbook to the installation, configuration, and usage of Python on a daily basis.

    diff --git a/docs/conf.py b/docs/conf.py index 81fcc6f91..d732ac29f 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -11,7 +11,9 @@ # All configuration values have a default; values that are commented out # serve to show the default. -import sys, os +import datetime +import os +import sys # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the @@ -45,8 +47,11 @@ master_doc = 'index' # General information about the project. +current_year = datetime.datetime.now().year project = u'pythonguide' -copyright = u'2011–2018 Kenneth Reitz & Real Python. CC BY-NC-SA 3.0' +copyright = (u'2011-{} Kenneth Reitz' + ' & Real Python.' + ' CC BY-NC-SA 3.0').format(current_year) # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -241,7 +246,7 @@ epub_title = u'pythonguide' epub_author = u'Kenneth Reitz' epub_publisher = u'Kenneth Reitz' -epub_copyright = u'2011–2018, Kenneth Reitz & Real Python' +epub_copyright = u'2011–{}, Kenneth Reitz & Real Python'.format(current_year) # The language of the text. It defaults to the language option # or en if the language is not set. @@ -279,5 +284,5 @@ todo_include_todos = True intersphinx_mapping = { - 'python': ('http://docs.python.org/', None), + 'python': ('https://docs.python.org/3', None), } diff --git a/docs/contents.rst.inc b/docs/contents.rst.inc index 76049aab6..29b076197 100644 --- a/docs/contents.rst.inc +++ b/docs/contents.rst.inc @@ -33,7 +33,7 @@ New to Python? Let's properly setup up your Python environment: Python Development Environments ------------------------------- -This part of the guide focus on the Python development environment, +This part of the guide focuses on the Python development environment, and the best-practice tools that are available for writing Python code. .. toctree:: @@ -97,11 +97,12 @@ different scenarios. Shipping Great Python Code -------------------------- -This part of the guide focuses on deploying your Python code. +This part of the guide focuses on sharing and deploying your Python code. .. toctree:: :maxdepth: 2 + shipping/publishing shipping/packaging shipping/freezing @@ -110,7 +111,7 @@ Additional Notes ---------------- This part of the guide, which is mostly prose, begins with some -background information about Python, then focuses on next steps. +background information about Python, and then focuses on next steps. .. toctree:: :maxdepth: 2 @@ -137,8 +138,3 @@ Contribution notes and legal information (for those interested). notes/contribute notes/license notes/styleguide - - - - - diff --git a/docs/dev/env.rst b/docs/dev/env.rst index de67a9ae9..7da63c0e6 100644 --- a/docs/dev/env.rst +++ b/docs/dev/env.rst @@ -7,7 +7,7 @@ Your Development Environment Text Editors :::::::::::: -Just about anything that can edit plain text will work for writing Python code, +Just about anything that can edit plain text will work for writing Python code; however, using a more powerful editor may make your life a bit easier. @@ -87,29 +87,29 @@ using ```` key or any other customized keys. .. _indent: http://www.vim.org/scripts/script.php?script_id=974 .. _syntax: http://www.vim.org/scripts/script.php?script_id=790 -.. _Pyflakes: http://pypi.python.org/pypi/pyflakes/ -.. _pycodestyle: https://pypi.python.org/pypi/pycodestyle/ -.. _syntastic: https://github.com/scrooloose/syntastic -.. _Python-mode: https://github.com/klen/python-mode +.. _Pyflakes: http://pypi.org/project/pyflakes/ +.. _pycodestyle: https://pypi.org/project/pycodestyle/ +.. _syntastic: https://github.com/vim-syntastic/syntastic +.. _Python-mode: https://github.com/python-mode/python-mode .. _SuperTab: http://www.vim.org/scripts/script.php?script_id=1643 .. _vim-flake8: https://github.com/nvie/vim-flake8 Emacs ----- -Emacs is another powerful text editor. It is fully programmable (lisp), but +Emacs is another powerful text editor. It is fully programmable (Lisp), but it can be some work to wire up correctly. A good start if you're already an Emacs user is `Python Programming in Emacs`_ at EmacsWiki. 1. Emacs itself comes with a Python mode. -.. _Python Programming in Emacs: http://emacswiki.org/emacs/PythonProgrammingInEmacs +.. _Python Programming in Emacs: https://www.emacswiki.org/emacs/PythonProgrammingInEmacs TextMate -------- `TextMate `_ brings Apple's approach to operating - systems into the world of text editors. By bridging UNIX underpinnings and + systems into the world of text editors. By bridging Unix underpinnings and GUI, TextMate cherry-picks the best of both worlds to the benefit of expert scripters and novice users alike. @@ -117,8 +117,8 @@ Sublime Text ------------ `Sublime Text `_ is a sophisticated text - editor for code, markup and prose. You'll love the slick user interface, - extraordinary features and amazing performance. + editor for code, markup, and prose. You'll love the slick user interface, + extraordinary features, and amazing performance. Sublime Text has excellent support for editing Python code and uses Python for its plugin API. It also has a diverse variety of plugins, @@ -133,11 +133,19 @@ Atom editors. Atom is web native (HTML, CSS, JS), focusing on modular design and easy plugin -development. It comes with native package control and plethora of packages. +development. It comes with native package control and a plethora of packages. Recommended for Python development is -`Linter `_ combined with +`Linter `_ combined with `linter-flake8 `_. +Python (on Visual Studio Code) +------------------------------ + +`Python for Visual Studio `_ is an extension for the `Visual Studio Code `_. +This is a free, lightweight, open source code editor, with support for Mac, Windows, and Linux. +Built using open source technologies such as Node.js and Python, with compelling features such as Intellisense (autocompletion), local and remote debugging, linting, and the like. + +MIT licensed. IDEs :::: @@ -152,18 +160,10 @@ features can be brought to IntelliJ with the free versions of PyCharm: Professional Edition (Free 30-day trial) and Community Edition (Apache 2.0 License) with fewer features. -Python (on Visual Studio Code) ------------------------------- - -`Python for Visual Studio `_ is an extension for the `Visual Studio Code IDE `_. -This is a free, light weight, open source IDE, with support for Mac, Windows, and Linux. -Built using open source technologies such as Node.js and Python, with compelling features such as Intellisense (autocompletion), local and remote debugging, linting, and the like. - -MIT licensed. Enthought Canopy ---------------- -`Enthought Canopy `_ is a Python +`Enthought Canopy `_ is a Python IDE which is focused towards Scientists and Engineers as it provides pre installed libraries for data analysis. @@ -171,13 +171,13 @@ Eclipse ------- The most popular Eclipse plugin for Python development is Aptana's -`PyDev `_. +`PyDev `_. Komodo IDE ---------- -`Komodo IDE `_ is developed by +`Komodo IDE `_ is developed by ActiveState and is a commercial IDE for Windows, Mac, and Linux. `KomodoEdit `_ is the open source alternative. @@ -188,11 +188,11 @@ Spyder `Spyder `_ is an IDE specifically geared toward working with scientific Python libraries (namely -`Scipy `_). It includes integration with pyflakes_, -`pylint `_ and +`SciPy `_). It includes integration with pyflakes_, +`pylint `_ and `rope `_. -Spyder is open-source (free), offers code completion, syntax highlighting, +Spyder is open source (free), offers code completion, syntax highlighting, a class and function browser, and object inspection. @@ -200,7 +200,7 @@ WingIDE ------- `WingIDE `_ is a Python specific IDE. It runs on Linux, -Windows and Mac (as an X11 application, which frustrates some Mac users). +Windows, and Mac (as an X11 application, which frustrates some Mac users). WingIDE offers code completion, syntax highlighting, source browser, graphical debugger and support for version control systems. @@ -211,11 +211,11 @@ NINJA-IDE `NINJA-IDE `_ (from the recursive acronym: "Ninja-IDE Is Not Just Another IDE") is a cross-platform IDE, specially designed to build -Python applications, and runs on Linux/X11, Mac OS X and Windows desktop +Python applications, and runs on Linux/X11, Mac OS X, and Windows desktop operating systems. Installers for these platforms can be downloaded from the website. -NINJA-IDE is open-source software (GPLv3 licence) and is developed +NINJA-IDE is open source software (GPLv3 licence) and is developed in Python and Qt. The source files can be downloaded from `GitHub `_. @@ -224,13 +224,25 @@ Eric (The Eric Python IDE) -------------------------- `Eric `_ is a full featured Python IDE -offering sourcecode autocompletion, syntax highlighting, support for version -control systems, python 3 support, integrated web browser, python shell, -integrated debugger and a flexible plug-in system. Written in python, it is -based on the Qt gui toolkit, integrating the Scintilla editor control. Eric -is an open-source software project (GPLv3 licence) with more than ten years of +offering source code autocompletion, syntax highlighting, support for version +control systems, Python 3 support, integrated web browser, python shell, +integrated debugger, and a flexible plug-in system. Written in Python, it is +based on the Qt GUI toolkit, integrating the Scintilla editor control. Eric +is an open source software project (GPLv3 licence) with more than ten years of active development. +Mu +-- + +`Mu `_ is a minimalist Python IDE which can run Python 3 code +locally and can also deploy code to the BBC micro:bit and to Adafruit boards running +CircuitPython. + +Intended for beginners, mu includes a Python 3 interpreter, and is easy to install +on Windows, OS/X and Linux. It runs well on the Raspberry Pi. + +There's an active support community on gitter. + Interpreter Tools ::::::::::::::::: @@ -242,19 +254,19 @@ Virtual Environments Virtual Environments provide a powerful way to isolate project package dependencies. This means that you can use packages particular to a Python project without installing them system wide and thus avoiding potential version conflicts. To start using and see more information: -`Virtual Environments `_ docs. +`Virtual Environments `_ docs. pyenv ----- -`pyenv `_ is a tool to allow multiple versions +`pyenv `_ is a tool to allow multiple versions of the Python interpreter to be installed at the same time. This solves the problem of having different projects requiring different versions of Python. For example, it becomes very easy to install Python 2.7 for compatibility in -one project, whilst still using Python 3.4 as the default interpreter. -pyenv isn't just limited to the CPython versions - it will also install PyPy, -anaconda, miniconda, stackless, jython, and ironpython interpreters. +one project, while still using Python 3.4 as the default interpreter. +pyenv isn't just limited to the CPython versions – it will also install PyPy, +Anaconda, miniconda, stackless, Jython, and IronPython interpreters. pyenv works by filling a ``shims`` directory with fake versions of the Python interpreter (plus other tools like ``pip`` and ``2to3``). When the system @@ -264,7 +276,7 @@ pyenv. pyenv then works out which version of Python should be run based on environment variables, ``.python-version`` files, and the global default. pyenv isn't a tool for managing virtual environments, but there is the plugin -`pyenv-virtualenv `_ which automates +`pyenv-virtualenv `_ which automates the creation of different environments, and also makes it possible to use the existing pyenv tools to switch to different environments based on environment variables or ``.python-version`` files. @@ -276,7 +288,7 @@ IDLE ---- :ref:`IDLE ` is an integrated development environment that is -part of Python standard library. It is completely written in Python and uses +part of the Python standard distribution. It is completely written in Python and uses the Tkinter GUI toolkit. Though IDLE is not suited for full-blown development using Python, it is quite helpful to try out small Python snippets and experiment with different features in Python. @@ -294,18 +306,18 @@ IPython `IPython `_ provides a rich toolkit to help you make the most out of using Python interactively. Its main components are: -* Powerful Python shells (terminal- and Qt-based). +* Powerful Python shells (terminal- and Qt-based) * A web-based notebook with the same core features but support for rich media, - text, code, mathematical expressions and inline plots. -* Support for interactive data visualization and use of GUI toolkits. -* Flexible, embeddable interpreters to load into your own projects. -* Tools for high level and interactive parallel computing. + text, code, mathematical expressions and inline plots +* Support for interactive data visualization and use of GUI toolkits +* Flexible, embeddable interpreters to load into your own projects +* Tools for high level and interactive parallel computing .. code-block:: console $ pip install ipython -To download and install IPython with all it's optional dependencies for the notebook, qtconsole, tests, and other functionalities +To download and install IPython with all its optional dependencies for the notebook, qtconsole, tests, and other functionalities: .. code-block:: console @@ -314,18 +326,18 @@ To download and install IPython with all it's optional dependencies for the note BPython ------- -`bpython `_ is an alternative interface to the +`bpython `_ is an alternative interface to the Python interpreter for Unix-like operating systems. It has the following features: -* In-line syntax highlighting. -* Readline-like autocomplete with suggestions displayed as you type. -* Expected parameter list for any Python function. -* "Rewind" function to pop the last line of code from memory and re-evaluate. -* Send entered code off to a pastebin. -* Save entered code to a file. -* Auto-indentation. -* Python 3 support. +* In-line syntax highlighting +* Readline-like autocomplete with suggestions displayed as you type +* Expected parameter list for any Python function +* "Rewind" function to pop the last line of code from memory and re-evaluate +* Send entered code off to a pastebin +* Save entered code to a file +* Auto-indentation +* Python 3 support .. code-block:: console @@ -334,19 +346,19 @@ features: ptpython -------- -`ptpython `_ is a REPL build -on top of the `prompt_toolkit `_ +`ptpython `_ is a REPL build +on top of the `prompt_toolkit `_ library. It is considered to be an alternative to BPython_. Features include: * Syntax highlighting * Autocompletion * Multiline editing -* Emacs and VIM Mode +* Emacs and Vim Modes * Embedding REPL inside of your code -* Syntax Validation +* Syntax validation * Tab pages * Support for integrating with IPython_'s shell, by installing IPython - ``pip install ipython`` and running ``ptipython``. + (``pip install ipython``) and running ``ptipython``. .. code-block:: console diff --git a/docs/dev/pip-virtualenv.rst b/docs/dev/pip-virtualenv.rst index 6971be876..667f30c7a 100644 --- a/docs/dev/pip-virtualenv.rst +++ b/docs/dev/pip-virtualenv.rst @@ -1,6 +1,6 @@ .. _pip-virtualenv: -Further Configuration of Pip and Virtualenv +Further Configuration of pip and Virtualenv =========================================== .. image:: /_static/photos/34018732105_f0e6758859_k_d.jpg @@ -51,7 +51,7 @@ can be found at: .. code-block:: console - %HOME%\pip\pip.ini + %USERPROFILE%\pip\pip.ini If you don't have a :file:`pip.conf` or :file:`pip.ini` file at these locations, you can create a new file with the correct name for your operating system. @@ -79,7 +79,7 @@ adding the following to your :file:`~/.bashrc` file: .. code-block:: console gpip() { - PIP_REQUIRE_VIRTUALENV="" pip "$@" + PIP_REQUIRE_VIRTUALENV=false pip "$@" } After saving the changes and sourcing your :file:`~/.bashrc` file you can now @@ -105,7 +105,7 @@ need any configuration. When using older versions, you can configure pip in such a way that it tries to reuse already installed packages, too. -On UNIX systems, you can add the following line to your :file:`.bashrc` or +On Unix systems, you can add the following line to your :file:`.bashrc` or :file:`.bash_profile` file. .. code-block:: console @@ -122,9 +122,9 @@ add the following line to your :file:`pip.ini` file under ``[global]`` settings: .. code-block:: console - download-cache = %HOME%\pip\cache + download-cache = %USERPROFILE%\pip\cache -Similarly, on UNIX systems you should simply add the following line to your +Similarly, on Unix systems you should simply add the following line to your :file:`pip.conf` file under ``[global]`` settings: .. code-block:: console diff --git a/docs/dev/virtualenvs.rst b/docs/dev/virtualenvs.rst index 11f924882..fe6bd3f9e 100644 --- a/docs/dev/virtualenvs.rst +++ b/docs/dev/virtualenvs.rst @@ -25,7 +25,7 @@ Make sure you've got Python & pip Before you go any further, make sure you have Python and that it's available from your command line. You can check this by simply running: -.. code-block:: bash +.. code-block:: console $ python --version @@ -50,7 +50,7 @@ install the latest 3.x version from `python.org`_ or refer to the Additionally, you'll need to make sure you have `pip`_ available. You can check this by running: -.. code-block:: bash +.. code-block:: console $ pip --version @@ -101,7 +101,7 @@ Use ``pip`` to install Pipenv: user ``PATH`` permanently in the `Control Panel`_. You may need to log out for the ``PATH`` changes to take effect. -.. _Pipenv: https://docs.pipenv.org/ +.. _Pipenv: https://pipenv.kennethreitz.org/ .. _npm: https://www.npmjs.com/ .. _bundler: http://bundler.io/ .. _user base: https://docs.python.org/3/library/site.html#site.USER_BASE @@ -116,9 +116,9 @@ Pipenv manages dependencies on a per-project basis. To install packages, change into your project's directory (or just an empty directory for this tutorial) and run: -.. code-block:: bash +.. code-block:: console - $ cd myproject + $ cd project_folder $ pipenv install requests Pipenv will install the excellent `Requests`_ library and create a ``Pipfile`` @@ -156,7 +156,7 @@ when you share your project with others. You should get output similar to this Adding requests to Pipfile's [packages]... P.S. You have excellent taste! ✨ 🍰 ✨ -.. _Requests: https://python-requests.org +.. _Requests: https://requests.readthedocs.io/en/latest/ Using installed packages @@ -175,7 +175,7 @@ use it: Then you can run this script using ``pipenv run``: -.. code-block:: bash +.. code-block:: console $ pipenv run python main.py @@ -200,7 +200,7 @@ Congratulations, you now know how to install and use Python packages! ✨ 🍰 Lower level: virtualenv ======================= -`virtualenv `_ is a tool to create +`virtualenv `_ is a tool to create isolated Python environments. virtualenv creates a folder which contains all the necessary executables to use the packages that a Python project would need. @@ -212,37 +212,40 @@ Install virtualenv via pip: $ pip install virtualenv -Test your installation +Test your installation: .. code-block:: console $ virtualenv --version Basic Usage -~~~~~~~~~~~ +----------- 1. Create a virtual environment for a project: .. code-block:: console - $ cd my_project_folder - $ virtualenv my_project + $ cd project_folder + $ virtualenv venv -``virtualenv my_project`` will create a folder in the current directory which will +``virtualenv venv`` will create a folder in the current directory which will contain the Python executable files, and a copy of the ``pip`` library which you can use to install other packages. The name of the virtual environment (in this -case, it was ``my_project``) can be anything; omitting the name will place the files +case, it was ``venv``) can be anything; omitting the name will place the files in the current directory instead. +.. note:: + 'venv' is the general convention used globally. As it is readily available in ignore files (eg: .gitignore') + This creates a copy of Python in whichever directory you ran the command in, -placing it in a folder named :file:`my_project`. +placing it in a folder named :file:`venv`. You can also use the Python interpreter of your choice (like ``python2.7``). .. code-block:: console - $ virtualenv -p /usr/bin/python2.7 my_project + $ virtualenv -p /usr/bin/python2.7 venv or change the interpreter globally with an env variable in ``~/.bashrc``: @@ -254,14 +257,22 @@ or change the interpreter globally with an env variable in ``~/.bashrc``: .. code-block:: console - $ source my_project/bin/activate + $ source venv/bin/activate The name of the current virtual environment will now appear on the left of -the prompt (e.g. ``(my_project)Your-Computer:your_project UserName$)`` to let you know +the prompt (e.g. ``(venv)Your-Computer:project_folder UserName$``) to let you know that it's active. From now on, any package that you install using pip will be -placed in the ``my_project`` folder, isolated from the global Python installation. +placed in the ``venv`` folder, isolated from the global Python installation. + +For Windows, the same command mentioned in step 1 can be used to create a virtual environment. However, activating the environment requires a slightly different command. -Install packages as usual, for example: +Assuming that you are in your project directory: + +.. code-block:: console + + C:\Users\SomeUser\project_folder> venv\Scripts\activate + +Install packages using the ``pip`` command: .. code-block:: console @@ -272,20 +283,23 @@ Install packages as usual, for example: .. code-block:: console - $ deactivate + $ deactivate This puts you back to the system's default Python interpreter with all its installed libraries. To delete a virtual environment, just delete its folder. (In this case, -it would be ``rm -rf my_project``.) +it would be ``rm -rf venv``.) After a while, though, you might end up with a lot of virtual environments -littered across your system, and its possible you'll forget their names or +littered across your system, and it's possible you'll forget their names or where they were placed. +.. note:: + Python has included venv module from version 3.3. For more details: `venv `_. + Other Notes -~~~~~~~~~~~ +----------- Running ``virtualenv`` with the option ``--no-site-packages`` will not include the packages that are installed globally. This can be useful @@ -293,7 +307,7 @@ for keeping the package list clean in case it needs to be accessed later. [This is the default behavior for ``virtualenv`` 1.7 and later.] In order to keep your environment consistent, it's a good idea to "freeze" -the current state of the environment packages. To do this, run +the current state of the environment packages. To do this, run: .. code-block:: console @@ -302,7 +316,7 @@ the current state of the environment packages. To do this, run This will create a :file:`requirements.txt` file, which contains a simple list of all the packages in the current environment, and their respective versions. You can see the list of installed packages without the requirements -format using "pip list". Later it will be easier for a different developer +format using ``pip list``. Later it will be easier for a different developer (or you, if you need to re-create the environment) to install the same packages using the same versions: @@ -343,7 +357,7 @@ To install (make sure **virtualenv** is already installed): $ pip install virtualenvwrapper-win -In Windows, the default path for WORKON_HOME is %USERPROFILE%\Envs +In Windows, the default path for WORKON_HOME is %USERPROFILE%\\Envs Basic Usage ~~~~~~~~~~~ @@ -352,23 +366,23 @@ Basic Usage .. code-block:: console - $ mkvirtualenv my_project + $ mkvirtualenv project_folder -This creates the :file:`my_project` folder inside :file:`~/Envs`. +This creates the :file:`project_folder` folder inside :file:`~/Envs`. 2. Work on a virtual environment: .. code-block:: console - $ workon my_project + $ workon project_folder Alternatively, you can make a project, which creates the virtual environment, -and also a project directory inside ``$WORKON_HOME``, which is ``cd`` -ed into -when you ``workon myproject``. +and also a project directory inside ``$WORKON_HOME``, which is ``cd``-ed into +when you ``workon project_folder``. .. code-block:: console - $ mkproject myproject + $ mkproject project_folder **virtualenvwrapper** provides tab-completion on environment names. It really helps when you have a lot of environments and have trouble remembering their @@ -413,20 +427,15 @@ virtualenv-burrito With `virtualenv-burrito `_, you can have a working virtualenv + virtualenvwrapper environment in a single command. -autoenv +direnv ------- -When you ``cd`` into a directory containing a :file:`.env`, `autoenv `_ +When you ``cd`` into a directory containing a :file:`.env`, `direnv `_ automagically activates the environment. Install it on Mac OS X using ``brew``: .. code-block:: console - $ brew install autoenv - -And on Linux: - -.. code-block:: console + $ brew install direnv - $ git clone git://github.com/kennethreitz/autoenv.git ~/.autoenv - $ echo 'source ~/.autoenv/activate.sh' >> ~/.bashrc +On Linux follow the instructions at `direnv.net `_ diff --git a/docs/index.rst b/docs/index.rst index aec53b975..df7fcb8ed 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -4,10 +4,12 @@ contain the root `toctree` directive. .. meta:: - :description: An opinionated guide to the Python programming language and a best practice handbook to the installation, configuration, and usage of Python on a daily basis. + :description: An opinionated guide to the Python programming language and a best practice handbook for the installation, configuration, and usage of Python on a daily basis. + +################################# The Hitchhiker's Guide to Python! -================================= +################################# Greetings, Earthling! Welcome to The Hitchhiker's Guide to Python. @@ -15,7 +17,7 @@ Greetings, Earthling! Welcome to The Hitchhiker's Guide to Python. `fork us on GitHub `_! This handcrafted guide exists to provide both novice and expert Python -developers a best practice handbook to the installation, configuration, and +developers a best practice handbook for the installation, configuration, and usage of Python on a daily basis. This guide is **opinionated** in a way that is almost, but not quite, entirely @@ -23,7 +25,7 @@ This guide is **opinionated** in a way that is almost, but not quite, entirely available here. Rather, you'll find a nice concise list of highly recommended options. -.. note:: The use of **Python 3** is *highly* preferred over Python 2. Consider upgrading your applications and infrastructure if you find yourself *still* using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste. +.. note:: The use of **Python 3** is *highly* recommended over Python 2. Consider upgrading your applications and infrastructures if you find yourself *still* using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste. —*Kenneth Reitz* Let's get started! But first, let's make sure you know where your towel is. diff --git a/docs/intro/community.rst b/docs/intro/community.rst index 2e54be330..4ce563658 100644 --- a/docs/intro/community.rst +++ b/docs/intro/community.rst @@ -1,20 +1,25 @@ .. _the-community: + +############# The Community -============= +############# .. image:: /_static/photos/34689432801_78d97ecec9_k_d.jpg + +**** BDFL ----- +**** Guido van Rossum, the creator of Python, is often referred to as the BDFL — the Benevolent Dictator For Life. - +************************** Python Software Foundation --------------------------- +************************** + The mission of the Python Software Foundation is to promote, protect, and advance the Python programming language, and to support and facilitate the @@ -23,8 +28,9 @@ growth of a diverse and international community of Python programmers. `Learn More about the PSF `_. +**** PEPs ----- +**** PEPs are *Python Enhancement Proposals*. They describe changes to Python itself, or the standards around it. @@ -70,8 +76,9 @@ Here's an overview of the PEP acceptance workflow: .. image:: ../_static/pep-0001-1.png +****************** Python Conferences --------------------------- +****************** The major events for the Python community are developer conferences. The two most notable conferences are PyCon, which is held in the US, and its European @@ -80,9 +87,29 @@ sibling, EuroPython. A comprehensive list of conferences is maintained at `pycon.org `_. +****************** Python User Groups --------------------------- +****************** User Groups are where a bunch of Python developers meet to present or talk about Python topics of interest. A list of local user groups is maintained at the `Python Software Foundation Wiki `_. + + +****************** +Online Communities +****************** + +`PythonistaCafe `_ is an invite-only, online community +of Python and software development enthusiasts helping each other succeed and grow. +Think of it as a club of mutual improvement for Pythonistas where a broad range of +programming questions, career advice, and other topics are discussed every day. + + +***************** +Python Job Boards +***************** + +`Python Jobs HQ `_ is a Python job board, by Python Developers +for Python Developers. The site aggregates Python job postings from across the web and +also allows employers to post Python job openings directly on the site. diff --git a/docs/intro/documentation.rst b/docs/intro/documentation.rst index ea247ce4f..887d0982b 100644 --- a/docs/intro/documentation.rst +++ b/docs/intro/documentation.rst @@ -1,10 +1,15 @@ + + +############# Documentation -============= +############# .. image:: /_static/photos/33928823133_2f3d32cf32_k_d.jpg + +********************** Official Documentation ----------------------- +********************** The official Python Language and Library documentation can be found here: @@ -12,8 +17,9 @@ The official Python Language and Library documentation can be found here: - `Python 3.x `_ +************* Read the Docs -------------- +************* Read the Docs is a popular community project that hosts documentation for open source software. It holds documentation for many Python modules, @@ -22,20 +28,21 @@ both popular and exotic. `Read the Docs `_ +***** pydoc ------ +***** :program:`pydoc` is a utility that is installed when you install Python. It allows you to quickly retrieve and search for documentation from your shell. For example, if you needed a quick refresher on the -:mod:`time` module, pulling up documentation would be as simple as +:mod:`time` module, pulling up documentation would be as simple as: .. code-block:: console $ pydoc time The above command is essentially equivalent to opening the Python REPL -and running +and running: .. code-block:: pycon diff --git a/docs/intro/duction.rst b/docs/intro/duction.rst index 4a6772d79..16baebc0d 100644 --- a/docs/intro/duction.rst +++ b/docs/intro/duction.rst @@ -1,5 +1,8 @@ + + +############ Introduction -============ +############ .. image:: /_static/photos/34725946825_0f85497e60_k_d.jpg @@ -25,11 +28,11 @@ include: object serialization, and much more. Additionally, the - `Python Package Index `_ is available + `Python Package Index `_ is available for users to submit their packages for widespread use, similar to - Perl's `CPAN `_. There is a thriving community + Perl's `CPAN `_. There is a thriving community of very powerful Python frameworks and tools like - the `Django `_ web framework and the + the `Django `_ web framework and the `NumPy `_ set of math routines. * **integration with other systems** @@ -56,8 +59,10 @@ include: .. _about-ref: + +**************** About This Guide ----------------- +**************** Purpose ~~~~~~~ diff --git a/docs/intro/learning.rst b/docs/intro/learning.rst index 22935e6d6..b0c957398 100644 --- a/docs/intro/learning.rst +++ b/docs/intro/learning.rst @@ -1,10 +1,15 @@ + + +############### Learning Python -=============== +############### .. image:: /_static/photos/32800783863_11a00db52c_k_d.jpg + +******** Beginner --------- +******** The Python Tutorial ~~~~~~~~~~~~~~~~~~~~ @@ -22,35 +27,36 @@ Real Python is a repository of free and in-depth Python tutorials created by a d `Real Python `_ - Python Basics -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~ -pythonbasics.org is an introductiory tutorial for beginners. The tutorial includes exercises. It covers the basics and there are also in-depth lessons like object oriented programming and regular expressions. +pythonbasics.org is an introductory tutorial for beginners. The tutorial includes exercises. It covers the basics and there are also in-depth lessons like object oriented programming and regular expressions. `Python basics `_ Python for Beginners -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -thepythonguru.com is a tutorial focuses on beginner programmers. It covers many python concepts -in depth. It also teaches you some advance constructs of python like lambda expression, regular expression. -At last it finishes off with tutorial "How to access MySQL db using python" +~~~~~~~~~~~~~~~~~~~~ +thepythonguru.com is a tutorial focused on beginner programmers. It covers many Python concepts +in depth. It also teaches you some advanced constructs of Python like lambda expressions and regular expressions. +And last it finishes off with the tutorial "How to access MySQL db using Python" - `Python for beginners `_ + `Python for Beginners `_ Learn Python Interactive Tutorial ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Learnpython.org is an easy non-intimidating way to get introduced to Python. The website takes the same approach used on the popular -`Try Ruby `_ website, it has an interactive Python +`Try Ruby `_ website. It has an interactive Python interpreter built into the site that allows you to go through the lessons without having to install Python locally. `Learn Python `_ +Python for You and Me +~~~~~~~~~~~~~~~~~~~~~ + If you want a more traditional book, *Python For You and Me* is an excellent resource for learning all aspects of the language. @@ -59,15 +65,15 @@ resource for learning all aspects of the language. Learn Python Step by Step ~~~~~~~~~~~~~~~~~~~~~~~~~ -Techbeamers.com provides step-by-step tutorials to teach Python. Each tutorial is supplemented with logically added coding snippets and equips with a follow-up quiz on the subject learned. There is a section for `Python interview questions `_ to help job seekers. You can also read essential `Python tips `_ and learn `best coding practices `_ for writing quality code. Here, you'll get the right platform to learn Python quickly. +Techbeamers.com provides step-by-step tutorials to teach Python. Each tutorial is supplemented with logically added coding snippets and equips with a follow-up quiz on the subject learned. There is a section for `Python interview questions `_ to help job seekers. You can also read essential `Python tips `_ and learn `best coding practices `_ for writing quality code. Here, you'll get the right platform to learn Python quickly. -`Learn Python Basic to Advanced `_ +`Learn Python Basic to Advanced `_ Online Python Tutor ~~~~~~~~~~~~~~~~~~~ -Online Python Tutor gives you a visual step by step +Online Python Tutor gives you a visual step-by-step representation of how your program runs. Python Tutor helps people overcome a fundamental barrier to learning programming by understanding what happens as the computer @@ -102,7 +108,7 @@ Learn Python the Hard Way This is an excellent beginner programmer's guide to Python. It covers "hello world" from the console to the web. - `Learn Python the Hard Way `_ + `Learn Python the Hard Way `_ Crash into Python @@ -111,7 +117,7 @@ Crash into Python Also known as *Python for Programmers with 3 Hours*, this guide gives experienced developers from other languages a crash course on Python. - `Crash into Python `_ + `Crash into Python `_ Dive Into Python 3 @@ -121,7 +127,7 @@ Dive Into Python 3 is a good book for those ready to jump in to Python 3. It's a good read if you are moving from Python 2 to 3 or if you already have some experience programming in another language. - `Dive Into Python 3 `_ + `Dive Into Python 3 `_ Think Python: How to Think Like a Computer Scientist @@ -129,7 +135,7 @@ Think Python: How to Think Like a Computer Scientist Think Python attempts to give an introduction to basic concepts in computer science through the use of the Python language. The focus was to create a book -with plenty of exercises, minimal jargon and a section in each chapter devoted +with plenty of exercises, minimal jargon, and a section in each chapter devoted to the subject of debugging. While exploring the various features available in the Python language the @@ -137,7 +143,7 @@ author weaves in various design patterns and best practices. The book also includes several case studies which have the reader explore the topics discussed in the book in greater detail by applying those topics to -real-world examples. Case studies include assignments in GUI and Markov +real-world examples. Case studies include assignments in GUI programming and Markov Analysis. `Think Python `_ @@ -147,7 +153,7 @@ Python Koans ~~~~~~~~~~~~ Python Koans is a port of Edgecase's Ruby Koans. It uses a test-driven -approach, q.v. TEST DRIVEN DESIGN SECTION to provide an interactive tutorial +approach to provide an interactive tutorial teaching basic Python concepts. By fixing assertion statements that fail in a test script, this provides sequential steps to learning Python. @@ -155,11 +161,11 @@ For those used to languages and figuring out puzzles on their own, this can be a fun, attractive option. For those new to Python and programming, having an additional resource or reference will be helpful. - `Python Koans `_ + `Python Koans `_ More information about test driven development can be found at these resources: - `Test Driven Development `_ + `Test Driven Development `_ A Byte of Python @@ -172,13 +178,15 @@ no previous programming experience. `A Byte of Python for Python 3.x `_ -Learn to Program in Python with Codeacademy +Computer Science Path on Codecademy ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -A Codeacademy course for the absolute Python beginner. This free and interactive course provides and teaches the basics (and beyond) of Python programming whilst testing the user's knowledge in between progress. +A Codecademy course for the absolute Python beginner. This free and interactive +course provides and teaches the basics (and beyond) of Python programming while +testing the user's knowledge in between progress. This course also features a built-in interpreter for receiving instant feedback on your learning. - `Learn to Program in Python with Codeacademy `_ + `Computer Science Path on Codecademy `_ Code the blocks @@ -187,21 +195,22 @@ Code the blocks *Code the blocks* provides free and interactive Python tutorials for beginners. It combines Python programming with a 3D environment where you "place blocks" and construct structures. The tutorials teach you -how to use Python to create progressively elaborate 3D structures, +how to use Python to create progressively more elaborate 3D structures, making the process of learning Python fun and engaging. `Code the blocks `_ +************ Intermediate ------------- +************ Python Tricks: The Book ~~~~~~~~~~~~~~~~~~~~~~~ -Discover Python's best practices with simple examples and start writing even more beautiful + Pythonic code. "Python Tricks: The Book" shows you exactly how. +Discover Python's best practices with simple examples and start writing even more beautiful + Pythonic code. *Python Tricks: The Book* shows you exactly how. -You’ll master intermediate and advanced-level features in Python with practical examples and a clear narrative: +You’ll master intermediate and advanced-level features in Python with practical examples and a clear narrative. `Python Tricks: The Book `_ @@ -209,15 +218,16 @@ Effective Python ~~~~~~~~~~~~~~~~ This book contains 59 specific ways to improve writing Pythonic code. At 227 -pages, it is a very brief overview of some of the most commons adapations +pages, it is a very brief overview of some of the most common adaptations programmers need to make to become efficient intermediate level Python programmers. - `Effective Python `_ + `Effective Python `_ +******** Advanced --------- +******** Pro Python ~~~~~~~~~~ @@ -226,7 +236,7 @@ This book is for intermediate to advanced Python programmers who are looking to understand how and why Python works the way it does and how they can take their code to the next level. - `Pro Python `_ + `Pro Python `_ Expert Python Programming @@ -235,15 +245,15 @@ Expert Python Programming deals with best practices in programming Python and is focused on the more advanced crowd. It starts with topics like decorators (with caching, proxy, and context manager -case-studies), method resolution order, using super() and meta-programming, and +case studies), method resolution order, using super() and meta-programming, and general :pep:`8` best practices. It has a detailed, multi-chapter case study on writing and releasing a package and eventually an application, including a chapter on using zc.buildout. Later chapters detail best practices such as writing documentation, test-driven -development, version control, optimization and profiling. +development, version control, optimization, and profiling. - `Expert Python Programming `_ + `Expert Python Programming `_ A Guide to Python's Magic Methods @@ -255,13 +265,13 @@ and can make classes and objects behave in different and magical ways. `A Guide to Python's Magic Methods `_ -.. note:: The Rafekettler.com is currently down, you can go to their Github version directly. Here you can find a PDF version: +.. note:: Rafekettler.com is currently down; you can go to their GitHub version directly. Here you can find a PDF version: `A Guide to Python's Magic Methods (repo on GitHub) `_ - +**************************** For Engineers and Scientists ----------------------------- +**************************** A Primer on Scientific Programming with Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -270,7 +280,7 @@ A Primer on Scientific Programming with Python, written by Hans Petter Langtangen, mainly covers Python's usage in the scientific field. In the book, examples are chosen from mathematics and the natural sciences. - `A Primer on Scientific Programming with Python `_ + `A Primer on Scientific Programming with Python `_ Numerical Methods in Engineering with Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -278,10 +288,12 @@ Numerical Methods in Engineering with Python Numerical Methods in Engineering with Python, written by Jaan Kiusalaas, puts the emphasis on numerical methods and how to implement them in Python. - `Numerical Methods in Engineering with Python `_ + `Numerical Methods in Engineering with Python `_ -Miscellaneous topics --------------------- + +******************** +Miscellaneous Topics +******************** Problem Solving with Algorithms and Data Structures ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -309,7 +321,7 @@ Transforming Code into Beautiful, Idiomatic Python Transforming Code into Beautiful, Idiomatic Python is a video by Raymond Hettinger. Learn to take better advantage of Python's best features and improve existing code -through a series of code transformations, "When you see this, do that instead." +through a series of code transformations: "When you see this, do that instead." `Transforming Code into Beautiful, Idiomatic Python `_ @@ -320,7 +332,7 @@ Fullstack Python Fullstack Python offers a complete top-to-bottom resource for web development using Python. -From setting up the webserver, to designing the front-end, choosing a database, +From setting up the web server, to designing the front-end, choosing a database, optimizing/scaling, etc. As the name suggests, it covers everything you need to build and run a complete @@ -329,14 +341,23 @@ web app from scratch. `Fullstack Python `_ +PythonistaCafe +~~~~~~~~~~~~~~ + +PythonistaCafe is an invite-only, online community of Python and software development enthusiasts helping each other succeed and grow. Think of it as a club of mutual improvement for Pythonistas where a broad range of programming questions, career advice, and other topics are discussed every day. + + `PythonistaCafe `_ + + +********** References ----------- +********** Python in a Nutshell ~~~~~~~~~~~~~~~~~~~~ Python in a Nutshell, written by Alex Martelli, covers most cross-platform -Python's usage, from its syntax to built-in libraries to advanced topics such +Python usage, from its syntax to built-in libraries to advanced topics such as writing C extensions. `Python in a Nutshell `_ @@ -344,7 +365,7 @@ as writing C extensions. The Python Language Reference ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This is Python's reference manual, it covers the syntax and the core semantics +This is Python's reference manual. It covers the syntax and the core semantics of the language. `The Python Language Reference `_ @@ -371,7 +392,7 @@ Python Cookbook ~~~~~~~~~~~~~~~ Python Cookbook, written by David Beazley and Brian K. Jones, is packed with -practical recipes. This book covers the core python language as well as tasks +practical recipes. This book covers the core Python language as well as tasks common to a wide variety of application domains. `Python Cookbook `_ @@ -379,13 +400,13 @@ common to a wide variety of application domains. Writing Idiomatic Python ~~~~~~~~~~~~~~~~~~~~~~~~ -"Writing Idiomatic Python", written by Jeff Knupp, contains the most common and +Writing Idiomatic Python, written by Jeff Knupp, contains the most common and important Python idioms in a format that maximizes identification and understanding. Each idiom is presented as a recommendation of a way to write some commonly used piece of code, followed by an explanation of why the idiom is important. It also contains two code samples for each idiom: the "Harmful" way to write it and the "Idiomatic" way. - `For Python 2.7.3+ `_ + `For Python 2.7.3+ `_ - `For Python 3.3+ `_ + `For Python 3.3+ `_ diff --git a/docs/intro/news.rst b/docs/intro/news.rst index 37c144a15..d1cda93be 100644 --- a/docs/intro/news.rst +++ b/docs/intro/news.rst @@ -1,72 +1,102 @@ + + +#### News -==== +#### .. image:: /_static/photos/33573767786_eececc5d27_k_d.jpg + +**************** +PyCoder’s Weekly +**************** + +PyCoder’s Weekly is a free weekly Python newsletter for Python developers +by Python developers (Projects, Articles, News, and Jobs). + + `PyCoder’s Weekly `_ + + +*********** +Real Python +*********** + +At Real Python you can learn all things Python from the ground up, with weekly free and in-depth tutorials. Everything from the absolute basics of Python, to web development and web scraping, to data visualization, and beyond. + + `Real Python `_ + + +************* Planet Python -~~~~~~~~~~~~~ +************* This is an aggregate of Python news from a growing number of developers. - `Planet Python `_ + `Planet Python `_ + +********* /r/python -~~~~~~~~~ +********* /r/python is the Reddit Python community where users contribute and vote on Python-related news. - `/r/python `_ + `/r/python `_ + +******************* Talk Python Podcast -~~~~~~~~~~~~~~~~~~~ +******************* The #1 Python-focused podcast covering the people and ideas in Python. `Talk Python To Me `_ + +******************** Python Bytes Podcast -~~~~~~~~~~~~~~~~~~~~ +******************** A short-form Python podcast covering recent developer headlines. `Python Bytes `_ -Pycoder's Weekly -~~~~~~~~~~~~~~~~ - -Pycoder's Weekly is a free weekly Python newsletter for Python developers -by Python developers (Projects, Articles, News, and Jobs). - - `Pycoder's Weekly `_ +************* Python Weekly -~~~~~~~~~~~~~ +************* Python Weekly is a free weekly newsletter featuring curated news, articles, new releases, jobs, etc. related to Python. - `Python Weekly `_ + `Python Weekly `_ + +*********** Python News -~~~~~~~~~~~~~ +*********** Python News is the news section in the official Python web site (www.python.org). It briefly highlights the news from the Python community. - `Python News `_ + `Python News `_ + +******************** Import Python Weekly -~~~~~~~~~~~~~~~~~~~~ +******************** -Weekly Python Newsletter containing Python Articles, Projects, Videos, Tweets +Weekly Python Newsletter containing Python Articles, Projects, Videos, and Tweets delivered in your inbox. Keep Your Python Programming Skills Updated. `Import Python Weekly Newsletter `_ + +************************* Awesome Python Newsletter -~~~~~~~~~~~~~~~~~~~~~~~~~ +************************* -A weekly overview of the most popular Python news, articles and packages. +A weekly overview of the most popular Python news, articles, and packages. `Awesome Python Newsletter `_ diff --git a/docs/notes/contribute.rst b/docs/notes/contribute.rst index 4045fb102..b6c72bb4f 100644 --- a/docs/notes/contribute.rst +++ b/docs/notes/contribute.rst @@ -1,5 +1,6 @@ +########## Contribute -~~~~~~~~~~ +########## .. image:: /_static/photos/33573769116_49c1ef51e7_k_d.jpg @@ -10,20 +11,24 @@ issue on GitHub_. To submit patches, please send a pull request on GitHub_. Once your changes get merged back in, you'll automatically be added to the `Contributors List `_. + +*********** Style Guide ------------ +*********** For all contributions, please follow the :ref:`guide-style-guide`. .. _todo-list-ref: + +********* Todo List ---------- +********* If you'd like to contribute, there's plenty to do. Here's a short todo_ list. .. include:: ../../TODO.rst -.. _GitHub: http://github.com/kennethreitz/python-guide/ +.. _GitHub: https://github.com/kennethreitz/python-guide/ .. _todo: https://github.com/kennethreitz/python-guide/blob/master/TODO.rst diff --git a/docs/notes/license.rst b/docs/notes/license.rst index 407e17358..1dd3d340a 100644 --- a/docs/notes/license.rst +++ b/docs/notes/license.rst @@ -1,6 +1,6 @@ -======= +####### License -======= +####### .. image:: /_static/photos/32800805573_568d6b72fd_k_d.jpg diff --git a/docs/notes/styleguide.rst b/docs/notes/styleguide.rst index 619c6daff..5a715332e 100644 --- a/docs/notes/styleguide.rst +++ b/docs/notes/styleguide.rst @@ -1,8 +1,8 @@ .. _guide-style-guide: -===================== +##################### The Guide Style Guide -===================== +##################### .. image:: /_static/photos/33573755856_7f43d43adf_k_d.jpg @@ -19,8 +19,10 @@ The Guide is written as :ref:`restructuredtext-ref`. .. note:: On any page of the rendered HTML you can click "Show Source" to see how authors have styled the page. + +********* Relevancy ---------- +********* Strive to keep any contributions relevant to the :ref:`purpose of The Guide `. @@ -29,15 +31,17 @@ Strive to keep any contributions relevant to the :ref:`purpose of The Guide relate to Python development. * Prefer to link to other sources if the information is already out there. Be sure to describe what and why you are linking. -* `Cite `_ +* `Cite `_ references where needed. * If a subject isn't directly relevant to Python, but useful in conjunction with Python (e.g., Git, GitHub, Databases), reference by linking to useful resources, and describe why it's useful to Python. * When in doubt, ask. + +******** Headings --------- +******** Use the following styles for headings. @@ -53,31 +57,35 @@ Page title: .. code-block:: rest - =================== + ******************* Time is an Illusion - =================== + ******************* Section headings: .. code-block:: rest Lunchtime Doubly So - ------------------- + =================== Sub section headings: .. code-block:: rest Very Deep - ~~~~~~~~~ + --------- + +***** Prose ------ +***** Wrap text lines at 78 characters. Where necessary, lines may exceed 78 characters, especially if wrapping would make the source text more difficult to read. +Use Standard American English, not British English. + Use of the `serial comma `_ (also known as the Oxford comma) is 100% non-optional. Any attempt to submit content with a missing serial comma will result in permanent banishment @@ -85,8 +93,10 @@ from this project, due to complete and total lack of taste. Banishment? Is this a joke? Hopefully we will never have to find out. + +************* Code Examples -------------- +************* Wrap all code examples at 70 characters to avoid horizontal scrollbars. @@ -99,7 +109,10 @@ Command line examples: $ run command --help $ ls .. -Be sure to include the ``$`` prefix before each line. +Be sure to include the ``$`` prefix before each line for Unix console examples. + +For Windows console examples, use ``doscon`` or ``powershell`` instead of +``console``, and omit the ``$`` prefix. Python interpreter examples: @@ -122,32 +135,36 @@ Python examples: def get_answer(): return 42 + +****************** Externally Linking ------------------- +****************** -* Prefer labels for well known subjects (ex: proper nouns) when linking: +* Prefer labels for well known subjects (e.g. proper nouns) when linking: .. code-block:: rest Sphinx_ is used to document Python. - .. _Sphinx: http://sphinx.pocoo.org + .. _Sphinx: https://www.sphinx-doc.org * Prefer to use descriptive labels with inline links instead of leaving bare links: .. code-block:: rest - Read the `Sphinx Tutorial `_ + Read the `Sphinx Tutorial `_ -* Avoid using labels such as "click here", "this", etc. preferring +* Avoid using labels such as "click here", "this", etc., preferring descriptive labels (SEO worthy) instead. + +******************************** Linking to Sections in The Guide --------------------------------- +******************************** To cross-reference other parts of this documentation, use the `:ref: -`_ +`_ keyword and labels. To make reference labels more clear and unique, always add a ``-ref`` suffix: @@ -159,11 +176,13 @@ To make reference labels more clear and unique, always add a ``-ref`` suffix: Some Section ------------ + +****************** Notes and Warnings ------------------- +****************** Make use of the appropriate `admonitions directives -`_ when making notes. +`_ when making notes. Notes: @@ -180,11 +199,13 @@ Warnings: .. warning:: DON'T PANIC + +***** TODOs ------ +***** Please mark any incomplete areas of The Guide with a `todo directive -`_. To +`_. To avoid cluttering the :ref:`todo-list-ref`, use a single ``todo`` for stub documents or large incomplete sections. @@ -193,4 +214,3 @@ documents or large incomplete sections. .. todo:: Learn the Ultimate Answer to the Ultimate Question of Life, The Universe, and Everything - diff --git a/docs/scenarios/admin.rst b/docs/scenarios/admin.rst index 1fe3b5358..f6433c213 100644 --- a/docs/scenarios/admin.rst +++ b/docs/scenarios/admin.rst @@ -1,10 +1,14 @@ + +###################### Systems Administration -====================== +###################### .. image:: /_static/photos/34435690580_3afec7d4cd_k_d.jpg + +****** Fabric ------- +****** `Fabric `_ is a library for simplifying system administration tasks. While Chef and Puppet tend to focus on managing servers @@ -19,7 +23,7 @@ Install Fabric: The following code will create two tasks that we can use: ``memory_usage`` and ``deploy``. The former will output the memory usage on each machine. The -latter will ssh into each server, cd to our project directory, activate the +latter will SSH into each server, cd to our project directory, activate the virtual environment, pull the newest codebase, and restart the application server. @@ -71,8 +75,10 @@ programs, and host grouping. `Fabric Documentation `_ + +**** Salt ----- +**** `Salt `_ is an open source infrastructure management tool. It supports remote command execution from a central point (master host) @@ -95,7 +101,7 @@ The following command lists all available minion hosts, using the ping module. $ salt '*' test.ping -The host filtering is accomplished by matching the minion id, +The host filtering is accomplished by matching the minion id or using the grains system. The `grains `_ system uses static host information like the operating system version or the @@ -125,16 +131,17 @@ it will install and start the Apache server: - require: - pkg: apache -State files can be written using YAML, the Jinja2 template system or pure Python. +State files can be written using YAML, the Jinja2 template system, or pure Python. `Salt Documentation `_ +****** Psutil ------- +****** `Psutil `_ is an interface to different -system information (e.g. CPU, memory, disks, network, users and processes). +system information (e.g. CPU, memory, disks, network, users, and processes). Here is an example to be aware of some server overload. If any of the tests (net, CPU) fail, it will send an email. @@ -148,49 +155,49 @@ tests (net, CPU) fail, it will send an email. # Package for email services: import smtplib import string - MAX_NET_USAGE = 400000 + MAX_NET_USAGE = 400000 # bytes per seconds MAX_ATTACKS = 4 attack = 0 - counter = 0 while attack <= MAX_ATTACKS: sleep(4) - counter = counter + 1 - # Check the cpu usage - if cpu_percent(interval = 1) > 70: - attack = attack + 1 - # Check the net usage - neti1 = net_io_counters()[1] - neto1 = net_io_counters()[0] + + # Check the net usage wit named tuples + neti1 = net_io_counters().bytes_recv + neto1 = net_io_counters().bytes_sent sleep(1) - neti2 = net_io_counters()[1] - neto2 = net_io_counters()[0] + neti2 = net_io_counters().bytes_recv + neto2 = net_io_counters().bytes_sent + # Calculate the bytes per second net = ((neti2+neto2) - (neti1+neto1))/2 - if net > MAX_NET_USAGE: - attack = attack + 1 - if counter > 25: - attack = 0 - counter = 0 + + # Check the net and cpu usage + if (net > MAX_NET_USAGE) or (cpu_percent(interval = 1) > 70): + attack+=1 + elif attack > 1: + attack-=1 + # Write a very important email if attack is higher than 4 TO = "you@your_email.com" FROM = "webmaster@your_domain.com" SUBJECT = "Your domain is out of system resources!" text = "Go and fix your server!" - BODY = string.join(("From: %s" %FROM,"To: %s" %TO,"Subject: %s" %SUBJECT, "",text), "\r\n") + string="\r\n" + BODY = string.join(("From: %s" %FROM,"To: %s" %TO, + "Subject: %s" %SUBJECT, "",text)) server = smtplib.SMTP('127.0.0.1') server.sendmail(FROM, [TO], BODY) server.quit() -A full terminal application like a widely extended top which is based on -psutil and with the ability of a client-server monitoring is -`glance `_. +A full terminal application like a widely extended top is `Glance `_, which is based on psutil and has the ability for client-server monitoring. +******* Ansible -------- +******* `Ansible `_ is an open source system automation tool. -The biggest advantage over Puppet or Chef is it does not require an agent on +Its biggest advantage over Puppet or Chef is that it does not require an agent on the client machine. Playbooks are Ansible’s configuration, deployment, and orchestration language and are written in YAML with Jinja2 for templating. @@ -238,8 +245,10 @@ about Ansible, read the `Ansible Docs `_. great and detailed introduction to getting started with Ansible. +**** Chef ----- +**** + `Chef `_ is a systems and cloud infrastructure automation framework that makes it easy to deploy servers and applications to any physical, virtual, or cloud location. In case this is your choice for configuration management, @@ -252,9 +261,9 @@ configures itself and this distributed approach makes Chef a scalable automation Chef works by using custom recipes (configuration elements), implemented in cookbooks. Cookbooks, which are basically packages for infrastructure choices, are usually stored in your Chef server. -Read the `Digital Ocean tutorial series +Read the `DigitalOcean tutorial series `_ -on chef to learn how to create a simple Chef Server. +on Chef to learn how to create a simple Chef Server. To create a simple cookbook the `knife `_ command is used: @@ -269,8 +278,10 @@ found on the `Chef Supermarket `_. - `Chef Documentation `_ + +****** Puppet ------- +****** `Puppet `_ is IT Automation and configuration management software from Puppet Labs that allows System Administrators to define the state @@ -286,8 +297,8 @@ Puppet Agents are installed on nodes whose state needs to be monitored or changed. A designated server known as the Puppet Master is responsible for orchestrating the agent nodes. -Agent nodes send basic facts about the system such as to the operating system, -kernel, architecture, ip address, hostname etc. to the Puppet Master. +Agent nodes send basic facts about the system such as the operating system, +kernel, architecture, IP address, hostname, etc. to the Puppet Master. The Puppet Master then compiles a catalog with information provided by the agents on how each node should be configured and sends it to the agent. The agent enforces the change as prescribed in the catalog and sends a report back @@ -307,7 +318,7 @@ your Puppet modules. Ubuntu Writing Modules in Puppet is pretty straight forward. Puppet Manifests together -form Puppet Modules. Puppet manifest end with an extension of ``.pp``. +form Puppet Modules. Puppet manifests end with an extension of ``.pp``. Here is an example of 'Hello World' in Puppet. .. code-block:: puppet @@ -321,7 +332,7 @@ Here is an example of 'Hello World' in Puppet. Here is another example with system based logic. Note how the operating system fact is being used as a variable prepended with the ``$`` sign. Similarly, this holds true for other facts such as hostname which can be referenced by -``$hostname`` +``$hostname``. .. code-block:: puppet @@ -333,10 +344,10 @@ holds true for other facts such as hostname which can be referenced by } There are several resource types for Puppet but the package-file-service -paradigm is all you need for undertaking majority of the configuration +paradigm is all you need for undertaking the majority of the configuration management. The following Puppet code makes sure that the OpenSSH-Server package is installed in a system and the sshd service is notified to restart -everytime the sshd configuration file is changed. +every time the sshd configuration file is changed. .. code-block:: puppet @@ -365,29 +376,22 @@ everytime the sshd configuration file is changed. For more information, refer to the `Puppet Labs Documentation `_ + +********* Blueprint ---------- +********* .. todo:: Write about Blueprint + +******** Buildout --------- +******** `Buildout `_ is an open source software build tool. Buildout is created using the Python programming language. It implements a -principle of separation of configuration from the scripts that do the setting up. -Buildout is primarily used to download and set up dependencies in Python eggs -format of the software being developed or deployed. Recipes for build tasks in any -environment can be created, and many are already available. - -Shinken -------- - -`Shinken `_ is a modern, Nagios compatible -monitoring framework written in Python. Its main goal is to give users a flexible -architecture for their monitoring system that is designed to scale to large -environments. - -Shinken is backwards-compatible with the Nagios configuration standard, and -plugins.It works on any operating system, and architecture that supports Python -which includes Windows, GNU/Linux, and FreeBSD. +principle of separation of configuration from the scripts that do the setting +up. Buildout is primarily used to download and set up dependencies in `Python +eggs `_ +format of the software being developed or deployed. Recipes for build tasks in +any environment can be created, and many are already available. diff --git a/docs/scenarios/ci.rst b/docs/scenarios/ci.rst index 676f0b805..4296679c7 100644 --- a/docs/scenarios/ci.rst +++ b/docs/scenarios/ci.rst @@ -1,16 +1,20 @@ + +###################### Continuous Integration -====================== +###################### .. image:: /_static/photos/33907150594_9abba7ad0a_k_d.jpg .. note:: For advice on writing your tests, see :doc:`/writing/tests`. + +**** Why? ----- +**** Martin Fowler, who first wrote about `Continuous Integration `_ -(short: CI) together with Kent Beck, describes the CI as follows: +(short: CI) together with Kent Beck, describes CI as follows: Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at @@ -20,27 +24,28 @@ Martin Fowler, who first wrote about `Continuous Integration `_ is an extensible continuous integration -engine. Use it. +******* +Jenkins +******* +`Jenkins CI `_ is an extensible Continuous Integration engine. Use it. +******** Buildbot --------- +******** `Buildbot `_ is a Python system to automate the compile/test cycle to validate code changes. - +*** Tox ---- +*** `tox `_ is an automation tool providing -packaging, testing and deployment of Python software right from the console or +packaging, testing, and deployment of Python software right from the console or CI server. It is a generic virtualenv management and test command line tool which provides the following features: @@ -49,17 +54,18 @@ which provides the following features: * Running tests in each of the environments, configuring your test tool of choice * Acting as a front-end to Continuous Integration servers, reducing boilerplate - and merging CI and shell-based testing. + and merging CI and shell-based testing +********* Travis-CI ---------- +********* `Travis-CI `_ is a distributed CI server which builds tests for open source projects for free. It provides multiple workers to run Python tests on and seamlessly integrates with GitHub. You can even have it comment on your Pull Requests whether this particular changeset breaks the -build or not. So if you are hosting your code on GitHub, travis-ci is a great +build or not. So, if you are hosting your code on GitHub, Travis-CI is a great and easy way to get started with Continuous Integration. In order to get started, add a :file:`.travis.yml` file to your repository with @@ -79,12 +85,12 @@ this example content:: This will get your project tested on all the listed Python versions by -running the given script, and will only build the master branch. There are a -lot more options you can enable, like notifications, before and after steps -and much more. The `travis-ci docs `_ +running the given script, and will only build the ``master`` branch. There are a +lot more options you can enable, like notifications, before and after steps, +and much more. The `Travis-CI docs `_ explain all of these options, and are very thorough. -In order to activate testing for your project, go to `the travis-ci site `_ +In order to activate testing for your project, go to `the Travis-CI site `_ and login with your GitHub account. Then activate your project in your profile settings and you're ready to go. From now on, your project's tests will be run on every push to GitHub. diff --git a/docs/scenarios/cli.rst b/docs/scenarios/cli.rst index 50bbbbc09..59f311e83 100644 --- a/docs/scenarios/cli.rst +++ b/docs/scenarios/cli.rst @@ -1,76 +1,91 @@ + +######################### Command-line Applications -========================= +######################### .. image:: /_static/photos/34435690330_11930b5987_k_d.jpg -Command-line applications, also referred to as -`Console Applications `_, -are computer programs designed to be used from a text interface, such as a -`shell `_. Command-line -applications usually accept various inputs as arguments, often referred to as -parameters or sub-commands, as well as options, often referred to as flags or -switches. +Command-line applications, also referred to as `Console Applications +`_, are computer programs +designed to be used from a text interface, such as a `shell +`_. Command-line applications +usually accept various inputs as arguments, often referred to as parameters or +sub-commands, as well as options, often referred to as flags or switches. Some popular command-line applications include: -* `Grep `_ - A plain-text data search utility +* `grep `_ - A plain-text data search utility * `curl `_ - A tool for data transfer with URL syntax -* `httpie `_ - A command line HTTP +* `httpie `_ - A command-line HTTP client, a user-friendly cURL replacement -* `git `_ - A distributed version control system -* `mercurial `_ - A distributed version control +* `Git `_ - A distributed version control system +* `Mercurial `_ - A distributed version control system primarily written in Python -Clint ------ - -`clint `_ is a Python module which is -filled with very useful tools for developing command-line applications. -It supports features such as; CLI colors and indents, simple and powerful -column printer, iterator based progress bars and implicit argument handling. +***** Click ------ +***** + +`click `_ is a Python package for creating +command-line interfaces in a composable way with as little code as possible. +This “Command-Line Interface Creation Kit” is highly configurable but comes +with good defaults out of the box. -`click `_ is a Python package for creating -command-line interfaces in a composable way with as little code as -possible. This “Command-line Interface Creation Kit” is highly -configurable but comes with good defaults out of the box. +****** docopt ------- +****** `docopt `_ is a lightweight, highly Pythonic package that allows creating command-line interfaces easily and intuitively, by parsing POSIX-style usage instructions. + +**** Plac ------- +**** -`Plac `_ is a simple wrapper +`Plac `_ is a simple wrapper over the Python standard library `argparse `_, which hides most of its complexity by using a declarative interface: the -argument parser is inferred rather than written down by imperatively. This -module targets especially unsophisticated users, programmers, sys-admins, -scientists and in general people writing throw-away scripts for themselves, +argument parser is inferred rather than written down imperatively. This +module targets unsophisticated users, programmers, sysadmins, +scientists, and in general people writing throw-away scripts for themselves, who choose to create a command-line interface because it is quick and simple. + +***** Cliff ------- +***** `Cliff `_ is a framework for building command-line programs. It uses setuptools entry points to provide subcommands, output formatters, and other extensions. The framework is meant -to be used to create multi-level commands such as subversion and git, where +to be used to create multi-level commands such as ``svn`` and ``git``, where the main program handles some basic argument parsing and then invokes a sub-command to do the work. + +****** Cement ------- - -`Cement `_ is an advanced CLI Application Framework. -Its goal is to introduce a standard, and feature-full platform -for both simple and complex command line applications as well -as support rapid development needs without sacrificing quality. -Cement is flexible, and it's use cases span from the simplicity of a micro-framework -to the complexity of a meg-framework. +****** + +`Cement `_ is an advanced CLI Application +Framework. Its goal is to introduce a standard and feature-full platform for +both simple and complex command line applications as well as support rapid +development needs without sacrificing quality. Cement is flexible, and its use +cases span from the simplicity of a micro-framework to the complexity of a +mega-framework. + + +*********** +Python Fire +*********** + +`Python Fire `_ is a library for +automatically generating command-line interfaces from absolutely any Python +object. It can help debug Python code more easily from the command line, +create CLI interfaces to existing code, allow you to interactively explore +code in a REPL, and simplify transitioning between Python and Bash (or any +other shell). diff --git a/docs/scenarios/clibs.rst b/docs/scenarios/clibs.rst index dba7d7127..13c9e4802 100644 --- a/docs/scenarios/clibs.rst +++ b/docs/scenarios/clibs.rst @@ -1,16 +1,20 @@ + +################################ Interfacing with C/C++ Libraries -================================ +################################ .. image:: /_static/photos/34725951345_c8f5959a2e_k_d.jpg + +**************************** C Foreign Function Interface ----------------------------- +**************************** `CFFI `_ provides a simple to use mechanism for interfacing with C from both CPython and PyPy. It supports two -modes: an inline ABI compatibility mode (example provided below), which allows +modes: an inline `ABI `_ compatibility mode (example provided below), which allows you to dynamically load and run functions from executable modules (essentially -exposing the same functionality as LoadLibrary or dlopen), and an API mode, +exposing the same functionality as `LoadLibrary `_ or `dlopen `_), and an API mode, which allows you to build C extension modules. ABI Interaction @@ -27,21 +31,23 @@ ABI Interaction # prints: 23 print("{}".format(length)) + +****** ctypes ------- +****** -`ctypes `_ is the de facto +`ctypes `_ is the de facto standard library for interfacing with C/C++ from CPython, and it provides not only full access to the native C interface of most major operating systems (e.g., kernel32 on Windows, or libc on \*nix), but also provides support for loading -and interfacing with dynamic libraries, such as DLLs or shared objects at -runtime. It does bring along with it a whole host of types for interacting +and interfacing with dynamic libraries, such as DLLs or shared objects, at +runtime. It brings along with it a whole host of types for interacting with system APIs, and allows you to rather easily define your own complex types, such as structs and unions, and allows you to modify things such as padding and alignment, if needed. It can be a bit crufty to use, but in -conjunction with the `struct `_ +conjunction with the `struct `_ module, you are essentially provided full control over how your data types get -translated into something usable by a pure C(++) method. +translated into something usable by a pure C/C++ method. Struct Equivalents ~~~~~~~~~~~~~~~~~~ @@ -66,15 +72,17 @@ Struct Equivalents _fields_ = [("a", c_int), ("b", c_int)] + +**** SWIG ----- +**** `SWIG `_, though not strictly Python focused (it supports a large number of scripting languages), is a tool for generating bindings for interpreted languages from C/C++ header files. It is extremely simple to use: the consumer simply needs to define an interface file (detailed in the tutorial and documentations), include the requisite C/C++ headers, and run -the build tool against them. While it does have some limits, (it currently +the build tool against them. While it does have some limits (it currently seems to have issues with a small subset of newer C++ features, and getting template-heavy code to work can be a bit verbose), it provides a great deal of power and exposes lots of features to Python with little effort. @@ -122,11 +130,12 @@ Example: Overloading __repr__ %include "MyClass.h" +************ Boost.Python ------------- +************ `Boost.Python `_ requires a bit more manual work to expose C++ object functionality, but it is capable of providing all the same features SWIG does and then some, -to include providing wrappers to access PyObjects in C++, extracting SWIG- +to include providing wrappers to access PyObjects in C++, extracting SWIG wrapper objects, and even embedding bits of Python into your C++ code. diff --git a/docs/scenarios/client.rst b/docs/scenarios/client.rst index 459feb243..c2d5c289b 100644 --- a/docs/scenarios/client.rst +++ b/docs/scenarios/client.rst @@ -1,10 +1,14 @@ + +#################### Network Applications -==================== +#################### .. image:: /_static/photos/34364815780_bea6614025_k_d.jpg + +**** HTTP -:::: +**** The Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, hypermedia information systems. HTTP is the @@ -24,13 +28,14 @@ your URLs, or to form-encode your POST data. Keep-alive and HTTP connection pooling are 100% automatic, powered by urllib3, which is embedded within Requests. -- `Documentation `_ -- `PyPi `_ +- `Documentation `_ +- `PyPi `_ - `GitHub `_ +******************* Distributed Systems -:::::::::::::::::::: +******************* ZeroMQ diff --git a/docs/scenarios/crypto.rst b/docs/scenarios/crypto.rst index 35330821c..262337204 100644 --- a/docs/scenarios/crypto.rst +++ b/docs/scenarios/crypto.rst @@ -1,18 +1,22 @@ + +############ Cryptography -============ +############ .. image:: /_static/photos/33907152824_bf91078cc1_k_d.jpg -Cryptography ------------- -`Cryptography `_ is an actively developed +************ +cryptography +************ + +`cryptography `_ is an actively developed library that provides cryptographic recipes and primitives. It supports -Python 2.6-2.7, Python 3.3+ and PyPy. +Python 2.6-2.7, Python 3.3+, and PyPy. -Cryptography is divided into two layers of recipes and hazardous materials -(hazmat). The recipes layer provides simple API for proper symmetric +cryptography is divided into two layers of recipes and hazardous materials +(hazmat). The recipes layer provides a simple API for proper symmetric encryption and the hazmat layer provides low-level cryptographic primitives. @@ -38,24 +42,24 @@ Example code using high level symmetric encryption recipe: plain_text = cipher_suite.decrypt(cipher_text) - - +************** GPGME bindings --------------- +************** + +The `GPGME Python bindings `_ provide Pythonic access to `GPG Made Easy `_, a C API for the entire GNU Privacy Guard suite of projects, including GPG, libgcrypt, and gpgsm (the S/MIME engine). It supports Python 2.6, 2.7, 3.4, and above. Depends on the SWIG C interface for Python as well as the GnuPG software and libraries. -The `GPGME Python bindings `_ provide pythonic access to `GPG Made Easy `_, a C API for the entire GNU Privacy Guard suite of projects, including GPG, libgcrypt and gpgsm (the S/MIME engine). It supports Python 2.6, 2.7, 3.4 and above. Depends on the SWIG C interface for Python as well as the GnuPG software and libraries. -A more comprehensive `GPGME Python Bindings HOWTO `_ is available with the source and a HTML version is available `here `_. Python 3 sample scripts from the examples in the HOWTO are also provided with the source and are accessible `here `_. +A more comprehensive `GPGME Python Bindings HOWTO `_ is available with the source, and an HTML version is available `at http://files.au.adversary.org `_. Python 3 sample scripts from the examples in the HOWTO are also provided with the source and are accessible `at gnupg.org `_. Available under the same terms as the rest of the GnuPG Project: GPLv2 and LGPLv2.1, both with the "or any later version" clause. Installation ------------- +~~~~~~~~~~~~ Included by default when compiling GPGME if the configure script locates a supported python version (which it will if it's in $PATH during configuration). Example -------- +~~~~~~~ .. code-block:: python3 @@ -85,34 +89,3 @@ Example print("Hang on ... did you say *all* of GnuPG? Yep.") else: pass - - - - -PyCrypto --------- - -`PyCrypto `_ is another library, -which provides secure hash functions and various encryption algorithms. It -supports Python version 2.1 through 3.3. - -Installation -~~~~~~~~~~~~ - -.. code-block:: console - - $ pip install pycrypto - -Example -~~~~~~~ - -.. code-block:: python - - from Crypto.Cipher import AES - # Encryption - encryption_suite = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456') - cipher_text = encryption_suite.encrypt("A really secret message. Not for prying eyes.") - - # Decryption - decryption_suite = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456') - plain_text = decryption_suite.decrypt(cipher_text) diff --git a/docs/scenarios/db.rst b/docs/scenarios/db.rst index af610e0f7..21217477a 100644 --- a/docs/scenarios/db.rst +++ b/docs/scenarios/db.rst @@ -1,22 +1,28 @@ + +######### Databases -========= +######### .. image:: /_static/photos/33907152464_a99fdcc8de_k_d.jpg + +****** DB-API ------- +****** The Python Database API (DB-API) defines a standard interface for Python database access modules. It's documented in :pep:`249`. -Nearly all Python database modules such as `sqlite3`, `psycopg` and +Nearly all Python database modules such as `sqlite3`, `psycopg`, and `mysql-python` conform to this interface. Tutorials that explain how to work with modules that conform to this interface can be found `here `__ and `here `__. + +********** SQLAlchemy ----------- +********** `SQLAlchemy `_ is a commonly used database toolkit. Unlike many database libraries it not only provides an ORM layer but also a @@ -26,12 +32,14 @@ generalized API for writing database-agnostic code without SQL. $ pip install sqlalchemy + +******* Records -------- +******* `Records `_ is minimalist SQL library, designed for sending raw SQL queries to various databases. Data can be used -programmatically, or exported to a number of useful data formats. +programmatically or exported to a number of useful data formats. .. code-block:: console @@ -39,10 +47,25 @@ programmatically, or exported to a number of useful data formats. Also included is a command-line tool for exporting SQL data. + +****** +PugSQL +****** + +`PugSQL `_ is a simple Python interface for organizing +and using parameterized, handwritten SQL. It is an anti-ORM that is +philosophically lo-fi, but it still presents a clean interface in Python. + +.. code-block:: console + + $ pip install pugsql + + +********** Django ORM ----------- +********** -The Django ORM is the interface used by `Django `_ +The Django ORM is the interface used by `Django `_ to provide database access. It's based on the idea of @@ -56,36 +79,40 @@ The basics: - Django gives you an automatically-generated database-access API; see `Making queries `__. + +****** peewee ------- +****** `peewee `_ is another ORM with a focus on being lightweight with support for Python 2.6+ and 3.2+ which supports -SQLite, MySQL and Postgres by default. The +SQLite, MySQL, and PostgreSQL by default. The `model layer `_ is similar to that of the Django ORM and it has `SQL-like methods `_ -to query data. While SQLite, MySQL and Postgres are supported out-of-the-box, +to query data. While SQLite, MySQL, and PostgreSQL are supported out-of-the-box, there is a `collection of add-ons `_ available. + +******* PonyORM -------- +******* `PonyORM `_ is an ORM that takes a different approach to querying the database. Instead of writing an SQL-like language or boolean -expressions, Python's generator syntax is used. There's also an graphical +expressions, Python's generator syntax is used. There's also a graphical schema editor that can generate PonyORM entities for you. It supports Python -2.6+ and Python 3.3+ and can connect to SQLite, MySQL, Postgres & Oracle - +2.6+ and Python 3.3+ and can connect to SQLite, MySQL, PostgreSQL, and Oracle. +********* SQLObject ---------- +********* `SQLObject `_ is yet another ORM. It supports a wide -variety of databases: Common database systems MySQL, Postgres and SQLite and -more exotic systems like SAP DB, SyBase and MSSQL. It only supports Python 2 +variety of databases: common database systems like MySQL, PostgreSQL, and SQLite and +more exotic systems like SAP DB, SyBase, and Microsoft SQL Server. It only supports Python 2 from Python 2.6 upwards. -.. There's no official information on this on their page, this information was gathered by looking at their source code +.. There's no official information on this on their page; this information was gathered by looking at their source code. diff --git a/docs/scenarios/gui.rst b/docs/scenarios/gui.rst index 28f1a9cd8..3ad0af364 100644 --- a/docs/scenarios/gui.rst +++ b/docs/scenarios/gui.rst @@ -1,105 +1,162 @@ + +################ GUI Applications -================ +################ .. image:: /_static/photos/33907143624_cd621b535c_k_d.jpg Alphabetical list of GUI Applications. + +******* Camelot -------- +******* + `Camelot `_ provides components for building -applications on top of Python, SQLAlchemy and Qt. It is inspired by -the Django admin interface. +applications on top of Python, SQLAlchemy, and Qt. It is inspired by the Django +admin interface. The main resource for information is the website: http://www.python-camelot.com -and the mailing list https://groups.google.com/forum/#!forum/project-camelot +and the mailing list https://groups.google.com/forum/#!forum/project-camelot. + +***** Cocoa ------ +***** + .. note:: The Cocoa framework is only available on OS X. Don't pick this if you're writing a cross-platform application! + +*** GTk ---- -PyGTK provides Python bindings for the GTK+ toolkit. Like the GTK+ library -itself, it is currently licensed under the GNU LGPL. It is worth noting that -PyGTK only currently supports the Gtk-2.X API (NOT Gtk-3.0). It is currently -recommended that PyGTK not be used for new projects and that existing -applications be ported from PyGTK to PyGObject. +*** + +.. note:: PyGTK provides Python bindings for the GTK+ toolkit. However, it has been superseded by PyGObject. PyGTK should not be used for new projects and existing projects should be ported to PyGObject. + +******************** PyGObject aka (PyGi) --------------------- -`PyGObject `_ provides Python bindings, which gives access to the entire GNOME software platform. -It is fully compatible with GTK+ 3. Here is a tutorial to get started with `Python GTK+ 3 Tutorial `_. +******************** + +`PyGObject `_ provides Python +bindings which gives access to the entire GNOME software platform. It is fully +compatible with GTK+ 3. Here is a tutorial to get started with `Python GTK+ 3 +Tutorial `_. `API Reference `_ + +**** Kivy ----- +**** + `Kivy `_ is a Python library for development of multi-touch enabled media rich applications. The aim is to allow for quick and easy -interaction design and rapid prototyping, while making your code reusable -and deployable. +interaction design and rapid prototyping, while making your code reusable and +deployable. -Kivy is written in Python, based on OpenGL and supports different input devices -such as: Mouse, Dual Mouse, TUIO, WiiMote, WM_TOUCH, HIDtouch, Apple's products +Kivy is written in Python, based on OpenGL, and supports different input devices +such as: Mouse, Dual Mouse, TUIO, WiiMote, WM_TOUCH, HIDtouch, Apple's products, and so on. Kivy is actively being developed by a community and is free to use. It operates -on all major platforms (Linux, OSX, Windows, Android). +on all major platforms (Linux, OS X, Windows, Android). The main resource for information is the website: http://kivy.org + +****** PyObjC ------- +****** + .. note:: Only available on OS X. Don't pick this if you're writing a cross-platform application. + +****** PySide ------- +****** + PySide is a Python binding of the cross-platform GUI toolkit Qt. +The package name depends on the major Qt version (`PySide` for Qt4, +`PySide2` for Qt5, and `PySide6` for Qt6). +This set of bindings is developed by `The Qt Company `_. + +.. code-block:: console + + $ pip install pyside6 - pip install pyside +https://pyside.org -https://wiki.qt.io/Category:LanguageBindings::PySide::Downloads +**** PyQt ----- +**** + .. note:: If your software does not fully comply with the GPL you will need a commercial license! PyQt provides Python bindings for the Qt Framework (see below). http://www.riverbankcomputing.co.uk/software/pyqt/download -PyjamasDesktop (pyjs Desktop) ------------------------------ -PyjamasDesktop is a port of Pyjamas. PyjamasDesktop is application widget set -for desktop and a cross-platform framework. (After release v0.6 PyjamasDesktop -is a part of Pyjamas (Pyjs)). Briefly, it allows the exact same Python web -application source code to be executed as a standalone desktop application. -`Python Wiki for PyjamasDesktop `_. +*************************************** +Pyjs Desktop (formerly Pyjamas Desktop) +*************************************** + +Pyjs Desktop is a application widget set for desktop and a cross-platform +framework. It allows the exact same Python web application source code to be +executed as a standalone desktop application. + -The main website; `pyjs Desktop `_. +The main website: `pyjs `_. + +** Qt --- -`Qt `_ is a cross-platform application framework that -is widely used for developing software with a GUI but can also be used for -non-GUI applications. +** + +`Qt `_ is a cross-platform application framework that is +widely used for developing software with a GUI but can also be used for non-GUI +applications. + + +*********** +PySimpleGUI +*********** + +`PySimpleGUI `_ is a wrapper for Tkinter +and Qt (others on the way). The amount of code required to implement custom +GUIs is much shorter using PySimpleGUI than if the same GUI were written +directly using Tkinter or Qt. PySimpleGUI code can be "ported" between GUI +frameworks by changing import statements. + +.. code-block:: console + $ pip install pysimplegui + +PySimpleGUI is contained in a single PySimpleGUI.py file. Should pip +installation be impossible, copying the PySimpleGUI.py file into a project's +folder is all that's required to import and begin using. + + +**** Toga ----- -`Toga `_ is a Python native, OS -native, cross platform GUI toolkit. Toga consists of a library of base -components with a shared interface to simplify platform-agnostic GUI -development. +**** + +`Toga `_ is a Python native, OS native, +cross platform GUI toolkit. Toga consists of a library of base components with a +shared interface to simplify platform-agnostic GUI development. -Toga is available on Mac OS, Windows, Linux (GTK), and mobile platforms such -as Android and iOS. +Toga is available on macOS, Windows, Linux (GTK), and mobile platforms such as +Android and iOS. + +** Tk --- +** + Tkinter is a thin object-oriented layer on top of Tcl/Tk. **It has the advantage of being included with the Python standard library, making it the most convenient and compatible toolkit to program with.** @@ -108,18 +165,21 @@ Both Tk and Tkinter are available on most Unix platforms, as well as on Windows and Macintosh systems. Starting with the 8.0 release, Tk offers native look and feel on all platforms. -There's a good multi-language Tk tutorial with Python examples at -`TkDocs `_. There's more information +There's a good multi-language Tk tutorial with Python examples at `TkDocs +`_. There's more information available on the `Python Wiki `_. + +******** wxPython --------- -wxPython is a GUI toolkit for the Python programming language. It allows -Python programmers to create programs with a robust, highly functional -graphical user interface, simply and easily. It is implemented as a Python -extension module (native code) that wraps the popular wxWidgets cross platform -GUI library, which is written in C++. +******** + +wxPython is a GUI toolkit for the Python programming language. It allows Python +programmers to create programs with a robust, highly functional graphical user +interface, simply and easily. It is implemented as a Python extension module +(native code) that wraps the popular wxWidgets cross platform GUI library, which +is written in C++. **Install (Stable) wxPython** -*go to http://www.wxpython.org/download.php#stable and download the appropriate +*go to https://www.wxpython.org/pages/downloads/ and download the appropriate package for your OS.* diff --git a/docs/scenarios/imaging.rst b/docs/scenarios/imaging.rst index ff620808e..8fe7e08fe 100644 --- a/docs/scenarios/imaging.rst +++ b/docs/scenarios/imaging.rst @@ -1,25 +1,28 @@ -================== + +################## Image Manipulation -================== +################## .. image:: /_static/photos/34575689432_3de8e9a348_k_d.jpg Most image processing and manipulation techniques can be carried out -effectively using two libraries: Python Imaging Library (PIL) and OpenSource +effectively using two libraries: Python Imaging Library (PIL) and Open Source Computer Vision (OpenCV). A brief description of both is given below. + +********************** Python Imaging Library ----------------------- +********************** The `Python Imaging Library `_, or PIL for short, is one of the core libraries for image manipulation in Python. Unfortunately, its development has stagnated, with its last release in 2009. Luckily for you, there's an actively-developed fork of PIL called -`Pillow `_ - it's easier to install, runs on -all operating systems, and supports Python 3. +`Pillow `_ -- it's easier to install, runs on +all major operating systems, and supports Python 3. Installation ~~~~~~~~~~~~ @@ -59,13 +62,14 @@ Example exif_data There are more examples of the Pillow library in the -`Pillow tutorial `_. +`Pillow tutorial `_. -OpenSource Computer Vision --------------------------- +*************************** +Open Source Computer Vision +*************************** -OpenSource Computer Vision, more commonly known as OpenCV, is a more advanced +Open Source Computer Vision, more commonly known as OpenCV, is a more advanced image manipulation and processing software than PIL. It has been implemented in several languages and is widely used. @@ -105,4 +109,4 @@ Example There are more Python-implemented examples of OpenCV in this `collection of tutorials -`_. +`_. diff --git a/docs/scenarios/json.rst b/docs/scenarios/json.rst index d555067db..1c3663777 100644 --- a/docs/scenarios/json.rst +++ b/docs/scenarios/json.rst @@ -1,14 +1,18 @@ + +#### JSON -==== +#### .. image:: /_static/photos/33928819683_97b5c6a184_k_d.jpg -The `json `_ library can parse +The `json `_ library can parse JSON from strings or files. The library parses JSON into a Python dictionary or list. It can also convert Python dictionaries or lists into JSON strings. + +************ Parsing JSON ------------- +************ Take the following string containing JSON data: @@ -42,26 +46,3 @@ You can also convert the following to JSON: print(json.dumps(d)) '{"first_name": "Guido", "last_name": "Rossum", "titles": ["BDFL", "Developer"]}' - - -simplejson ----------- - -The JSON library was added to Python in version 2.6. -If you're using an earlier version of Python, the -`simplejson `_ library is -available via PyPI. - -simplejson mimics the json standard library. It is available so that developers -that use older versions of Python can use the latest features available in the -json lib. - -You can start using simplejson when the json library is not available by -importing simplejson under a different name: - -.. code-block:: python - - import simplejson as json - -After importing simplejson as json, the above examples will all work as if you -were using the standard json library. diff --git a/docs/scenarios/ml.rst b/docs/scenarios/ml.rst index 30eefcbdf..b3655c819 100644 --- a/docs/scenarios/ml.rst +++ b/docs/scenarios/ml.rst @@ -1,17 +1,21 @@ -================ + + +################ Machine Learning -================ +################ .. image:: /_static/photos/34018729885_002ced9b54_k_d.jpg -Python has a vast number of libraries for data analysis, statistics and Machine Learning itself, making it a language of choice for many data scientists. +Python has a vast number of libraries for data analysis, statistics, and Machine Learning itself, making it a language of choice for many data scientists. -Some widely used packages for Machine Learning and other Data Science applications are enlisted below. +Some widely used packages for Machine Learning and other data science applications are listed below. -Scipy Stack ------------ -The Scipy stack consists of a bunch of core helper packages used in data science, for statistical analysis and visualising data. Because of its huge number of functionalities and ease of use, the Stack is considered a must-have for most data science applications. +*********** +SciPy Stack +*********** + +The SciPy stack consists of a bunch of core helper packages used in data science for statistical analysis and visualising data. Because of its huge number of functionalities and ease of use, the Stack is considered a must-have for most data science applications. The Stack consists of the following packages (link to documentation given): @@ -32,12 +36,14 @@ For installing the full stack, or individual packages, you can refer to the inst **NB:** `Anaconda `_ is highly preferred and recommended for installing and maintaining data science packages seamlessly. + +************ scikit-learn ------------- +************ -Scikit is a free and open-source machine learning library for Python. It offers off-the-shelf functions to implement many algorithms like linear regression, classifiers, SVMs, k-means, Neural Networks etc. It also has a few sample datasets which can be directly used for training and testing. +Scikit is a free and open source machine learning library for Python. It offers off-the-shelf functions to implement many algorithms like linear regression, classifiers, SVMs, k-means, Neural Networks, etc. It also has a few sample datasets which can be directly used for training and testing. -Because of its speed, robustness and easiness to use, it's one of the most widely-used libraries for many Machine Learning applications. +Because of its speed, robustness, and ease of, it's one of the most widely-used libraries for many Machine Learning applications. Installation ~~~~~~~~~~~~ @@ -54,16 +60,16 @@ Through conda: conda install scikit-learn -scikit-learn also comes in shipped with Anaconda (mentioned above). For more installation instructions, refer to `this link `_. +scikit-learn also comes shipped with Anaconda (mentioned above). For more installation instructions, refer to `this link `_. Example ~~~~~~~ For this example, we train a simple classifier on the `Iris dataset `_, which comes bundled in with scikit-learn. -The dataset takes four features of flowers: sepal length, sepal width, petal length and petal width, and classifies them into three flower species (labels): setosa, versicolor or virginica. The labels have been represented as numbers in the dataset: 0 (setosa), 1 (versicolor) and 2 (virginica). +The dataset takes four features of flowers: sepal length, sepal width, petal length, and petal width, and classifies them into three flower species (labels): setosa, versicolor, or virginica. The labels have been represented as numbers in the dataset: 0 (setosa), 1 (versicolor), and 2 (virginica). -We shuffle the Iris dataset, and divide it into separate training and testing sets: keeping the last 10 data points for testing and rest for training. We then train the classifier on the training set, and predict on the testing set. +We shuffle the Iris dataset and divide it into separate training and testing sets, keeping the last 10 data points for testing and rest for training. We then train the classifier on the training set and predict on the testing set. .. code-block:: python @@ -114,6 +120,6 @@ Since we're splitting randomly and the classifier trains on every iteration, the [0 1 1 1 0 2 0 2 2 2] 100.0 -The first line contains the labels (i.e flower species) of the testing data as predicted by our classifier, and the second line contains the actual flower species as given in the dataset. We thus get an accuracy of 100% this time. +The first line contains the labels (i.e. flower species) of the testing data as predicted by our classifier, and the second line contains the actual flower species as given in the dataset. We thus get an accuracy of 100% this time. More on scikit-learn can be read in the `documentation `_. diff --git a/docs/scenarios/network.rst b/docs/scenarios/network.rst index 1cea42dd5..bb751b78e 100644 --- a/docs/scenarios/network.rst +++ b/docs/scenarios/network.rst @@ -1,22 +1,28 @@ + +########## Networking -========== +########## .. image:: /_static/photos/34151833832_6bdfd930af_k_d.jpg + +******* Twisted -------- +******* -`Twisted `_ is an event-driven networking +`Twisted `_ is an event-driven networking engine. It can be used to build applications around many different networking -protocols, including http servers and clients, applications using SMTP, POP3, -IMAP or SSH protocols, instant messaging -and `much more `_. +protocols, including HTTP servers and clients, applications using SMTP, POP3, +IMAP, or SSH protocols, instant messaging, +and `much more `_. + +***** PyZMQ ------ +***** -`PyZMQ `_ is the Python binding for -`ZeroMQ `_, which is a high-performance asynchronous +`PyZMQ `_ is the Python binding for +`ZeroMQ `_, which is a high-performance asynchronous messaging library. One great advantage of ZeroMQ is that it can be used for message queuing without a message broker. The basic patterns for this are: @@ -24,14 +30,16 @@ message queuing without a message broker. The basic patterns for this are: remote procedure call and task distribution pattern. - publish-subscribe: connects a set of publishers to a set of subscribers. This is a data distribution pattern. -- push-pull (or pipeline): connects nodes in a fan-out / fan-in pattern that - can have multiple steps, and loops. This is a parallel task distribution +- push-pull (or pipeline): connects nodes in a fan-out/fan-in pattern that + can have multiple steps and loops. This is a parallel task distribution and collection pattern. For a quick start, read the `ZeroMQ guide `_. + +****** gevent ------- +****** `gevent `_ is a coroutine-based Python networking library that uses greenlets to provide a high-level synchronous API on top of diff --git a/docs/scenarios/scientific.rst b/docs/scenarios/scientific.rst index 7276420d3..bb0547323 100644 --- a/docs/scenarios/scientific.rst +++ b/docs/scenarios/scientific.rst @@ -1,11 +1,14 @@ -======================= + +####################### Scientific Applications -======================= +####################### .. image:: /_static/photos/33925223870_97e44f5629_k_d.jpg + +******* Context -::::::: +******* Python is frequently used for high-performance scientific applications. It is widely used in academia and scientific projects because it is easy to write @@ -13,15 +16,16 @@ and performs well. Due to its high performance nature, scientific computing in Python often utilizes external libraries, typically written in faster languages (like C, or -FORTRAN for matrix operations). The main libraries used are `NumPy`_, `SciPy`_ +Fortran for matrix operations). The main libraries used are `NumPy`_, `SciPy`_ and `Matplotlib`_. Going into detail about these libraries is beyond the scope of the Python guide. However, a comprehensive introduction to the scientific Python ecosystem can be found in the `Python Scientific Lecture Notes -`_ +`_. +***** Tools -::::: +***** IPython ------- @@ -31,19 +35,20 @@ which provides features of great interest to scientists. The `inline mode` allows graphics and plots to be displayed in the terminal (Qt based version). Moreover, the `notebook` mode supports literate programming and reproducible science generating a web-based Python notebook. This notebook allows you to -store chunks of Python code along side the results and additional comments +store chunks of Python code alongside the results and additional comments (HTML, LaTeX, Markdown). The notebook can then be shared and exported in various file formats. +********* Libraries -::::::::: +********* NumPy ----- `NumPy `_ is a low level library written in C (and -FORTRAN) for high level mathematical functions. NumPy cleverly overcomes the +Fortran) for high level mathematical functions. NumPy cleverly overcomes the problem of running slower algorithms on Python by using multidimensional arrays and functions that operate on arrays. Any algorithm can then be expressed as a function on arrays, allowing the algorithms to be run quickly. @@ -52,7 +57,7 @@ NumPy is part of the SciPy project, and is released as a separate library so people who only need the basic requirements can use it without installing the rest of SciPy. -NumPy is compatible with Python versions 2.4 through to 2.7.2 and 3.1+. +NumPy is compatible with Python versions 2.4 through 2.7.2 and 3.1+. Numba ----- @@ -70,7 +75,7 @@ SciPy functions. SciPy uses NumPy arrays as the basic data structure, and comes with modules for various commonly used tasks in scientific programming, including linear algebra, integration (calculus), ordinary differential equation -solving and signal processing. +solving, and signal processing. Matplotlib ---------- @@ -79,20 +84,31 @@ Matplotlib library for creating interactive 2D and 3D plots that can also be saved as manuscript-quality figures. The API in many ways reflects that of `MATLAB `_, easing transition of MATLAB -users to Python. Many examples, along with the source code to re-create them, +users to Python. Many examples, along with the source code to recreate them, are available in the `matplotlib gallery `_. Pandas ------ -`Pandas `_ is data manipulation library -based on Numpy which provides many useful functions for accessing, -indexing, merging and grouping data easily. The main data structure (DataFrame) +`Pandas `_ is a data manipulation library +based on NumPy which provides many useful functions for accessing, +indexing, merging, and grouping data easily. The main data structure (DataFrame) is close to what could be found in the R statistical package; that is, -heterogeneous data tables with name indexing, time series operations and +heterogeneous data tables with name indexing, time series operations, and auto-alignment of data. +xarray +------ + +`xarray `_ is similar to Pandas, but it +is intended for wrapping multidimensional scientific data. By labelling the +data with dimensions, coordinates, and attributes, it makes complex +multidimensional operations clearer and more intuitive. It also wraps +matplotlib for quick plotting, and can apply most operations in parallel using +`dask `_. + + Rpy2 ---- @@ -107,12 +123,13 @@ PsychoPy `PsychoPy `_ is a library for cognitive scientists allowing the creation of cognitive psychology and neuroscience experiments. -The library handles presentation of stimuli, scripting of experimental design +The library handles presentation of stimuli, scripting of experimental design, and data collection. +********* Resources -::::::::: +********* Installation of scientific Python packages can be troublesome, as many of these packages are implemented as Python C extensions which need to be compiled. @@ -125,7 +142,7 @@ Unofficial Windows Binaries for Python Extension Packages Many people who do scientific computing are on Windows, yet many of the scientific computing packages are notoriously difficult to build and install on -this platform. `Christoph Gohlke `_ +this platform. `Christoph Gohlke `_, however, has compiled a list of Windows binaries for many useful Python packages. The list of packages has grown from a mainly scientific Python resource to a more general list. If you're on Windows, you may want to check it @@ -134,17 +151,16 @@ out. Anaconda -------- -`Continuum Analytics `_ offers the `Anaconda -Python Distribution `_ which +The `Anaconda Python Distribution `_ includes all the common scientific Python packages as well as many packages -related to data analytics and big data. Anaconda itself is free, and -Continuum sells a number of proprietary add-ons. Free licenses for the +related to data analytics and big data. Anaconda itself is free, and a number +of proprietary add-ons are available for a fee. Free licenses for the add-ons are available for academics and researchers. Canopy ------ -`Canopy `_ is another scientific +`Canopy `_ is another scientific Python distribution, produced by `Enthought `_. A limited 'Canopy Express' variant is available for free, but Enthought charges for the full distribution. Free licenses are available for academics. diff --git a/docs/scenarios/scrape.rst b/docs/scenarios/scrape.rst index 01d56f4a3..527719200 100644 --- a/docs/scenarios/scrape.rst +++ b/docs/scenarios/scrape.rst @@ -1,28 +1,34 @@ + +############# HTML Scraping -============= +############# .. image:: /_static/photos/34268661876_442428e122_k_d.jpg + +************ Web Scraping ------------- +************ Web sites are written using HTML, which means that each web page is a structured document. Sometimes it would be great to obtain some data from them and preserve the structure while we're at it. Web sites don't always -provide their data in comfortable formats such as ``csv`` or ``json``. +provide their data in comfortable formats such as CSV or JSON. This is where web scraping comes in. Web scraping is the practice of using a computer program to sift through a web page and gather the data that you need in a format most useful to you while at the same time preserving the structure of the data. + +***************** lxml and Requests ------------------ +***************** `lxml `_ is a pretty extensive library written for parsing XML and HTML documents very quickly, even handling messed up tags in the process. We will also be using the -`Requests `_ module instead of the +`Requests `_ module instead of the already built-in urllib2 module due to improvements in speed and readability. You can easily install both using ``pip install lxml`` and ``pip install requests``. @@ -35,7 +41,7 @@ Let's start with the imports: import requests Next we will use ``requests.get`` to retrieve the web page with our data, -parse it using the ``html`` module and save the results in ``tree``: +parse it using the ``html`` module, and save the results in ``tree``: .. code-block:: python @@ -56,10 +62,10 @@ HTML or XML documents. A good introduction to XPath is on There are also various tools for obtaining the XPath of elements such as FireBug for Firefox or the Chrome Inspector. If you're using Chrome, you can right click an element, choose 'Inspect element', highlight the code, -right click again and choose 'Copy XPath'. +right click again, and choose 'Copy XPath'. After a quick analysis, we see that in our page the data is contained in -two elements - one is a div with title 'buyer-name' and the other is a +two elements -- one is a div with title 'buyer-name' and the other is a span with class 'item-price': .. code-block:: html @@ -81,8 +87,8 @@ Let's see what we got exactly: .. code-block:: python - print 'Buyers: ', buyers - print 'Prices: ', prices + print('Buyers: ', buyers) + print('Prices: ', prices) :: diff --git a/docs/scenarios/serialization.rst b/docs/scenarios/serialization.rst index 5d5f41517..3edc6aaa2 100644 --- a/docs/scenarios/serialization.rst +++ b/docs/scenarios/serialization.rst @@ -1,20 +1,213 @@ -================== + +################## Data Serialization -================== +################## .. image:: /_static/photos/33467946364_3e59bd376a_k_d.jpg + +*************************** What is data serialization? ---------------------------- +*************************** + +Data serialization is the process of converting structured data to a format +that allows sharing or storage of the data in a form that allows recovery of its original +structure. In some cases, the secondary intention of data +serialization is to minimize the data's size which then +reduces disk space or bandwidth requirements. + +******************** +Flat vs. Nested data +******************** + +Before beginning to serialize data, it is important to identify or decide how the +data should be structured during data serialization - flat or nested. +The differences in the two styles are shown in the below examples. + +Flat style: + +.. code-block:: python + + { "Type" : "A", "field1": "value1", "field2": "value2", "field3": "value3" } + + +Nested style: + +.. code-block:: python + + {"A" + { "field1": "value1", "field2": "value2", "field3": "value3" } } + + +For more reading on the two styles, please see the discussion on +`Python mailing list `__, +`IETF mailing list `__ and +`in stackexchange `__. + +**************** +Serializing Text +**************** + +======================= +Simple file (flat data) +======================= + +If the data to be serialized is located in a file and contains flat data, Python offers two methods to serialize data. + +repr +---- + +The repr method in Python takes a single object parameter and returns a printable representation of the input: + +.. code-block:: python + + # input as flat text + a = { "Type" : "A", "field1": "value1", "field2": "value2", "field3": "value3" } + + # the same input can also be read from a file + a = open('/tmp/file.py', 'r') + + # returns a printable representation of the input; + # the output can be written to a file as well + print(repr(a)) + + # write content to files using repr + with open('/tmp/file.py') as f:f.write(repr(a)) + + +ast.literal_eval +---------------- + +The literal_eval method safely parses and evaluates an expression for a Python datatype. +Supported data types are: strings, numbers, tuples, lists, dicts, booleans, and None. + +.. code-block:: python + + with open('/tmp/file.py', 'r') as f: inp = ast.literal_eval(f.read()) + +==================== +CSV file (flat data) +==================== + +The CSV module in Python implements classes to read and write tabular +data in CSV format. + +Simple example for reading: + +.. code-block:: python + + # Reading CSV content from a file + import csv + with open('/tmp/file.csv', newline='') as f: + reader = csv.reader(f) + for row in reader: + print(row) + +Simple example for writing: -Data serialization is the concept of converting structured data into a format -that allows it to be shared or stored in such a way that its original -structure to be recovered. In some cases, the secondary intention of data -serialization is to minimize the size of the serialized data which then -minimizes disk space or bandwidth requirements. +.. code-block:: python + + # Writing CSV content to a file + import csv + with open('/temp/file.csv', 'w', newline='') as f: + writer = csv.writer(f) + writer.writerows(iterable) + + +The module's contents, functions, and examples can be found +`in the Python documentation `__. + +================== +YAML (nested data) +================== + +There are many third party modules to parse and read/write YAML file +structures in Python. One such example is below. + +.. code-block:: python + + # Reading YAML content from a file using the load method + import yaml + with open('/tmp/file.yaml', 'r', newline='') as f: + try: + print(yaml.load(f)) + except yaml.YAMLError as ymlexcp: + print(ymlexcp) + +Documentation on the third party module can be found +`in the PyYAML Documentation `__. + +======================= +JSON file (nested data) +======================= + +Python's JSON module can be used to read and write JSON files. +Example code is below. + +Reading: + +.. code-block:: python + + # Reading JSON content from a file + import json + with open('/tmp/file.json', 'r') as f: + data = json.load(f) + +Writing: + +.. code-block:: python -Pickle ------- + # Writing JSON content to a file using the dump method + import json + with open('/tmp/file.json', 'w') as f: + json.dump(data, f, sort_keys=True) + +================= +XML (nested data) +================= + +XML parsing in Python is possible using the `xml` package. + +Example: + +.. code-block:: python + + # reading XML content from a file + import xml.etree.ElementTree as ET + tree = ET.parse('country_data.xml') + root = tree.getroot() + +More documentation on using the `xml.dom` and `xml.sax` packages can be found +`in the Python XML library documentation `__. + + +******* +Binary +******* + +======================= +NumPy Array (flat data) +======================= + +Python's NumPy array can be used to serialize and deserialize data to and from byte representation. + +Example: + +.. code-block:: python + + import NumPy as np + + # Converting NumPy array to byte format + byte_output = np.array([ [1, 2, 3], [4, 5, 6], [7, 8, 9] ]).tobytes() + + # Converting byte format back to NumPy array + array_format = np.frombuffer(byte_output) + + + +==================== +Pickle (nested data) +==================== The native data serialization module for Python is called `Pickle `_. @@ -34,8 +227,10 @@ Here's an example: #Use loads to de-serialize an object received_grades = pickle.loads( serial_grades ) + +******** Protobuf --------- +******** If you're looking for a serialization module that has support in multiple languages, Google's `Protobuf diff --git a/docs/scenarios/speed.rst b/docs/scenarios/speed.rst index dcb141a71..5dc0cd845 100644 --- a/docs/scenarios/speed.rst +++ b/docs/scenarios/speed.rst @@ -1,5 +1,7 @@ + +##### Speed -===== +##### .. image:: /_static/photos/33175625804_e225b90f3e_k_d.jpg @@ -35,8 +37,10 @@ and PyPy's processing. 1.54693889618 1.60109114647 + +******* Context -::::::: +******* The GIL @@ -63,14 +67,16 @@ The GIL `Special care`_ must be taken when writing C extensions to make sure you register your threads with the interpreter. + +************ C Extensions -:::::::::::: +************ Cython ------ -`Cython `_ implements a superset of the Python language +`Cython `_ implements a superset of the Python language with which you are able to write C and C++ modules for Python. Cython also allows you to call functions from compiled C libraries. Using Cython allows you to take advantage of Python's strong typing of variables and operations. @@ -162,25 +168,23 @@ What's the difference in speed? Let's try it! .. code-block:: python import time - #activate pyx compiler + # Activate pyx compiler import pyximport - pyximport.install() - #primes implemented with Cython - import primesCy - #primes implemented with Python - import primes - - print "Cython:" - t1= time.time() - print primesCy.primes(500) - t2= time.time() - print "Cython time: %s" %(t2-t1) - print "" - print "Python" - t1= time.time() - print primes.primes(500) - t2= time.time() - print "Python time: %s" %(t2-t1) + pyximport.install() + import primesCy # primes implemented with Cython + import primes # primes implemented with Python + + print("Cython:") + t1 = time.time() + print(primesCy.primes(500)) + t2 = time.time() + print("Cython time: %s" % (t2 - t1)) + print("") + print("Python") + t1 = time.time() + print(primes.primes(500)) + t2 = time.time() + print("Python time: %s" % (t2 - t1)) These lines both need a remark: @@ -194,8 +198,8 @@ These lines both need a remark: The `pyximport` module allows you to import :file:`*.pyx` files (e.g., :file:`primesCy.pyx`) with the Cython-compiled version of the `primes` function. The `pyximport.install()` command allows the Python interpreter to -start the Cython compiler directly to generate C-code, which is automatically -compiled to a :file:`*.so` C-library. Cython is then able to import this +start the Cython compiler directly to generate C code, which is automatically +compiled to a :file:`*.so` C library. Cython is then able to import this library for you in your Python code, easily and efficiently. With the `time.time()` function you are able to compare the time between these 2 different calls to find 500 prime numbers. On a standard notebook (dual core @@ -208,7 +212,7 @@ AMD E-450 1.6 GHz), the measured values are: Python time: 0.0566 seconds -And here the output of an embedded `ARM beaglebone `_ machine: +And here is the output of an embedded `ARM beaglebone `_ machine: .. code-block:: console @@ -224,8 +228,10 @@ Pyrex Shedskin? --------- + +*********** Concurrency -::::::::::: +*********** Concurrent.futures @@ -245,7 +251,7 @@ available. The ProcessPoolExecutor works in the same way, except instead of using multiple threads for its workers, it will use multiple processes. This makes it possible -to side-step the GIL, however because of the way things are passed to worker +to side-step the GIL; however, because of the way things are passed to worker processes, only picklable objects can be executed and returned. Because of the way the GIL works, a good rule of thumb is to use a @@ -256,7 +262,7 @@ executor when the task is computationally expensive. There are two main ways of executing things in parallel using the two Executors. One way is with the `map(func, iterables)` method. This works almost exactly like the builtin `map()` function, except it will execute -everything in parallel. : +everything in parallel. .. code-block:: python @@ -296,7 +302,7 @@ result() the scheduled callable returns by default. exception() Return the exception raised by the call. If no exception was raised then - this returns `None`. Note that this will block just like `result()`. + this returns None. Note that this will block just like `result()`. add_done_callback(fn) Attach a callback function that will be executed (as `fn(future)`) when the scheduled callable returns. @@ -349,14 +355,14 @@ futures provided have completed. For more information, on using the `concurrent.futures`_ module, consult the official documentation. -Threading +threading --------- The standard library comes with a `threading`_ module that allows a user to work with multiple threads manually. Running a function in another thread is as simple as passing a callable and -it's arguments to `Thread`'s constructor and then calling `start()`: +its arguments to `Thread`'s constructor and then calling `start()`: .. code-block:: python @@ -391,7 +397,7 @@ there might be situations where two or more threads are trying to write to the same resource at the same time or where the output is dependent on the sequence or timing of certain events. This is called a `data race`_ or race condition. When this happens, the output will be garbled or you may encounter problems -which are difficult to debug. A good example is this `stackoverflow post`_. +which are difficult to debug. A good example is this `Stack Overflow post`_. The way this can be avoided is by using a `Lock`_ that each thread needs to acquire before writing to a shared resource. Locks can be acquired and released @@ -440,14 +446,14 @@ Multiprocessing .. _`PyPy`: http://pypy.org -.. _`The GIL`: http://wiki.python.org/moin/GlobalInterpreterLock +.. _`The GIL`: https://wiki.python.org/moin/GlobalInterpreterLock .. _`guide`: http://www.dabeaz.com/python/UnderstandingGIL.pdf .. _`New GIL`: http://www.dabeaz.com/python/NewGIL.pdf -.. _`Special care`: http://docs.python.org/c-api/init.html#threads +.. _`Special care`: https://docs.python.org/c-api/init.html#threads .. _`David Beazley's`: http://www.dabeaz.com/GIL/gilvis/measure2.py .. _`concurrent.futures`: https://docs.python.org/3/library/concurrent.futures.html .. _`Future`: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future .. _`threading`: https://docs.python.org/3/library/threading.html -.. _`stackoverflow post`: http://stackoverflow.com/questions/26688424/python-threads-are-printing-at-the-same-time-messing-up-the-text-output +.. _`Stack Overflow post`: https://stackoverflow.com/questions/26688424/python-threads-are-printing-at-the-same-time-messing-up-the-text-output .. _`data race`: https://en.wikipedia.org/wiki/Race_condition .. _`Lock`: https://docs.python.org/3/library/threading.html#lock-objects diff --git a/docs/scenarios/web.rst b/docs/scenarios/web.rst index 923c9423b..a2c84818b 100644 --- a/docs/scenarios/web.rst +++ b/docs/scenarios/web.rst @@ -1,6 +1,7 @@ -============================= + +############################# Web Applications & Frameworks -============================= +############################# .. image:: /_static/photos/34309496175_b82d104282_k_d.jpg @@ -8,8 +9,10 @@ As a powerful scripting language adapted to both fast prototyping and bigger projects, Python is widely used in web application development. + +******* Context -::::::: +******* @@ -24,8 +27,9 @@ can be deployed in any :ref:`WSGI-compliant web server `. WSGI is documented in :pep:`3333`. +********** Frameworks -:::::::::: +********** Broadly speaking, a web framework consists of a set of libraries and a main handler within which you can build custom code to implement a web application @@ -37,7 +41,7 @@ URL Routing be invoked Request and Response Objects - Encapsulate the information received from or sent to a user's browser + Encapsulates the information received from or sent to a user's browser Template Engine Allows for separating Python code implementing an application's logic from @@ -51,7 +55,7 @@ Development Web Server Django ------ -`Django `_ is a "batteries included" web +`Django `_ is a "batteries included" web application framework, and is an excellent choice for creating content-oriented websites. By providing many utilities and patterns out of the box, Django aims to make it possible to build complex, database-backed web applications quickly, @@ -101,14 +105,14 @@ It is a reliable, high-performance Python web framework for building large-scale app backends and microservices. Falcon encourages the REST architectural style of mapping URIs to resources, trying to do as little as possible while remaining highly effective. -Falcon highlights four main focuses: speed, reliability, flexibility and debuggability. +Falcon highlights four main focuses: speed, reliability, flexibility, and debuggability. It implements HTTP through "responders" such as ``on_get()``, ``on_put()``, etc. These responders receive intuitive request and response objects. Tornado -------- -`Tornado `_ is an asyncronous web framework +`Tornado `_ is an asynchronous web framework for Python that has its own event loop. This allows it to natively support WebSockets, for example. Well-written Tornado applications are known to have excellent performance characteristics. @@ -122,15 +126,46 @@ Pyramid focus on modularity. It comes with a small number of libraries ("batteries") built-in, and encourages users to extend its base functionality. A set of provided cookiecutter templates helps making new project decisions for users. -It poweres one of the most important parts of python infrastucture -`PyPI `_. +It powers one of the most important parts of python infrastructure +`PyPI `_. Pyramid does not have a large user base, unlike Django and Flask. It's a capable framework, but not a very popular choice for new Python web applications today. +Masonite +-------- + +`Masonite `_ is a modern and developer centric, "batteries included", web framework. + +The Masonite framework follows the MVC (Model-View-Controller) architecture pattern and is heavily inspired by frameworks such as Rails and Laravel, so if you are coming to Python from a Ruby or PHP background then you will feel right at home! + +Masonite comes with a lot of functionality out of the box including a powerful IOC container with auto resolving dependency injection, craft command line tools, and the Orator active record style ORM. + +Masonite is perfect for beginners or experienced developers alike and works hard to be fast and easy from install through to deployment. Try it once and you’ll fall in love. + +FastAPI +------- + +`FastAPI `_ is a modern web framework for building +APIs with Python 3.6+. + +It has very high performance as it is based on `Starlette `_ +and `Pydantic `_. + +FastAPI takes advantage of standard Python type declarations in function parameters +to declare request parameters and bodies, perform data conversion (serialization, +parsing), data validation, and automatic API documentation with **OpenAPI 3** +(including **JSON Schema**). + +It includes tools and utilities for security and authentication (including OAuth2 with JWT +tokens), a dependency injection system, automatic generation of interactive API +documentation, and other features. + + +*********** Web Servers -::::::::::: +*********** .. _nginx-ref: @@ -138,7 +173,7 @@ Nginx ----- `Nginx `_ (pronounced "engine-x") is a web server and -reverse-proxy for HTTP, SMTP and other protocols. It is known for its +reverse-proxy for HTTP, SMTP, and other protocols. It is known for its high performance, relative simplicity, and compatibility with many application servers (like WSGI servers). It also includes handy features like load-balancing, basic authentication, streaming, and others. Designed @@ -147,20 +182,22 @@ to serve high-load websites, Nginx is gradually becoming quite popular. .. _wsgi-servers-ref: + +************ WSGI Servers -:::::::::::: +************ Stand-alone WSGI servers typically use less resources than traditional web -servers and provide top performance [3]_. +servers and provide top performance [1]_. .. _gunicorn-ref: Gunicorn -------- -`Gunicorn `_ (Green Unicorn) is a pure-python WSGI +`Gunicorn `_ (Green Unicorn) is a pure-Python WSGI server used to serve Python applications. Unlike other Python web servers, -it has a thoughtful user-interface, and is extremely easy to use and +it has a thoughtful user interface, and is extremely easy to use and configure. Gunicorn has sane and reasonable defaults for configurations. However, some @@ -173,7 +210,7 @@ Gunicorn is the recommended choice for new Python web applications today. Waitress -------- -`Waitress `_ is a pure-python WSGI server +`Waitress `_ is a pure-Python WSGI server that claims "very acceptable performance". Its documentation is not very detailed, but it does offer some nice functionality that Gunicorn doesn't have (e.g. HTTP request buffering). @@ -188,13 +225,13 @@ uWSGI `uWSGI `_ is a full stack for building hosting services. In addition to process management, process monitoring, and other functionality, uWSGI acts as an application server for various -programming languages and protocols - including Python and WSGI. uWSGI can +programming languages and protocols -- including Python and WSGI. uWSGI can either be run as a stand-alone web router, or be run behind a full web server (such as Nginx or Apache). In the latter case, a web server can configure uWSGI and an application's operation over the `uwsgi protocol `_. uWSGI's web server support allows for dynamically configuring -Python, passing environment variables and further tuning. For full details, +Python, passing environment variables, and further tuning. For full details, see `uWSGI magic variables `_. @@ -203,8 +240,9 @@ I do not recommend using uWSGI unless you know why you need it. .. _server-best-practices-ref: +********************* Server Best Practices -::::::::::::::::::::: +********************* The majority of self-hosted Python applications today are hosted with a WSGI server such as :ref:`Gunicorn `, either directly or behind a @@ -214,8 +252,10 @@ The WSGI servers serve the Python applications while the web server handles tasks better suited for it such as static file serving, request routing, DDoS protection, and basic authentication. + +******* Hosting -::::::: +******* Platform-as-a-Service (PaaS) is a type of cloud computing infrastructure which abstracts and manages infrastructure, routing, and scaling of web @@ -226,32 +266,27 @@ details. Heroku ------ -`Heroku `_ offers first-class support for +`Heroku `_ offers first-class support for Python 2.7–3.5 applications. Heroku supports all types of Python web applications, servers, and frameworks. Applications can be developed on Heroku for free. Once your application is ready for production, you can upgrade to a Hobby or Professional application. -Heroku maintains `detailed articles `_ +Heroku maintains `detailed articles `_ on using Python with Heroku, as well as `step-by-step instructions `_ on how to set up your first application. Heroku is the recommended PaaS for deploying Python web applications today. -Eldarion --------- - -`Eldarion `_ (formely known as Gondor) is a PaaS powered -by Kubernetes, CoreOS, and Docker. They support any WSGI application and have a -guide on deploying `Django projects `_. +********** Templating -:::::::::: +********** Most WSGI applications are responding to HTTP requests to serve content in HTML -or other markup languages. Instead of generating directly textual content from +or other markup languages. Instead of directly generating textual content from Python, the concept of separation of concerns advises us to use templates. A template engine manages a suite of template files, with a system of hierarchy and inclusion to avoid unnecessary repetition, and is in charge of rendering @@ -289,10 +324,10 @@ Jinja2 `Jinja2 `_ is a very well-regarded template engine. It uses a text-based template language and can thus be used to generate any -type markup, not just HTML. It allows customization of filters, tags, tests +type of markup, not just HTML. It allows customization of filters, tags, tests, and globals. It features many improvements over Django's templating system. -Here some important html tags in Jinja2: +Here some important HTML tags in Jinja2: .. code-block:: html @@ -312,7 +347,7 @@ Here some important html tags in Jinja2: {% endfor %} -The next listings is an example of a web site in combination with the Tornado +The next listings are an example of a web site in combination with the Tornado web server. Tornado is not very complicated to use. .. code-block:: python @@ -406,16 +441,16 @@ Chameleon --------- `Chameleon `_ Page Templates are an HTML/XML template -engine implementation of the `Template Attribute Language (TAL) `_, +engine implementation of the `Template Attribute Language (TAL) `_, `TAL Expression Syntax (TALES) `_, and `Macro Expansion TAL (Metal) `_ syntaxes. -Chameleon is available for Python 2.5 and up (including 3.x and pypy), and -is commonly used by the `Pyramid Framework `_. +Chameleon is available for Python 2.5 and up (including 3.x and PyPy), and +is commonly used by the `Pyramid Framework `_. Page Templates add within your document structure special element attributes and text markup. Using a set of simple language constructs, you control the -document flow, element repetition, text replacement and translation. Because +document flow, element repetition, text replacement, and translation. Because of the attribute-based syntax, unrendered page templates are valid HTML and can be viewed in a browser and even edited in WYSIWYG editors. This can make round-trip collaboration with designers and prototyping with static files in a @@ -469,7 +504,7 @@ Mako ---- `Mako `_ is a template language that compiles to Python -for maximum performance. Its syntax and api is borrowed from the best parts of other +for maximum performance. Its syntax and API are borrowed from the best parts of other templating languages like Django and Jinja2 templates. It is the default template language included with the `Pylons and Pyramid `_ web frameworks. @@ -507,6 +542,4 @@ Mako is well respected within the Python web community. .. rubric:: References -.. [1] `The mod_python project is now officially dead `_ -.. [2] `mod_wsgi vs mod_python `_ -.. [3] `Benchmark of Python WSGI Servers `_ +.. [1] `Benchmark of Python WSGI Servers `_ diff --git a/docs/scenarios/xml.rst b/docs/scenarios/xml.rst index 51a784d20..24dab5869 100644 --- a/docs/scenarios/xml.rst +++ b/docs/scenarios/xml.rst @@ -1,10 +1,14 @@ + +########### XML parsing -=========== +########### .. image:: /_static/photos/33888714601_a1f7d020a2_k_d.jpg + +******** untangle --------- +******** `untangle `_ is a simple library which takes an XML document and returns a Python object which mirrors the nodes and @@ -26,18 +30,20 @@ can be loaded like this: import untangle obj = untangle.parse('path/to/file.xml') -and then you can get the child elements name like this: +and then you can get the child element's name attribute like this: .. code-block:: python obj.root.child['name'] -untangle also supports loading XML from a string or an URL. +untangle also supports loading XML from a string or a URL. + +********* xmltodict ---------- +********* -`xmltodict `_ is another simple +`xmltodict `_ is another simple library that aims at making XML feel like working with JSON. An XML file like this: @@ -63,7 +69,7 @@ can be loaded into a Python dict like this: with open('path/to/file.xml') as fd: doc = xmltodict.parse(fd.read()) -and then you can access elements, attributes and values like this: +and then you can access elements, attributes, and values like this: .. code-block:: python @@ -73,5 +79,32 @@ and then you can access elements, attributes and values like this: doc['mydocument']['plus']['#text'] # == u'element as well' xmltodict also lets you roundtrip back to XML with the unparse function, -has a streaming mode suitable for handling files that don't fit in memory -and supports namespaces. +has a streaming mode suitable for handling files that don't fit in memory, +and supports XML namespaces. + +********** +xmlschema +********** + +`xmlschema `_ provides support for using XSD-Schemas in Python. +Unlike other XML libraries, automatic type parsing is available, so f.e. if the schema defines an element to be of type ``int``, the parsed ``dict`` will contain also an ``int`` value for that element. +Moreover the library supports automatic and explicit validation of XML documents against a schema. + +.. code-block:: python + + from xmlschema import XMLSchema, etree_tostring + + # load a XSD schema file + schema = XMLSchema("your_schema.xsd") + + # validate against the schema + schema.validate("your_file.xml") + + # or + schema.is_valid("your_file.xml") + + # decode a file + data = schmema.decode("your_file.xml") + + # encode to string + s = etree_tostring(schema.encode(data)) diff --git a/docs/shipping/freezing.rst b/docs/shipping/freezing.rst index 4514a8540..841fa2040 100644 --- a/docs/shipping/freezing.rst +++ b/docs/shipping/freezing.rst @@ -1,8 +1,9 @@ .. _freezing-your-code-ref: -================== + +################## Freezing Your Code -================== +################## .. image:: /_static/photos/33907151034_e0a9e53402_k_d.jpg @@ -22,12 +23,14 @@ Besides, end-user software should always be in an executable format. Files ending in ``.py`` are for software engineers and system administrators. One disadvantage of freezing is that it will increase the size of your -distribution by about 2–12MB. Also, you will be responsible for shipping +distribution by about 2–12 MB. Also, you will be responsible for shipping updated versions of your application when security vulnerabilities to Python are patched. + +************************ Alternatives to Freezing ------------------------- +************************ :ref:`Packaging your code ` is for distributing libraries or tools to other developers. @@ -39,33 +42,37 @@ On Linux, an alternative to freezing is to .. todo:: Fill in "Freezing Your Code" stub +**************************** Comparison of Freezing Tools ----------------------------- +**************************** +Date of this writing: Oct 5, 2019 Solutions and platforms/features supported: -=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== -Solution Windows Linux OS X Python 3 License One-file mode Zipfile import Eggs pkg_resources support -=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== -bbFreeze yes yes yes no MIT no yes yes yes -py2exe yes no no yes MIT yes yes no no -pyInstaller yes yes yes yes GPL yes no yes no -cx_Freeze yes yes yes yes PSF no yes yes no -py2app no no yes yes MIT no yes yes yes -=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== +=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== ===================== +Solution Windows Linux OS X Python 3 License One-file mode Zipfile import Eggs pkg_resources support Latest release date +=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== ===================== +bbFreeze yes yes yes no MIT no yes yes yes Jan 20, 2014 +py2exe yes no no yes MIT yes yes no no Oct 21, 2014 +pyInstaller yes yes yes yes GPL yes no yes no Jul 9, 2019 +cx_Freeze yes yes yes yes PSF no yes yes no Aug 29, 2019 +py2app no no yes yes MIT no yes yes yes Mar 25, 2019 +=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== ===================== .. note:: Freezing Python code on Linux into a Windows executable was only once - supported in PyInstaller `and later dropped. - `_. + supported in PyInstaller `and later dropped + `_. .. note:: - All solutions need MS Visual C++ dll to be installed on target machine, except py2app. - Only Pyinstaller makes self-executable exe that bundles the dll when + All solutions need a Microsoft Visual C++ to be installed on the target machine, except py2app. + Only PyInstaller makes a self-executable exe that bundles the appropriate DLL when passing ``--onefile`` to :file:`Configure.py`. + +******* Windows -------- +******* bbFreeze ~~~~~~~~ @@ -91,7 +98,7 @@ Prerequisite is to install :ref:`Python, Setuptools and pywin32 dependency on Wi .. note:: This will work for the most basic one file scripts. For more advanced freezing you will have to provide - include and exclude paths like so + include and exclude paths like so: .. code-block:: python @@ -103,8 +110,8 @@ Prerequisite is to install :ref:`Python, Setuptools and pywin32 dependency on Wi freezer.setIcon('my_awesome_icon.ico') -4. Provide the Microsoft Visual C runtime DLL for the freezer. It might be possible to append your :code:`sys.path` -with Microsoft Visual Studio path but I find it easier to drop :file:`msvcp90.dll` in the same folder where your script +4. Provide the Microsoft Visual C++ runtime DLL for the freezer. It might be possible to append your :code:`sys.path` +with the Microsoft Visual Studio path but I find it easier to drop :file:`msvcp90.dll` in the same folder where your script resides. 5. Freeze! @@ -116,7 +123,7 @@ resides. py2exe ~~~~~~ -Prerequisite is to install :ref:`Python on Windows `. The last release of py2exe is from the year 2014. There is not active development. +Prerequisite is to install :ref:`Python on Windows `. The last release of py2exe is from the year 2014. There is not active development. 1. Download and install http://sourceforge.net/projects/py2exe/files/py2exe/ @@ -133,7 +140,7 @@ Prerequisite is to install :ref:`Python on Windows `. The last 3. (Optionally) `include icon `_ -4. (Optionally) `one-file mode `_ +4. (Optionally) `one-file mode `_ 5. Generate :file:`.exe` into :file:`dist` directory: @@ -141,7 +148,7 @@ Prerequisite is to install :ref:`Python on Windows `. The last $ python setup.py py2exe -6. Provide the Microsoft Visual C runtime DLL. Two options: `globally install dll on target machine `_ or `distribute dll alongside with .exe `_. +6. Provide the Microsoft Visual C++ runtime DLL. Two options: `globally install dll on target machine `_ or `distribute dll alongside with .exe `_. PyInstaller ~~~~~~~~~~~ @@ -152,8 +159,9 @@ Prerequisite is to have installed :ref:`Python, Setuptools and pywin32 dependenc - `Manual `_ +**** OS X ----- +**** py2app @@ -176,19 +184,19 @@ To create a standard Unix executable, from say :code:`script.py`, use: $ pyinstaller script.py -This creates, +This creates: - a :code:`script.spec` file, analogous to a :code:`make` file - a :code:`build` folder, that holds some log files -- a :code:`dist` folder, that holds the main executable :code:`script`, and some dependent Python libraries, +- a :code:`dist` folder, that holds the main executable :code:`script`, and some dependent Python libraries all in the same folder as :code:`script.py`. PyInstaller puts all the Python libraries used in :code:`script.py` into the :code:`dist` folder, so when distributing the executable, distribute the whole :code:`dist` folder. -The :code:`script.spec` file can be edited to `customise the build `_, with options such as +The :code:`script.spec` file can be edited to `customise the build `_, with options such as: - bundling data files with the executable - including run-time libraries (:code:`.dll` or :code:`.so` files) that PyInstaller can't infer automatically -- adding Python run-time options to the executable, +- adding Python run-time options to the executable Now :code:`script.spec` can be run with :code:`pyinstaller` (instead of using :code:`script.py` again): @@ -196,7 +204,7 @@ Now :code:`script.spec` can be run with :code:`pyinstaller` (instead of using :c $ pyinstaller script.spec -To create a standalone windowed OS X application, use the :code:`--windowed` option +To create a standalone windowed OS X application, use the :code:`--windowed` option: .. code-block:: console @@ -207,12 +215,52 @@ This creates a :code:`script.app` in the :code:`dist` folder. Make sure to use G There are several options in :code:`script.spec` related to Mac OS X app bundles `here `_. For example, to specify an icon for the app, use the :code:`icon=\path\to\icon.icns` option. +***** Linux ------ +***** bbFreeze ~~~~~~~~ +.. warning:: bbFreeze will ONLY work in Python 2.x environment, since it's no longer being maintained as stated by it's former maintainer. If you're interested in it, check the repository in `here `_. + +bbFreeze can be used with all distributions that has Python installed along with pip2 and/or easy_install. + +For pip2, use the following: + +.. code-block:: console + + $ pip2 install bbfreeze + +Or, for easy_install: + +.. code-block:: console + + $ easy_install bbfreeze + +With bbFreeze installed, you're ready to freeze your applications. + +Let's assume you have a script, say, "hello.py" and a module called "module.py" and you have a function in it that's being used in your script. +No need to worry, you can just ask to freeze the main entrypoint of your script and it should freeze entirely: + +.. code-block:: console + + $ bbfreeze script.py + +With this, it creates a folder called dist/, of which contains the executable of the script and required .so (shared objects) files linked against libraries used within the Python script. + +Alternatively, you can create a script that does the freezing for you. An API for the freezer is available from the library within: + +.. code-block:: python + + from bbfreeze import Freezer + + freezer = Freezer(distdir='dist') + freezer.addScript('script.py', gui_only=True) # Enable gui_only kwarg for app that uses GUI packages. + freezer() PyInstaller ~~~~~~~~~~~ +PyInstaller can be used in a similar fashion as in OS X. The installation goes in the same manner as shown in the OS X section. + +Don't forget to have dependencies such as Python and pip installed for usage. diff --git a/docs/shipping/packaging.rst b/docs/shipping/packaging.rst index 79d2cc0c7..34674cff2 100644 --- a/docs/shipping/packaging.rst +++ b/docs/shipping/packaging.rst @@ -1,12 +1,13 @@ .. _packaging-your-code-ref: -=================== + +################### Packaging Your Code -=================== +################### .. image:: /_static/photos/36137234682_be6898bf57_k_d.jpg -Package your code to share it with other developers. For example +Package your code to share it with other developers. For example, to share a library for other developers to use in their application, or for development tools like 'py.test'. @@ -17,7 +18,7 @@ large, professional systems. It is a well-established convention for Python code to be shared this way. If your code isn't packaged on PyPI, then it will be harder -for other developers to find it, and to use it as part of their existing +for other developers to find it and to use it as part of their existing process. They will regard such projects with substantial suspicion of being either badly managed or abandoned. @@ -30,8 +31,10 @@ makes this method unsuitable for distributing applications to end-users. The `Python Packaging Guide `_ provides an extensive guide on creating and maintaining Python packages. + +************************* Alternatives to Packaging -::::::::::::::::::::::::: +************************* To distribute applications to end-users, you should :ref:`freeze your application `. @@ -40,10 +43,12 @@ On Linux, you may also want to consider :ref:`creating a Linux distro package ` (e.g. a .deb file for Debian or Ubuntu.) + +********************* For Python Developers -::::::::::::::::::::: +********************* -If you're writing an open source Python module, `PyPI `_ +If you're writing an open source Python module, `PyPI `_ , more properly known as *The Cheeseshop*, is the place to host it. @@ -51,15 +56,15 @@ If you're writing an open source Python module, `PyPI `_ Pip vs. easy_install -------------------- -Use `pip `_. More details -`here `_ +Use `pip `_. More details +`here `_. Personal PyPI ------------- -If you want to install packages from a source other than PyPI, (say, if -your packages are *proprietary*), you can do it by hosting a simple http +If you want to install packages from a source other than PyPI (say, if +your packages are *proprietary*), you can do it by hosting a simple HTTP server, running from the directory which holds those packages which need to be installed. @@ -78,11 +83,11 @@ Go to your command prompt and type: .. code-block:: console $ cd archive - $ python -m SimpleHTTPServer 9000 + $ python -m http.server 9000 -This runs a simple http server running on port 9000 and will list all packages +This runs a simple HTTP server running on port 9000 and will list all packages (like **MyPackage**). Now you can install **MyPackage** using any Python -package installer. Using Pip, you would do it like: +package installer. Using pip, you would do it like: .. code-block:: console @@ -90,7 +95,7 @@ package installer. Using Pip, you would do it like: Having a folder with the same name as the package name is **crucial** here. I got fooled by that, one time. But if you feel that creating a folder called -:file:`MyPackage` and keeping :file:`MyPackage.tar.gz` inside that, is +:file:`MyPackage` and keeping :file:`MyPackage.tar.gz` inside that is *redundant*, you can still install MyPackage using: .. code-block:: console @@ -100,7 +105,7 @@ I got fooled by that, one time. But if you feel that creating a folder called pypiserver ++++++++++ -`Pypiserver `_ is a minimal PyPI +`pypiserver `_ is a minimal PyPI compatible server. It can be used to serve a set of packages to easy_install or pip. It includes helpful features like an administrative command (``-U``) which will update all its packages to their latest versions @@ -110,7 +115,7 @@ found on PyPI. S3-Hosted PyPi ++++++++++++++ -One simple option for a personal PyPi server is to use Amazon S3. A +One simple option for a personal PyPI server is to use Amazon S3. A prerequisite for this is that you have an Amazon AWS account with an S3 bucket. 1. **Install all your requirements from PyPi or another source** @@ -125,8 +130,8 @@ prerequisite for this is that you have an Amazon AWS account with an S3 bucket. 4. **Upload the new files** -* Use a client like Cyberduck to sync the entire :file:`packages` folder to your s3 bucket -* Make sure you upload :code:`packages/simple/index.html` as well as all new files and directories +* Use a client like Cyberduck to sync the entire :file:`packages` folder to your s3 bucket. +* Make sure you upload :code:`packages/simple/index.html` as well as all new files and directories. 5. **Fix new file permissions** @@ -136,18 +141,20 @@ prerequisite for this is that you have an Amazon AWS account with an S3 bucket. 6. **All done** -* You can now install your package with :code:`pip install --index-url=http://your-s3-bucket/packages/simple/ YourPackage` +* You can now install your package with :code:`pip install --index-url=http://your-s3-bucket/packages/simple/ YourPackage`. .. _packaging-for-linux-distributions-ref: + +*********************** For Linux Distributions -:::::::::::::::::::::::: +*********************** Creating a Linux distro package is arguably the "right way" to distribute code on Linux. Because a distribution package doesn't include the Python interpreter, it -makes the download and install about 2MB smaller than +makes the download and install about 2-12 MB smaller than :ref:`freezing your application `. Also, if a distribution releases a new security update for Python, then your @@ -158,13 +165,13 @@ for use by distributions like Red Hat or SuSE trivially easy. However, creating and maintaining the different configurations required for each distribution's format (e.g. .deb for Debian/Ubuntu, .rpm for Red -Hat/Fedora, etc) is a fair amount of work. If your code is an application that +Hat/Fedora, etc.) is a fair amount of work. If your code is an application that you plan to distribute on other platforms, then you'll also have to create and maintain the separate config required to freeze your application for Windows -and OSX. It would be much less work to simply create and maintain a single +and OS X. It would be much less work to simply create and maintain a single config for one of the cross platform :ref:`freezing tools `, which will produce stand-alone executables for all -distributions of Linux, as well as Windows and OSX. +distributions of Linux, as well as Windows and OS X. Creating a distribution package is also problematic if your code is for a version of Python that isn't currently supported by a distribution. @@ -186,4 +193,4 @@ Useful Tools - `fpm `_ - `alien `_ -- `dh-virtualenv `_ (for APT/DEB omnibus packaging) +- `dh-virtualenv `_ (for APT/DEB omnibus packaging) diff --git a/docs/shipping/publishing.rst b/docs/shipping/publishing.rst new file mode 100644 index 000000000..4b480f801 --- /dev/null +++ b/docs/shipping/publishing.rst @@ -0,0 +1,66 @@ +.. _publishing-your-code-ref: + + +#################### +Publishing Your Code +#################### + +.. todo:: Replace this kitten with the photo we want. + +.. image:: https://placekitten.com/800/600 + +A healthy open source project needs a place to publish its code and project +management stuff so other developers can collaborate with you. This lets your +users gain a better understanding of your code, keep up with new developments, +report bugs, and contribute code. + +This development web site should include the source code history itself, a bug +tracker, a patch submission (aka "Pull Request") queue, and possibly additional +developer-oriented documentation. + +There are several free open source project hosting sites (aka "forges"). These +include GitHub, SourceForge, Bitbucket, and GitLab. GitHub is currently the best. +Use GitHub. + + +********************************* +Creating a Project Repo on GitHub +********************************* + +To publish your Python project on GitHub: + +1. Create a GitHub account if you don't already have one. + +2. Create a new repo for your project. + + 1. Click on the "+" menu next to your avatar in the upper right of the page and choose "New repository". + + 2. Name it after your project and give it an SEO-friendly description. + + 3. If you don't have an existing project repo, choose the settings to add a + README, `.gitignore`, and license. Use the Python `.gitignore` option. + +3. On the newly created repo page, click "Manage topics" and add the tags "python" and "python3" and/or "python2" as appropriate. + +4. Include a link to your new GitHub repo in your project's README file so people who just have the project distribution know where to find it. + +If this is a brand new repo, clone it to your local machine and start working: + +.. code-block:: console + + $ git clone https://github.com// + +Or, if you already have a project Git repo, add your new GitHub repo as a remote: + +.. code-block:: console + + $ cd + $ git remote add origin https://github.com// + $ git push --tags + +*********************** +When Your Project Grows +*********************** + +For more information about managing an open source software project, see the book +`Producing Open Source Software `_. diff --git a/docs/starting/install/linux.rst b/docs/starting/install/linux.rst index 65cff2abe..4198be8be 100644 --- a/docs/starting/install/linux.rst +++ b/docs/starting/install/linux.rst @@ -1,7 +1,9 @@ .. _install-linux: + +############################ Installing Python 2 on Linux -============================= +############################ .. image:: /_static/photos/34435688560_4cc2a7bcbb_k_d.jpg @@ -31,10 +33,12 @@ described in the next section before you start building Python applications for real-world use. In particular, you should always install Setuptools and pip, as it makes it much easier for you to use other third-party Python libraries. + +**************** Setuptools & Pip ----------------- +**************** -The two most crucial third-party Python packages are `setuptools `_ and `pip `_. +The two most crucial third-party Python packages are `setuptools `_ and `pip `_. Once installed, you can download, install and uninstall any compliant Python software product with a single command. It also enables you to add this network installation @@ -51,8 +55,10 @@ To see if pip is installed, open a command prompt and run To install pip, `follow the official pip installation guide `_ - this will automatically install the latest version of setuptools. + +******************** Virtual Environments --------------------- +******************** A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them. It solves the @@ -69,6 +75,5 @@ manage your virtual environments. -------------------------------- -This page is a remixed version of `another guide `_, +This page is a remixed version of `another guide `_, which is available under the same license. - diff --git a/docs/starting/install/osx.rst b/docs/starting/install/osx.rst index 5cd3db300..4a8a8e03b 100644 --- a/docs/starting/install/osx.rst +++ b/docs/starting/install/osx.rst @@ -1,16 +1,19 @@ .. _install-osx: + +############################### Installing Python 2 on Mac OS X -=============================== +############################### .. image:: /_static/photos/34435688560_4cc2a7bcbb_k_d.jpg .. note:: Check out our :ref:`guide for installing Python 3 on OS X`. -The latest version of Mac OS X, High Sierra, **comes with Python 2.7 out of the box**. +**Mac OS X comes with Python 2.7 out of the box between versions 10.8 and 12.3.** -You do not need to install or configure anything else to use Python. Having said +If your Mac OS X version is between the above versions, +you do not need to install or configure anything else to use Python. Having said that, I would strongly recommend that you install the tools and libraries described in the next section before you start building Python applications for real-world use. In particular, you should always install Setuptools, as it makes @@ -21,35 +24,37 @@ good for development. The version shipped with OS X may be out of date from the `official current Python release `_, which is considered the stable production version. + +************** Doing it Right --------------- +************** Let's install a real version of Python. Before installing Python, you'll need to install a C compiler. The fastest way is to install the Xcode Command Line Tools by running ``xcode-select --install``. You can also download the full version of -`Xcode `_ from the Mac App Store, or the +`Xcode `_ from the Mac App Store, or the minimal but unofficial `OSX-GCC-Installer `_ package. .. note:: - If you already have XCode installed, do not install OSX-GCC-Installer. + If you already have Xcode installed, do not install OSX-GCC-Installer. In combination, the software can cause issues that are difficult to diagnose. .. note:: - If you perform a fresh install of XCode, you will also need to add the + If you perform a fresh install of Xcode, you will also need to add the commandline tools by running ``xcode-select --install`` on the terminal. -While OS X comes with a large number of UNIX utilities, those familiar with +While OS X comes with a large number of Unix utilities, those familiar with Linux systems will notice one key component missing: a decent package manager. -`Homebrew `_ fills this void. +`Homebrew `_ fills this void. -To `install Homebrew `_, open :file:`Terminal` or -your favorite OSX terminal emulator and run +To `install Homebrew `_, open :file:`Terminal` or +your favorite OS X terminal emulator and run .. code-block:: console @@ -87,8 +92,9 @@ Homebrew names the executable ``python2`` so that you can still run the system P $ python3 -V # Homebrew installed Python 3 interpreter (if installed) +**************** Setuptools & Pip ----------------- +**************** Homebrew installs Setuptools and ``pip`` for you. @@ -108,9 +114,9 @@ and is actively maintained. $ pip -V # pip pointing to the Homebrew installed Python 3 interpreter (if installed) - +******************** Virtual Environments --------------------- +******************** A Virtual Environment (commonly referred to as a 'virtualenv') is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them. It solves the @@ -124,5 +130,5 @@ To start using this and see more information: :ref:`Virtual Environments `_, +This page is a remixed version of `another guide `_, which is available under the same license. diff --git a/docs/starting/install/win.rst b/docs/starting/install/win.rst index 113a1740c..67f8885d6 100644 --- a/docs/starting/install/win.rst +++ b/docs/starting/install/win.rst @@ -1,17 +1,19 @@ .. _install-windows: + +############################## Installing Python 2 on Windows -============================== +############################## .. image:: /_static/photos/34435688560_4cc2a7bcbb_k_d.jpg .. note:: Check out our :ref:`guide for installing Python 3 on Windows`. -First, download the `latest version `_ +First, download the `latest version `_ of Python 2.7 from the official website. If you want to be sure you are installing a fully up-to-date version, click the Downloads > Windows link from the home page of the -`Python.org web site `_ . +`Python.org web site `_ . The Windows version is provided as an MSI package. To install it manually, just double-click the file. The MSI package format allows Windows administrators to @@ -30,13 +32,13 @@ tedious, so add the directories for your default Python version to the :envvar:` Assuming that your Python installation is in :file:`C:\\Python27\\`, add this to your :envvar:`PATH`: -.. code-block:: console +.. code-block:: doscon C:\Python27\;C:\Python27\Scripts\ You can do this easily by running the following in ``powershell``: -.. code-block:: console +.. code-block:: powershell [Environment]::SetEnvironmentVariable("Path", "$env:Path;C:\Python27\;C:\Python27\Scripts\", "User") @@ -50,10 +52,12 @@ described in the next section before you start building Python applications for real-world use. In particular, you should always install Setuptools, as it makes it much easier for you to use other third-party Python libraries. + +**************** Setuptools + Pip ----------------- +**************** -The two most crucial third-party Python packages are `setuptools `_ and `pip `_. +The two most crucial third-party Python packages are `setuptools `_ and `pip `_. Once installed, you can download, install and uninstall any compliant Python software product with a single command. It also enables you to add this network installation @@ -64,15 +68,16 @@ pip by default. To see if pip is installed, open a command prompt and run -.. code-block:: console +.. code-block:: doscon - $ command -v pip + command -v pip To install pip, `follow the official pip installation guide `_ - this will automatically install the latest version of setuptools. +******************** Virtual Environments --------------------- +******************** A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them. It solves the @@ -87,5 +92,5 @@ To start using this and see more information: :ref:`Virtual Environments `_, +This page is a remixed version of `another guide `_, which is available under the same license. diff --git a/docs/starting/install3/linux.rst b/docs/starting/install3/linux.rst index fa1b01c2e..b02204bc0 100644 --- a/docs/starting/install3/linux.rst +++ b/docs/starting/install3/linux.rst @@ -1,11 +1,13 @@ .. _install3-linux: + +############################ Installing Python 3 on Linux -============================ +############################ .. image:: /_static/photos/34435689480_2e6f358510_k_d.jpg -This document describes how to install Python 3.6 on Ubuntu Linux machines. +This document describes how to install Python 3.6 or 3.8 on Ubuntu Linux machines. To see which version of Python 3 you have installed, open a command prompt and run @@ -18,12 +20,12 @@ If you are using Ubuntu 16.10 or newer, then you can easily install Python 3.6 w $ sudo apt-get update $ sudo apt-get install python3.6 -If you're using another version of Ubuntu (e.g. the latest LTS release), we recommend using the `deadsnakes PPA `_ to install Python 3.6:: +If you're using another version of Ubuntu (e.g. the latest LTS release) or you want to use a more current Python, we recommend using the `deadsnakes PPA `_ to install Python 3.8:: $ sudo apt-get install software-properties-common $ sudo add-apt-repository ppa:deadsnakes/ppa $ sudo apt-get update - $ sudo apt-get install python3.6 + $ sudo apt-get install python3.8 If you are using other Linux distribution, chances are you already have Python 3 pre-installed as well. If not, use your distribution's package manager. @@ -35,15 +37,16 @@ For example on Fedora, you would use `dnf`: Note that if the version of the ``python3`` package is not recent enough for you, there may be ways of installing more recent versions as well, -depending on you distribution. For example installing the ``python36`` package -on Fedora 25 to get Python 3.6. If you are a Fedora user, you might want +depending on you distribution. For example installing the ``python3.9`` package +on Fedora 32 to get Python 3.9. If you are a Fedora user, you might want to read about `multiple Python versions available in Fedora`_. .. _multiple Python versions available in Fedora: https://developer.fedoraproject.org/tech/languages/python/multiple-pythons.html +********************* Working with Python 3 ---------------------- +********************* At this point, you may have system Python 2.7 available as well. @@ -51,18 +54,20 @@ At this point, you may have system Python 2.7 available as well. $ python -This will launch the Python 2 interpreter. +This might launch the Python 2 interpreter. .. code-block:: console $ python3 -This will launch the Python 3 interpreter. +This will always launch the Python 3 interpreter. + +**************** Setuptools & Pip ----------------- +**************** -The two most crucial third-party Python packages are `setuptools `_ and `pip `_. +The two most crucial third-party Python packages are `setuptools `_ and `pip `_. Once installed, you can download, install and uninstall any compliant Python software product with a single command. It also enables you to add this network installation @@ -90,8 +95,9 @@ However, when using virtual environments (described below), you don't need to care about that. +***************************** Pipenv & Virtual Environments ------------------------------ +***************************** The next step is to install Pipenv, so you can install dependencies and manage virtual environments. @@ -107,6 +113,5 @@ So, onward! To the :ref:`Pipenv & Virtual Environments -------------------------------- -This page is a remixed version of `another guide `_, +This page is a remixed version of `another guide `_, which is available under the same license. - diff --git a/docs/starting/install3/osx.rst b/docs/starting/install3/osx.rst index e330964ac..fa90fb93a 100644 --- a/docs/starting/install3/osx.rst +++ b/docs/starting/install3/osx.rst @@ -2,14 +2,17 @@ .. _install3-osx: + +############################### Installing Python 3 on Mac OS X -=============================== +############################### .. image:: /_static/photos/34435689480_2e6f358510_k_d.jpg -The latest version of Mac OS X, High Sierra, **comes with Python 2.7 out of the box**. +**Mac OS X comes with Python 2.7 out of the box between versions 10.8 and 12.3.** -You do not need to install or configure anything else to use Python 2. These +If your Mac OS X version is between the above versions, +you do not need to install or configure anything else to use Python 2. These instructions document the installation of Python 3. The version of Python that ships with OS X is great for learning, but it's not @@ -17,36 +20,38 @@ good for development. The version shipped with OS X may be out of date from the `official current Python release `_, which is considered the stable production version. + +************** Doing it Right --------------- +************** Let's install a real version of Python. Before installing Python, you'll need to install GCC. GCC can be obtained -by downloading `XCode `_, the smaller +by downloading `Xcode `_, the smaller `Command Line Tools `_ (must have an Apple account) or the even smaller `OSX-GCC-Installer `_ package. .. note:: - If you already have XCode installed, do not install OSX-GCC-Installer. + If you already have Xcode installed, do not install OSX-GCC-Installer. In combination, the software can cause issues that are difficult to diagnose. .. note:: - If you perform a fresh install of XCode, you will also need to add the + If you perform a fresh install of Xcode, you will also need to add the commandline tools by running ``xcode-select --install`` on the terminal. -While OS X comes with a large number of UNIX utilities, those familiar with +While OS X comes with a large number of Unix utilities, those familiar with Linux systems will notice one key component missing: a package manager. -`Homebrew `_ fills this void. +`Homebrew `_ fills this void. -To `install Homebrew `_, open :file:`Terminal` or -your favorite OSX terminal emulator and run +To `install Homebrew `_, open :file:`Terminal` or +your favorite OS X terminal emulator and run .. code-block:: console - $ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" + $ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" The script will explain what changes it will make and prompt you before the installation begins. @@ -54,6 +59,12 @@ Once you've installed Homebrew, insert the Homebrew directory at the top of your :envvar:`PATH` environment variable. You can do this by adding the following line at the bottom of your :file:`~/.profile` file +.. code-block:: console + + export PATH="/usr/local/opt/python/libexec/bin:$PATH" + +If you have OS X 10.12 (Sierra) or older use this line instead + .. code-block:: console export PATH=/usr/local/bin:/usr/local/sbin:$PATH @@ -66,14 +77,16 @@ Now, we can install Python 3: This will take a minute or two. - +*** Pip ---- +*** Homebrew installs ``pip`` pointing to the Homebrew'd Python 3 for you. + +********************* Working with Python 3 ---------------------- +********************* At this point, you have the system Python 2.7 available, potentially the :ref:`Homebrew version of Python 2 ` installed, and the Homebrew @@ -83,19 +96,19 @@ version of Python 3 as well. $ python -will launch the homebrew-installed Python 3 interpreter. +will launch the Homebrew-installed Python 3 interpreter. .. code-block:: console $ python2 -will launch the homebrew-installed Python 2 interpreter (if any). +will launch the Homebrew-installed Python 2 interpreter (if any). .. code-block:: console $ python3 -will launch the homebrew-installed Python 3 interpreter. +will launch the Homebrew-installed Python 3 interpreter. If the Homebrew version of Python 2 is installed then ``pip2`` will point to Python 2. If the Homebrew version of Python 3 is installed then ``pip`` will point to Python 3. @@ -106,11 +119,12 @@ The rest of the guide will assume that ``python`` references Python 3. # Do I have a Python 3 installed? $ python --version - Python 3.6.4 # Success! - # If you still see 2.7 ensure in PATH /usr/local/bin/ takes precedence over /usr/bin/ + Python 3.7.1 # Success! + +***************************** Pipenv & Virtual Environments ------------------------------ +***************************** The next step is to install Pipenv, so you can install dependencies and manage virtual environments. @@ -126,5 +140,5 @@ So, onward! To the :ref:`Pipenv & Virtual Environments -------------------------------- -This page is a remixed version of `another guide `_, +This page is a remixed version of `another guide `_, which is available under the same license. diff --git a/docs/starting/install3/win.rst b/docs/starting/install3/win.rst index 1a93221e4..db78d9240 100644 --- a/docs/starting/install3/win.rst +++ b/docs/starting/install3/win.rst @@ -1,37 +1,44 @@ .. _install3-windows: + +############################## Installing Python 3 on Windows -============================== +############################## .. image:: /_static/photos/34435689480_2e6f358510_k_d.jpg First, follow the installation instructions for `Chocolatey `_. -It's a community system packager manager for Windows 7+. (It's very much like Homebrew on OSX.) +It's a community system packager manager for Windows 7+. (It's very much like Homebrew on OS X.) Once done, installing Python 3 is very simple, because Chocolatey pushes Python 3 as the default. -.. code-block:: console +.. code-block:: doscon choco install python Once you've run this command, you should be able to launch Python directly from to the console. (Chocolatey is fantastic and automatically adds Python to your path.) + +**************** Setuptools + Pip ----------------- +**************** -The two most crucial third-party Python packages are `setuptools `_ and `pip `_, +The two most crucial third-party Python packages are `setuptools `_ and `pip `_, which let you download, install and uninstall any compliant Python software product with a single command. It also enables you to add this network installation capability to your own Python software with very little work. -All supported versions of Python 3 include pip, so just make sure it's up to date:: +All supported versions of Python 3 include pip, so just make sure it's up to date: + +.. code-block:: doscon python -m pip install -U pip +***************************** Pipenv & Virtual Environments ------------------------------ +***************************** The next step is to install Pipenv, so you can install dependencies and manage virtual environments. @@ -47,5 +54,5 @@ So, onward! To the :ref:`Pipenv & Virtual Environments -------------------------------- -This page is a remixed version of `another guide `_, +This page is a remixed version of `another guide `_, which is available under the same license. diff --git a/docs/starting/installation.rst b/docs/starting/installation.rst index d48165f78..5b84a3ecf 100644 --- a/docs/starting/installation.rst +++ b/docs/starting/installation.rst @@ -1,7 +1,9 @@ .. _installation: + +########################## Properly Installing Python -========================== +########################## .. image:: /_static/photos/36137232412_fdcb0f84eb_k_d.jpg @@ -17,9 +19,9 @@ other third-party Python libraries. .. note:: The use of **Python 3** is *highly* preferred over Python 2. Consider upgrading your applications and infrastructure if you find yourself *still* using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste. —*Kenneth Reitz* - +******************* Installation Guides -------------------- +******************* These guides go over the proper installation of :ref:`Python ` for development purposes, as well as setuptools, pip and virtualenv. diff --git a/docs/starting/which-python.rst b/docs/starting/which-python.rst index 85f35239e..59d42352e 100644 --- a/docs/starting/which-python.rst +++ b/docs/starting/which-python.rst @@ -1,12 +1,17 @@ + + +##################################### Picking a Python Interpreter (3 vs 2) -===================================== +##################################### .. image:: /_static/photos/34484834733_5b80f65ab1_k_d.jpg .. _which-python: + +*************************** The State of Python (3 & 2) -~~~~~~~~~~~~~~~~~~~~~~~~~~~ +*************************** When choosing a Python interpreter, one looming question is always present: "Should I choose Python 2 or Python 3"? The answer is a bit more subtle than @@ -15,16 +20,18 @@ one might think. The basic gist of the state of things is as follows: -1. Most production applications today use Python 2.7. +1. Most production applications today use Python 3. 2. Python 3 is ready for the production deployment of applications today. -3. Python 2.7 will only receive necessary security updates until 2020 [#pep373_eol]_. +3. Python 2 reached the end of its life on January 1, 2020 [#pep373_eol]_. 4. The brand name "Python" encapsulates both Python 3 and Python 2. + +*************** Recommendations -~~~~~~~~~~~~~~~ +*************** -.. note:: The use of **Python 3** is *highly* preferred over Python 2. Consider upgrading your applications and infrastructure if you find yourself *still* using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste. +.. note:: The use of **Python 3** is *highly* recommended over Python 2. Consider upgrading your applications and infrastructure if you find yourself *still* using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste. —*Kenneth Reitz* I'll be blunt: @@ -33,13 +40,11 @@ I'll be blunt: - If you're learning Python for the first time, familiarizing yourself with Python 2.7 will be very useful, but not more useful than learning Python 3. - Learn both. They are both "Python". -- Software that is already built often depends on Python 2.7. -- If you are writing a new open source Python library, it's best to write it for both Python 2 and 3 - simultaneously. Only supporting Python 3 for a new library you want to be widely adopted is a - political statement and will alienate many of your users. This is not a problem — slowly, over the next three years, this will become less the case. + +********* So.... 3? -~~~~~~~~~ +********* If you're choosing a Python interpreter to use, I recommend you use the newest Python 3.x, since every version brings new and @@ -49,8 +54,6 @@ Given such, only use Python 2 if you have a strong reason to, such as a pre-existing code-base, a Python 2 exclusive library, simplicity/familiarity, or, of course, you absolutely love and are inspired by Python 2. No harm in that. -Check out `Can I Use Python 3? `_ to see if any -software you're depending on will block your adoption of Python 3. `Further Reading `_ @@ -58,12 +61,12 @@ It is possible to `write code that works on Python 2.6, 2.7, and Python 3 `_. This ranges from trivial to hard depending upon the kind of software you are writing; if you're a beginner there are far more important things to -worry about. Note that Python 2.6 is end-of-life upstream, so you shouldn't -try to write 2.6-compatible code unless you're being paid specifically to -do that. +worry about. + +*************** Implementations -~~~~~~~~~~~~~~~ +*************** When people speak of *Python* they often mean not just the language but also the CPython implementation. *Python* is actually a specification for a language @@ -77,7 +80,7 @@ written in C. It compiles Python code to intermediate bytecode which is then interpreted by a virtual machine. CPython provides the highest level of compatibility with Python packages and C extension modules. -If you are writing open-source Python code and want to reach the widest possible +If you are writing open source Python code and want to reach the widest possible audience, targeting CPython is best. To use packages which rely on C extensions to function, CPython is your only implementation option. @@ -124,7 +127,8 @@ expose Python code to other languages in the .NET framework. IronPython directly into the Visual Studio development environment, making it an ideal choice for Windows developers. -IronPython supports Python 2.7. [#iron_ver]_ +IronPython supports Python 2.7. [#iron_ver]_ IronPython 3 [#iron_ver3]_ +is being developed, but is not ready for use as of September 2020. PythonNet --------- @@ -140,16 +144,16 @@ installations on non-Windows operating systems, such as OS X and Linux, to operate within the .NET framework. It can be run in addition to IronPython without conflict. -Pythonnet supports from Python 2.6 up to Python 3.5. [#pythonnet_ver1]_ [#pythonnet_ver2]_ +Pythonnet is compatible with Python 2.7 and 3.5-3.8. [#pythonnet_ver1]_ -.. [#pypy_ver] http://pypy.org/compat.html +.. [#pypy_ver] https://pypy.org/compat.html .. [#jython_ver] https://hg.python.org/jython/file/412a8f9445f7/NEWS -.. [#iron_ver] http://ironpython.codeplex.com/releases/view/81726 +.. [#iron_ver] https://ironpython.net/download/ -.. [#pythonnet_ver1] https://travis-ci.org/pythonnet/pythonnet +.. [#iron_ver3] https://github.com/IronLanguages/ironpython3 -.. [#pythonnet_ver2] https://ci.appveyor.com/project/TonyRoberts/pythonnet-480xs +.. [#pythonnet_ver1] https://pythonnet.github.io/ .. [#pep373_eol] https://www.python.org/dev/peps/pep-0373/#id2 diff --git a/docs/writing/documentation.rst b/docs/writing/documentation.rst index a712db8af..27d85405a 100644 --- a/docs/writing/documentation.rst +++ b/docs/writing/documentation.rst @@ -1,5 +1,8 @@ + + +############# Documentation -============= +############# .. image:: /_static/photos/35620636012_f66aa88f93_k_d.jpg @@ -7,8 +10,10 @@ Readability is a primary focus for Python developers, in both project and code documentation. Following some simple best practices can save both you and others a lot of time. + +********************* Project Documentation ---------------------- +********************* A :file:`README` file at the root directory should give general information to both users and maintainers of a project. It should be raw text or @@ -20,7 +25,7 @@ information. This file is the main entry point for readers of the code. An :file:`INSTALL` file is less necessary with Python. The installation instructions are often reduced to one command, such as ``pip install -module`` or ``python setup.py install`` and added to the :file:`README` +module`` or ``python setup.py install``, and added to the :file:`README` file. A :file:`LICENSE` file should *always* be present and specify the license @@ -32,13 +37,15 @@ planned development for the code. A :file:`CHANGELOG` file or section in :file:`README` should compile a short overview of the changes in the code base for the latest versions. + +******************* Project Publication -------------------- +******************* Depending on the project, your documentation might include some or all of the following components: -- An *introduction* should show a very short overview of what can be +- An *introduction* should give a very short overview of what can be done with the product, using one or two extremely simplified use cases. This is the thirty-second pitch for your project. @@ -68,7 +75,7 @@ your source repository so that rebuilding your documentation will happen automatically. When run, Sphinx_ will import your code and using Python's introspection -features it will extract all function, method and class signatures. It will +features it will extract all function, method, and class signatures. It will also extract the accompanying docstrings, and compile it all into well structured and easily readable documentation for your project. @@ -78,7 +85,7 @@ structured and easily readable documentation for your project. for general project documentation. This Guide is built with Sphinx_ and is hosted on `Read The Docs`_ -.. _Sphinx: http://sphinx.pocoo.org +.. _Sphinx: https://www.sphinx-doc.org .. _Read The Docs: http://readthedocs.org .. _restructuredtext-ref: @@ -87,18 +94,19 @@ reStructuredText ~~~~~~~~~~~~~~~~ Most Python documentation is written with reStructuredText_. It's like -Markdown with all the optional extensions built in. +Markdown, but with all the optional extensions built in. The `reStructuredText Primer`_ and the `reStructuredText Quick Reference`_ should help you familiarize yourself with its syntax. .. _reStructuredText: http://docutils.sourceforge.net/rst.html -.. _reStructuredText Primer: http://sphinx.pocoo.org/rest.html +.. _reStructuredText Primer: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html .. _reStructuredText Quick Reference: http://docutils.sourceforge.net/docs/user/rst/quickref.html +************************* Code Documentation Advice -------------------------- +************************* Comments clarify the code and they are added with purpose of making the code easier to understand. In Python, comments begin with a hash @@ -141,8 +149,8 @@ a project's documentation. Additionally, Doctest_ will read all embedded docstrings that look like input from the Python commandline (prefixed with ">>>") and run them, checking to see if the output of the command matches the text on the following line. This -allows developers to embed real examples and usage of functions alongside -their source code, and as a side effect, it also ensures that their code is +allows developers to embed real examples and usage of functions alongside +their source code. As a side effect, it also ensures that their code is tested and works. :: @@ -174,13 +182,13 @@ comment block is a programmer's note. The docstring describes the Unlike block comments, docstrings are built into the Python language itself. This means you can use all of Python's powerful introspection capabilities to -access docstrings at runtime, compared with comments which are optimised out. +access docstrings at runtime, compared with comments which are optimized out. Docstrings are accessible from both the `__doc__` dunder attribute for almost every Python object, as well as with the built in `help()` function. While block comments are usually used to explain *what* a section of code is -doing, or the specifics of an algorithm, docstrings are more intended for -explaining to other users of your code (or you in 6 months time) *how* a +doing, or the specifics of an algorithm, docstrings are more intended towards +explaining other users of your code (or you in 6 months time) *how* a particular function can be used and the general purpose of a function, class, or module. @@ -206,8 +214,8 @@ In larger or more complex projects however, it is often a good idea to give more information about a function, what it does, any exceptions it may raise, what it returns, or relevant details about the parameters. -For more detailed documentation of code a popular style is the one used for the -Numpy project, often called `Numpy style`_ docstrings. While it can take up more +For more detailed documentation of code a popular style used, is the one used by the +NumPy project, often called `NumPy style`_ docstrings. While it can take up more lines than the previous example, it allows the developer to include a lot more information about a method, function, or class. :: @@ -237,8 +245,8 @@ docstrings, making it easy to incorporate NumPy style docstrings into your project. At the end of the day, it doesn't really matter what style is used for writing -docstrings, their purpose is to serve as documentation for anyone who may need -to read or make changes to your code. As long as it is correct, understandable +docstrings; their purpose is to serve as documentation for anyone who may need +to read or make changes to your code. As long as it is correct, understandable, and gets the relevant points across then it has done the job it was designed to do. @@ -249,8 +257,10 @@ For further reading on docstrings, feel free to consult :pep:`257` .. _sphinx.ext.napoleon: https://sphinxcontrib-napoleon.readthedocs.io/ .. _`NumPy style`: http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html + +*********** Other Tools ------------ +*********** You might see these in the wild. Use :ref:`sphinx-ref`. diff --git a/docs/writing/gotchas.rst b/docs/writing/gotchas.rst index b230640af..f717858ca 100644 --- a/docs/writing/gotchas.rst +++ b/docs/writing/gotchas.rst @@ -1,23 +1,28 @@ + + +############## Common Gotchas -============== +############## .. image:: /_static/photos/34435688380_b5a740762b_k_d.jpg For the most part, Python aims to be a clean and consistent language that -avoids surprises. However, there are a few cases that can be confusing to +avoids surprises. However, there are a few cases that can be confusing for newcomers. Some of these cases are intentional but can be potentially surprising. Some could arguably be considered language warts. In general, what follows is a collection of potentially tricky behavior that might seem strange at first -glance, but is generally sensible once you're aware of the underlying cause for +glance, but are generally sensible, once you're aware of the underlying cause for the surprise. .. _default_args: + +************************* Mutable Default Arguments -------------------------- +************************* Seemingly the *most* common surprise new Python programmers encounter is Python's treatment of mutable default arguments in function definitions. @@ -48,8 +53,8 @@ isn't provided, so that the output is:: [12] [42] -What Does Happen -~~~~~~~~~~~~~~~~ +What Actually Happens +~~~~~~~~~~~~~~~~~~~~~ .. testoutput:: @@ -88,12 +93,14 @@ to maintain state between calls of a function. This is often done when writing a caching function. +**************************** Late Binding Closures ---------------------- +**************************** Another common source of confusion is the way Python binds its variables in closures (or in the surrounding global scope). + What You Wrote ~~~~~~~~~~~~~~ @@ -119,8 +126,8 @@ variable that multiplies their argument, producing:: 6 8 -What Does Happen -~~~~~~~~~~~~~~~~ +What Actually Happens +~~~~~~~~~~~~~~~~~~~~~ .. testoutput:: @@ -189,9 +196,9 @@ lots of situations. Looping to create unique functions is unfortunately a case where they can cause hiccups. - +********************************* Bytecode (.pyc) Files Everywhere! ---------------------------------- +********************************* By default, when executing Python code from files, the Python interpreter will automatically write a bytecode version of that file to disk, e.g. @@ -199,8 +206,8 @@ will automatically write a bytecode version of that file to disk, e.g. These ``.pyc`` files should not be checked into your source code repositories. -Theoretically, this behavior is on by default, for performance reasons. -Without these bytecode files present, Python would re-generate the bytecode +Theoretically, this behavior is on by default for performance reasons. +Without these bytecode files, Python would re-generate the bytecode every time the file is loaded. diff --git a/docs/writing/license.rst b/docs/writing/license.rst index c48194ecb..820cc9a05 100644 --- a/docs/writing/license.rst +++ b/docs/writing/license.rst @@ -1,10 +1,13 @@ + + +################## Choosing a License -================== +################## .. image:: /_static/photos/33907149294_82d7535a6c_k_d.jpg -Your source publication *needs* a license. In the US, if no license is -specified, users have no legal right to download, modify, or distribute. +Your source publication *needs* a license. In the US, unless a license is +specified, users have no legal right to download, modify, or distribute the product. Furthermore, people can't contribute to your code unless you tell them what rules to play by. Choosing a license is complicated, so here are some pointers: @@ -16,18 +19,18 @@ In general, these licenses tend to fall into one of two categories: 1. licenses that focus more on the user's freedom to do with the software as they please (these are the more permissive open - source licenses such as the MIT, BSD, & Apache). + source licenses such as the MIT, BSD, and Apache) 2. licenses that focus more on making sure that the code itself — including any changes made to it and distributed along with it — always remains free (these are the less permissive free software - licenses such as the GPL and LGPL). + licenses such as the GPL and LGPL) The latter are less permissive in the sense that they don't permit someone to add code to the software and distribute it without also including the source code for their changes. -To help you choose one for your project, there's a `license chooser `_, +To help you choose one for your project, there's a `license chooser `_; **use it**. **More Permissive** diff --git a/docs/writing/logging.rst b/docs/writing/logging.rst index 21623594b..b6110f7f7 100644 --- a/docs/writing/logging.rst +++ b/docs/writing/logging.rst @@ -1,5 +1,8 @@ + + +####### Logging -======= +####### .. image:: /_static/photos/35254379756_c9fe23f843_k_d.jpg @@ -7,6 +10,8 @@ The :mod:`logging` module has been a part of Python's Standard Library since version 2.3. It is succinctly described in :pep:`282`. The documentation is notoriously hard to read, except for the `basic logging tutorial`_. +As an alternative, `loguru `_ provides an approach for logging, nearly as simple as using a simple ``print`` statement. + Logging serves two purposes: - **Diagnostic logging** records events related to the application's @@ -17,8 +22,9 @@ Logging serves two purposes: reports or to optimize a business goal. +************* ... or Print? -------------- +************* The only time that ``print`` is a better option than logging is when the goal is to display a help statement for a command line application. @@ -34,8 +40,9 @@ Other reasons why logging is better than ``print``: :attr:`logging.Logger.disabled` to ``True``. +******************** Logging in a Library --------------------- +******************** Notes for `configuring logging for a library`_ are in the `logging tutorial`_. Because the *user*, not the library, should @@ -52,8 +59,8 @@ using the ``__name__`` global variable: the :mod:`logging` module creates a hierarchy of loggers using dot notation, so using ``__name__`` ensures no name collisions. -Here is an example of best practice from the `requests source`_ -- place -this in your ``__init__.py`` +Here is an example of the best practice from the `requests source`_ -- place +this in your ``__init__.py``: .. code-block:: python @@ -61,12 +68,13 @@ this in your ``__init__.py`` logging.getLogger(__name__).addHandler(logging.NullHandler()) +************************* Logging in an Application -------------------------- +************************* -The `twelve factor app `_, an authoritative reference +The `twelve factor app `_, an authoritative reference for good practice in application development, contains a section on -`logging best practice `_. It emphatically +`logging best practice `_. It emphatically advocates for treating log events as an event stream, and for sending that event stream to standard output to be handled by the application environment. @@ -75,9 +83,9 @@ application environment. There are at least three ways to configure a logger: - Using an INI-formatted file: - - **Pro**: possible to update configuration while running using the + - **Pro**: possible to update configuration while running, using the function :func:`logging.config.listen` to listen on a socket. - - **Con**: less control (*e.g.* custom subclassed filters or loggers) + - **Con**: less control (e.g. custom subclassed filters or loggers) than possible when configuring a logger in code. - Using a dictionary or a JSON-formatted file: - **Pro**: in addition to updating while running, it is possible to load @@ -86,13 +94,13 @@ There are at least three ways to configure a logger: - **Con**: less control than when configuring a logger in code. - Using code: - **Pro**: complete control over the configuration. - - **Con**: modifications require a change to source code. + - **Con**: modifications require a change to the source code. Example Configuration via an INI File ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Let us say the file is named ``logging_config.ini``. +Let us say that the file is named ``logging_config.ini``. More details for the file format are in the `logging configuration`_ section of the `logging tutorial`_. @@ -186,9 +194,9 @@ Example Configuration Directly in Code logger.debug('often makes a very good meal of %s', 'visiting tourists') -.. _basic logging tutorial: http://docs.python.org/howto/logging.html#logging-basic-tutorial -.. _logging configuration: https://docs.python.org/howto/logging.html#configuring-logging -.. _logging tutorial: http://docs.python.org/howto/logging.html -.. _configuring logging for a library: https://docs.python.org/howto/logging.html#configuring-logging-for-a-library -.. _log record: https://docs.python.org/library/logging.html#logrecord-attributes +.. _basic logging tutorial: http://docs.python.org/3/howto/logging.html#logging-basic-tutorial +.. _logging configuration: https://docs.python.org/3/howto/logging.html#configuring-logging +.. _logging tutorial: http://docs.python.org/3/howto/logging.html +.. _configuring logging for a library: https://docs.python.org/3/howto/logging.html#configuring-logging-for-a-library +.. _log record: https://docs.python.org/3/library/logging.html#logrecord-attributes .. _requests source: https://github.com/kennethreitz/requests diff --git a/docs/writing/reading.rst b/docs/writing/reading.rst index a74b64b9c..1e03a4def 100644 --- a/docs/writing/reading.rst +++ b/docs/writing/reading.rst @@ -1,12 +1,11 @@ + + +################## Reading Great Code -================== +################## .. image:: /_static/photos/34689452831_93d7fd0571_k_d.jpg -One of the core tenets behind the design of Python is creating -readable code. The motivation behind this design is simple: The number -one thing that Python programmers do is read code. - One of the secrets of becoming a great Python programmer is to read, understand, and comprehend excellent code. @@ -26,28 +25,28 @@ reading. Each one of these projects is a paragon of Python coding. best intentions in mind. - `Diamond `_ - Diamond is a python daemon that collects metrics + Diamond is a Python daemon that collects metrics and publishes them to Graphite or other backends. - It is capable of collecting cpu, memory, network, i/o, load and disk metrics. + It is capable of collecting CPU, memory, network, I/O, load, and disk metrics. Additionally, it features an API for implementing custom collectors for gathering metrics from almost any source. - `Werkzeug `_ - Werkzeug started as simple collection of various utilities for WSGI + Werkzeug started as a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility modules. It includes a powerful debugger, full-featured request and response objects, HTTP utilities to handle entity tags, cache control headers, HTTP dates, - cookie handling, file uploads, a powerful URL routing system and a bunch + cookie handling, file uploads, a powerful URL routing system, and a bunch of community-contributed addon modules. - `Requests `_ Requests is an Apache2 Licensed HTTP library, written in Python, for human beings. -- `Tablib `_ +- `Tablib `_ Tablib is a format-agnostic tabular dataset library, written in Python. .. todo:: Include code examples of exemplary code from each of the projects listed. Explain why it is excellent code. Use complex examples. -.. todo:: Explain techniques to rapidly identify data structures, algorithms and determine what the code is doing. +.. todo:: Explain techniques to rapidly identify data structures and algorithms and determine what the code is doing. diff --git a/docs/writing/structure.rst b/docs/writing/structure.rst index c59f4dd00..5d9645acd 100644 --- a/docs/writing/structure.rst +++ b/docs/writing/structure.rst @@ -1,5 +1,8 @@ + + +######################## Structuring Your Project -======================== +######################## .. image:: /_static/photos/33907151224_0574e7dfc2_k_d.jpg @@ -15,21 +18,21 @@ the project? What features and functions can be grouped together and isolated? By answering questions like these you can begin to plan, in a broad sense, what your finished product will look like. -In this section we take a closer look at Python's module and import +In this section, we take a closer look at Python's modules and import systems as they are the central elements to enforcing structure in your project. We then discuss various perspectives on how to build code which can be extended and tested reliably. - +*************************** Structure of the Repository ---------------------------- +*************************** It's Important. ::::::::::::::: Just as Code Style, API Design, and Automation are essential for a -healthy development cycle, Repository structure is a crucial part of +healthy development cycle. Repository structure is a crucial part of your project's `architecture `__. @@ -51,13 +54,12 @@ documentation. Of course, first impressions aren't everything. You and your colleagues will spend countless hours working with this repository, eventually -becoming intimately familiar with every nook and cranny. The layout of -it is important. +becoming intimately familiar with every nook and cranny. The layout is important. Sample Repository ::::::::::::::::: -**tl;dr**: This is what `Kenneth Reitz `_ recommends. +**tl;dr**: This is what `Kenneth Reitz recommended in 2013 `__. This repository is `available on GitHub `__. @@ -123,7 +125,7 @@ If you aren't sure which license you should use for your project, check out `choosealicense.com `_. Of course, you are also free to publish code without a license, but this -would prevent many people from potentially using your code. +would prevent many people from potentially using or contributing to your code. Setup.py :::::::: @@ -154,8 +156,8 @@ should be placed at the root of the repository. It should specify the dependencies required to contribute to the project: testing, building, and generating documentation. -If your project has no development dependencies, or you prefer -development environment setup via ``setup.py``, this file may be +If your project has no development dependencies, or if you prefer +setting up a development environment via ``setup.py``, this file may be unnecessary. Documentation @@ -208,7 +210,7 @@ I highly recommend the latter. Requiring a developer to run codebase also requires them to have an isolated environment setup for each instance of the codebase. -To give the individual tests import context, create a tests/context.py +To give the individual tests import context, create a ``tests/context.py`` file: :: @@ -244,8 +246,8 @@ Makefile If you look at most of my projects or any Pocoo project, you'll notice a -Makefile laying around. Why? These projects aren't written in C... In -short, make is a incredibly useful tool for defining generic tasks for +Makefile lying around. Why? These projects aren't written in C... In +short, make is an incredibly useful tool for defining generic tasks for your project. **Sample Makefile:** @@ -313,9 +315,9 @@ The resulting structure: - +************************ Structure of Code is Key ------------------------- +************************ Thanks to the way imports and modules are handled in Python, it is relatively easy to structure a Python project. Easy, here, means @@ -328,46 +330,47 @@ Easy structuring of a project means it is also easy to do it poorly. Some signs of a poorly structured project include: -- Multiple and messy circular dependencies: if your classes +- Multiple and messy circular dependencies: If the classes Table and Chair in :file:`furn.py` need to import Carpenter from :file:`workers.py` to answer a question such as ``table.isdoneby()``, - and if conversely the class Carpenter needs to import Table and Chair, + and if conversely the class Carpenter needs to import Table and Chair to answer the question ``carpenter.whatdo()``, then you have a circular dependency. In this case you will have to resort to - fragile hacks such as using import statements inside + fragile hacks such as using import statements inside your methods or functions. -- Hidden coupling: each and every change in Table's implementation +- Hidden coupling: Each and every change in Table's implementation breaks 20 tests in unrelated test cases because it breaks Carpenter's code, - which requires very careful surgery to adapt the change. This means + which requires very careful surgery to adapt to the change. This means you have too many assumptions about Table in Carpenter's code or the reverse. -- Heavy usage of global state or context: instead of explicitly +- Heavy usage of global state or context: Instead of explicitly passing ``(height, width, type, wood)`` to each other, Table and Carpenter rely on global variables that can be modified and are modified on the fly by different agents. You need to - scrutinize all access to these global variables to understand why + scrutinize all access to these global variables in order to understand why a rectangular table became a square, and discover that remote template code is also modifying this context, messing with - table dimensions. + the table dimensions. - Spaghetti code: multiple pages of nested if clauses and for loops with a lot of copy-pasted procedural code and no proper segmentation are known as spaghetti code. Python's - meaningful indentation (one of its most controversial features) make - it very hard to maintain this kind of code. So the good news is that + meaningful indentation (one of its most controversial features) makes + it very hard to maintain this kind of code. The good news is that you might not see too much of it. - Ravioli code is more likely in Python: it consists of hundreds of similar little pieces of logic, often classes or objects, without - proper structure. If you never can remember if you have to use + proper structure. If you never can remember, if you have to use FurnitureTable, AssetTable or Table, or even TableNew for your - task at hand, you might be swimming in ravioli code. + task at hand, then you might be swimming in ravioli code. +******* Modules -------- +******* Python modules are one of the main abstraction layers available and probably the most natural one. Abstraction layers allow separating code into parts holding @@ -380,13 +383,13 @@ in one file, and all low-level operations in another file. In this case, the interface file needs to import the low-level file. This is done with the ``import`` and ``from ... import`` statements. -As soon as you use `import` statements you use modules. These can be either +As soon as you use `import` statements, you use modules. These can be either built-in modules such as `os` and `sys`, third-party modules you have installed in your environment, or your project's internal modules. To keep in line with the style guide, keep module names short, lowercase, and be sure to avoid using special symbols like the dot (.) or question mark (?). -So a file name like :file:`my.spam.py` is one you should avoid! Naming this way +A file name like :file:`my.spam.py` is the one you should avoid! Naming this way will interfere with the way Python looks for modules. In the case of `my.spam.py` Python expects to find a :file:`spam.py` file in a @@ -394,11 +397,11 @@ folder named :file:`my` which is not the case. There is an `example `_ of how the dot notation should be used in the Python docs. -If you'd like you could name your module :file:`my_spam.py`, but even our -friend the underscore should not be seen often in module names. However, using other +If you like, you could name your module :file:`my_spam.py`, but even our trusty +friend the underscore, should not be seen that often in module names. However, using other characters (spaces or hyphens) in module names will prevent importing -(- is the subtract operator), so try to keep module names short so there is -no need to separate words. And, most of all, don't namespace with underscores, use submodules instead. +(- is the subtract operator). Try to keep module names short so there is +no need to separate words. And, most of all, don't namespace with underscores; use submodules instead. .. code-block:: python @@ -408,15 +411,15 @@ no need to separate words. And, most of all, don't namespace with underscores, u import library.foo_plugin Aside from some naming restrictions, nothing special is required for a Python -file to be a module, but you need to understand the import mechanism in order +file to be a module. But you need to understand the import mechanism in order to use this concept properly and avoid some issues. Concretely, the ``import modu`` statement will look for the proper file, which -is :file:`modu.py` in the same directory as the caller if it exists. If it is +is :file:`modu.py` in the same directory as the caller, if it exists. If it is not found, the Python interpreter will search for :file:`modu.py` in the "path" -recursively and raise an ImportError exception if it is not found. +recursively and raise an ImportError exception when it is not found. -Once :file:`modu.py` is found, the Python interpreter will execute the module in +When :file:`modu.py` is found, the Python interpreter will execute the module in an isolated scope. Any top-level statement in :file:`modu.py` will be executed, including other imports if any. Function and class definitions are stored in the module's dictionary. @@ -433,12 +436,12 @@ unwanted effects, e.g. override an existing function with the same name. It is possible to simulate the more standard behavior by using a special syntax of the import statement: ``from modu import *``. This is generally considered -bad practice. **Using** ``import *`` **makes code harder to read and makes +bad practice. **Using** ``import *`` **makes the code harder to read and makes dependencies less compartmentalized**. Using ``from modu import func`` is a way to pinpoint the function you want to -import and put it in the global namespace. While much less harmful than ``import -*`` because it shows explicitly what is imported in the global namespace, its +import and put it in the local namespace. While much less harmful than ``import +*`` because it shows explicitly what is imported in the local namespace, its only advantage over a simpler ``import modu`` is that it will save a little typing. @@ -469,15 +472,16 @@ typing. As mentioned in the :ref:`code_style` section, readability is one of the main features of Python. Readability means to avoid useless boilerplate text and -clutter, therefore some efforts are spent trying to achieve a certain level of +clutter; therefore some efforts are spent trying to achieve a certain level of brevity. But terseness and obscurity are the limits where brevity should stop. Being able to tell immediately where a class or function comes from, as in the ``modu.func`` idiom, greatly improves code readability and understandability in all but the simplest single file projects. +******** Packages --------- +******** Python provides a very straightforward packaging system, which is simply an extension of the module mechanism to a directory. @@ -488,20 +492,20 @@ modules, but with a special behavior for the :file:`__init__.py` file, which is used to gather all package-wide definitions. A file :file:`modu.py` in the directory :file:`pack/` is imported with the -statement ``import pack.modu``. This statement will look for an -:file:`__init__.py` file in :file:`pack`, execute all of its top-level +statement ``import pack.modu``. This statement will look for +:file:`__init__.py` file in :file:`pack` and execute all of its top-level statements. Then it will look for a file named :file:`pack/modu.py` and execute all of its top-level statements. After these operations, any variable, function, or class defined in :file:`modu.py` is available in the pack.modu namespace. -A commonly seen issue is to add too much code to :file:`__init__.py` +A commonly seen issue is adding too much code to :file:`__init__.py` files. When the project complexity grows, there may be sub-packages and sub-sub-packages in a deep directory structure. In this case, importing a single item from a sub-sub-package will require executing all :file:`__init__.py` files met while traversing the tree. -Leaving an :file:`__init__.py` file empty is considered normal and even a good +Leaving an :file:`__init__.py` file empty is considered normal and even good practice, if the package's modules and sub-packages do not need to share any code. @@ -509,49 +513,50 @@ Lastly, a convenient syntax is available for importing deeply nested packages: ``import very.deep.module as mod``. This allows you to use `mod` in place of the verbose repetition of ``very.deep.module``. + +*************************** Object-oriented programming ---------------------------- +*************************** Python is sometimes described as an object-oriented programming language. This -can be somewhat misleading and needs to be clarified. +can be somewhat misleading and requires further clarifications. In Python, everything is an object, and can be handled as such. This is what is meant when we say, for example, that functions are first-class objects. Functions, classes, strings, and even types are objects in Python: like any object, they have a type, they can be passed as function arguments, and they -may have methods and properties. In this understanding, Python is an -object-oriented language. +may have methods and properties. In this understanding, Python can be considered +as an object-oriented language. However, unlike Java, Python does not impose object-oriented programming as the main programming paradigm. It is perfectly viable for a Python project to not be object-oriented, i.e. to use no or very few class definitions, class inheritance, or any other mechanisms that are specific to object-oriented -programming. +programming languages. Moreover, as seen in the modules_ section, the way Python handles modules and namespaces gives the developer a natural way to ensure the encapsulation and separation of abstraction layers, both being the most common reasons to use object-orientation. Therefore, Python programmers have more -latitude to not use object-orientation, when it is not required by the business +latitude as to not use object-orientation, when it is not required by the business model. There are some reasons to avoid unnecessary object-orientation. Defining -custom classes is useful when we want to glue together some state and some -functionality. The problem, as pointed out by the discussions about functional +custom classes is useful when we want to glue some state and some +functionality together. The problem, as pointed out by the discussions about functional programming, comes from the "state" part of the equation. In some architectures, typically web applications, multiple instances of Python -processes are spawned to respond to external requests that can happen at the -same time. In this case, holding some state into instantiated objects, which +processes are spawned as a response to external requests that happen simultaneously. +In this case, holding some state in instantiated objects, which means keeping some static information about the world, is prone to concurrency -problems or race-conditions. Sometimes, between the initialization of the state +problems or race conditions. Sometimes, between the initialization of the state of an object (usually done with the ``__init__()`` method) and the actual use of the object state through one of its methods, the world may have changed, and the retained state may be outdated. For example, a request may load an item in memory and mark it as read by a user. If another request requires the deletion -of this item at the same time, it may happen that the deletion actually occurs -after the first process loaded the item, and then we have to mark as read a -deleted object. +of this item at the same time, the deletion may actually occur after the first +process loaded the item, and then we have to mark a deleted object as read. This and other issues led to the idea that using stateless functions is a better programming paradigm. @@ -565,7 +570,7 @@ or deletes data in a global variable or in the persistence layer, it is said to have a side-effect. Carefully isolating functions with context and side-effects from functions with -logic (called pure functions) allow the following benefits: +logic (called pure functions) allows the following benefits: - Pure functions are deterministic: given a fixed input, the output will always be the same. @@ -573,7 +578,7 @@ logic (called pure functions) allow the following benefits: - Pure functions are much easier to change or replace if they need to be refactored or optimized. -- Pure functions are easier to test with unit-tests: There is less +- Pure functions are easier to test with unit tests: There is less need for complex context setup and data cleaning afterwards. - Pure functions are easier to manipulate, decorate, and pass around. @@ -587,8 +592,9 @@ things that are manipulated (windows, buttons, avatars, vehicles) have a relatively long life of their own in the computer's memory. +********** Decorators ----------- +********** The Python language provides a simple yet powerful syntax called 'decorators'. A decorator is a function or a class that wraps (or decorates) a function @@ -614,15 +620,17 @@ clearer and thus preferred. # bar() is decorated This mechanism is useful for separating concerns and avoiding -external un-related logic 'polluting' the core logic of the function +external unrelated logic 'polluting' the core logic of the function or method. A good example of a piece of functionality that is better handled with decoration is `memoization `__ or caching: you want to store the results of an expensive function in a table and use them directly instead of recomputing them when they have already been computed. This is clearly not part of the function logic. + +**************** Context Managers ----------------- +**************** A context manager is a Python object that provides extra contextual information to an action. This extra information takes the form of running a callable upon @@ -665,7 +673,7 @@ by the ``with`` statement. CustomOpen is first instantiated and then its is finished executing, the ``__exit__`` method is then called. And now the generator approach using Python's own -`contextlib `_: +`contextlib `_: .. code-block:: python @@ -694,15 +702,17 @@ to decide when to use which. The class approach might be better if there's a considerable amount of logic to encapsulate. The function approach might be better for situations where we're dealing with a simple action. + +************** Dynamic typing --------------- +************** Python is dynamically typed, which means that variables do not have a fixed type. In fact, in Python, variables are very different from what they are in many other languages, specifically statically-typed languages. Variables are not a segment of the computer's memory where some value is written, they are 'tags' or 'names' pointing to objects. It is therefore possible for the variable 'a' to -be set to the value 1, then to the value 'a string', then to a function. +be set to the value 1, then the value 'a string', to a function. The dynamic typing of Python is often considered to be a weakness, and indeed it can lead to complexities and hard-to-debug code. Something named 'a' can be @@ -732,7 +742,7 @@ Some guidelines help to avoid this issue: def func(): pass # Do something -Using short functions or methods helps reduce the risk +Using short functions or methods helps to reduce the risk of using the same name for two unrelated things. It is better to use different names even for things that are related, @@ -758,8 +768,10 @@ a `final` keyword and it would be against its philosophy anyway. However, it may be a good discipline to avoid assigning to a variable more than once, and it helps in grasping the concept of mutable and immutable types. + +*************************** Mutable and immutable types ---------------------------- +*************************** Python has two kinds of built-in or user-defined types. @@ -776,7 +788,7 @@ compute x + 1, you have to create another integer and give it a name. my_list = [1, 2, 3] my_list[0] = 4 - print my_list # [4, 2, 3] <- The same list has changed + print(my_list) # [4, 2, 3] <- The same list has changed x = 6 x = x + 1 # The new x is another object @@ -795,16 +807,12 @@ and can be used as a key for a dictionary. One peculiarity of Python that can surprise beginners is that strings are immutable. This means that when constructing a string from -its parts, it is much more efficient to accumulate the parts in a list, -which is mutable, and then glue ('join') the parts together when the -full string is needed. One thing to notice, however, is that list -comprehensions are better and faster than constructing a list in a loop -with calls to ``append()``. - -One other option is using the map function, which can 'map' a function -('str') to an iterable ('range(20)'). This results in a map object, -which you can then ('join') together just like the other examples. -The map function can be even faster than a list comprehension in some cases. +its parts, appending each part to the string is inefficient because +the entirety of the string is copied on each append. +Instead, it is much more efficient to accumulate the parts in a list, +which is mutable, and then glue (``join``) the parts together when the +full string is needed. List comprehensions are usually the fastest and +most idiomatic way to do this. **Bad** @@ -814,9 +822,9 @@ The map function can be even faster than a list comprehension in some cases. nums = "" for n in range(20): nums += str(n) # slow and inefficient - print nums + print(nums) -**Good** +**Better** .. code-block:: python @@ -824,27 +832,19 @@ The map function can be even faster than a list comprehension in some cases. nums = [] for n in range(20): nums.append(str(n)) - print "".join(nums) # much more efficient - -**Better** - -.. code-block:: python - - # create a concatenated string from 0 to 19 (e.g. "012..1819") - nums = [str(n) for n in range(20)] - print "".join(nums) + print("".join(nums)) # much more efficient **Best** .. code-block:: python # create a concatenated string from 0 to 19 (e.g. "012..1819") - nums = map(str, range(20)) - print "".join(nums) + nums = [str(n) for n in range(20)] + print("".join(nums)) One final thing to mention about strings is that using ``join()`` is not always best. In the instances where you are creating a new string from a pre-determined -number of strings, using the addition operator is actually faster, but in cases +number of strings, using the addition operator is actually faster. But in cases like above or in cases where you are adding to an existing string, using ``join()`` should be your preferred method. @@ -860,7 +860,7 @@ like above or in cases where you are adding to an existing string, using .. note:: You can also use the :ref:`% ` formatting operator to concatenate a pre-determined number of strings besides :py:meth:`str.join` - and ``+``. However, :pep:`3101`, discourages the usage of the ``%`` operator + and ``+``. However, :pep:`3101` discourages the usage of the ``%`` operator in favor of the :py:meth:`str.format` method. .. code-block:: python @@ -873,16 +873,20 @@ like above or in cases where you are adding to an existing string, using foobar = '{foo}{bar}'.format(foo=foo, bar=bar) # It is best +************************ Vendorizing Dependencies ------------------------- +************************ + +******* Runners -------- +******* +*************** Further Reading ---------------- +*************** -- http://docs.python.org/2/library/ -- http://www.diveintopython.net/toc/index.html +- http://docs.python.org/3/library/ +- https://diveintopython3.net/ diff --git a/docs/writing/style.rst b/docs/writing/style.rst index 59781c6ee..d8c096a05 100644 --- a/docs/writing/style.rst +++ b/docs/writing/style.rst @@ -1,12 +1,14 @@ .. _code_style: + +########## Code Style -========== +########## .. image:: /_static/photos/33907150054_5ee79e8940_k_d.jpg If you ask Python programmers what they like most about Python, they will -often cite its high readability. Indeed, a high level of readability +often cite its high readability. Indeed, a high level of readability is at the heart of the design of the Python language, following the recognized fact that code is read much more often than it is written. @@ -21,8 +23,10 @@ what is considered the best (hear: most readable) way. On some border cases, no best way has been agreed upon on how to express an intent in Python code, but these cases are rare. + +**************** General concepts ----------------- +**************** Explicit code ~~~~~~~~~~~~~ @@ -61,9 +65,9 @@ it is bad practice to have two disjointed statements on the same line of code. .. code-block:: python - print 'one'; print 'two' + print('one'); print('two') - if x == 1: print 'one' + if x == 1: print('one') if and : # do something @@ -72,11 +76,11 @@ it is bad practice to have two disjointed statements on the same line of code. .. code-block:: python - print 'one' - print 'two' + print('one') + print('two') if x == 1: - print 'one' + print('one') cond1 = cond2 = @@ -111,7 +115,7 @@ compared to the more straightforward calls to ``send('Hello', 'World')`` and optional, and evaluate to ``None`` when they are not passed another value. Calling a function with keyword arguments can be done in multiple ways in -Python, for example it is possible to follow the order of arguments in the +Python; for example, it is possible to follow the order of arguments in the definition without explicitly naming the arguments, like in ``send('Hello', 'World', 'Cthulhu', 'God')``, sending a blind carbon copy to God. It would also be possible to name arguments in another order, like in @@ -120,7 +124,7 @@ possibilities are better avoided without any strong reason to not follow the syntax that is the closest to the function definition: ``send('Hello', 'World', cc='Cthulhu', bcc='God')``. -As a side note, following `YAGNI `_ +As a side note, following the `YAGNI `_ principle, it is often harder to remove an optional argument (and its logic inside the function) that was added "just in case" and is seemingly never used, than to add a new optional argument and its logic when needed. @@ -130,7 +134,7 @@ than to add a new optional argument and its logic when needed. an extensible number of positional arguments, it can be defined with the ``*args`` constructs. In the function body, ``args`` will be a tuple of all the remaining positional arguments. For example, ``send(message, *args)`` - can be called with each recipient as an argument:``send('Hello', 'God', + can be called with each recipient as an argument: ``send('Hello', 'God', 'Mom', 'Cthulhu')``, and in the function body ``args`` will be equal to ``('God', 'Mom', 'Cthulhu')``. @@ -177,7 +181,7 @@ possible to do each of the following: * change how the Python interpreter imports modules -* it is even possible (and recommended if needed) to embed C routines in Python. +* It is even possible (and recommended if needed) to embed C routines in Python. However, all these options have many drawbacks and it is always better to use the most straightforward way to achieve your goal. The main drawback is that @@ -204,7 +208,7 @@ are all responsible users". This doesn't mean that, for example, no properties are considered private, and that no proper encapsulation is possible in Python. Rather, instead of relying -on concrete walls erected by the developers between their code and other's, the +on concrete walls erected by the developers between their code and others', the Python community prefers to rely on a set of conventions indicating that these elements should not be accessed directly. @@ -222,7 +226,7 @@ but making a public property private might be a much harder operation. Returning values ~~~~~~~~~~~~~~~~ -When a function grows in complexity it is not uncommon to use multiple return +When a function grows in complexity, it is not uncommon to use multiple return statements inside the function's body. However, in order to keep a clear intent and a sustainable readability level, it is preferable to avoid returning meaningful values from many output points in the body. @@ -260,12 +264,14 @@ is needed. return x # One single exit point for the returned value x will help # when maintaining the code. + +****** Idioms ------- +****** A programming idiom, put simply, is a *way* to write code. The notion of programming idioms is discussed amply at `c2 `_ -and at `Stack Overflow `_. +and at `Stack Overflow `_. Idiomatic Python code is often referred to as being *Pythonic*. @@ -351,9 +357,7 @@ Instead, use a list comprehension: .. code-block:: python - four_lists = [[] for __ in xrange(4)] - -Note: Use range() instead of xrange() in Python 3 + four_lists = [[] for __ in range(4)] Create a string from a list ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -397,7 +401,7 @@ hand, the hash of the item will tell Python where in the set to look for a matching item. As a result, the search can be done quickly, even if the set is large. Searching in dictionaries works the same way. For more information see this -`StackOverflow `_ +`StackOverflow `_ page. For detailed information on the amount of time various common operations take on each of these data structures, see `this page `_. @@ -417,8 +421,9 @@ hashtable will often be greater than the time saved by the improved search speed. +************* Zen of Python -------------- +************* Also known as :pep:`20`, the guiding principles for Python's design. @@ -448,18 +453,19 @@ Also known as :pep:`20`, the guiding principles for Python's design. Namespaces are one honking great idea -- let's do more of those! For some examples of good Python style, see `these slides from a Python user -group `_. +group `_. + +***** PEP 8 ------ +***** -:pep:`8` is the de-facto code style guide for Python. A high quality, +:pep:`8` is the de facto code style guide for Python. A high quality, easy-to-read version of PEP 8 is also available at `pep8.org `_. This is highly recommended reading. The entire Python community does their best to adhere to the guidelines laid out within this document. Some project -may sway from it from time to time, while others may -`amend its recommendations `_. +may sway from it from time to time, while others may amend its recommendations. That being said, conforming your Python code to PEP 8 is generally a good idea and helps make code more consistent when working on projects with other @@ -487,7 +493,15 @@ Then run it on a file or series of files to get a report of any violations. optparse.py:472:29: E221 multiple spaces before operator optparse.py:544:21: W601 .has_key() is deprecated, use 'in' -The program `autopep8 `_ can be used to +Auto-Formatting +~~~~~~~~~~~~~~~ + +There are several auto-formatting tools that can reformat your code, +in order to comply with PEP 8. + +**autopep8** + +The program `autopep8 `_ can be used to automatically reformat code in the PEP 8 style. Install the program with: .. code-block:: console @@ -504,15 +518,60 @@ Excluding the ``--in-place`` flag will cause the program to output the modified code directly to the console for review. The ``--aggressive`` flag will perform more substantial changes and can be applied multiple times for greater effect. +**yapf** + +While autopep8 focuses on solving the PEP 8 violations, `yapf `_ +tries to improve the format of your code aside from complying with PEP 8. +This formatter aims at providing as good looking code as a programmer who +writes PEP 8 compliant code. +It gets installed with: + +.. code-block:: console + + $ pip install yapf + +Run the auto-formatting of a file with: + +.. code-block:: console + + $ yapf --in-place optparse.py + +Similar to autopep8, running the command without the ``--in-place`` flag will +output the diff for review before applying the changes. + +**black** + +The auto-formatter `black `_ offers an +opinionated and deterministic reformatting of your code base. +Its main focus lies in providing a uniform code style without the need of +configuration throughout its users. Hence, users of black are able to forget +about formatting altogether. Also, due to the deterministic approach minimal +git diffs with only the relevant changes are guaranteed. You can install the +tool as follows: + +.. code-block:: console + + $ pip install black + +A python file can be formatted with: + +.. code-block:: console + + $ black optparse.py + +Adding the ``--diff`` flag provides the code modification for review without +direct application. + +*********** Conventions ----------------- +*********** Here are some conventions you should follow to make your code easier to read. -Check if variable equals a constant -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Check if a variable equals a constant +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You don't need to explicitly compare a value to True, or None, or 0 - you can +You don't need to explicitly compare a value to True, or None, or 0 -- you can just add it to the if statement. See `Truth Value Testing `_ for a list of what is considered false. @@ -522,10 +581,10 @@ list of what is considered false. .. code-block:: python if attr == True: - print 'True!' + print('True!') if attr == None: - print 'attr is None!' + print('attr is None!') **Good**: @@ -533,15 +592,15 @@ list of what is considered false. # Just check the value if attr: - print 'attr is truthy!' + print('attr is truthy!') # or check for the opposite if not attr: - print 'attr is falsey!' + print('attr is falsey!') # or, since None is considered false, explicitly check for it if attr is None: - print 'attr is None!' + print('attr is None!') Access a Dictionary Element ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -555,9 +614,9 @@ or pass a default argument to :py:meth:`dict.get`. d = {'hello': 'world'} if d.has_key('hello'): - print d['hello'] # prints 'world' + print(d['hello']) # prints 'world' else: - print 'default_value' + print('default_value') **Good**: @@ -565,59 +624,155 @@ or pass a default argument to :py:meth:`dict.get`. d = {'hello': 'world'} - print d.get('hello', 'default_value') # prints 'world' - print d.get('thingy', 'default_value') # prints 'default_value' + print(d.get('hello', 'default_value')) # prints 'world' + print(d.get('thingy', 'default_value')) # prints 'default_value' # Or: if 'hello' in d: - print d['hello'] + print(d['hello']) Short Ways to Manipulate Lists ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `List comprehensions `_ -provide a powerful, concise way to work with lists. Also, the :py:func:`map` and -:py:func:`filter` functions can perform operations on lists using a different, -more concise syntax. +provides a powerful, concise way to work with lists. + +`Generator expressions +`_ +follows almost the same syntax as list comprehensions but return a generator +instead of a list. + +Creating a new list requires more work and uses more memory. If you are just going +to loop through the new list, prefer using an iterator instead. **Bad**: +.. code-block:: python + + # needlessly allocates a list of all (gpa, name) entires in memory + valedictorian = max([(student.gpa, student.name) for student in graduates]) + +**Good**: + +.. code-block:: python + + valedictorian = max((student.gpa, student.name) for student in graduates) + + +Use list comprehensions when you really need to create a second list, for +example if you need to use the result multiple times. + + +If your logic is too complicated for a short list comprehension or generator +expression, consider using a generator function instead of returning a list. + +**Good**: + +.. code-block:: python + + def make_batches(items, batch_size): + """ + >>> list(make_batches([1, 2, 3, 4, 5], batch_size=3)) + [[1, 2, 3], [4, 5]] + """ + current_batch = [] + for item in items: + current_batch.append(item) + if len(current_batch) == batch_size: + yield current_batch + current_batch = [] + yield current_batch + + +Never use a list comprehension just for its side effects. + +**Bad**: + +.. code-block:: python + + [print(x) for x in sequence] + +**Good**: + +.. code-block:: python + + for x in sequence: + print(x) + + +Filtering a list +~~~~~~~~~~~~~~~~ + +**Bad**: + +Never remove items from a list while you are iterating through it. + .. code-block:: python # Filter elements greater than 4 a = [3, 4, 5] - b = [] for i in a: if i > 4: - b.append(i) + a.remove(i) + +Don't make multiple passes through the list. + +.. code-block:: python + + while i in a: + a.remove(i) **Good**: +Use a list comprehension or generator expression. + .. code-block:: python - a = [3, 4, 5] - b = [i for i in a if i > 4] - # Or: - b = filter(lambda x: x > 4, a) + # comprehensions create a new list object + filtered_values = [value for value in sequence if value != x] + + # generators don't create another list + filtered_values = (value for value in sequence if value != x) + + +Possible side effects of modifying the original list +:::::::::::::::::::::::::::::::::::::::::::::::::::: + +Modifying the original list can be risky if there are other variables referencing it. But you can use *slice assignment* if you really want to do that. +.. code-block:: python + + # replace the contents of the original list + sequence[::] = [value for value in sequence if value != x] + + +Modifying the values in a list +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Bad**: +Remember that assignment never creates a new object. If two or more variables refer to the same list, changing one of them changes them all. + .. code-block:: python # Add three to all list members. a = [3, 4, 5] + b = a # a and b refer to the same list object + for i in range(len(a)): - a[i] += 3 + a[i] += 3 # b[i] also changes **Good**: +It's safer to create a new list object and leave the original alone. + .. code-block:: python a = [3, 4, 5] + b = a + + # assign the variable "a" to a new list without changing "b" a = [i + 3 for i in a] - # Or: - a = map(lambda i: i + 3, a) Use :py:func:`enumerate` keep a count of your place in the list. @@ -625,7 +780,7 @@ Use :py:func:`enumerate` keep a count of your place in the list. a = [3, 4, 5] for i, item in enumerate(a): - print i, item + print(i, item) # prints # 0 3 # 1 4 @@ -646,7 +801,7 @@ files for you. f = open('file.txt') a = f.read() - print a + print(a) f.close() **Good**: @@ -655,7 +810,7 @@ files for you. with open('file.txt') as f: for line in f: - print line + print(line) The ``with`` statement is better because it will ensure you always close the file, even if an exception is raised inside the ``with`` block. @@ -671,7 +826,7 @@ a white space added to the end of the line, after the backslash, will break the code and may have unexpected results. A better solution is to use parentheses around your elements. Left with an -unclosed parenthesis on an end-of-line the Python interpreter will join the +unclosed parenthesis on an end-of-line, the Python interpreter will join the next line until the parentheses are closed. The same behavior holds for curly and square braces. diff --git a/docs/writing/tests.rst b/docs/writing/tests.rst index 7da42e788..28eb3260b 100644 --- a/docs/writing/tests.rst +++ b/docs/writing/tests.rst @@ -1,13 +1,16 @@ + + +################# Testing Your Code -================= +################# .. image:: /_static/photos/34435687940_8f73fc1fa6_k_d.jpg Testing your code is very important. Getting used to writing testing code and running this code in parallel is now -considered a good habit. Used wisely, this method helps you define more -precisely your code's intent and have a more decoupled architecture. +considered a good habit. Used wisely, this method helps to define your +code's intent more precisely and have a more decoupled architecture. Some general rules of testing: @@ -71,11 +74,14 @@ Some general rules of testing: be to add a test to ensure that the new functionality is not already a working path that has not been plugged into the interface. + + +********** The Basics -:::::::::: +********** -Unittest +unittest -------- :mod:`unittest` is the batteries-included test module in the Python standard @@ -136,8 +142,10 @@ When running this module from the command line as in ``python module.py``, the doctests will run and complain if anything is not behaving as described in the docstrings. + +***** Tools -::::: +***** py.test @@ -162,7 +170,7 @@ functions: def test_answer(): assert func(3) == 5 -and then running the `py.test` command +and then running the `py.test` command: .. code-block:: console @@ -193,7 +201,7 @@ the unittest module! Hypothesis ---------- -Hypothesis is a library which lets you write tests that are parametrized by +Hypothesis is a library which lets you write tests that are parameterized by a source of examples. It then generates simple and comprehensible examples that make your tests fail, letting you find more bugs with less work. @@ -217,7 +225,7 @@ minimal example of each bug (distinguished exception type and location): xs=[1.7976321109618856e+308, 6.102390043022755e+303] ) -Hypothesis is practical as well as very powerful, and will often find bugs +Hypothesis is practical as well as very powerful and will often find bugs that escaped all other forms of testing. It integrates well with py.test, and has a strong focus on usability in both simple and advanced scenarios. @@ -228,47 +236,18 @@ tox --- tox is a tool for automating test environment management and testing against -multiple interpreter configurations +multiple interpreter configurations. .. code-block:: console $ pip install tox tox allows you to configure complicated multi-parameter test matrices via a -simple ini-style configuration file. +simple INI-style configuration file. `tox `_ -Unittest2 ---------- - -unittest2 is a backport of Python 2.7's unittest module which has an improved -API and better assertions over the one available in previous versions of Python. - -If you're using Python 2.6 or below, you can install it with pip - -.. code-block:: console - - $ pip install unittest2 - -You may want to import the module under the name unittest to make porting code -to newer versions of the module easier in the future - -.. code-block:: python - - import unittest2 as unittest - - class MyTest(unittest.TestCase): - ... - -This way if you ever switch to a newer Python version and no longer need the -unittest2 module, you can simply change the import in your test module without -the need to change any other code. - - `unittest2 `_ - - mock ---- @@ -315,7 +294,6 @@ always returns the same result (but only for the duration of the test). # get_search_results runs a search and iterates over the result self.assertEqual(len(myapp.get_search_results(q="fish")), 3) -Mock has many other ways you can configure it and control its behavior. +Mock has many other ways with which you can configure and control its behaviour. `mock `_ - diff --git a/requirements.txt b/requirements.txt index c2886a7fe..4a246ac76 100644 --- a/requirements.txt +++ b/requirements.txt @@ -6,7 +6,7 @@ docutils==0.14 idna==2.7 imagesize==1.0.0 Jinja2==2.10 -MarkupSafe==1.0 +MarkupSafe==1.1.1 packaging==17.1 Pygments==2.2.0 pyparsing==2.2.0 diff --git a/runtime.txt b/runtime.txt index d70c8f8d8..cc1923a40 100644 --- a/runtime.txt +++ b/runtime.txt @@ -1 +1 @@ -3.6 +3.8