Two Scoops of Django 3x Compress 10
Two Scoops of Django 3x Compress 10
Two Scoops of Django 3x Compress 10
for years. We’ve always found his works enlightening and useful.
An incredible book by Python luminaries David Beazley and Brian Jones, it’s filled
with delicious ice cream recipes... err... incredibly useful Python recipes for any devel-
oper using Python 3.3 or greater.
Treading on Python Volume 2
amzn.to/1kVWi2a
Covers more advanced Python structures.
Writing Idiomatic Python 3.3
amzn.to/1aS5df4
Great tips for optimizing your code and increasing the legibility of your work. There
are a few places it differs from our practices (imports being the largest area of differ-
ence), but overall we concur. A 2.7 version is available at amzn.to/1fj9j7z
JavaScript Resources
WARNING: ATTENTION: Help Wanted!
Books:
Web Resources:
We scan the results page for the Mozilla Developer Network (MDN) link, usually
around the third position, and click on that one.
Appendix D: Internationalization and
Localization
Django and Python provides a lot of very useful tools for dealing with internationalization,
localization, and Unicode.
This appendix contains a list of things helpful for preparing your Django application for
non-English readers and non-USA users. This list is by no means complete, and we invite
the reader to provide additional feedback.
Start Early
It is always easier to start with and grow an internationalized, localized project than to
convert an existing project.
– Patrick McLoughlan
We used to construct translation strings all the time, going so far as to include it in the 1.5
edition of the book. This is when you use slightly-clever code to construct sentences out of
various Python objects. For reference, this was part of Example 8.7:
# DON'T DO THIS!
# Skipping the rest of imports for the sake of brevity
class FlavorActionMixin:
@property
def action(self):
msg = '{0} is missing action.'.format(self.__class__)
raise NotImplementedError(msg)
While seemingly handy in that it makes for a self-maintaining mixin, it is overly clever in
we can’t internationalize the result of calling self.__class__. In other words, you can’t
just add django.utils.translation the following and expect it to produce anything
meaningful for translators to work from:
# DON'T DO THIS!
from django.utils.translations import gettext as _
Rather than writing code that constructs sentences out of various Python constructs, now
we write more meaningful dialogues that can be readily translated. This means a little more
work, but the result is a more easily translatable project. Hence we now follow this pattern:
Appendix D: Internationalization and Localization
class FlavorActionMixin:
@property
def success_msg(self):
return NotImplemented
For reference, you can combine individual strings representing meaningful sentences and
dialogues into larger values. However, you shouldn’t build sentences by concatenating pieces,
because other languages may require a different order. For the same reason, you should
always include punctuation in translated strings. See as follows:
class FlavorActionMixin:
@property
def success_msg(self):
return NotImplemented
Unicode Tricks
Here are some things we’ve learned when dealing with unicode-related issues.
A good Django-based example is Mozilla and their various sites for supporting tools like
Firefox. On these sites they handle translations for over 80 languages. Unfortunately, a title
that fits the page in English breaks the site in more verbose languages such as German.
Mozilla’s answer is to determine the width of a title container, then use JavaScript to adjust
the font size of the title text downwards until the text fits into the container with wrapping.
A simpler way of handling this issue is to assume that other languages can take up twice as
Appendix D: Internationalization and Localization
much space as English. English is a pretty concise language that, because of its short words,
handles text wrapping very well.
ä yourcalendricalfallacyis.com
Appendix E: Settings Alternatives
Here a couple of alternative patterns for managing settings that we feel can be recommended.
They avoid the local_settings anti-pattern and allow for management of configuration that
will work with either the Environment Variables Pattern or the Secrets File Pattern.
For these reasons, we suggest being conservative about switching to new settings
approaches. Only do it when the current settings management approach has become
a pain point, not when a new method becomes popular.
The argument for this approach is that when using the multiple settings files approach,
you end up with environment-specific code. For instance, when doing local development,
you’re not running the code with production settings. This increases the chance of running
into production-specific bugs when you update some code without updating the production
settings accordingly.
This style involves using sensible default settings and as few environment specific values as
possible. When combined with tools like Vagrant and Docker, it means that mirroring
production is trivial.
Appendix E: Settings Alternatives
It results in a much simpler settings file, and for Twelve Factor App fans, it’s right in line
with that approach.
If you want to see an example of the approach in action, check out FeedHQ’s settings mod-
ule: github.com/feedhq/feedhq/blob/master/feedhq/settings.py
We’ve enjoyed this approach for new and smaller projects. When done right, it makes things
elegantly simple.
If you would like to know more about this approach, we recommend the following articles:
ä bruno.im/2013/may/18/django-stop-writing-settings-files/
ä 12factor.net/config
Appendix G: Security
Settings Reference
In Django, knowing which setting should be set to what value in development vs production
requires an unfortunate amount of domain knowledge. This appendix is a reference for better
understanding how to configure a Django project for both development and production.
vanilla.twoscoopspress.com to chocolate.twoscoopspress.com):
docs.djangoproject.com/en/3.2/ref/settings/#csrf-cookie-domain
ä Changing the default CSRF failure view: docs.djangoproject.com/en/3.2/
ref/settings/#csrf-failure-view
Email SSL
Django supports secure connections to SMTP servers. We strongly suggest using this fea-
ture. Documentation on the following settings begins at docs.djangoproject.com/en/
3.2/ref/settings/#email-use-tls
ä EMAIL_USE_TLS
ä EMAIL_USE_SSL
ä EMAIL_SSL_CERTFILE
ä EMAIL_SSL_KEYFILE
SESSION_SERIALIZER
Per subsection 28.10.4:
SESSION_SERIALIZER = django.contrib.sessions.serializers.JSONSerializer.
SECURE_PROXY_SSL_HEADER
For some setups, most notably Heroku, this should be:
Therefore, it’s critical that a point-by-point plan be written and made available to maintain-
ers and even non-technical participants of a project. Here is a sample plan:
On Heroku:
\$ heroku maintenance:on
Enabling maintenance mode for myapp... done
Appendix G: Handling Security Failures
For projects you deploy yourself or with automated tools, you’re going to have create this
capability yourself. Fortunately, other people have faced this before so we come prepared
with reference material:
ä cyberciti.biz/faq/custom-nginx-maintenance-page-with-http503
for putting up maintenance 503 pages.
ä Other tools can be found at djangopackages.org/grids/g/
emergency-management
Back Everything Up
Get a copy of the code and then the data off the servers and keep it on a local hard drive or
SSD. You might also consider a bonded, professional storage company.
Why? First, when you back things up at this stage, you are protecting your audit trail. This
might provide you with the capability to determine where and when things went wrong.
Second, and this might be unpleasant to hear, but malignant staff can cause as many prob-
lems as any bug or successful penetration. What that means is that the best software-based
security is useless against a developer who creates a backdoor or a non-technical staff level
user who decides to cause trouble.
ä Writing up a quick summary will help you focus and gather your thoughts. You’re
going to be under an amazing amount of stress. The stress and urgency of the situation
can make you attempt stupid things that can aggravate the problem.
ä You never know, the Django security team might have good advice or even an answer
for you.
ä It might be a Django problem! If that is the case, the Django security team needs to
know so they can mitigate the problem for everyone else before it becomes public.
TIP: Jacob Kaplan-Moss on Reporting to the Django Project
Former Django BDFL and former Director of Security for Heroku Jacob Kaplan-
Moss says, “I’d much rather have people send things that aren’t actual problems
in Django to security@djangoproject.com than accidentally disclose security issues
publicly because they don’t know better.”
This will be a stressful time and people will be on the edge of panic. Start doing research,
perhaps in this book, ask questions as per Chapter 36: Where and How to Ask Django
Questions, and find a resolution.
Before you implement a correction, it’s often better to make sure you have a real, proper
fix for the problem then do a rushed emergency patch that destroys everything. Yes, this is
where tests and continuous integration shine.
Stay positive: now is the time for everyone to come together and fix the problem. Start taking
notes, ask for help from the best people you know, remind yourself (or the team) that you
have the will and the smarts to fix things, and make things right!
ä Attacks the day of a software update release, right before most people get
around to installing the software update.
ä Attacks right after a user writes up a blog post about a vulnerability they found,
before package maintainers have the chance to write and release a patch.
ä Attacks targeting vulnerabilities in discussion on a closed security mailing list.
With zero-day attacks, there is often no time to address and patch the vulnerability,
making the compromise especially difficult to manage. If there was ever a reason to
have a battle plan for handling security issues, this is it.
See en.wikipedia.org/wiki/0day
Appendix H: WebSockets with Channels
Appendix H: WebSockets with Channels
Already mentioned in Chapter 27: Asynchronous Task Queues, Channels provides the ca-
pability for Django to handle WebSockets. The advantage of using Channels for this pur-
pose is that unlike alternative approaches such as FastAPI or Node.js , Channels allows us
to use Websockets in a way very similar to views. Better yet, by using Channels, we can ac-
cess our project’s code base, allowing us to access our models and other custom code. We’ve
found Channels to be a very powerful tool, especially when we follow the same practices we
espouse in the rest of this book.
Keep in mind that the advantage of WebSockets is much more than providing a real-time
interface. It’s that the protocol is lighter than HTTP. This makes it a faster way to transmit
data back and forth between client and server.
Here are our thoughts on using Channels based off lessons learned:
Another option is to track the number of WebSockets a particular user has to your system.
If they have more than twenty (or any number of your choosing) open, then close the con-
nections that don’t seem to be doing anything. We found this to be a pretty effective way of
protecting servers from unnecessary load. Unfortunately, there isn’t yet a stock solution for
resolving things in this manner.
The problem is that the odds are stacked against the connection’s survival. Let’s go over
some of what threatens it, shall we?
Rather than address this problem with some kind of long poll fallback like socket.io or
SockJS, Django Channels just lets the connection die. When this happens, the client has
to trigger a new connection. Conveniently, Django Channels provides a small JavaScript
library that creates a new connection when the old one dies.
The important thing in all this is to remember is that when using Channels we have to take
into account that long polling isn’t an option we can use.
Later we took our time, leaned on the concept of fat models and helper files documented
as we worked, wrote better tests, and embraced Generic Consumers as if they were Class-
Based Views. In other words, we practiced what we preach.
In talking to other coders we discovered we weren’t the only ones whose first Channels
project turned out messy. The lesson learned from this is that in the excitement to play
with Django Channels, it’s easy to make basic mistakes. So do yourself a favor the first
time you write Channels code: Remember to take the time and use standard best practices.
Especially when it comes to writing tests: channels.readthedocs.io/en/stable/
topics/testing.html
Acknowledgments
This book was not written in a vacuum. We would like to express our thanks to everyone
who had a part in putting it together.
Adeyinka Adegbenro Adeyinka Adegbenro is a Software Engineer with a passion for com-
puter science and problem-solving. Outside of work, she enjoys writing(essays and
technical tutorials), working out, and watching documentaries. She lives in Lagos,
Nigeria.
Carol Willing is a member of Python’s Steering Council and a core developer of
CPython. She’s a Python Software Foundation Fellow and former Director. In 2019,
she was awarded the Frank Willison Award python.org/community/awards/
frank-willison/#carol-willing-2019 for technical and community contri-
butions to Python. With a strong commitment to community outreach, Carol co-
organizes PyLadies San Diego and San Diego Python User Group.
Enrique Matías Sánchez is a Software Engineer with a soft spot for System Administra-
tion. Besides programming, he enjoys outdoor sports (cycling, hiking, running, climb-
ing...) and learning to gibber in foreign languages. He lives in Zaragoza, Spain and
shares a black cat with his charming girlfriend Elena.
Haris Ibrahim K V is a human being working as a Software Developer since 2014 building
websites using primarily & Django in order to support his family. Enjoys teaching,
writing, singing, religion, learning, and banana roasted in ghee (as opposed to ice
cream, so there!).
Acknowledgments
Iacopo Spalletti tinkers with his computer for more time he would admit, Iacopo met
Python and Django in the late ’00s and he felt like home. He split his time building
things with clients and as opensource, working with the community as a speaker and
as organizer and mentor (meetups, Pycon Italia, DjangoCon Europe, DjangoGirls).
Lately focusing on avoiding too many bugs in raising a little human being.
Madelene Campos is a recovering professional musician, having earned 2 degrees in music.
Despite not studying computer science at University, web development seemed like a
natural extension to her musical studies. Programming and music are both powerful
and important communication tools. She started on this path in 2015, and has been a
professional developer since 2016. Maddie has been happily using Python and Django
for a little over 2 years, and hopes to continue on this glorious path!
Modasser Billah is a djangonaut whose favorite pet is Python. He loves to build enterprise
software systems in the cloud and currently works as a Lead Software Engineer at
BriteCore. Modasser is an AWS certified developer associate and a proponent of
leveraging the collective wisdom of software engineers known as best practices. He
loves remote work and teams without borders. Modasser has a Bachelor’s degree in
CS from Bangladesh University of Engineering & Technology and lives in Cumilla,
Bangladesh. When he is not coding, he is probably either enjoying today’s edition of
Dilbert or playing with his toddler daughters Naseeha and Naaera.
Renato Oliveira is a Brazilian Software Engineer and co-founder at Labcodes Software
Studio. He has been working with Python and Django for almost 10 years. He loves
creating sustainable web applications that don’t take forever to add a feature and also
helping others to accomplish their tasks.
Florian Apolloner us a Software Engineer who loves debugging hopeless cases and find-
ing the root causes of weird bugs. He started contributing to Django around 2007,
became a member of the Django Team in 2012 and nowadays works as part of the Se-
curity Team. He enjoys biking and various winter sports and is living in the kangaroo-
free Austria (not Australia ;)).
James Bennett has been using and loving Django since almost the day it was released. He
spent five years working at the Lawrence Journal-World where he became directly
involved in the project, and has since served as a committer and release manager, a
member of the framework’s security team and technical board, and on the Board of
Directors of the Django Software Foundation. He now lives in California. Daniel and
Audrey met in his PyCon 2010 Djang Deep Dive tutorial, hence he is responsible for Two
Scoops of Django.
Markus Holterman (Security) - Markus Holtermann has been a member of several Open
Source communities and projects for over a decade. He gained experience in engineer-
ing, application and infrastructure security, and community management by taking on
tasks in these communities. Markus is a member of the security team for the Django
web framework and has given several conference talks about security in and around
Django. In his free time, Markus is a hobby photographer.
Michael Manfre is a software engineer with over two decades of involvement in Open
Source projects. He is a member of the Django web framework’s security team.
Michael’s Django contributions started in 2008 and initially focused on maintaining
a Microsoft SQL Server database backend and running Django on Windows.
Matt Braymer-Hayes
Nathan Cox
Ola Sendecka
Jannis Gebauer
Haris Ibrahim K V
Tom Christie
Michael Herman
Humphrey Butau
Sasha Romijn - Security Reviewer
James Bennett - Security Reviewer
Florian Apolloner - Security Reviewer
Aymeric Augustin - Security Reviewer
Acknowledgments
Contributors to 1.11
The following individuals helped us improve this edition: Andrés Pérez-Albela H., Leila
Loezer, Martin Koistinen, Miguel Pachas, José Augusto Costa Martins Jr., Daniel Bond,
John Carter, Bernard ‘BJ’ Jauregui, Peter Inglesby, Michael Scharf, Jason Wolosonovich,
Dipanjan Sarkar, Anish Menon, Ramon Maria Gallart Escolà, You Zhou, Khaled Alqe-
naei, Xus Zoyo, Joris Derese, Laurent Steffan, Louie Pascual, Charlie Tian, Thijs van
Dien, Paulina Rybicka, Qi Ying Koo, Bassem Ali, Jonathan Mitchell, Anton Backer, Mar-
tijn Mhkuu, Photong, Michael John Barr, Kevin Marsh, Greg Smith, Muraoka Yusuke,
Michael Helmick, Zachery Tapp, Jesús Gómez, Klemen Strušnik, Peter Brooks, Bernat
Bonet, Danilo Cabello, Alenajk, Piotr Szpetkowski, Nick Wright, Michael Sanders, Nate
Guerin, David Adam Hernandez, Brendan M. Sleight, Maksim Iakovlev, and David Da-
han
Bartek Ogryczak
Barry Morrison
Kevin Stone
Paul Hallett
Saurabh Kumar
Sasha Romijn - Security Reviewer
Contributors to 1.8
Kenneth Love, Patrick McLoughlan, Sebastián J. Seba, Kevin Campbell, Doug Folland,
Kevin London, Ramon Maria Gallart Escolà, Eli Bendersky, Dan O’Donovan, Ryan Cur-
rah, Shafique Jamal, Russ Ferriday, Charles L. Johnson, Josh Wiegand, William Vincent,
Tom Atkins, Martey Dodoo, Krace Kumar Ramaraju, Felipe Arruda Pontes, Ed Patrick
Tan, Sven Aßmann, Christopher Lambacher, Colin O’Brien, Sebastien de Menten, Evan-
gelos Mantadakis, Silas Wegg, Michal Hoftich, Markus Holterman, Pat Curry, Gaston
Keller, Mihail Russu, Jean-Baptiste Lab, Kaleb Elwert, Tim Bell, Zuhair Parvez, Ger
Schinkel, Athena Yao, Norberto Bensa, Abhaya Agarwal, Steve Sarjeant, Karlo Tamayo,
Cary Kempston, José Padilla, Konstantinos Faliagkas, Kelsey Gilmore-Innis, Adam Bog-
dał, Tyler Davis, Javier Liendo, Kevin Xu, Michael Barr, Caroline Simpson, John Might,
Tom Christie, Nicolas Pannetier, Marc Tamlyn, Loïc Bistuer, Arnaud Limbourg, Alasdair
Nicol, and Ludvig Wadenstein.
Malcolm Tredinnick lived in Sydney, Australia and spent much of his time travelling in-
ternationally. He was a Python user for over 15 years and Django user since just after
it was released to the public in mid-2005, becoming a Django core developer in 2006.
A user of many programming languages, he felt that Django was one of the better
web libraries/frameworks that he used professionally and was glad to see its incredibly
broad adoption over the years. In 2012 when he found out that we were co-hosting
the first PyCon Philippines, he immediately volunteered to fly out, give two talks,
and co-run the sprints. Sadly, he passed away in March of 2013, just two months
after this book was released. His leadership and generosity in the Python and Django
community will always be remembered.
The following were also critical in supporting the 1.5 edition of this book.
Kenneth Love
Lynn Root
Barry Morrison
Jacob Kaplan-Moss
Jeff Triplett
Lennart Regebro
Randall Degges
Sean Bradley
Contributors to 1.5
The following individuals sent us corrections, cleanups, bug fixes, and suggestions. This in-
cludes: Álex González, Alex Gaynor, Amar Šahinović, Andrew Halloran, Andrew Jordan,
Acknowledgments
Anthony Burke, Aymeric Augustin, Baptiste Mispelon, Bernardo Brik, Branko Vukelic,
Brian Shumate, Carlos Cardoso, Charl Botha, Charles Denton, Chris Foresman, Chris
Jones, Dan Loewenherz, Dan Poirier, Darren Ma, Daryl Yu, Dave Castillo, Dave Murphy,
David Beazley, David Sauve, Davide Rizzo, Deric Crago, Dolugen Buuraldaa, Dominik
Aumayr, Douglas Miranda, Eric Woudenberg, Sasha Romijn, Esteban Gaviota, Fabio Na-
tali, Farhan Syed, Felipe Coelho, Felix Ingram, Florian Apolloner, Francisco Barros, Gabe
Jackson, Gabriel Duman, Garry Cairns, Graham Dumpleton, Hamid Hoorzad, Hamish
Downer, Harold Ekstrom, Hrayr Artunyan, Jacinda Shelly, Jamie Norrish, Jason Best, Ja-
son Bittel, Jason Novinger, Jannis Leidel, Jax, Jim Kalafut, Jim Munro, João Oliveira, Joe
Golton, John Goodleaf, John Jensen, Jonas Obrist, Jonathan Hartley, Jonathan Miller,
Josh Schreuder, Kal Sze, Karol Breguła, Kelly Nicholes, Kelly Nichols, Kevin Londo,
Khee Chin, Lachlan Musicman, Larry Prince, Lee Hinde, Maik Hoepfel, Marc Tam-
lyn, Marcin Pietranik, Martin Báchtold, Matt Harrison, Matt Johnson, Michael Reczek,
Mickey Cheong, Mike Dewhirst, Myles Braithwaite, Nick August, Nick Smith, Nicola
Marangon, Olav Andreas Lindekleiv, Patrick Jacobs, Patti Chen, Peter Heise, Peter Valdez,
Phil Davis, Prahlad Nrsimha Das, R. Michael Herberge, Richard Cochrane, Richard Cor-
den, Richard Donkin, Robbie Totten, Robert Węglarek, Rohit Aggarwa, Russ Ferriday,
Saul Shanabrook, Simon Charettes, Stefane Fermigier, Steve Klass, Tayfun Sen, Tiberiu
Ana, Tim Baxter, Timothy Goshinski, Tobias G. Waaler, Tyler Perkins, Vinay Sajip, Vinod
Kurup, Vraj Mohan, Wee Liat, William Adams, Xianyi Lin, Yan Kalchevskiy, Zed Shaw,
and Zoltán Árokszállási.
Typesetting
We thank Laura Gelsomino for helping us with all of our LaTeX issues and for improving
upon the book layout.
Laura Gelsomino is an economist keen about art and writing, and with a soft spot for
computers, who found the meeting point between her interests the day she discovered
LaTeX. Since that day, she habitually finds any excuse to vent her aesthetic sense on
any text she can lay her hands on, beginning with her economic models.
We originally typeset the alpha version of the 1.5 edition with iWork Pages. Later editions
of the book were written using LaTeX. All editions up to 1.11 have been written on 2011
Macbook Airs. The 3.x edition was written on a mix of Macbook Pros and Macbook Airs.
List of Figures
4.1 It’ll make more sense when you see the next figure. . . . . . . . . . . . . 31
4.2 Did that make sense? If not, read it again. . . . . . . . . . . . . . . . . . 32
4.3 Our vision for Icecreamlandia. . . . . . . . . . . . . . . . . . . . . . . . 33
4.4 Two small, single-flavor pints are better than a giant, 100-flavor container. 35
5.1 As your project grows, your Django settings can get pretty complex. . . . 43
5.2 While we’re at it, let’s go down this path. . . . . . . . . . . . . . . . . . . 60
6.1 Cones migrating south for the winter. Django’s built-in migration system
started out as an external project called South. . . . . . . . . . . . . . . . 70
6.2 A common source of confusion. . . . . . . . . . . . . . . . . . . . . . . 72
7.1 This flavor of ice cream contains raw SQL. It’s a bit chewy. . . . . . . . . 91
7.2 Because no one loves ice cream quite like a database. . . . . . . . . . . . . 95
9.1 If you look at sprinkles closely, you’ll see that they’re Python decorators. . 118
12.1 At Tasty Research, every flavor must begin with “Tasty”. . . . . . . . . . 145
12.2 Why would they do this to us? . . . . . . . . . . . . . . . . . . . . . . . 152
15.1 This filter transforms 1-2 flavors of ice cream into vanilla, outputting to a
cone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
20.1 Replacing more core components of cake with ice cream seems like a good
idea. Which cake would win? The one on the right! . . . . . . . . . . . . 248
23.1 A jar of Django’s mysterious secret sauce. Most don’t have a clue what this is.275
23.2 The secret is out. It’s just hot fudge. . . . . . . . . . . . . . . . . . . . . . 276
24.1 Test as much of your project as you can, as if it were free ice cream. . . . . 301
26.1 With your site running smoothly, you’ll be feeling as cool as a cone. . . . . 326
Webpack, 10
webpack, 241
Websockets with Django Channels, 429–
430
wheel, 289, 292
Windows, 12, 15, 17, 25, 51, 368, 408