In this tutorial I’m going to show you how I usually start and organize a new Django project nowadays. I’ve tried many different configurations and ways to organize the project, but for the past 4 years or so this has been consistently my go-to setup.
Please note that this is not intended to be a “best practice” guide or to fit every use case. It’s just the way I like to use Django and that’s also the way that I found that allow your project to grow in healthy way.
Index
Usually those are the premises I take into account when setting up a project:
Usually I work with three environment dimensions in my code: local, tests and production. I like to see it
as a “mode” how I run the project. What dictates which mode I’m running the project is which settings.py
I’m currently
using.
The local dimension always come first. It is the settings and setup that a developer will use on their local machine.
All the defaults and configurations must be done to attend the local development environment first.
The reason why I like to do it that way is that the project must be as simple as possible for a new hire to clone the repository, run the project and start coding.
The production environment usually will be configured and maintained by experienced developers and by those who are more familiar with the code base itself. And because the deployment should be automated, there is no reason for people being re-creating the production server over and over again. So it is perfectly fine for the production setup require a few extra steps and configuration.
The tests environment will be also available locally, so developers can test the code and run the static checks.
But the idea of the tests environment is to expose it to a CI environment like Travis CI, Circle CI, AWS Code Pipeline, etc.
It is a simple setup that you can install the project and run all the unit tests.
The production dimension is the real deal. This is the environment that goes live without the testing and debugging utilities.
I also use this “mode” or dimension to run the staging server.
A staging server is where you roll out new features and bug fixes before applying to the production server.
The idea here is that your staging server should run in production mode, and the only difference is going to be your static/media server and database server. And this can be achieved just by changing the configuration to tell what is the database connection string for example.
But the main thing is that you should not have any conditional in your code that checks if it is the production or staging server. The project should run exactly in the same way as in production.
Right from the beginning it is a good idea to setup a remote version control service. My go-to option is Git on GitHub. Usually I create the remote repository first then clone it on my local machine to get started.
Let’s say our project is called simple
, after creating the repository on GitHub I will create a directory named
simple
on my local machine, then within the simple
directory I will clone the repository, like shown on the
structure below:
Then I create the virtualenv
outside of the Git repository:
Then alongside the simple
and venv
directories I may place some other support files related to the project which I
do not plan to commit to the Git repository.
The reason I do that is because it is more convenient to destroy and re-create/re-clone both the virtual environment or the repository itself.
It is also good to store your virtual environment outside of the git repository/project root so you don’t need to bother ignoring its path when using libs like flake8, isort, black, tox, etc.
You can also use tools like virtualenvwrapper
to manage your virtual environments, but I prefer doing it that way
because everything is in one place. And if I no longer need to keep a given project on my local machine, I can delete
it completely without leaving behind anything related to the project on my machine.
The next step is installing Django inside the virtualenv so we can use the django-admin
commands.
Inside the simple
directory (where the git repository was cloned) start a new project:
Attention to the .
in the end of the command. It is necessary to not create yet another directory called simple
.
So now the structure should be something like this:
At this point I already complement the project package directory with three extra directories for templates
, static
and locale
.
Both templates
and static
we are going to manage at a project-level and app-level. Those are refer to the global
templates and static files.
The locale
is necessary in case you are using i18n
to translate your application to other languages. So here
is where you are going to store the .mo
and .po
files.
So the structure now should be something like this:
Inside the project root (2) I like to create a directory called requirements
with all the .txt
files, breaking down
the project dependencies like this:
base.txt
: Main dependencies, strictly necessary to make the project run. Common to all environmentstests.txt
: Inherits from base.txt
+ test utilitieslocal.txt
: Inherits from tests.txt
+ development utilitiesproduction.txt
: Inherits from base.txt
+ production only dependenciesNote that I do not have a staging.txt
requirements file, that’s because the staging environment is going to use the
production.txt
requirements so we have an exact copy of the production environment.
Now let’s have a look inside each of those requirements file and what are the python libraries that I always use no matter what type of Django project I’m developing.
base.txt
.env
files in a safe waysettings.py
module. It also helps with decoupling configuration from source codetests.txt
The -r base.txt
inherits all the requirements defined in the base.txt
file
local.txt
The -r tests.txt
inherits all the requirements defined in the base.txt
and tests.txt
file
production.txt
The -r base.txt
inherits all the requirements defined in the base.txt
file
Also following the environments and modes premise I like to setup multiple settings modules. Those are going to serve as the entry point to determine in which mode I’m running the project.
Inside the simple
project package, I create a new directory called settings
and break down the files like this:
Note that I removed the settings.py
that used to live inside the simple/ (3)
directory.
The majority of the code will live inside the base.py
settings module.
Everything that we can set only once in the base.py
and change its value using python-decouple
we should keep in the
base.py
and never repeat/override in the other settings modules.
After the removal of the main settings.py
a nice touch is to modify the manage.py
file to set the
local.py
as the default settings module so we can still run commands like python manage.py runserver
without any
further parameters:
manage.py
Now let’s have a look on each of those settings modules.
base.py
A few comments on the overall base settings file contents:
config()
are from the python-decouple
library. It is exposing the configuration to an environment variable and
retrieving its value accordingly to the expected data type. Read more about python-decouple
on this guide:
How to Use Python DecoupleSECRET_KEY
, DEBUG
and ALLOWED_HOSTS
defaults to local/development environment values.
That means a new developer won’t need to set a local .env
and provide some initial value to run locallydj_database_url
to translate this one line string to a Python
dictionary as Django expectsMEDIA_ROOT
we are navigating two directories up to create a media
directory outside the git
repository but inside our project workspace (inside the directory simple/ (1)
). So everything is handy and we won’t
be committing test uploads to our repositorybase.py
settings I reserve two blocks for third-party Django libraries that I may install, such
as Django Rest Framework or Django Crispy Forms. And the first-party settings refer to custom settings that I may create
exclusively for our project. Usually I will prefix them with the project name, like SIMPLE_XXX
local.py
Here is where I will setup Django Debug Toolbar for example. Or set the email backend to display the sent emails on console instead of having to setup a valid email server to work on the project.
All the code that is only relevant for the development process goes here.
You can use it to setup other libs like Django Silk to run profiling without exposing it to production.
tests.py
Here I add configurations that help us run the test cases faster. Sometimes disabling the migrations may not work if you have interdependencies between the apps models so Django may fail to create a database without the migrations.
In some projects it is better to keep the test database after the execution.
production.py
The most important part here on the production settings is to enable all the security settings Django offer. I like to do it that way because you can’t run the development server with most of those configurations on.
The other thing is the Sentry configuration.
Note the simple.__version__
on the release. Next we are going to explore how I usually manage the version of the
project.
I like to reuse Django’s get_version
utility for a simple and PEP 440 complaint version identification.
Inside the project’s __init__.py
module:
You can do something like this:
The only down side of using the get_version
directly from the Django module is that it won’t be able to resolve the
git hash for alpha versions.
A possible solution is making a copy of the django/utils/version.py
file to your project, and then you import it
locally, so it will be able to identify your git repository within the project folder.
But it also depends what kind of versioning you are using for your project. If the version of your project is not really relevant to the end user and you want to keep track of it for internal management like to identify the release on a Sentry issue, you could use a date-based release versioning.
A Django app is a Python package that you “install” using the INSTALLED_APPS
in your settings file. An app can live pretty
much anywhere: inside or outside the project package or even in a library that you installed using pip
.
Indeed, your Django apps may be reusable on other projects. But that doesn’t mean it should. Don’t let it destroy your project design or don’t get obsessed over it. Also, it shouldn’t necessarily represent a “part” of your website/web application.
It is perfectly fine for some apps to not have models, or other apps have only views. Some of your modules doesn’t even need to be a Django app at all. I like to see my Django projects as a big Python package and organize it in a way that makes sense, and not try to place everything inside reusable apps.
The general recommendation of the official Django documentation is to place your apps in the project root (alongside
the manage.py file, identified here in this tutorial by the simple/ (2)
folder).
But actually I prefer to create my apps inside the project package (identified in this tutorial by the simple/ (3)
folder). I create a module named apps
and then inside the apps
I create my Django apps. The main reason why is that
it creates a nice namespace for the app. It helps you easily identify that a particular import is part of your
project. Also this namespace helps when creating logging rules to handle events in a different way.
Here is an example of how I do it:
In the example above the folders accounts/
and core/
are Django apps created with the command django-admin startapp
.
Those two apps are also always in my project. The accounts
app is the one that I use the replace the default Django
User
model and also the place where I eventually create password reset, account activation, sign ups, etc.
The core
app I use for general/global implementations. For example to define a model that will be used across most
of the other apps. I try to keep it decoupled from other apps, not importing other apps resources. It usually is a good
place to implement general purpose or reusable views and mixins.
Something to pay attention when using this approach is that you need to change the name
of the apps configuration,
inside the apps.py
file of the Django app:
accounts/apps.py
You should rename it like this, to respect the namespace:
Then on your INSTALLED_APPS
you are going to create a reference to your models like this:
The namespace also helps to organize your INSTALLED_APPS
making your project apps easily recognizable.
This is what my app structure looks like:
The first thing I do is create a folder named tests
so I can break down my tests into several files. I always add a
factories.py
to create my model factories using the factory-boy
library.
For both static
and templates
always create first a directory with the same name as the app to avoid name collisions
when Django collect all static files and try to resolve the templates.
The admin.py
may be there or not depending if I’m using the Django Admin contrib app.
Other common modules that you may have is a utils.py
, forms.py
, managers.py
, services.py
etc.
Now I’m going to show you the configuration that I use for tools like isort
, black
, flake8
, coverage
and tox
.
The .editorconfig
file is a standard recognized by all major IDEs and code editors. It helps the editor understand
what is the file formatting rules used in the project.
It tells the editor if the project is indented with tabs or spaces. How many spaces/tabs. What’s the max length for a line of code.
I like to use Django’s .editorconfig
file. Here is what it looks like:
.editorconfig
Flake8 is a Python library that wraps PyFlakes, pycodestyle and Ned Batchelder’s McCabe script. It is a great toolkit for checking your code base against coding style (PEP8), programming errors (like “library imported but unused” and “Undefined name”) and to check cyclomatic complexity.
To learn more about flake8, check this tutorial I posted a while a go: How to Use Flake8.
setup.cfg
isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections.
To learn more about isort, check this tutorial I posted a while a go: How to Use Python isort Library.
setup.cfg
Pay attention to the known_first_party
, it should be the name of your project so isort can group your project’s
imports.
Black is a life changing library to auto-format your Python applications. There is no way I’m coding with Python nowadays without using Black.
Here is the basic configuration that I use:
pyproject.toml
In this tutorial I described my go-to project setup when working with Django. That’s pretty much how I start all my projects nowadays.
Here is the final project structure for reference:
You can also explore the code on GitHub: django-production-template.
Chart.js is a cool open source JavaScript library that helps you render HTML5 charts. It is responsive and counts with 8 different chart types.
In this tutorial we are going to explore a little bit of how to make Django talk with Chart.js and render some simple charts based on data extracted from our models.
For this tutorial all you are going to do is add the Chart.js lib to your HTML page:
You can download it from Chart.js official website and use it locally, or you can use it from a CDN using the URL above.
I’m going to use the same example I used for the tutorial How to Create Group By Queries With Django ORM which is a good complement to this tutorial because actually the tricky part of working with charts is to transform the data so it can fit in a bar chart / line chart / etc.
We are going to use the two models below, Country
and City
:
And the raw data stored in the database:
cities | |||
---|---|---|---|
id | name | country_id | population |
1 | Tokyo | 28 | 36,923,000 |
2 | Shanghai | 13 | 34,000,000 |
3 | Jakarta | 19 | 30,000,000 |
4 | Seoul | 21 | 25,514,000 |
5 | Guangzhou | 13 | 25,000,000 |
6 | Beijing | 13 | 24,900,000 |
7 | Karachi | 22 | 24,300,000 |
8 | Shenzhen | 13 | 23,300,000 |
9 | Delhi | 25 | 21,753,486 |
10 | Mexico City | 24 | 21,339,781 |
11 | Lagos | 9 | 21,000,000 |
12 | São Paulo | 1 | 20,935,204 |
13 | Mumbai | 25 | 20,748,395 |
14 | New York City | 20 | 20,092,883 |
15 | Osaka | 28 | 19,342,000 |
16 | Wuhan | 13 | 19,000,000 |
17 | Chengdu | 13 | 18,100,000 |
18 | Dhaka | 4 | 17,151,925 |
19 | Chongqing | 13 | 17,000,000 |
20 | Tianjin | 13 | 15,400,000 |
21 | Kolkata | 25 | 14,617,882 |
22 | Tehran | 11 | 14,595,904 |
23 | Istanbul | 2 | 14,377,018 |
24 | London | 26 | 14,031,830 |
25 | Hangzhou | 13 | 13,400,000 |
26 | Los Angeles | 20 | 13,262,220 |
27 | Buenos Aires | 8 | 13,074,000 |
28 | Xi'an | 13 | 12,900,000 |
29 | Paris | 6 | 12,405,426 |
30 | Changzhou | 13 | 12,400,000 |
31 | Shantou | 13 | 12,000,000 |
32 | Rio de Janeiro | 1 | 11,973,505 |
33 | Manila | 18 | 11,855,975 |
34 | Nanjing | 13 | 11,700,000 |
35 | Rhine-Ruhr | 16 | 11,470,000 |
36 | Jinan | 13 | 11,000,000 |
37 | Bangalore | 25 | 10,576,167 |
38 | Harbin | 13 | 10,500,000 |
39 | Lima | 7 | 9,886,647 |
40 | Zhengzhou | 13 | 9,700,000 |
41 | Qingdao | 13 | 9,600,000 |
42 | Chicago | 20 | 9,554,598 |
43 | Nagoya | 28 | 9,107,000 |
44 | Chennai | 25 | 8,917,749 |
45 | Bangkok | 15 | 8,305,218 |
46 | Bogotá | 27 | 7,878,783 |
47 | Hyderabad | 25 | 7,749,334 |
48 | Shenyang | 13 | 7,700,000 |
49 | Wenzhou | 13 | 7,600,000 |
50 | Nanchang | 13 | 7,400,000 |
51 | Hong Kong | 13 | 7,298,600 |
52 | Taipei | 29 | 7,045,488 |
53 | Dallas–Fort Worth | 20 | 6,954,330 |
54 | Santiago | 14 | 6,683,852 |
55 | Luanda | 23 | 6,542,944 |
56 | Houston | 20 | 6,490,180 |
57 | Madrid | 17 | 6,378,297 |
58 | Ahmedabad | 25 | 6,352,254 |
59 | Toronto | 5 | 6,055,724 |
60 | Philadelphia | 20 | 6,051,170 |
61 | Washington, D.C. | 20 | 6,033,737 |
62 | Miami | 20 | 5,929,819 |
63 | Belo Horizonte | 1 | 5,767,414 |
64 | Atlanta | 20 | 5,614,323 |
65 | Singapore | 12 | 5,535,000 |
66 | Barcelona | 17 | 5,445,616 |
67 | Munich | 16 | 5,203,738 |
68 | Stuttgart | 16 | 5,200,000 |
69 | Ankara | 2 | 5,150,072 |
70 | Hamburg | 16 | 5,100,000 |
71 | Pune | 25 | 5,049,968 |
72 | Berlin | 16 | 5,005,216 |
73 | Guadalajara | 24 | 4,796,050 |
74 | Boston | 20 | 4,732,161 |
75 | Sydney | 10 | 5,000,500 |
76 | San Francisco | 20 | 4,594,060 |
77 | Surat | 25 | 4,585,367 |
78 | Phoenix | 20 | 4,489,109 |
79 | Monterrey | 24 | 4,477,614 |
80 | Inland Empire | 20 | 4,441,890 |
81 | Rome | 3 | 4,321,244 |
82 | Detroit | 20 | 4,296,611 |
83 | Milan | 3 | 4,267,946 |
84 | Melbourne | 10 | 4,650,000 |
countries | |
---|---|
id | name |
1 | Brazil |
2 | Turkey |
3 | Italy |
4 | Bangladesh |
5 | Canada |
6 | France |
7 | Peru |
8 | Argentina |
9 | Nigeria |
10 | Australia |
11 | Iran |
12 | Singapore |
13 | China |
14 | Chile |
15 | Thailand |
16 | Germany |
17 | Spain |
18 | Philippines |
19 | Indonesia |
20 | United States |
21 | South Korea |
22 | Pakistan |
23 | Angola |
24 | Mexico |
25 | India |
26 | United Kingdom |
27 | Colombia |
28 | Japan |
29 | Taiwan |
For the first example we are only going to retrieve the top 5 most populous cities and render it as a pie chart. In this strategy we are going to return the chart data as part of the view context and inject the results in the JavaScript code using the Django Template language.
views.py
Basically in the view above we are iterating through the City
queryset and building a list of labels
and a list of
data
. Here in this case the data
is the population count saved in the City
model.
For the urls.py
just a simple routing:
urls.py
Now the template. I got a basic snippet from the Chart.js Pie Chart Documentation.
pie_chart.html
In the example above the base.html
template is not important but you can see it in the code example I shared in the
end of this post.
This strategy is not ideal but works fine. The bad thing is that we are using the Django Template Language to interfere
with the JavaScript logic. When we put {{ data|safe}}
we are injecting a variable that came from
the server directly in the JavaScript code.
The code above looks like this:
As the title says, we are now going to render a bar chart using an async call.
views.py
So here we are using two views. The home
view would be the main page where the chart would be loaded at. The other
view population_chart
would be the one with the sole responsibility to aggregate the data the return a JSON response
with the labels and data.
If you are wondering about what this queryset is doing, it is grouping the cities by the country and aggregating the total population of each country. The result is going to be a list of country + total population. To learn more about this kind of query have a look on this post: How to Create Group By Queries With Django ORM
urls.py
home.html
Now we have a better separation of concerns. Looking at the chart container:
We added a reference to the URL that holds the chart rendering logic. Later on we are using it to execute the Ajax call.
Inside the success
callback we then finally execute the Chart.js related code using the JsonResponse
data.
I hope this tutorial helped you to get started with working with charts using Chart.js. I published another tutorial on the same subject a while ago but using the Highcharts library. The approach is pretty much the same: How to Integrate Highcharts.js with Django.
If you want to grab the code I used in this tutorial you can find it here: github.com/sibtc/django-chartjs-example.
In this tutorial you are going to learn how to pass extra data to your serializer, before saving it to the database.
When using regular Django forms, there is this common pattern where we save the form with commit=False
and then pass
some extra data to the instance before saving it to the database, like this:
This is very useful because we can save the required information using only one database query and it also make it possible to handle not nullable columns that was not defined in the form.
To simulate this pattern using a Django REST Framework serializer you can do something like this:
You can also pass several parameters at once:
In this example I created an app named core
.
models.py
serializers.py
views.py
Very similar example, using the same models.py and serializers.py as in the previous example.
views.py
In this tutorial we are going to explore three date/datetime pickers options that you can easily use in a Django project. We are going to explore how to do it manually first, then how to set up a custom widget and finally how to use a third-party Django app with support to datetime pickers.
The implementation of a date picker is mostly done on the front-end.
The key part of the implementation is to assure Django will receive the date input value in the correct format, and also that Django will be able to reproduce the format when rendering a form with initial data.
We can also use custom widgets to provide a deeper integration between the front-end and back-end and also to promote better reuse throughout a project.
In the next sections we are going to explore following date pickers:
Tempus Dominus Bootstrap 4 Docs Source
XDSoft DateTimePicker Docs Source
Fengyuan Chen’s Datepicker Docs Source
This is a great JavaScript library and it integrate well with Bootstrap 4. The downside is that it requires moment.js and sort of need Font-Awesome for the icons.
It only make sense to use this library with you are already using Bootstrap 4 + jQuery, otherwise the list of CSS and JS may look a little bit overwhelming.
To install it you can use their CDN or download the latest release from their GitHub Releases page.
If you downloaded the code from the releases page, grab the processed code from the build/ folder.
Below, a static HTML example of the datepicker:
The challenge now is to have this input snippet integrated with a Django form.
forms.py
template
The script tag can be placed anywhere because the snippet $(function () { ... });
will run the datetimepicker
initialization when the page is ready. The only requirement is that this script tag is placed after the jQuery script
tag.
You can create the widget in any app you want, here I’m going to consider we have a Django app named core.
core/widgets.py
In the implementation above we generate a unique ID datetimepicker_id
and also include it in the widget context.
Then the front-end implementation is done inside the widget HTML snippet.
widgets/bootstrap_datetimepicker.html
Note how we make use of the built-in django/forms/widgets/input.html
template.
Now the usage:
core/forms.py
Now simply render the field:
template
The good thing about having the widget is that your form could have several date fields using the widget and you could simply render the whole form like:
The XDSoft DateTimePicker is a very versatile date picker and doesn’t rely on moment.js or Bootstrap, although it looks good in a Bootstrap website.
It is easy to use and it is very straightforward.
You can download the source from GitHub releases page.
Below, a static example so you can see the minimum requirements and how all the pieces come together:
A basic integration with Django would look like this:
forms.py
Simple form, default widget, nothing special.
Now using it on the template:
template
The id_date
is the default ID Django generates for the form fields (id_
+ name
).
core/widgets.py
widgets/xdsoft_datetimepicker.html
To have a more generic implementation, this time we are selecting the field to initialize the component using its name instead of its id, should the user change the id prefix.
Now the usage:
core/forms.py
template
This is a very beautiful and minimalist date picker. Unfortunately there is no time support. But if you only need dates this is a great choice.
To install this datepicker you can either use their CDN or download the sources from their GitHub releases page. Please note that they do not provide a compiled/processed JavaScript files. But you can download those to your local machine using the CDN.
A basic integration with Django (note that we are now using DateField
instead of DateTimeField
):
forms.py
template
core/widgets.py
widgets/fengyuanchen_datepicker.html
Usage:
core/forms.py
template
The implementation is very similar no matter what date/datetime picker you are using. Hopefully this tutorial provided some insights on how to integrate this kind of frontend library to a Django project.
As always, the best source of information about each of those libraries are their official documentation.
I also created an example project to show the usage and implementation of the widgets for each of the libraries presented in this tutorial. Grab the source code at github.com/sibtc/django-datetimepicker-example.
The Django forms API have two field types to work with multiple options: ChoiceField
and ModelChoiceField
.
Both use select input as the default widget and they work in a similar way, except that ModelChoiceField
is designed
to handle QuerySets and work with foreign key relationships.
A basic implementation using a ChoiceField
would be:
You can also organize the choices in groups to generate the <optgroup>
tags like this:
When you are using a ModelChoiceField
unfortunately there is no built-in solution.
Recently I found a nice solution on Django’s ticket tracker, where
someone proposed adding an opt_group
argument to the ModelChoiceField
.
While the discussion is still ongoing, Simon Charette proposed a really good solution.
Let’s see how we can integrate it in our project.
First consider the following models:
models.py
So now our category instead of being a regular choices field it is now a model and the Expense
model have a
relationship with it using a foreign key.
If we create a ModelForm
using this model, the result will be very similar to our first example.
To simulate a grouped categories you will need the code below. First create a new module named fields.py:
fields.py
And here is how you use it in your forms:
forms.py
Because in the example above I used a self-referencing relationship I had to add the exclude(parent=None)
to hide
the “group categories” from showing up in the select input as a valid option.
You can download the code used in this tutorial from GitHub: github.com/sibtc/django-grouped-choice-field-example
Credits to the solution Simon Charette on Django Ticket Track.
JWT stand for JSON Web Token and it is an authentication strategy used by client/server applications where the client is a Web application using JavaScript and some frontend framework like Angular, React or VueJS.
In this tutorial we are going to explore the specifics of JWT authentication. If you want to learn more about Token-based authentication using Django REST Framework (DRF), or if you want to know how to start a new DRF project you can read this tutorial: How to Implement Token Authentication using Django REST Framework. The concepts are the same, we are just going to switch the authentication backend.
The JWT is just an authorization token that should be included in all requests:
The JWT is acquired by exchanging an username + password for an access token and an refresh token.
The access token is usually short-lived (expires in 5 min or so, can be customized though).
The refresh token lives a little bit longer (expires in 24 hours, also customizable). It is comparable to an authentication session. After it expires, you need a full login with username + password again.
Why is that?
It’s a security feature and also it’s because the JWT holds a little bit more information. If you look closely the example I gave above, you will see the token is composed by three parts:
Those are three distinctive parts that compose a JWT:
So we have here:
This information is encoded using Base64. If we decode, we will see something like this:
header
payload
signature
The signature is issued by the JWT backend, using the header base64 + payload base64 + SECRET_KEY
. Upon each request
this signature is verified. If any information in the header or in the payload was changed by the client it will
invalidate the signature. The only way of checking and validating the signature is by using your application’s
SECRET_KEY
. Among other things, that’s why you should always keep your SECRET_KEY
secret!
For this tutorial we are going to use the djangorestframework_simplejwt
library, recommended by the DRF developers.
settings.py
urls.py
For this tutorial I will use the following route and API view:
views.py
urls.py
I will be using HTTPie to consume the API endpoints via the terminal. But you can also use cURL (readily available in many OS) to try things out locally.
Or alternatively, use the DRF web interface by accessing the endpoint URLs like this:
First step is to authenticate and obtain the token. The endpoint is /api/token/
and it only accepts POST requests.
So basically your response body is the two tokens:
After that you are going to store both the access token and the refresh token on the client side, usually in the localStorage.
In order to access the protected views on the backend (i.e., the API endpoints that require authentication), you should include the access token in the header of all requests, like this:
You can use this access token for the next five minutes.
After five min, the token will expire, and if you try to access the view again, you are going to get the following error:
To get a new access token, you should use the refresh token endpoint /api/token/refresh/
posting the
refresh token:
The return is a new access token that you should use in the subsequent requests.
The refresh token is valid for the next 24 hours. When it finally expires too, the user will need to perform a full authentication again using their username and password to get a new set of access token + refresh token.
At first glance the refresh token may look pointless, but in fact it is necessary to make sure the user still have the correct permissions. If your access token have a long expire time, it may take longer to update the information associated with the token. That’s because the authentication check is done by cryptographic means, instead of querying the database and verifying the data. So some information is sort of cached.
There is also a security aspect, in a sense that the refresh token only travel in the POST data. And the access token is sent via HTTP header, which may be logged along the way. So this also give a short window, should your access token be compromised.
This should cover the basics on the backend implementation. It’s worth checking the djangorestframework_simplejwt settings for further customization and to get a better idea of what the library offers.
The implementation on the frontend depends on what framework/library you are using. Some libraries and articles covering popular frontend frameworks like angular/react/vue.js:
The code used in this tutorial is available at github.com/sibtc/drf-jwt-example.
[Django 2.1.3 / Python 3.6.5 / Bootstrap 4.1.3]
In this tutorial we are going to explore some of the Django Crispy Forms features to handle advanced/custom forms rendering. This blog post started as a discussion in our community forum, so I decided to compile the insights and solutions in a blog post to benefit a wider audience.
Table of Contents
Throughout this tutorial we are going to implement the following Bootstrap 4 form using Django APIs:
This was taken from Bootstrap 4 official documentation as an example of how to use form rows.
NOTE!
The examples below refer to a base.html
template. Consider the code below:
base.html
Install it using pip:
Add it to your INSTALLED_APPS
and select which styles to use:
settings.py
For detailed instructions about how to install django-crispy-forms
, please refer to this tutorial:
How to Use Bootstrap 4 Forms With Django
The Python code required to represent the form above is the following:
In this case I’m using a regular Form
, but it could also be a ModelForm
based on a Django model with similar
fields. The state
field and the STATES
choices could be either a foreign key or anything else. Here I’m just using
a simple static example with three Brazilian states.
Template:
Rendered HTML:
Rendered HTML with validation state:
Same form code as in the example before.
Template:
Rendered HTML:
Rendered HTML with validation state:
Same form code as in the first example.
Template:
Rendered HTML:
Rendered HTML with validation state:
We could use the crispy forms layout helpers to achieve the same result as above. The implementation is done inside
the form __init__
method:
forms.py
The template implementation is very minimal:
The end result is the same.
Rendered HTML:
Rendered HTML with validation state:
You may also customize the field template and easily reuse throughout your application. Let’s say we want to use the custom Bootstrap 4 checkbox:
From the official documentation, the necessary HTML to output the input above:
Using the crispy forms API, we can create a new template for this custom field in our “templates” folder:
custom_checkbox.html
Now we can create a new crispy field, either in our forms.py module or in a new Python module named fields.py or something.
forms.py
We can use it now in our form definition:
forms.py
(PS: the AddressForm
was defined here and is the same as in the previous example.)
The end result:
There is much more Django Crispy Forms can do. Hopefully this tutorial gave you some extra insights on how to use the form helpers and layout classes. As always, the official documentation is the best source of information:
Django Crispy Forms layouts docs
Also, the code used in this tutorial is available on GitHub at github.com/sibtc/advanced-crispy-forms-examples.
In this tutorial you are going to learn how to implement Token-based authentication using Django REST Framework (DRF). The token authentication works by exchanging username and password for a token that will be used in all subsequent requests so to identify the user on the server side.
The specifics of how the authentication is handled on the client side vary a lot depending on the technology/language/framework you are working with. The client could be a mobile application using iOS or Android. It could be a desktop application using Python or C++. It could be a Web application using PHP or Ruby.
But once you understand the overall process, it’s easier to find the necessary resources and documentation for your specific use case.
Token authentication is suitable for client-server applications, where the token is safely stored. You should never expose your token, as it would be (sort of) equivalent of a handing out your username and password.
Table of Contents
So let’s start from the very beginning. Install Django and DRF:
Create a new Django project:
Navigate to the myapi folder:
Start a new app. I will call my app core:
Here is what your project structure should look like:
Add the core app (you created) and the rest_framework app (you installed) to the INSTALLED_APPS
, inside the
settings.py module:
myapi/settings.py
Return to the project root (the folder where the manage.py script is), and migrate the database:
Let’s create our first API view just to test things out:
myapi/core/views.py
Now register a path in the urls.py module:
myapi/urls.py
So now we have an API with just one endpoint /hello/
that we can perform GET
requests. We can use the browser to
consume this endpoint, just by accessing the URL http://127.0.0.1:8000/hello/
:
We can also ask to receive the response as plain JSON data by passing the format
parameter in the querystring like
http://127.0.0.1:8000/hello/?format=json
:
Both methods are fine to try out a DRF API, but sometimes a command line tool is more handy as we can play more easily with the requests headers. You can use cURL, which is widely available on all major Linux/macOS distributions:
But usually I prefer to use HTTPie, which is a pretty awesome Python command line tool:
Now let’s protect this API endpoint so we can implement the token authentication:
myapi/core/views.py
Try again to access the API endpoint:
And now we get an HTTP 403 Forbidden error. Now let’s implement the token authentication so we can access this endpoint.
We need to add two pieces of information in our settings.py module. First include rest_framework.authtoken to
your INSTALLED_APPS
and include the TokenAuthentication
to REST_FRAMEWORK
:
myapi/settings.py
Migrate the database to create the table that will store the authentication tokens:
Now we need a user account. Let’s just create one using the manage.py
command line utility:
The easiest way to generate a token, just for testing purpose, is using the command line utility again:
This piece of information, the random string 9054f7aa9305e012b3c2300408c3dfdf390fcddf
is what we are going to use
next to authenticate.
But now that we have the TokenAuthentication
in place, let’s try to make another request to our /hello/
endpoint:
Notice how our API is now providing some extra information to the client on the required authentication method.
So finally, let’s use our token!
And that’s pretty much it. For now on, on all subsequent request you should include the header Authorization: Token 9054f7aa9305e012b3c2300408c3dfdf390fcddf
.
The formatting looks weird and usually it is a point of confusion on how to set this header. It will depend on the client and how to set the HTTP request header.
For example, if we were using cURL, the command would be something like this:
Or if it was a Python requests call:
Or if we were using Angular, you could implement an HttpInterceptor
and set a header:
The DRF provide an endpoint for the users to request an authentication token using their username and password.
Include the following route to the urls.py module:
myapi/urls.py
So now we have a brand new API endpoint, which is /api-token-auth/
. Let’s first inspect it:
It doesn’t handle GET requests. Basically it’s just a view to receive a POST request with username and password.
Let’s try again:
The response body is the token associated with this particular user. After this point you store this token and apply it to the future requests.
Then, again, the way you are going to make the POST request to the API depends on the language/framework you are using.
If this was an Angular client, you could store the token in the localStorage
, if this was a Desktop CLI application
you could store in a text file in the user’s home directory in a dot file.
Hopefully this tutorial provided some insights on how the token authentication works. I will try to follow up this tutorial providing some concrete examples of Angular applications, command line applications and Web clients as well.
It is important to note that the default Token implementation has some limitations such as only one token per user, no built-in way to set an expiry date to the token.
You can grab the code used in this tutorial at github.com/sibtc/drf-token-auth-example.
Django comes with a variety of command line utilities that can be either invoked using django-admin.py
or the
convenient manage.py
script. A nice thing about it is that you can also add your own commands. Those management
commands can be very handy when you need to interact with your application via command line using a terminal and it can
also serve as an interface to execute cron jobs. In this tutorial you are going to learn how to code your own commands.
Just before we get started, let’s take a moment to familiarize with Django’s command line interface. You are probably
already familiar with commands like startproject
, runserver
or collectstatic
. To see a complete list of commands
you can run the command below:
Output:
We can create our own commands for our apps and include them in the list by creating a management/commands directory inside an app directory, like below:
The name of the command file will be used to invoke using the command line utility. For example, if our command was
called my_custom_command.py
, then we will be able to execute it via:
Let’s explore next our first example.
Below, a basic example of what the custom command should look like:
management/commands/what_time_is_it.py
Basically a Django management command is composed by a class named Command
which inherits from BaseCommand
. The
command code should be defined inside the handle()
method.
See how we named our module what_time_is_it.py
. This command can be executed as:
Output:
You may be asking yourself, how is that different from a regular Python script, or what’s the benefit of it. Well, the main advantage is that all Django machinery is loaded and ready to be used. That means you can import models, execute queries to the database using Django’s ORM and interact with all your project’s resources.
Django make use of the argparse, which is part of Python’s standard
library. To handle arguments in our custom command we should define a method named add_arguments
.
The next example is a command that create random user instances. It takes a mandatory argument named total
, which
will define the number of users that will be created by the command.
management/commands/create_users.py
Here is how one would use it:
The optional (and named) arguments can be passed in any order. In the example below you will find the definition of an argument named “prefix”, which will be used to compose the username field:
management/commands/create_users.py
Usage:
or
If the prefix is used, the username field will be created as custom_user_oYwoxtt4vNHR
. If not prefix, it will be
created simply as oYwoxtt4vNHR
– a random string.
Another type of optional arguments are flags, which are used to handle boolean values. Let’s say we want to add an
--admin
flag, to instruct our command to create a super user or to create a regular user if the flag is not present.
management/commands/create_users.py
Usage:
Or
Let’s create a new command now named delete_users. In this new command we will be able to pass a list of user ids and the command should delete those users from the database.
management/commands/delete_users.py
Usage:
Output:
We can also pass a number of ids separated by spaces, so the command will delete the users in a single call:
Output:
We could improve the previous example a little big by setting an appropriate color to the output message:
management/commands/delete_users.py
Usage is the same as before, difference now is just the output:
Output:
Below a list of all available styles, in form of a management command:
If you have a task that must run periodically, like generating a report every Monday. Or let’s say you have a Web scrapper that collects data from some Website every 10 minutes. You can define this code as a management command and simply add it to your server’s crontab like this:
The example above will execute the my_custom_command
every day at 4 a.m.
The examples above should be enough to get you started. More advanced usage will boil down to knowing how to use the argparse features. And of course, Django’s official documentation on management commands is the best resource.
You can find all the code used in this tutorial on GitHub.
This is a quick tutorial to get you start with django-crispy-forms and never look back. Crispy-forms is a great application that gives you control over how you render Django forms, without breaking the default behavior. This tutorial is going to be tailored towards Bootstrap 4, but it can also be used with older Bootstrap versions as well as with the Foundation framework.
The main reason why I like to use it on my projects is because you can simply render a Django form using `` and it will be nicely rendered with Bootstrap 4, with very minimal setup. It’s a really life saver.
Install it using pip:
Add it to your INSTALLED_APPS
and select which styles to use:
settings.py
You can either download the latest Bootstrap 4 version at getbootstrap.com. In that case, go to download page and get the Compiled CSS and JS version.
Or you can use the hosted Bootstrap CDN:
For simplicity, I will be using the CDN version. Here is my base.html template that will be referenced in the following examples:
I only added the CSS file because we won’t be using any JavaScript feature.
Suppose we have a model named Person
as follows:
models.py
Let’s say we wanted to create a view to add new Person
objects. In that case we could use the built-in CreateView
:
views.py
Without any further change, Django will try to use a template named people/person_form.html
. In that case “people”
is the name of my Django app:
people/person_form.html
This is a very basic form rendering, and as it is, Django will render it like this, with no style, just plain form fields:
To render the same form using Bootstrap 4 CSS classes you can do the following:
people/person_form.html
Now the result, much better:
There are some cases where you may want more freedom to render your fields. You can do so by rendering the fields
manually and using the as_crispy_field
template filter:
And the result is something like the screen shot below:
The django-crispy-forms app have a special class named FormHelper
to make your life easier and to give you complete
control over how you want to render your forms.
Here is an example of an update view:
forms.py
The job is done inside the __init__()
method. The rest is just a regular Django model form. Here I’m defining that
this form should handle the request using the POST method and the form should have an submit button with label
“Save person”.
Now our view, just regular Django code:
views.py
Then in our template:
people/person_update_form.html
Here we can simply call the {% crispy %}
template tag and pass our form instance as parameter.
And that’s all you need to render the form:
That’s pretty much it for the basics. Honestly that’s about all that I use. Usually I don’t even go for the
FormHelper
objects. But there are much more about it. If you are interested, you can check their official
documentation: django-crispy-forms.readthedocs.io.
If you are not sure about where you should create a certain file, or want to explore the sample project I created for this tutorial, you can grab the source code on GitHub at github.com/sibtc/bootstrap-forms-example.
Highcharts is, in my opinion, one of the best JavaScript libraries to work with data visualization and charts out there. Even though Highcharts is open source, it’s a commercial library. It’s free for use in non-commercial applications though.
In this tutorial we are going to explore how to integrate it with a Django project to render dynamically generated charts. In relation to drawing the charts and rendering it to the client, all the hard work is done by Highcharts at the client side. The configuration and setup is pure JavaScript.
The main challenge here is on how to translate the data from your backend to a format that Highcharts will understand. This data may come from a database or an external API, and probably is represented as Python objects (like in a QuerySet), or simply be represented as Python dictionaries or lists.
Generally speaking, there are two ways to do it:
The first option is like a brute force and in many cases the easiest way. The second option requires a slightly complicated setup, but you will also benefit from the page loading speed and from the maintainability of the code.
Basically we just need to include the Highcharts library in our template and we are ready to go. You can either download and serve it locally or simply use their CDN:
Now we need some data. I thought that it would be fun to play with an existing dataset. The Titanic dataset is pretty famous one, and easy to access.
What I did here was loading the dataset (1300~ rows) into a model named Passenger
:
If you are familiar with data mining, data science, or machine learning probably you already know this data set. This dataset is usually used for learning purpose. It’s composed by the list of passengers of the famous RMS Titanic tragedy.
We won’t be doing anything smart with it, just querying the database and displaying the data using Highcharts.
I won’t dive into deep details about Highcharts. The goal is to understand how to make Django and Highcharts talk. For details about how to do this or that with Highcharts, best thing is to consult the official documentation.
Here is a working example of a column chart using Highcharts:
The code above generates the following chart:
The basic structure here is:
The most straightforward way to do it is by writing directly in the template (which is not recommended).
Let’s write an query to display the number of survivors and deaths organized by ticket class.
views.py
The queryset above generates a data in the following format:
Then we could just write it in the template, inside the JavaScript tags:
ticket_class.html
This kind of strategy is not really a good idea because the code is hard to read, hard to maintain and it is too easy
to shoot in the foot. Because we are using Python to generate JavaScript code, we have to format it properly.
For example, the code {% if not forloop.last %}, {% endif %}
is to not append a comma (,) after
the last item of the array (otherwise the result would be [200, 119, 181,]
). The newest JavaScript versions are
forgiving and accepts an extra comma (like Python does), but older versions don’t, so it might cause problem in old
browsers. Anyway, the point is you have to make sure your Python code is writing valid JavaScript code.
A slightly better way to do it would be processing the data a little bit more in the view:
views.py
ticket_class_2.html
Here’s what we are doing: first run through the queryset and create three separate lists, append the values and do the
formatting. After that, use the json
module and dump the Python lists into JSON format. The result are Python strings
properly formatted as JSON data.
We have to use the safe
template filter to properly render the categories
because Django automatically escape
characters like '
and "
for safety reason, so we have to instruct Django to trust and render it as it is.
We could also do all the configuration in the backend, like this:
views.py
ticket_class_3.html
As you can see, that way we move all the configuration to the server side. But we are still interacting with the JavaScript code directly.
Now this is how I usually like to work with Highcharts (or any other JavaScript library that interacts with the server).
The idea here is to render the chart using an asynchronous call, returning a JsonResponse
from the server.
This time, we are going to need two routes:
urls.py
The json_example
URL route is pointing to a regular view, which will render the template which will invoke the
chart_data
view. This call can be automatic upon page load, or it can be triggered by an action (a button click
for example).
views.py
Here we can see the json_example
is nothing special, just returning the json_example.html
template, which we are
going to explore in a minute.
The chart_data
is the one doing all the hard work. Here we have the database query and building the chart
dictionary. In the end we return the chart data as a JSON object.
json_example.html
Here is where the magic happens. The div with id container
is where the chart is going to be rendered. Now, observe
that I included a custom attribute named data-url
. Inside this attribute I stored the path to the view that will be
used to load the chart data.
Inside the ajax call, we make the request based on the URL provided in the data-url
and instruct the ajax request
that we are expecting a JSON object in return (defined by the dataType
). When the request completes, the JSON
response will be inside the data
parameter in the success function. Finally, inside the success
function, we
render the chart using the Highcharts API.
This is extremely useful because now we can decouple all the JavaScript from our template. In this example we used a single file for simplicity, but nothing stops us now from saving the script tag content in a separate file. This is great because we are no longer mixing the Django template language with JavaScript.
The result is the following screen shot:
In this tutorial we explored the basics on how to integrate Highcharts.js with Django. The implementation concepts used in this tutorial can be applied in other charts libraries such as Charts.js. The process should be very similar.
Whenever possible, try to avoid interacting with JavaScript code using the Django Template Language. Prefer returning the data as JSON objects already processed and ready to use.
Usually when working with charts and data visualization the most challenging part is to squeeze the data in the format required to render the chart. What I usually do is first create a static example hardcoding the data so I can have an idea about the data format. Next, I start creating the QuerySet using in the Python terminal. After I get it right, I finally write the view function.
If you want to learn more, the best way is to get your hands dirty. Here’s something you can do: