Connexion
I have a tutorial that explains the basics of the SSH client configuration file and some of the most common configuration options for Linux and Unix developers or sysadmin. In this quick tip, I will explain how and why you sometimes need to ignore or exclude the ssh client config file under Linux, Unix, *BSD and macOS.
Love this? sudo share_on: Twitter - Facebook - LinkedIn - Whatsapp - Reddit
The post How to tell ssh client to ignore ssh config file appeared first on nixCraft.
Sometimes I need a specific Ansible module from ansible-galaxy to carry out a particular type of automation. But I don't know if it is already installed or not or bundled with Ansible when I installed the latest version on Ubuntu Linux. So here is a quick command-line tip for finding out if the Ansible module is installed or not on your development workstations powered by Linux, Unix or macOS.
Love this? sudo share_on: Twitter - Facebook - LinkedIn - Whatsapp - Reddit
The post How to check if Ansible collection is installed or not appeared first on nixCraft.
The post How to Change Hostname on Ubuntu 20.04 appeared first on nixCraft.
How do I change my default shell in Linux?
The post Change my default shell in Linux using chsh appeared first on nixCraft.
How do I see how many processes are running in Linux? How can count all running processes in Linux?
The post How do I count how many processes are running in Linux? appeared first on nixCraft.
I am running file /usr/sbin/arp command on a Debian VM minimal but getting an error that read as:
bash: file: command not foundHow do I fix this error on a Debian or Ubuntu Linux system? How can I install file command on Linux cloud?
The post bash: file: command not found. How to install file appeared first on nixCraft.
Every Django website needs some automatic background tasks to execute regularly. The outdated sessions need to be cleaned up, search index needs to be updated, some data needs to be imported from RSS feeds or APIs, backups need to be created, you name it. Usually, if not all the time, those regular tasks are being set as cron jobs. However, when some task is run in the background, by default, you don't get any feedback whether it was successfully completed, or whether it crashed on the way. In this post I will show you how I handle the results of cron jobs.
In a Django project, all those tasks are usually implemented as management commands. For each such command I write a short bash script, that will call the management command with specific parameters and will print the verbose output to a log file.
Let's say my project structure is like this on a remote server:
Lateo.net - Flux RSS en pagaille (pour en ajouter : @ moi) 
/home/myproject
├── bin
├── include
├── lib
├── public_html
├── backups
├── project
│ └── myproject
├── scripts
└── logs
A virtual environment is created in the home directory of myproject linux user. The Django project itself is kept under project directory. The scripts directory is for my bash scripts. And the logs directory is for the verbose output of the bash scripts.
For example, for the default clearsessions command that removes outdated sessions, I would create scripts/cleanup.sh bash script as follows:
#!/usr/bin/env bash
SECONDS=0
PROJECT_PATH=/home/myproject
CRON_LOG_FILE=${PROJECT_PATH}/logs/cleanup.log
echo "Cleaning up the database" > ${CRON_LOG_FILE}
date >> ${CRON_LOG_FILE}
cd ${PROJECT_PATH}
source bin/activate
cd project/myproject
python manage.py clearsessions --verbosity=2 --traceback >> ${CRON_LOG_FILE} 2>&1
echo "Finished." >> ${CRON_LOG_FILE}
duration=$SECONDS
echo "$(($duration / 60)) minutes and $(($duration % 60)) seconds elapsed." >> ${CRON_LOG_FILE}
To run this command every night at 1 AM, you could create a file myproject_crontab with the following content:
MAILTO=""
00 01 * * * /home/myproject/scripts/cleanup.sh
Then register the cron jobs with:
$ crontab myproject_crontab
By such a bash script, I can track:
In addition, this gives me information whether the crontab was registered and whether the cron service was running at all. As I get the total time of execution in minutes and seconds, I can decide how often I can call the cron job regularly so that it doesn't clash with another cron job.
When you have multiple Django management commands, you can group them thematically into single bash script, or you can wrap them into individual bash scripts. After putting them into the crontab, the only thing left is manually checking the logs from time to time.
If you have any suggestions how I could even improve this setup, I would be glad to hear your opinion in the comments.
Here is the Gist of the scripts in this post. To see some examples of custom Django management commands, you can check Chapter 9, Data Import and Export in my book Web Development with Django Cookbook - Second Edition.
Cover photo by Redd Angelo
This week the post office delivered a package that made me very satisfied. It was a box with three paper versions of my "Web Development with Django Cookbook - Second Edition". The book was published at the end of January after months of hard, but fulfilling work in the late evenings and at weekends.
The first Django Cookbook was dealing with Django 1.6. Unfortunately, the support for that version is over. So it made sense to write an update for a newer Django version. The second edition was adapted for Django 1.8 which has a long-term support until April 2018 or later. This edition introduces new features added to Django 1.7 and Django 1.8, such as database migrations, QuerySet expressions, or System Check Framework. Most concepts in this new book should also be working with Django 1.9.
My top 5 favourite new recipes are these:
The book is worth reading for any Django developer, but will be best understood by those who already know the basics of web development with Django. You can learn more about the book and buy it at the Packt website or Amazon.
I thank the Packt Publishing very much for long cooperation in the development of this book. I am especially thankful to acquisition editor Nadeem N. Bagban, content development editors Arwa Manasawala and Sumeet Sawant, and technical editor Bharat Patil. Also I am grateful for insightful feedback from the reviewer Jake Kronika.
What 5 recipes do you find the most useful?
I recently switched from a Windows server 2016 to a FreeBSD Unix server. I need to check disk space on Unix command line. How do I find out disk space utilization information using the Unix command-line option?
The post How to check disk space on Unix servers appeared first on nixCraft.
Django comes with a variety of command line utilities that can be either invoked using django-admin.py or the
convenient manage.py script. A nice thing about it is that you can also add your own commands. Those management
commands can be very handy when you need to interact with your application via command line using a terminal and it can
also serve as an interface to execute cron jobs. In this tutorial you are going to learn how to code your own commands.
Just before we get started, let’s take a moment to familiarize with Django’s command line interface. You are probably
already familiar with commands like startproject, runserver or collectstatic. To see a complete list of commands
you can run the command below:
python manage.py helpOutput:
Type 'manage.py help <subcommand>' for help on a specific subcommand.
Available subcommands:
[auth]
changepassword
createsuperuser
[contenttypes]
remove_stale_contenttypes
[django]
check
compilemessages
createcachetable
dbshell
diffsettings
dumpdata
flush
inspectdb
loaddata
makemessages
makemigrations
migrate
sendtestemail
shell
showmigrations
sqlflush
sqlmigrate
sqlsequencereset
squashmigrations
startapp
startproject
test
testserver
[sessions]
clearsessions
[staticfiles]
collectstatic
findstatic
runserver
We can create our own commands for our apps and include them in the list by creating a management/commands directory inside an app directory, like below:
mysite/ <-- project directory
|-- core/ <-- app directory
| |-- management/
| | +-- commands/
| | +-- my_custom_command.py <-- module where command is going to live
| |-- migrations/
| | +-- __init__.py
| |-- __init__.py
| |-- admin.py
| |-- apps.py
| |-- models.py
| |-- tests.py
| +-- views.py
|-- mysite/
| |-- __init__.py
| |-- settings.py
| |-- urls.py
| |-- wsgi.py
+-- manage.pyThe name of the command file will be used to invoke using the command line utility. For example, if our command was
called my_custom_command.py, then we will be able to execute it via:
python manage.py my_custom_commandLet’s explore next our first example.
Below, a basic example of what the custom command should look like:
management/commands/what_time_is_it.py
from django.core.management.base import BaseCommand
from django.utils import timezone
class Command(BaseCommand):
help = 'Displays current time'
def handle(self, *args, **kwargs):
time = timezone.now().strftime('%X')
self.stdout.write("It's now %s" % time)Basically a Django management command is composed by a class named Command which inherits from BaseCommand. The
command code should be defined inside the handle() method.
See how we named our module what_time_is_it.py. This command can be executed as:
python manage.py what_time_is_itOutput:
It's now 18:35:31You may be asking yourself, how is that different from a regular Python script, or what’s the benefit of it. Well, the main advantage is that all Django machinery is loaded and ready to be used. That means you can import models, execute queries to the database using Django’s ORM and interact with all your project’s resources.
Django make use of the argparse, which is part of Python’s standard
library. To handle arguments in our custom command we should define a method named add_arguments.
The next example is a command that create random user instances. It takes a mandatory argument named total, which
will define the number of users that will be created by the command.
management/commands/create_users.py
from django.contrib.auth.models import User
from django.core.management.base import BaseCommand
from django.utils.crypto import get_random_string
class Command(BaseCommand):
help = 'Create random users'
def add_arguments(self, parser):
parser.add_argument('total', type=int, help='Indicates the number of users to be created')
def handle(self, *args, **kwargs):
total = kwargs['total']
for i in range(total):
User.objects.create_user(username=get_random_string(), email='', password='123')Here is how one would use it:
python manage.py create_users 10The optional (and named) arguments can be passed in any order. In the example below you will find the definition of an argument named “prefix”, which will be used to compose the username field:
management/commands/create_users.py
from django.contrib.auth.models import User
from django.core.management.base import BaseCommand
from django.utils.crypto import get_random_string
class Command(BaseCommand):
help = 'Create random users'
def add_arguments(self, parser):
parser.add_argument('total', type=int, help='Indicates the number of users to be created')
# Optional argument
parser.add_argument('-p', '--prefix', type=str, help='Define a username prefix', )
def handle(self, *args, **kwargs):
total = kwargs['total']
prefix = kwargs['prefix']
for i in range(total):
if prefix:
username = '{prefix}_{random_string}'.format(prefix=prefix, random_string=get_random_string())
else:
username = get_random_string()
User.objects.create_user(username=username, email='', password='123')Usage:
python manage.py create_users 10 --prefix custom_useror
python manage.py create_users 10 -p custom_userIf the prefix is used, the username field will be created as custom_user_oYwoxtt4vNHR. If not prefix, it will be
created simply as oYwoxtt4vNHR – a random string.
Another type of optional arguments are flags, which are used to handle boolean values. Let’s say we want to add an
--admin flag, to instruct our command to create a super user or to create a regular user if the flag is not present.
management/commands/create_users.py
from django.contrib.auth.models import User
from django.core.management.base import BaseCommand
from django.utils.crypto import get_random_string
class Command(BaseCommand):
help = 'Create random users'
def add_arguments(self, parser):
parser.add_argument('total', type=int, help='Indicates the number of users to be created')
parser.add_argument('-p', '--prefix', type=str, help='Define a username prefix')
parser.add_argument('-a', '--admin', action='store_true', help='Create an admin account')
def handle(self, *args, **kwargs):
total = kwargs['total']
prefix = kwargs['prefix']
admin = kwargs['admin']
for i in range(total):
if prefix:
username = '{prefix}_{random_string}'.format(prefix=prefix, random_string=get_random_string())
else:
username = get_random_string()
if admin:
User.objects.create_superuser(username=username, email='', password='123')
else:
User.objects.create_user(username=username, email='', password='123')Usage:
python manage.py create_users 2 --adminOr
python manage.py create_users 2 -aLet’s create a new command now named delete_users. In this new command we will be able to pass a list of user ids and the command should delete those users from the database.
management/commands/delete_users.py
from django.contrib.auth.models import User
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = 'Delete users'
def add_arguments(self, parser):
parser.add_argument('user_id', nargs='+', type=int, help='User ID')
def handle(self, *args, **kwargs):
users_ids = kwargs['user_id']
for user_id in users_ids:
try:
user = User.objects.get(pk=user_id)
user.delete()
self.stdout.write('User "%s (%s)" deleted with success!' % (user.username, user_id))
except User.DoesNotExist:
self.stdout.write('User with id "%s" does not exist.' % user_id)Usage:
python manage.py delete_users 1Output:
User "SMl5ISqAsIS8 (1)" deleted with success!We can also pass a number of ids separated by spaces, so the command will delete the users in a single call:
python manage.py delete_users 1 2 3 4Output:
User with id "1" does not exist.
User "9teHR4Y7Bz4q (2)" deleted with success!
User "ABdSgmBtfO2t (3)" deleted with success!
User "BsDxOO8Uxgvo (4)" deleted with success!We could improve the previous example a little big by setting an appropriate color to the output message:
management/commands/delete_users.py
from django.contrib.auth.models import User
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = 'Delete users'
def add_arguments(self, parser):
parser.add_argument('user_id', nargs='+', type=int, help='User ID')
def handle(self, *args, **kwargs):
users_ids = kwargs['user_id']
for user_id in users_ids:
try:
user = User.objects.get(pk=user_id)
user.delete()
self.stdout.write(self.style.SUCCESS('User "%s (%s)" deleted with success!' % (user.username, user_id)))
except User.DoesNotExist:
self.stdout.write(self.style.WARNING('User with id "%s" does not exist.' % user_id))Usage is the same as before, difference now is just the output:
python manage.py delete_users 3 4 5 6Output:
Below a list of all available styles, in form of a management command:
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = 'Show all available styles'
def handle(self, *args, **kwargs):
self.stdout.write(self.style.ERROR('error - A major error.'))
self.stdout.write(self.style.NOTICE('notice - A minor error.'))
self.stdout.write(self.style.SUCCESS('success - A success.'))
self.stdout.write(self.style.WARNING('warning - A warning.'))
self.stdout.write(self.style.SQL_FIELD('sql_field - The name of a model field in SQL.'))
self.stdout.write(self.style.SQL_COLTYPE('sql_coltype - The type of a model field in SQL.'))
self.stdout.write(self.style.SQL_KEYWORD('sql_keyword - An SQL keyword.'))
self.stdout.write(self.style.SQL_TABLE('sql_table - The name of a model in SQL.'))
self.stdout.write(self.style.HTTP_INFO('http_info - A 1XX HTTP Informational server response.'))
self.stdout.write(self.style.HTTP_SUCCESS('http_success - A 2XX HTTP Success server response.'))
self.stdout.write(self.style.HTTP_NOT_MODIFIED('http_not_modified - A 304 HTTP Not Modified server response.'))
self.stdout.write(self.style.HTTP_REDIRECT('http_redirect - A 3XX HTTP Redirect server response other than 304.'))
self.stdout.write(self.style.HTTP_NOT_FOUND('http_not_found - A 404 HTTP Not Found server response.'))
self.stdout.write(self.style.HTTP_BAD_REQUEST('http_bad_request - A 4XX HTTP Bad Request server response other than 404.'))
self.stdout.write(self.style.HTTP_SERVER_ERROR('http_server_error - A 5XX HTTP Server Error response.'))
self.stdout.write(self.style.MIGRATE_HEADING('migrate_heading - A heading in a migrations management command.'))
self.stdout.write(self.style.MIGRATE_LABEL('migrate_label - A migration name.'))
If you have a task that must run periodically, like generating a report every Monday. Or let’s say you have a Web scrapper that collects data from some Website every 10 minutes. You can define this code as a management command and simply add it to your server’s crontab like this:
# m h dom mon dow command
0 4 * * * /home/mysite/venv/bin/python /home/mysite/mysite/manage.py my_custom_commandThe example above will execute the my_custom_command every day at 4 a.m.
The examples above should be enough to get you started. More advanced usage will boil down to knowing how to use the argparse features. And of course, Django’s official documentation on management commands is the best resource.
You can find all the code used in this tutorial on GitHub.
