Custom Django management commands

Jul 2017

In many computer applications there is usually a need to execute some actions by use of command line interfaces. The inbuilt argparse python module provides a simple way of writing such interfaces. Click is a third party python module which solves the same problem but provides a richer set of features and arguably a better developer experience compared to argparse. Generally, when there is a need to build complex python command-line interfaces it is advisable to use Click or any of the other third party solutions because of the many useful abstractions they provide.

Django management commands are Django’s own way of writing command-line interfaces. The manage.py utilitiy is used to register and run these commands.

Django comes with several management commands such as runserver, migrate etc which are used to perform actions during the Django development process.

Users can also write and register their own management commands which can be run like the commands which ship with Django. Each Django app in a Django project can provide and register its own management commands.

Invoking python manage.py or python manage.py help will show all registered commands grouped by app.

[auth]
    changepassword
    createsuperuser

[contenttypes]
    remove_stale_contenttypes

[django]
    check
    compilemessages
    createcachetable
    .....

In this tutorial, we’ll create a simple Django management command which we’ll use to fill a database table with randomly generated data.

Set up

To begin with, lets create a new Django project called stockrecords and within this project a Django app which we’ll call main for simplicity.

$ django-admin startproject stockrecords
$ cd stockrecords
$ python manage.py startapp main

Next lets add the main app to the installed apps list in settings.py

# stockrecords/settings.py
# ...
INSTALLED_APPS = [
    'main',

    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

In the main app’s models.py lets create a simple model which will record highly ficticious stock values.

# main/models.py
from django.db import models


class StockRecord(models.Model):
    day = models.DateField()
    closing_record = models.PositiveSmallIntegerField()

    def __str__(self):
        return '{}'.format(self.closing_record)

Next lets run makemigrations and migrate to persist our changes in the database.

$ python manage.py makemigrations main
$ python manage.py migrate

Lets register our model in the Django admin so as to make CRUD operations easier.

# main/admin.py
from django.contrib import admin

from .models import StockRecord


class StockRecordAdmin(admin.ModelAdmin):
    list_display = ('day', 'closing_record')


admin.site.register(StockRecord, StockRecordAdmin)

For custom commands to be picked by Django, a conventional file naming pattern is used. All custom management commands of a given Django app should be stored in a management/commands package within the app.

Thus the custom commands of our main app will be stored in main/management/commands:

├── admin.py
├── apps.py
├── __init__.py
├── management
│   ├── commands
│   │   ├── __init__.py
│   │   ├── mycommand.py
│   │   ├── _utils.py
│   ├── __init__.py
├── migrations
│   ├── 0001_initial.py
| .....

On Python 2, be sure to include __init__.py files in both the management and
management/commands directories as done above or your command will not be detected. Spelling errors in either of the directory’s name will also cause the same issue.

In the case above, the python files in /commands will be our management commands, except by convention all files beginning with an underscore. This is useful if there is a need to have something like a utility file within the commands package which you do not want to be registered as a custom command. For the example above, mycommand.py will be registered as a management command while _utils.pywon’t.

In the example above, the mycommand command will be made available to any project that includes the main application in INSTALLED_APPS.

The command line invocation of the management command is based on the name of the file. Lets call our custom command populatestocks.py. This is the name which will be used when invoking this command with manage.py. Like so:

$ Python manage.py populatestocks

When we invoke populatestocks, Django looks for the management/commands/ subdirectory in all installed apps inside of which it will look for the populatestocks.py file. Inside populatestocks.py it will look for a Command class. If populatestocks.py is not found or the Command class is not found within it, a CommandError will be raised.

The Command class mentioned above must be defined and it must extend BaseCommand or one of its subclasses; AppCommand or LabelCommand. These classes are found in django.core.management.base

AppCommand takes one or more installed application labels as arguments, and does something with each of them while LabelCommand takes one or more arbitrary arguments (labels) on the command line, and does something with each of them.

BaseCommand is usually enough for most custom command actions. Its what we’ll use.

Our actual Command class will look like this:

# main/management/commands/populatestocks.py
import datetime
import random

from django.core.management.base import BaseCommand

from main.models import StockRecord


class Command(BaseCommand):
    help = "Save randomly generated stock record values."

    def get_date(self):
        # Naively generating a random date
        day = random.randint(1, 28)
        month = random.randint(1, 12)
        year = random.randint(2014, 2017)
        return datetime.date(year, month, day)

    def handle(self, *args, **options):
        records = []
        for _ in range(100):
            kwargs = {
                'day': self.get_date(),
                'closing_record': random.randint(1, 1000)
            }
            record = StockRecord(**kwargs)
            records.append(record)
        StockRecord.objects.bulk_create(records)
        self.stdout.write(self.style.SUCCESS('Stock records saved successfully.'))

When we run this command, 100 stock record objects will be generated and saved in the database.

$ python manage.py populatestocks
Stock records saved successfully.

The Command class has a help attribute which is used as a short description of the command. It will be printed in the help message when the user runs the command python manage.py help populatestocks

We’ve also implemented a helper method(get_date()) to generated random datetime.date objects.

Sub-classes of BaseCommand should implement the handle method which is called when the command is invoked. In our implementation above, 100 stock records will be randomly generated and saved to the database whenever the populatestocks command is invoked.

Accepting positional arguments

To spice things up, what if we want to allow the user to specify the actual number of stock records to save? We can do this by allowing them to pass the number to be generated as an argument when invoking the command.

To do this, we need to implement the add_arguments() method. This method receives a parser argument which is an instance of ArgumentParser, part of Python’s argparse package.

# ...
class Command(BaseCommand):
    help = "Save randomly generated stock record values."

    def add_arguments(self, parser):
        # Positional arguments
        parser.add_argument(
            'number_of_stock_records',
            type=int,
            help="Number of stock records to generate and save to database"
        )

    def get_date(self):
        # ....

    def handle(self, *args, **options):
        records = []
        size = options["number_of_stock_records"]
        for _ in range(size):
            kwargs = {
                'day': self.get_date(),
                'closing_record': random.randint(1, 1000)
            }
            record = StockRecord(**kwargs)
            records.append(record)
        StockRecord.objects.bulk_create(records)
        self.stdout.write(self.style.SUCCESS('{} Stock records saved.'.format(size)))

We use the parser.add_argument() method to add the arguments we need. In the case above, we specify that the name of the argument is number_of_stock_records, it must be an int and we add a help message to be shown next to the argument when the command is invoked with the help option. The argument is added to the options dict and can be accessed using its name as key.

Example usage:

$ python manage.py populatestocks 20
20 Stock records saved.

The number_of_stock_records is a mandatory argument. Optional arguments are also supported.

Accepting optional arguments

Lets add an optional argument which will be used to decide if exisiting stock records are to be deleted before generating new ones. Optional arguments work as boolean flags which can be used to control code execution.

# ...
class Command(BaseCommand):
    help = "Save randomly generated stock record values."

    def add_arguments(self, parser):
        # ...

        # Named (optional) arguments
        parser.add_argument(
            '--delete-existing',
            action='store_true',
            dest='delete_existing',
            default=False,
            help='Delete existing stock records before generating new ones',
        )

    def get_date(self):
        # ....

    def handle(self, *args, **options):
        records = []
        size = options["number_of_stock_records"]
        for _ in range(size):
            kwargs = {
                'day': self.get_date(),
                'closing_record': random.randint(1, 1000)
            }
            record = StockRecord(**kwargs)
            records.append(record)
        if options["delete_existing"]:
            StockRecord.objects.all().delete()
            self.stdout.write(self.style.SUCCESS('Existing stock records deleted.'))
        StockRecord.objects.bulk_create(records)
        self.stdout.write(self.style.SUCCESS('{} Stock records saved.'.format(size)))

Example usage:

$ python manage.py populatestocks 20
20 Stock records saved.

$ python manage.py populatestocks 200 --delete-existing
Existing stock records deleted.
200 Stock records saved.

Raising command errors

Lets add some validation to restrict generation of stock records to 1-10000 at a go.

# ...
from django.core.management.base import BaseCommand, CommandError
# ...
class Command(BaseCommand):
    # ....

    def handle(self, *args, **options):
        records = []
        size = options["number_of_stock_records"]
        if size < 0 or size > 10000:
            raise CommandError("You can only generate 1-10000 stock records at a go")
        for _ in range(size):
            # ...

A CommandError will be raised if the condition is not met.

Testing

Management commands can be tested with the call_command() function. This function provides a simple way of calling the command programatically. The following are some tests for our populatestocks command.

from django.core.management import call_command
from django.core.management.base import CommandError
from django.test import TestCase
from django.utils import timezone

from .models import StockRecord


def create_records():
    day = timezone.now()
    record2 = StockRecord(day=day, closing_record=10)
    record3 = StockRecord(day=day, closing_record=20)
    record1 = StockRecord(day=day, closing_record=30)
    StockRecord.objects.bulk_create([record1, record2, record3])


class PopulateStocksTestCase(TestCase):
    '''Tests for the populatestocks management command'''

    def test_positional_args_required(self):
        # Test that the command cannot be called without the number_of_stock_records
        msg = 'Error: the following arguments are required: number_of_stock_records'
        with self.assertRaisesMessage(CommandError, msg):
            call_command('populatestocks')

    def test_input_validation(self):
        # Test validation of number_of_stock_records positional argument
        msg = 'You can only generate 1-10000 stock records at a go'
        with self.assertRaisesMessage(CommandError, msg):
            call_command('populatestocks', -4)

        with self.assertRaisesMessage(CommandError, msg):
            call_command('populatestocks', 1000000)

    def test_records_saved(self):
        call_command('populatestocks', 10)

        self.assertEqual(StockRecord.objects.count(), 10)

    def test_existing_records_not_deleted_before_save(self):
        create_records()

        self.assertEqual(StockRecord.objects.count(), 3)

        call_command('populatestocks', 10)

        self.assertEqual(StockRecord.objects.count(), 13)

    def test_existing_records_deleted_before_save(self):
        create_records()

        self.assertEqual(StockRecord.objects.count(), 3)

        call_command('populatestocks', 10, delete_existing=True)

        self.assertEqual(StockRecord.objects.count(), 10)

Scheduled Management commands

It is a very common task to need a way to run custom management commands at sheduled intervals of time such as daily at midnight, annually etc. This can be done by periodically executing the management command from the UNIX crontab or from Windows scheduled tasks control panel.

Example, to execute a custom management command using the UNIX crontab: Add the following in your crontab file.

General syntax:

cron_shedule python_path manage_py_path command_name positional_args optional_args

For example lets say we want to run the populatestocks command daily at midnight(though our example does not make much sense for this use case) using UNIX crontab:

# Run the populatestocks command everyday at midnight(Ubuntu)
$ sudo crontab -e
# Append the following, make changes as appropriate to suit your system
@midnight /home/erick/.virtualenvs/stockrecords/bin/python3.5 /home/erick/projects/stockrecords/manage.py populatestocks 100

The above is a very simple way of running a custom management command at scheduled intervals. In mission critical production evironments it is often better to make use of more robust scheduling tools to handle such tasks.

Thats it for django management commands.

The code for this tutorial is on github. Happy coding!

Subscribe to our Newsletter

Receive updates when we add new content. No spam or some funny tricks.