Django
******

.. highlight:: python

- `Field Types`_
- :doc:`dev-django-media`
- :doc:`dev-django-migrations`
- :doc:`dev-django-static`
- :doc:`dev-django-thumbnails`

Exceptions
==========

::

  from django.core.exceptions import ObjectDoesNotExist

Fields
======

Parameters
----------

``blank``

  If ``True``, the field is allowed to be blank.  Default is ``False``.

**Note**: See ``null`` notes below.  To allow a date to be empty, you will need
to set both ``blank`` and ``null`` (see *Sample* below):

*Sample*::

  completed = models.DateTimeField(blank=True, null=True)
  notes = models.TextField(blank=True)

``null``

  If ``True``, Django will store empty values as ``NULL`` in the database.
  The default is ``False``.

**Note**:

- Empty string values will always get stored as empty strings, not as ``NULL``.
- Only use ``null=True`` for non-string fields such as integers, booleans and
  dates.
- For both types of fields, you will also need to set ``blank=True`` if you
  wish to permit empty values in forms, as the ``null`` parameter only affects
  database storage (see ``blank`` above).

Sample::

  num_pages = models.IntegerField(blank=True, null=True)

Field Types
-----------

:doc:`dev-django-field`

Settings
========

In your ``settings/base.py`` file::

  from pathlib import Path
  BASE_DIR = Path(__file__).resolve().parent.parent

In your ``settings/production.py`` file::

  if get_env_variable_bool('SSL'):
      SESSION_COOKIE_SECURE = True
      CSRF_COOKIE_SECURE = True

  ALLOWED_HOSTS = get_env_variable('ALLOWED_HOSTS').split(',')
  # https://docs.djangoproject.com/en/1.10/releases/1.9/#csrf
  CSRF_TRUSTED_ORIGINS = ALLOWED_HOSTS

  DOMAIN = get_env_variable('DOMAIN')
  DATABASE = DOMAIN.replace('.', '_').replace('-', '_')

  DATABASES = {
      'default': {
          'ENGINE': 'django.db.backends.postgresql_psycopg2',
          'NAME': DATABASE,
          'USER': DATABASE,
          'PASSWORD': get_env_variable('DB_PASS'),
          'HOST': get_env_variable('DB_IP'),
          'PORT': '',
      }
  }

Run development environment without debug
-----------------------------------------

In settings/local.py set::

  DEBUG = False

Then run development server::

  django-admin runserver 0.0.0.0:8000 --insecure

Management Command
==================

One argument::

  def add_arguments(self, parser):
      parser.add_argument('issue_pk', type=int)

  def handle(self, *args, **options):
      self.stdout.write("{}...".format(self.help))
      issue_pk = options['issue_pk']

An optional argument::

  def add_arguments(self, parser):
      parser.add_argument("content_id", type=int, nargs="?")

Boolean argument (Python 3.9 and above)::

  import argparse

  def add_arguments(self, parser):
      parser.add_argument("process-step-id", type=int)
      parser.add_argument(
          "--ignore-drive-id",
          action=argparse.BooleanOptionalAction,
          default=True,
      )

  def handle(self, *args, **options):
      # note... we use underscores to get the value
      ignore_drive_id = options["ignore_drive_id"]

To call this, use ``--no``::

  django-admin one-drive-download --ignore-drive-id
  # for *not*, use '--no'
  django-admin one-drive-download --no-ignore-drive-id

Migrations
==========

:doc:`dev-django-migrations`

Next
====

The :doc:`app-base` app has a ``get_context_data`` mixin to help with navigating
back to the previous page using the ``next`` parameter.

.. _django_transactions:

Transactions
============

I have started using ``transaction.atomic`` in several of the views.  Make sure
the transaction is committed before adding a task to the queue or returning the
HTTP response.

- For example patterns, see :ref:`celery_tasks`
- For documentation, see on_commit_.

Updates
=======

Django 4.2
----------

The default value for ``DEFAULT_AUTO_FIELD`` is::

  DEFAULT_AUTO_FIELD = "django.db.models.AutoField"

Upstream support for PostgreSQL 10 ends in November 2022.
https://www.kbsoftware.co.uk/crm/ticket/6389/

Django 4.1 supports PostgreSQL 11 and higher.
https://www.kbsoftware.co.uk/crm/ticket/6065/

::

  DATABASES = {
      "default": {
          "ENGINE": "django.db.backends.postgresql",

::

  # .gitlab-ci.yml
  services:
    - postgres:14.8

Django 4
--------

``is_ajax``

*The* ``HttpRequest.is_ajax()`` *method is deprecated*,
https://docs.djangoproject.com/en/4.0/releases/3.1/#id2

diff for ``requirements/base.txt``::

  -django-braces==1.15.0
  +git+https://github.com/brack3t/django-braces.git@df847c9a517a09d81ccd27609af8b96d20887922#egg=django-braces

``django-sendfile``

.. warning:: Do **not** use this in ``apps`` until the project is on Django 4

`diff (update to 'django-sendfile2')`_

In ``requirements/base.txt``, replace ``django-sendfile``
with ``django-sendfile2``.

In ``settings/base.py``::

  from pathlib import Path
  BASE_DIR = Path(__file__).resolve(strict=True).parent.parent

In ``settings/local.py``::

  # https://django-sendfile2.readthedocs.io/en/
  SENDFILE_BACKEND = "django_sendfile.backends.development"
  SENDFILE_ROOT = BASE_DIR / "media-private"

In ``settings/production.py``::

  # https://django-sendfile2.readthedocs.io/en/latest/backends.html#nginx-backend
  SENDFILE_BACKEND = "django_sendfile.backends.nginx"
  SENDFILE_ROOT = get_env_variable("SENDFILE_ROOT")
  SENDFILE_URL = "/private"

In the view, change ``sendfile`` to ``django_sendfile``::

  from django_sendfile import sendfile

In tests, make sure to put any test files into ``SENDFILE_ROOT``::

  from django.conf import settings

  file_name = os.path.join(
      settings.SENDFILE_ROOT,
      "data",
      "empty-text-file.txt",
  )

``ifequal``

  Replace
  ``{% ifequal x y %}`` with ``{% if x == y %}`` and
  ``{% endifequal %}` with ``{% endif %}``.

Django 3.2
----------

``DEFAULT_AUTO_FIELD``::

  DEFAULT_AUTO_FIELD = "django.db.models.AutoField"

``JSONField``::

  # remove
  # from django.contrib.postgres.fields import JSONField

  # use ``models.JSONField``
  parameters = models.JSONField(blank=True, null=True, encoder=DjangoJSONEncoder)

.. warning:: ``models.JSONField`` does not appear to work with Django==3.1.x

``re_path``::

  # change
  url(regex=r"^", view=include("login.urls")),
  url(regex=r"^$", view=HomeView.as_view(), name="project.home"),
  url(regex=r"^dash/$", view=DashView.as_view(), name="project.dash"),

  # to...
  from django.urls import include, re_path

  re_path(r"^", include("login.urls")),
  re_path(r"^$", HomeView.as_view(), name="project.home"),
  re_path(r"^dash/$", DashView.as_view(), name="project.dash"),

Django 2.0
----------

``admin``: To include the ``admin`` URLs, replace::

  urlpatterns = [
      url(regex=r'^admin/',
          view=include(admin.site.urls)
          ),

with::

  from django.urls import path
  urlpatterns = [
      path('admin/', admin.site.urls),

``braces``: Use the ``2.0`` branch::

  git+https://github.com/brack3t/django-braces.git@2.0#egg=braces

``on_delete``: For a ``ForeignKey``, add ``on_delete``::

  user_deleted = models.ForeignKey(
      ...
      on_delete=models.CASCADE,

.. note:: The default is ``models.CASCADE``

``pytest``: If ``pytest-django`` is looking for ``manage.py``, add the
following to ``setup.cfg``::

  [tool:pytest]
  django_find_project = false

``pytest`` error::

  INTERNALERROR> AttributeError: 'NoneType' object has no attribute 'testscollected'

::

  pip install pytest==3.3.2

``pytest-django``:

If you get ``django.core.urlresolvers`` from ``pytest-django``, then add the
following to your requirements::

  git+https://github.com/pytest-dev/pytest-django.git@94cccb956435dd7a719606744ee7608397e1eafb#egg=pytest_django

``taggit``: https://github.com/alex/django-taggit/issues/520::

  AttributeError: 'TaggableRel' object has no attribute 'related_query_name'

To fix the issue, install::

  django-taggit

``urls``: The import for ``reverse`` is just ``django.urls``::

  from django.urls import reverse
  from django.urls import reverse_lazy

``WSGIRequest``::

  AttributeError: 'WSGIRequest' object has no attribute 'user'

``MIDDLEWARE``: Use ``MIDDLEWARE`` in the settings file rather than
``MIDDLEWARE_CLASSES``.

Django 1.11
-----------

To include the admin URLs in Django 1.11::

  url(regex=r'^admin/',
      view=admin.site.urls
      ),

Older versions of ``easy-thumbnails`` and ``django-bootstrap3`` have
compatiblity issues with version 1.11. Upgrade to ``easy-thumbnails==2.4.1``
and ``django-bootstrap3==8.2.2``.

There is a change to the ``build_attrs`` method signature from
``django.forms.Widget``.  Whereas before we could do this::

  from django.forms import Widget

  widget = Widget()
  final_attrs = widget.build_attrs(attrs, type='file', name='image')

We now need to do this::

  from django.forms import Widget

  widget = Widget()
  extra_attrs = attrs.copy() if attr else {}
  extra_attrs.update(name='image', type='file')
  final_attrs = widget.build_attrs(widget.attrs, extra_attrs)

Use ``MIDDLEWARE`` in the settings file rather than ``MIDDLEWARE_CLASSES``.

In old migrations, the ``on_delete`` parameter is now required, so change::

  ('mail', models.ForeignKey(to='mail.Mail')),

to::

  ('mail', models.ForeignKey(to='mail.Mail', on_delete=models.CASCADE)),

Django 1.10
-----------

::

  from django.conf.urls import patterns, url

  urlpatterns = patterns(
      '',

Is now::

  from django.conf.urls import url

  urlpatterns = [

.. note:: Make sure you remove ``''``

The use of strings to define a view in the URLConf has been removed see
`this stackoverflow post`_

A url definition such as this::

  url(regex=r'^sitemap\.xml$',
      view='django.contrib.sitemaps.views.sitemap',
      kwargs={'sitemaps': sitemaps},
      ),

should be replaced with::

  from django.contrib.sitemaps.views import sitemap
  ...

  url(regex=r'^sitemap\.xml$',
      view=sitemap,
      kwargs={'sitemaps': sitemaps},
      ),

Django 1.9
----------

The contenttypes framework::

  from django.contrib.contenttypes.fields import GenericForeignKey
  from django.contrib.contenttypes.models import ContentType

  content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
  object_id = models.PositiveIntegerField()
  content_object = GenericForeignKey()

Database

``postgresql_psycopg2`` is now ``postgresql``::

  DATABASES = {
      'default': {
          'ENGINE': 'django.db.backends.postgresql',

Forms

Using a ``forms.ModelForm`` with a ``FormView`` will raise this error::

  TypeError: isinstance() arg 2 must be a type or tuple of types

To solve the issue use a ``forms.Form`` with a ``FormView``.

Logging

From `Cannot resolve 'django.utils.log.NullHandler'`_::

  'handlers': {
    'null': {
       'level': 'DEBUG',
       'class': 'logging.NullHandler',
    },

.. _`Cannot resolve 'django.utils.log.NullHandler'`: http://stackoverflow.com/questions/34348360/cannot-resolve-django-utils-log-nullhandler

Django 1.8
----------

Remove ``TEMPLATE_DIRS`` and replace with::

  TEMPLATES = [
      {
          'BACKEND': 'django.template.backends.django.DjangoTemplates',
          'DIRS': [],
          'APP_DIRS': True,
          'OPTIONS': {
              'context_processors': [
                  'django.contrib.auth.context_processors.auth',
                  'django.template.context_processors.debug',
                  'django.template.context_processors.i18n',
                  'django.template.context_processors.media',
                  'django.template.context_processors.static',
                  'django.template.context_processors.tz',
                  'django.contrib.messages.context_processors.messages',
              ],
              'string_if_invalid': '**** INVALID EXPRESSION: %s ****',
          },
      },
  ]

Remove::

  TEMPLATE_DEBUG = DEBUG
  TEMPLATE_STRING_IF_INVALID = '**** INVALID EXPRESSION: %s ****'

``formtools``

The ``formtools`` package has been removed, so if you need wizards replace::

  from django.contrib.formtools.wizard.views import SessionWizardView

with::

  from formtools.wizard.views import SessionWizardView

Install::

  django-formtools

Add ``formtools`` to ``INSTALLED_APPS``::

  INSTALLED_APPS = (
      # ...
      'formtools',
  )

``get_model``

::

  from django.apps import apps
  model = apps.get_model('compose', 'Article')

Management Commands

Django 1.8 uses ``argparse`` rather than ``optparse``.  For details, see
`Custom Management Commands`_ and `Argparse Tutorial`.  You are encouraged to
exclusively use ``**options`` for new commands::

  def add_arguments(self, parser):
      parser.add_argument(
          '--path',
          help="path to the 'name.csv' file"
      )

  def handle(self, *args, **options):
      file_name = options['path']

Django 1.7
----------

Remove ``south`` from requirements and ``INSTALLED_APPS``

Update django and reversion in the requirements::

  django-reversion==1.8.5
  Django==1.7.1

To get rid of ``(1_6.W001) Some project unittests may not execute as expected``
remove the following from your settings::

  SITE_ID = 1
  DJANGO_APPS = (
      'django.contrib.sites',

  TEMPLATE_LOADERS = (
      'django.template.loaders.filesystem.Loader',
      'django.template.loaders.app_directories.Loader',
      # 'django.template.loaders.eggs.Loader',
  )

.. note:: only remove the ``SITE_ID`` and ``sites`` if you are not using
          ``Site`` in your project (I am using ``Site`` in
          ``hatherleigh_net``).

Remove ``migrations`` folders... then create new version 1.7 migrations:
:doc:`dev-django-migrations`.

Deploy

If you get the error::

  relation "easy_thumbnails_thumbnaildimensions" already exists

Then, just drop the table::

  DROP TABLE easy_thumbnails_thumbnaildimensions;

.. _django_generate_url:

Generate URL
============

To generate a complete URL (including the domain) when you don't have a request
object:

Add the ``HOST_NAME`` to ``base.py`` in your settings::

  HOST_NAME = get_env_variable("HOST_NAME")

For testing, add ``HOST_NAME`` to your environment e.g::

  set -x HOST_NAME "http://localhost:8000"

To generate the URL::

  import urllib.parse
  from django.conf import settings
  absolute_url = contact.get_absolute_url()
  url = urllib.parse.urljoin(settings.HOST_NAME, absolute_url)

A ``host_name`` for your site will automatically be generated by Salt.  If you
need to override this, then you can set the ``host_name`` in the pillar.  For
more information, see :ref:`pillar_host_name`.

.. note:: Our standard is to use ``host_name`` rather than the Django
          ``build_absolute_uri`` method for adding links to email messages.

From `How to use Django to get the name for the host server?`_:

  If you have a request (i.e. inside a view), you can look at
  ``request.get_host()`` which gets you a complete host and port, taking into
  account reverse proxy headers if any.

  If you don't have a request, you should configure the hostname somewhere in
  your settings.

  Just looking at the system hostname can be ambiguous in a lot of cases,
  virtual hosts being the most common.

Users
=====

::

  from django.contrib.auth import get_user_model
  get_user_model().objects.get(username=user_name)

Watchman
========

.. warning:: I couldn't get ``watchman`` installed on Ubuntu 20.04.

Add ``pywatchman`` to ``requirements/local.txt``

Create a ``.watchmanconfig`` file in the root of your Django project::

  {
    "ignore_dirs": ["node_modules"]
  }

Install watchman on Ubuntu 20.04 (from `Installing from source`_)

.. warning:: This doesn't seem to work...

::

  git clone https://github.com/facebook/watchman.git -b v4.9.0 --depth 1
  cd watchman
  ./autogen.sh
  ./configure
  make
  sudo make install


.. _`Argparse Tutorial`: https://docs.python.org/3/howto/argparse.html#argparse-tutorial
.. _`Custom Management Commands`: https://docs.djangoproject.com/en/1.8/howto/custom-management-commands/
.. _`diff (update to 'django-sendfile2')`: https://gitlab.com/kb/job/-/commit/da8718c1eeaede3f93ea014987412f98b7911c7f
.. _`How to use Django to get the name for the host server?`: https://stackoverflow.com/questions/4093999/how-to-use-django-to-get-the-name-for-the-host-server
.. _`Installing from source`: https://facebook.github.io/watchman/docs/install.html#installing-from-source
.. _`this stackoverflow post`: http://stackoverflow.com/questions/34096424/django-support-for-string-view-arguments-to-url-is-deprecated-and-will-be-rem
.. _on_commit: https://docs.djangoproject.com/en/1.10/topics/db/transactions/#django.db.transaction.on_commit