CUCC Expedition Handbook

Django and Troggle

Django Versions and troggle

Troggle is built on top of the django framework. In 2019 we had let ourselves get rather behind in keeping up to date with the current django version and this held us back from upgrading the operating system - which we needed to do. Troggle uses several community-contributed django plugins and each of these had their own version dependencies. We also got caught up with out of date and deprecated python language features.

A ridiculous amount of work was required to get to a situation where troggle was running on relatively recent and nearly security-in-date versions of python, django, plugins and linux by Summer 2020. We don't want that to happen again.

Three sentence intro to Django for python programmers

• There are many 'magic' function calls autogenerated. So when we create a 'Cave' class (subclass of Django Model), with an instance 'caveenetrance' we automagically define functions such as:
  • Cave.objects.filter()
  • Cave.objects.all()
  • Cave.caveandentrance_set.all
  note that the suffix on a name ''_set actually denotes a function call to the database. Don't let this upset you.
• You will get very familiar with the online documentation. be sure to be looking that the flavour for the right release of Django (indicated on the bottom-right of the screen: the link here goes to v2.2). You can read it in Polish, Indonesian, Brazilian-Portuguese or whatever if you prefer.
• Django has a good process for migrating both the python code and the SQL database to later versions of Django via 'migrations'. Note that even though Expo imports all the data from files and its database is ephemeral (but it is intensely used for cross-referencing), the database schema still needs to be migrated.

Django versions and deprecation

• Django has a mature and reliable release programme: docs.djangoproject.com/en/dev/releases/
• Django pre-announces which features will be deprecated: docs.djangoproject.com/en/dev/internals/deprecation/
• Django release schedule: supported versions timetable
• Significant new Django versions are released every 8 months. Deprecated features after removed after the two subsequent versions.

Django plugins

We do not just use our own code and django. We also use django plugins (known as "apps") too. These can do such things as image re-sizing, user authentication with captchas, PayPal processing or mailing list integration. Unfortunately django plugins are volunteer efforts and do not have the same defined update management process as django itself, so release schedules are rather random. They cause merry hell when upgrading troggle to use a new version of django.

Every extra plugin increases the "vulnerability surface" of troggle with respect to django upgrade problems so we now only install a new plugin if it is really, really necessary and we have removed as many as we could. For example, when django-staticfiles broke during one upgrade we discovered that we could use our own troggle.expopages as a workaround, so we are not planning on reinstalling staticfiles if we don't have to.

Why we still use django

Well we might not use django indefinitely, but unlike many frameworks the necessary functions are separately replaceable. So this gives us an evolution path. We can incrementally reduce the amount of django we use, replacing it with our own simpler python that does only what we need.

The separate functions within the framework form a "stack". These functions exist in all web application frameworks but in Django they are loosely coupled.

The stack is:

1. SQL database: mapping to/from python objects and indexing/access routines for all the entries (sqlite, MySQL, MariaDB, postgres all work). This is where the multi-user synchronisation happens too.
2. request/response (http GET/POST) including access control and security middleware. We could never keep up with web security if we didn't use a well-supported system for this.
3. URL mapping/dispatch which matches a requested URL with a chunk of python code, and vice versa (declarative and bidrectional). "Cool URIs don't change."
4. templates which format the results of data queries into HTML pages or JSON data exports

See the django design philosophy on 'loose coupling and tight cohesion'. Note that having a URL dispatcher that supports reverse resolution is important: it is a key reason why we need a framework and not just a mess of javascript. We have nearly 100 URL patterns. We should try to reduce the number of distinct URLs perhaps, but they help document the structure too.