Troggle status update in handbook

handbook/troggle/trogdjangup.html

@@ -69,10 +69,15 @@
 </table>
 </div>
 <h4>Major, minor and releases</h4>
-<p>Django release 2.2.20 is major-version 2, minor-version 2, and patch-release 20. 2.2 is the "feature release" and patch releases within each feature release are not meant to break anything. They are just to fix bugs.
+<p>Django release 3.2.10 is major-version 3, minor-version 2, and patch-release 10. 3.2 is the "feature release" and patch releases within each 
-<p>Things <em>will break</em> between <a href="https://docs.djangoproject.com/en/dev/internals/release-process/">feature releases</a> which come out every 8 months. We plan on upgrading troggle on the server whenever we upgrade the server operating system, which we do only every 2-3 years between <a href="https://wiki.debian.org/LTS">Debian LTS releases</a>.
+feature release are not meant to break anything. They are just to fix bugs.
-<p>You will come to rely extensively on <a href="https://docs.djangoproject.com/en/3.2/releases/">the release notes and versions documentation</a> which is maintained by django.
+<p>Things <em>will break</em> between <a href="https://docs.djangoproject.com/en/dev/internals/release-process/">feature releases</a> which come 
-<p>You will also need to read the django <a href="https://docs.djangoproject.com/en/3.2/howto/upgrade-version/">guide to upgrading to newer versions</a>.
+out every 8 months. We plan on upgrading troggle on the server whenever we upgrade the server operating system, which we do only every 2-3 years 
+between <a href="https://wiki.debian.org/LTS">Debian LTS releases</a>.
+<p>You will come to rely extensively on <a href="https://docs.djangoproject.com/en/3.2/releases/">the release notes and versions 
+documentation</a> which is maintained by django.
+<p>You will also need to read the django <a href="https://docs.djangoproject.com/en/3.2/howto/upgrade-version/">guide to upgrading to newer 
+versions</a>.
 
 <h4>Django plugins ("apps")</h4>
 <p>Documentation for the plugins is highly variable and plugin projects, being run by volunteers, can just die unexpectedly. For the django-registration plugin there are two sources of current information:<br />
@@ -86,14 +91,17 @@
 <a href="https://pypi.org/project/django-extensions/">pypi.org/project/django-extensions/</a> (only available for django 2.2 and later). ]
 
 <h3 id="dev">Troggle software development</h3>
-<p>Upgrading the version of django used by troggle is a serious programming job. It is not just a matter of editing a few config files. You will need a full troggle software development environment set up on your machine including, most definitely, the capability of running the troggle test suite. See <a href="troglaptop.html">how to set up a troggle software development laptop</a>. Note particularly that you will find sqlite database browser software very helpful and that you will need to know git.
+<p>Upgrading the version of django used by troggle can be a serious programming job. It is not just a matter of editing a few config files. You 
+will need a full troggle software development environment set up on your machine including, most definitely, the capability of running the 
+troggle test suite. See <a href="troglaptop.html">how to set up a troggle software development laptop</a>. Note particularly that you will find 
+sqlite database browser software very helpful and that you will need to know git.
 
 <h3 id="djangotricks">Important Tricks</h3>
 <p>There are six critical tricks that make everything much, much easier:
 <ol>
 <li>Use <a href="https://docs.python.org/3/library/venv.html">pip venv</a> or virtualenv to set up a virtual python environment within which you can easily and quickly change the specific releases of python, django, django's dependencies and django plugins.
-   With a previously created venv <var>py37d22</var> start it up like this:<br />
+   With a previously created venv <var>py39d32</var> start it up like this:<br />
-        <var>cd py37d22<br />
+        <var>cd py39d32<br />
         source bin/activate<br />
         python --version<br />
         cd troggle<br />
@@ -102,21 +110,31 @@
         python manage.py check -v 3<br />
         django-admin check <br />
         python manage.py test -v 3 --traceback</var>
-<li>Use the <em>highest release number</em> when upgrading between minor-versions of django. <br />So we went from 1.8.19 to 1.9.13 to 1.10.8 to 1.11.29 to 2.0.13 to 2.1.15 then 2.2.19 (.20 was not released then) and next we will go to 3.2.x .
+<li>Use the <em>highest release number</em> when upgrading between minor-versions of django. <br />
+So we went from 1.8.19 to 1.9.13 to 1.10.8 to 1.11.29 to 2.0.13 to 2.1.15 then 2.2.19 (variously patched at .20, .24, .25) and then 3.2.10 when 
+we upgraded debian 10 (buster) to 11 (bullseye).
 <li>Use the django 'check' maintenance system at the most verbose setting at each release<br />
 <var>troggle$ python manage.py check -v 3 --deploy</var><br>
 <var>troggle$ python -Wall manage.py check</var><br>
 <li>Use our test suite (and if you see errors, run it with <var>-v 3</var>)<br />
 <var>troggle$ python manage.py test [-v 3]</var><br>
-<li>Read all the release notes for <em>all</em> the intermediate releases. So from 1.1.29 we  read <a href="https://docs.djangoproject.com/en/3.0/releases/">14 sets of notes</a>: for <a href="https://docs.djangoproject.com/en/3.0/releases/2.0/">2.0</a>, 2.0.1, 2.0.2... up to 2.0.13  .
+<li>Read all the release notes for <em>all</em> the intermediate releases. So from 1.1.29 we  read <a 
-<li>Seriously learn how to use the traceback webpage produced by django when a page crashes. There is a full record of every variable value hidden in there.
+href="https://docs.djangoproject.com/en/3.0/releases/">14 sets of notes</a>: for <a 
-<li>You will be doing a lot of local testing just on your development machine. The default is to use the webserver built into Django: <var>python manage.py runserver 0.0.0.0:8000 -v 3</var>. Alternatively you can install and use gunicorn: 
+href="https://docs.djangoproject.com/en/3.0/releases/2.0/">2.0</a>, 2.0.1, 2.0.2... up to 2.0.13  .
+<li>Seriously learn how to use the traceback webpage produced by django when a page crashes. There is a full record of every variable value 
+hidden in there.
+<li>You will be doing a lot of local testing just on your development machine. The default is to use the webserver built into Django: 
+<var>python manage.py runserver 0.0.0.0:8000 -v 3</var>. Alternatively you can install and use gunicorn: 
 <var><a href="https://docs.gunicorn.org/en/stable/design.html">gunicorn</a> --reload -w 9 -b :8000 wsgi</var>. the '-w' flag is for the number of worker threads and should be 2n+1 where n is the number of cpu cores on your machine.
 </ol>
+
 <h3>Deprecation warnings and python versions</h3>
-<p>The individual releases within a minor version don't break anything but do fix bugs. So if you are on 1.10.x there is no point in getting 1.11.1 to work and you should go straight to 1.11.29 .
+<p>The individual releases within a minor version don't break anything but do fix bugs. So if you are on 3.1.x there is no point 
+in getting 3.2.1 to work and you should go straight to the latest available on your machine of the 3.2 feature release, i.e. 3.2.10 on debian 11.
 <p><var>check --deploy</var> gives django warnings about security issues in your settings as well as django deprecation warnings.<br>
-<var>-Wall</var> is a standard python option and gives warnings of deprecated python features used by django and all the current plugins. So it tells us that django 1.11.29 is using a deprecated python language feature which will be removed from the language in python 3.9 . Python version compatibilities are documented at the top of each x.0 release note. From Django 3.0 it requires python 3.6.
+<var>-Wall</var> is a standard python option and gives warnings of deprecated python features used by django and all the current plugins. So it 
+tells us that django 1.11.29 is using a deprecated python language feature which will be removed from the language in python 3.9 . Python 
+version compatibilities are documented at the top of each x.0 release note. From Django 4.0 it requires python 3.8.
 
 <h3>The upgrade process</h3>
 <h4>Django migrations - prior reading</h4>
@@ -133,7 +151,7 @@
 <h4>Running troggle before and after a Django migration</h4>
 <ol>
 <li>ensure that you have the exact version of python installed on your machine as is live for troggle on the server, e.g. do  <var>$ sudo apt install python3.7.5</var>.
-<li>create a venv using the version of python to be used.
+<li>create a venv using the version of python to be used. <br>We have a script to help you do this in <var>:troggle:venv-trog.sh</var>
 <li>do <a href="troglaptop.html">a clean install</a> of django and troggle.
 <li>check the versions of plugins using <var>pip list -o</a>
 <li>open two terminal windows: 
@@ -175,7 +193,7 @@ pytz==2021.1
 sqlparse==0.4.1
 Unidecode==1.2.0</code></pre>
 
-Although the above all work fine, on debian <var>buster</var> we are actually using the standard installs which are older:
+Although the above all work fine, on debian <var>buster</var> we were actually using the standard installs which are older:
 <pre><code>$ pip freeze
 confusable-homoglyphs==3.2.0
 Django==2.2.19
@@ -188,10 +206,13 @@ Unidecode==1.0.23</code></pre>
 <p>To upgrade pip istelf, do <var>$ pip install --upgrade pip</var>
 
 <h3>Django database dependencies</h3>
-<p>On the expo server we run MariaDB as the database which has its own dependencies. In particular
+<p>On the expo server we run MariaDB as the database which has its own dependencies. In particular under debian buster we
+had to use MariaDB v1.3.10 which was the latest version supported by buster:
 
 <pre><code>mysqlclient==1.3.10
-</code></pre>which is technically incompatible with Django 2.2 which requires 1.3.13. This incompatibility is a policy choice by the Django team. Wookey ran the Django system tests with this (April 2020) and it works fine.
+</code></pre>which is technically incompatible with Django 2.2 which requires 1.3.13. 
+This incompatibility is a policy choice by the Django team. Wookey ran the Django system tests with 1.3.10 (April 2020) and it worked fine. 
+Django just didn't support it because they couldn't spare the effort to test it.
 
 <h4>To run Django's system test suite:</h4>
 <p>This is only necessary if we hit a policy-imposed incomatibility, as we did with mysqlclient==1.3.10 . We have only ever had to do this once.
@@ -200,7 +221,7 @@ Follow the instructions in the "Unit tests" section installed locally at docs/in
 This involves running <var>git clone</var> on the django source repo to download the tests.  We hope we never have to do this again but in case we get incomprehensible bugs in future, we should be prepared to do so.
 
 <hr />
-<p>We should not need to anything until we move from django 2.2 LTS to 3.2 LTS which <a href="trog2030.html">we may never do</a>.
+
 <p>Current upgrade status is documented: <a href="trogstatus.html">here</a>.
 <p>
 

handbook/troggle/trogstatus.html

@@ -27,8 +27,23 @@ In the course of these migrations several unused or partly-used django plugins w
 <h4>April 2021</h4>
 <p>Lockdown has been good to troggle. During March and April Philip migrated troggle up to Django 2.2.19, excising the ancient and unused user registration system on the way. Django 2.2 LTS is a long-term stable relase which will be in-support by Django until April next year. Wookey discovered and ran the Django system testsuite on the Debian server thus enabling us to use a necessary (but obstensibly outdated) link between Django and the database MariaDB. As of April 9th troggle is now running on software which is actually 'in date'.</p>
 <p>We plan to stick with this configuration for a year.</p>
-<p>Next April (2022) we may upgrade to the forthcoming debian stable release 11 <var>bullseye</var>. At that point debian will have python 3.8 as standard. We also hope to migrate to django 3.2 LTS which will be a year old by then and which will be <a href="https://www.djangoproject.com/download/#supported-versions">supported until April 2025</a>.
+
-    <p>With any luck that will be the last of our involvement with django migrations as we may not move on from using django 3.2 until we stop using django altogether, see <a href="trogspeculate.html">troggle architecture speculations</a>.
+
+<h4>March 2022</h4>
+
+<p>The week beginning 7th March we plan to upgrade to the debian release 11 <var>bullseye</var>. 
+At this point 'debian stable' is bullseye and has python 3.9 as 
+standard. We will also migrate to Django 3.2 LTS which will be a year old by then and which will be <a 
+href="https://www.djangoproject.com/download/#supported-versions">supported until April 2024</a>. 
+<var>Bullseye</var> will be <a href="https://wiki.debian.org/LTS">in support until June 2026</a>.
+
+<p>Django migrations are not nearly as painful as they used to be, and troggle is already compatible with Django 4.0.3 (though we won't use that).
+So the presure to migrate from Django is now very greatly lessened. However, see <a href="trogspeculate.html">troggle architecture speculations</a>
+and possible <a href="trog2030.html">migration from Django</a>.
+
+
+<p>We should not need to anything until we move from django 3.2 LTS to 4.2 LTS before April 2024.
+
 <hr />
 <img  border="1" class="onright" width="150px" src='tricky-troggle.jpg' alt='git logo'/></a>
 Return to: <br />

handbook/website-history.html

@@ -33,7 +33,7 @@ summary {
 </style>
 <h2>Early history</h2>
 <p>
-Over 42 years, CUCC has developed methods for handling such information. Refinements in data 
+Over more than 4 decades, CUCC has developed methods for handling such information. Refinements in data 
 management were made necessary by improved quantity and quality of survey; but refinements in 
 data management also helped to drive those improvements. The first CUCC Austria expedition, in 
 1976, produced only Grade 1 survey for the most part (ref <a href="http://expo.survex.com/years/1977/report.htm">