expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2024-11-22 07:11:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

documentation updates

Browse Source
This commit is contained in:
Philip Sargent 2021-04-09 13:50:23 +01:00
parent 55d8aeb423
commit f2d83f1926
8 changed files with 44 additions and 23 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
handbook/computing/onlinesystems.html
View File

@ -12,7 +12,7 @@ body {
}
/* Use grey for system/programming/nerd pages as a visual reminder */
</style>
<h2 id="tophead">CUCC Expedition Handbook - Online overview</h2>
<h2 id="tophead">CUCC Expedition Handbook - Online systems</h2>
<h1>Expo Online Systems Overview</h1>

2
handbook/troggle/trogdesign.html
View File

@ -12,7 +12,7 @@
<p>Specific, immediate problems:
<ul>
<li>New systems for <a href="menudesign.html">website menus</a>
<li>Short-term note on "logon" <a href="trogregistr.html">django-registration</a>
<li><s>Short-term note on "logon" <a href="trogregistr.html">django-registration</a></s>
</ul>
<p>Open architectural issues being worked on:

4
handbook/troggle/trogdjango.html
View File

@ -39,7 +39,7 @@ note that the suffix on a name '<var>'_set</var> actually denotes a function cal
<h4>Django plugins</h4>
<p>We do not just use our own code and django. We also use django plugins (known as "apps") too. These can do such things as image re-sizing, user authentication with captchas, PayPal processing or mailing list integration. Unfortunately django plugins are volunteer efforts and do not have the same defined update management process as django itself, so release schedules are rather random. They cause merry hell when upgrading troggle to use a new version of django.
<p>Every extra plugin increases the "vulnerability surface" of troggle with respect to django upgrade problems so we now only install a new plugin if it is really, really necessary and we have removed as many as we could. For example, when django-staticfiles broke during one upgrade we discovered that we could use troggle.flatfiles as a workaround, so we are not planning on reinstalling staticfiles again.
<p>Every extra plugin increases the "vulnerability surface" of troggle with respect to django upgrade problems so we now only install a new plugin if it is really, really necessary and we have removed as many as we could. For example, when django-staticfiles broke during one upgrade we discovered that we could use our own troggle.expopages as a workaround, so we are not planning on reinstalling staticfiles if we don't have to.
<h4>Why we still use django</h4>
<p>Well we <a href="trog2030.html">might not use django indefinitely</a>, but unlike many frameworks the necessary functions are separately replaceable. So this gives us an evolution path. We can incrementally reduce the amount of django we use, replacing it with our own simpler python that does only what we need.
@ -52,7 +52,7 @@ note that the suffix on a name '<var>'_set</var> actually denotes a function cal
<li><a href="https://docs.djangoproject.com/en/dev/topics/templates/">templates</a> which format the results of data queries into HTML pages or JSON data exports
</ol>
<p>See the <a href="https://docs.djangoproject.com/en/dev/misc/design-philosophies/">django design philosophy</a>. Note that having a URL dispatcher that supports <a href="https://docs.djangoproject.com/en/dev/topics/http/urls/#s-reverse-resolution-of-urls">reverse resolution</a> is important: it is a key reason why we need a framework and not just a mess of javascript. We have nearly 100 URL patterns.
<p>See the <a href="https://docs.djangoproject.com/en/dev/misc/design-philosophies/">django design philosophy</a> on 'loose coupling and tight cohesion'. Note that having a URL dispatcher that supports <a href="https://docs.djangoproject.com/en/dev/topics/http/urls/#s-reverse-resolution-of-urls">reverse resolution</a> is important: it is a key reason why we need a framework and not just a mess of javascript. We have nearly 100 URL patterns.
<hr />

31
handbook/troggle/trogdjangup.html
View File

@ -73,7 +73,7 @@
<h3>Important Tricks</h3>
<p>There are six critical tricks that make everything much, much easier:
<ol>
<li>Use <a href="https://docs.python.org/3/library/venv.html">pip venv</a> to set up a virtual python environment within which you can easily and quickly change the specific releases of python, django, django's dependencies and django plugins.
<li>Use <a href="https://docs.python.org/3/library/venv.html">pip venv</a> or virtualenv to set up a virtual python environment within which you can easily and quickly change the specific releases of python, django, django's dependencies and django plugins.
With a previously created venv <var>t37</var> start it up like this:<br />
<var>cd t37<br />
source bin/activate<br />
@ -92,6 +92,8 @@
<var>troggle$ python manage.py test [-v 3]</var><br>
<li>Read all the release notes for <em>all</em> the intermediate releases. So from 1.1.29 we will read <a href="https://docs.djangoproject.com/en/3.0/releases/">14 sets of notes</a>: for <a href="https://docs.djangoproject.com/en/3.0/releases/2.0/">2.0</a>, 2.0.1, 2.0.2... up to 2.0.13 .
<li>Seriously learn how to use the traceback webpage produced by django when a page crashes. There is a full record of every variable value hidden in there.
<li>You will be doing a lot of local testing just on your development machine. The default is to use the webserver built into Django: <var>python manage.py runserver 0.0.0.0:8000 -v 3</var>. Alternatively you can install and use gunicorn:
<var><a href="https://docs.gunicorn.org/en/stable/design.html">gunicorn</a> --reload -w 9 -b :8000 wsgi</var>. the '-w' flag is for the number of worker threads and should be 2n+1 where n is the number of cpu cores on your machine.
</ol>
<p>The individual releases within a minor version don't break anything but do fix bugs. So if you are on 1.10.x there is no point in getting 1.11.1 to work and you should go straight to 1.11.29 (29 is the highest release number in minor version 1.11).
<p><var>--deploy</var> gives django warnings about security issues in your settings as well as django deprecation warnings.<br>
@ -104,11 +106,15 @@
<li>do <a href="/repositories/troggle/.git/tree/README.txt">a clean install</a> of django and troggle using pip.
<li>check the versions of plugins using <var>pip list -o</a>
<li>open two terminal windows:
<br>- one will be used for installations and tests.
<br>- the other one will be used for <var>python manage.py runserver 0.0.0.0:8000 -v 3</var> (or whatever portnumber you like)
<li>rename the most recent localsettingsXXX.py to fit your local system and make it symlink to localsettings.py
<li>edit your localsettingsXXX.py to use sqlite with a file <var>troggle.sqlite</var>
<li>edit your localsettingsXXX.py to set a password for user "expo" (or just use default "nnn:gggggg")
<ul>
<li>- one will be used for installations and tests.
<li>- the other one will be used for <var>python manage.py runserver 0.0.0.0:8000 -v 3</var> (or whatever portnumber you like)
</ul>
<li>rename the most recent localsettingsXXX.py to fit your local system and copy it to localsettings.py
<li>run a script to make a sanitised copy (no passwords) of localsettings.py as localsettingsXXX.py before every commit
<li>edit your localsettings.py to use sqlite with a file <var>troggle.sqlite</var>
<li>edit your localsettings.py to set a password for users "expo" & "expoadmin" (or just use default "nnn:gggggg")
<li>testing:
<ul>
<li>run <var>troggle$ python manage.py test</var>
@ -141,21 +147,22 @@
<pre><code>$ pip list -o
Package Version Latest Type
------------------- ------- ------ -----
Django 1.11.29 3.0.8 wheel
django-registration 2.5.2 3.1 wheel
Pillow 7.1.2 7.2.0 wheel
setuptools 45.2.0 49.2.0 wheel</code></pre>
Django 1.11.29 3.2 wheel
docutils 0.16 0.17 wheel
Pillow 7.2.0 8.2.0 wheel
pytz 2020.1 2021.1 wheel
setuptools 45.2.0 54.2.0 wheel
Unidecode 1.1.1 1.2.0 wheel</code></pre>
<p>This only gives any output for packages which are not the most recent. To see what is actually installed use
<pre><code>$ pip freeze
confusable-homoglyphs==3.2.0
Django==1.11.29
django-registration==2.5.2
docutils==0.16
Pillow==7.2.0
pytz==2020.1
Unidecode==1.1.1</code></pre>
<p>To upgrade pip istelf, do <var>$ pip install --upgrade pip</var>
<p>We will not need to update the django-registration until we move from django 2.2.15+ to 3.0.9+ which <a href="trog2030.html">we may never do</a>.
<p>We will not need to anything until we move from django 2.2.15+ to 3.2+ which <a href="trog2030.html">we may never do</a>.
<p>Current upgrade status is documented: <a href="trogstatus.html">here</a>.
<hr />

4
handbook/troggle/trogregistr.html
View File

@ -9,7 +9,9 @@
<h2 id="tophead">CUCC Expedition Handbook: Troggle</h2>
<h1>Troggle Login and user registration</h1>
<h2>Current position (June 2020) </h2>
<h2>Done as of April 2021</h2>
<p>We did option #1. No problem.
<h2>Position as of June 2020 </h2>
<p>Troggle has two "users", each with a password. They are managed by entirely separate systems:
<ol>
<li>The user "expo" with the "cavey:beery" password on the website that gives access to

7
handbook/troggle/trogspeculate.html
View File

@ -14,7 +14,7 @@
From: Philip Sargent (Gmail) [mailto:philip.sargent@gmail.com]
Sent: 19 April 2020 01:28 [original - since edited with extra refs.]
To: expo-tech@lists.wookware.org
Subject: vague thoughts about future troggle architecture
Subject: vague thoughts about future troggle architecture
</pre>
<p>
@ -27,8 +27,9 @@ These front-end frameworks get out of date in couple of years or so. So they do
give us the decade-long stability we need to match available maintenance
effort. [ See <a href="https://en.wikipedia.org/wiki/Comparison_of_JavaScript_frameworks">
Wikipedia list of javascript frameworks</a>.] With our deep historical perspective ("cough"),
we can expect this menagerie to sort itself out into a stable, standardised foundation
within the next 10 years.
we can expect <a href="https://www.circleid.com/posts/20201031-the-javascript-ecosystem/">this
menagerie to sort itself</a> out into a stable, standardised foundation
within the next couple of decades but probably not within the next 10 years.
<p>
A web API to expose the troggle database (read-only) would allow some keen
person to write a special-purpose app on a phone, e.g. an entrance-locator

11
handbook/troggle/trogstatus.html
View File

@ -32,7 +32,7 @@ the same version of django, then upgrade to as recent a version of django as we
discovered that django1.7 works just fine with <a href="https://docs.djangoproject.com/en/1.10/topics/python3/">python3</a>, so we will move the development version to python3 during June and
then upgrade the server operating system from Debian <var>stretch</var> to <var>buster</var> before
tackling the next step: thinking deeply about when we migrate from django
<a href="trogdesignx.html">to something else</a>.
<a href="trogdesign.html">to something else</a>.
</p>
<p>
Enforced time at home is giving us a new impetus to writing and restructuring the documentation for everything.
@ -43,8 +43,13 @@ In the course of these migrations several unused or partly-used django plugins w
<p>Missing troggle functions were repaired and partly-implemented pages, such as the list of all cavers and their surveyed passages, were finished and made to work. The logbook parsing acquired a cacheing system to re-load pre-parsed files. The survex file parsing was completely rebuilt to reduce the excessive memory footprint. While doing so the parser was extended to cover nearly the full range of survex syntax and modified to parse, but not store, all the survey stations locations. A great many unused classes and some partly written code ideas were deleted.
<h4>July 2020</h4>
<p>Wookey upgraded debian on the server from 9 <var>stretch</var> to 10 <var>buster</var> and we got the python3 development of troggle running as the public version (with some http:// and https:// glitches) by 23rd July. <var>Buster</var> will be in-support definitely until June 2024 so we are rather pleased to be on a "not ancient" version of the operating system at last. This concided with a last tweak at improving the full cave data file import so now it runs on the server in ~80 seconds. Which is considerably more useful than the ~5 hours it was taking earlier this year.
<p>We plan to stick with debian 10 <var>buster</var>, django 1.11.29 and python 3.7.5 (the standard on <var>buster</var>) until spring 2021 when we may upgrade debian to the forthcoming stable release 11 <var>bullseye</var>. At that point debian will have python 3.8 as standard and we will migrate to django 2.x, hopefully getting as far as django 2.2 which is an LTS and actually in support until April 2022.
<p>With any luck that will be the last of our involvement with django migrations as we may not move on from using django 2.2 until we stop using django altogether, see <a href="trogspeculate.html">troggle architecture speculations</a>.
<h4>April 2021</h4>
<p>Lockdown has been good to troggle. During March and April Philip migrated troggle up to Django 2.2.19, excising the ancient and unused user registration system on the way. Django 2.2 LTS is a long-term stable relase which will be in-support by Django until April next year. Wookie discovered and ran the Django system testsuite on the Debian server thus enabling us to use a necessary obstensibly outdated link between Django and the database MariaDB. As of April 9th troggle is now running on software which is actually 'in date'.</p>
<p>We plan to stick with this configuration for a year.</p>
<p>Next April (2022) we may to the forthcoming debian stable release 11 <var>bullseye</var>. At that point debian will have python 3.8 as standard and we also hope to migrate to django 3.2 LTS which will be a year old by then and which will be <a href="https://www.djangoproject.com/download/#supported-versions">supported until April 2025</a>.
<p>With any luck that will be the last of our involvement with django migrations as we may not move on from using django 3.2 until we stop using django altogether, see <a href="trogspeculate.html">troggle architecture speculations</a>.
<hr />
<img border="1" class="onright" width="150px" src='tricky-troggle.jpg' alt='git logo'/></a>
Return to: <br />

6
handbook/website-history.html
View File

@ -60,6 +60,12 @@ webpages from data files reduced the need for repetitive manual HTML coding. Cen
of all caves in a large .CSV file with a cave on each row made the storage of new information
more straightforward.
<h3>HTML and browser incompatibilities</h3>
<p>This was a much bigger problem in the past than it is possible to imagine now
(most of the early editing is done on an Acorn platform using !Zap). So much so
that we have a separate page all about it: "<a href="computing/useany.html">Platform portability</a> - making the website work widely".
<h3>Version control</h3>
<p>Another important element of this system was <a href="computing/repos.html">version control</a>. The entire data structure was
stored initially in a <a href="https://en.wikipedia.org/wiki/Concurrent_Versions_System">Concurrent Version System</a> repository, and later migrated to