expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2024-11-25 08:41:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

More emphasis and explanation and links

Browse Source
This commit is contained in:
Philip Sargent 2020-05-12 19:59:10 +01:00
parent 9765be6f16
commit 2cb287ca81
3 changed files with 52 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
handbook/troggle/trogdesignx.html
View File

@ -47,14 +47,14 @@ enthusiast must have done this already surely ? Ah yes, Brython.<br>
<a href="https://www.brython.info/">www.brython.info</a>
<p>
Which is fun, but not useful. And not just because it is immature. None of
this addresses our biggest problem: devising something that can be
this addresses <strong>our biggest problem: devising something that can be
maintained by fewer, less-expert people who can only devote short snippets
of time and not long-duration immersion.
of time and not long-duration immersion</strong>.
<p>
I know Wookey has been thinking of a loose federation of independent scripts
working on the same data, but the more I look at troggle and the tasks it
does the less I feel that would work. At the core there is a common data
model that everything must understand - and the only unambiguous way of
does the less I feel that would work. <strong>At the core there is a common data
model that everything must understand</strong> - and the only unambiguous way of
presenting that data model is working code, e.g. see
<a href="http://expo.survex.com/handbook/troggle/trogarch.html">Troggle architecture</a> and click on the image
to see a bigger copy. [It is out of date - if someone can quickly generate

56
handbook/troggle/trogsimpler.html
View File

@ -9,7 +9,10 @@
<h2 id="tophead">CUCC Expedition Handbook</h2>
<h1>Troggle - a kinder simpler troggle?</h1>
<p>Troggle runs much of the the cave survey data management, presents the data on the website and manages the Expo Handbook.
<p>This part of the handbook is intended for people
maintaining the troggle software.
Day to day cave recording and surveying tasks are documented
in the expo "survey handbook"
<h3>stroggle</h3>
<p>At one time Martin Green attempted to reimplement troggle as "stroggle" using <a href="https://www.fullstackpython.com/flask.html">flask</a> instead of Django at
<a href="https://en.wikipedia.org/wiki/Gitorious">git@gitorious.org:stroggle/stroggle.git</a> (but gitorious has been deleted).</p>
@ -22,10 +25,7 @@
<h3>Radost's proposal</h3>
<p>This part of the handbook is intended for people
maintaining the troggle software.
Day to day cave recording and surveying tasks are documented
in the expo "survey handbook"
<div style="font-family: monospace; font-size: large">
<p>Radost Waszkiewicz (CUCC member 2016-2019) proposed a plan for superceding troggle</p>
@ -41,13 +41,18 @@ unnecessary - we have about 200 caves, about 250 entrances, about 200
people and couple dozen expos. We don't need efficient lookups at all. We
can write something which will be 'slow' and make only things we actually
care about.
<div style="margin-left: 4em; margin-right:4em; font-family: sanserif"><em>
[What Rad has misunderstood here is that the database is not for speed. We use it mostly so that we can manage 'referential integrity' i.e. have all the different bits of information match up correctly. While the total size of the data is small, the interrelationships and complexity is quite large. From the justification for troggle: <p>"A less obvious but more deeply rooted problem was the lack of relational information. One table named folk.csv stored
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
[What Rad has misunderstood here is that the database is not for speed. We use it mostly so that we can manage
'referential integrity' i.e. have all the different bits of information match up correctly.
While the total size of the data is small, the interrelationships and complexity is quite large.
From the justification for troggle:
<p></em>"A less obvious but more deeply rooted problem was the lack of relational information. One table named folk.csv stored
names of all expedition members, the years in which they were present, and a link to a biography page. This was great
for displaying a table of members by expedition year, but what if you wanted to display a list of people who wrote in the
logbook about a certain cave in a certain expedition year? Theoretically, all of the necessary information to produce that
list has been recorded in the logbook, but there is no way to access it because there is no connection between the
person's name in folk.csv and the entries he wrote in the logbook".
person's name in folk.csv and the entries he wrote in the logbook".
[<a href="/expofiles/documents/troggle/troggle_paper.pdf" download>Adrian Curtis</a>]<em>
<p>And for ensuring survey data does not get lost we need to coordinate people, trips, survex blocks,
survex files, drawing files (several formats), QMs, wallet-progress pages, rigging guides, entrance photos, GPS tracks, kataster boundaries, scans of sketches, scans of underground notes, and dates for all those - Philip Sargent]
</em>
@ -57,8 +62,21 @@ survex files, drawing files (several formats), QMs, wallet-progress pages, riggi
template pages. These contain mainly nested for loops which could just as
well be written in e.g. python.
<div style="margin-left: 4em; margin-right:4em; font-family: sanserif"><em>
[He could indeed. But for most people producing HTML while writing in python is just unnecessarily difficult.]
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
[He could indeed. But for most people producing HTML while writing in python is just unnecessarily difficult.
But it has to be said that the
<a href="https://docs.djangoproject.com/en/2.1/ref/templates/builtins/#std:templatefilter-dictsort">django
HTML templating mechanism</a> is sufficiently powerful that it does almost
amount to an additional language to learn.
<p>Troggle has 66 different url recognisersand there are 71 HTML django
template files which the recognisers direct to.
Not all page templates are currently used but still some kind of templating system would seem to be
probably necessary.</p>
<p>The django system is sufficiently well-thought-of
that it forms the basis for the framework-independent templaing engine
<a href="https://jinja.palletsprojects.com/en/2.11.x/">Jinja</a> - and that site has a good discussion
on whether templating is a good thing or not. There are about
<a href="https://wiki.python.org/moin/Templating">20 different python template engines</a>.]
</em>
</div>
@ -68,9 +86,9 @@ I'd advocate following solution:
<li> have a small number of .py scripts which generate static webpages with
content we want
</ul>
<div style="margin-left: 4em; margin-right:4em; font-family: sanserif"><em>
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
[A reasonable proposal, but needs quantifying with all the things troggle does
which Rad was unaware of. This will not be a "small" number but it needs estimating. We don't need
which Rad was unaware of. This will not be a "small number" but it needs estimating. We don't need
everything troggle does for us of course, but that doesn't mean that removing django/troggle
will reduce the total amount of code. The input data parsers will be nearly the same size obviously.]
</em>
@ -99,8 +117,10 @@ anymore
<li> get's rid of the horrifying url rewrites that correspond to no files that
exist
</ul>
<div style="margin-left: 4em; margin-right:4em; font-family: sanserif"><em>
[This vastly underestimates the number of things that troggle does for us. See "<a href="/expofiles/documents/troggle/troggle_paper.pdf" download>Troggle: a novel system for cave exploration information management</a>". And a VM is not required to run and debug troggle.
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
[This vastly underestimates the number of things that troggle does for us.
See "<a href="/expofiles/documents/troggle/troggle_paper.pdf" download>Troggle: a novel system for
cave exploration information management</a>". And a VM is not required to run and debug troggle.
Sam has produced a docker variant which he uses extensively.
<p>Troggle today has 8,200 lines of python (including comments and blank lines), plus 600 in imagekit and 200 in flatpages. 2,200 of those are in the parsers. Django itself has a lot more, including integration with TinyMCE in-browser HTML editor. </em>]
</div>
@ -118,11 +138,11 @@ changed to do this instead of modifying SQL database with minimal effort
<li> html/css side of things already exists, but it would be nice to go for a
more modern look there as well
</ul>
<div style="margin-left: 4em; margin-right:4em; font-family: sanserif"><em>
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
[The effort estimate is similarly a gross underestimate because (a) he assumes one script per page of output, forgetting all the core work to create a central consistent dataset, and (b) he is missing out most
of the functionality we use without realizing it because it is built into
django's SQL system, such as multi-user operations.
<p><span style="color:red">Eventually we will have to migrate from django of course</span>, as it will eventually
<p><span style="color:red">Eventually we will have to <a href="trogdesignx.html">migrate from django</a> of course</span>, as it will eventually
fail to keep up with the rest of the world. Right now we need to get ourselves onto python3
so that we can use an LTS release which has current security updates. This is
<a href="https://docs.djangoproject.com/en/3.0/internals/release-process/#supported-versions-policy">more urgent for django</a> than for Linux. In Ubuntu terms we are on 18.04 LTS (Debian 10) which has no free maintenance updates from 2023. <span style="color:red">We should plan to migrate troggle from django to another framework in about 2025. See stroggle below.</span>]
@ -137,7 +157,7 @@ solution to this
minified automatically)
</ul>
Rad
<div style="margin-left: 4em; margin-right:4em; font-family: sanserif"><em>
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
[Creating a cave description for a new cave, and especially linking in images,
is currently so difficult that only a couple of people can do it. Fixing this is
a matter of urgency. No one should have to imagine where the path to a file will be but isn't now.
@ -151,7 +171,7 @@ We need a file uploading system to put things in the right place; and this would
<hr />
Go on to: <a href="trogarch.html">Troggle architecture</a><br />
Return to: <a href="trogdesign.html">Troggle design notes</a><br />
Return to: <a href="trogdesign.html">Troggle design and future implementations</a><br />
<hr />
</body>
</html>

12
handbook/troggle/trogstatus.html
View File

@ -30,13 +30,21 @@ The loser repo and expoweb repo need more care in migration (expoweb is the webs
<p>
Similarly expoweb is full of bloat from fat images and surveys and one 82MB thesis that got checked in and then removed. Clearing that out is a good idea. I have a set of 'unused fat blob' lists which can be stripped out with git-gilter. It's not hard to make a 'do the conversion' script, ready for sometime after expo 2019 has calmed down.
<h4>April 2020</h4>
<h4>April/May 2020 and django versions</h4>
<p>
Wookey has now moved 'expoweb' from mercurial to git largely "as-is" and will to use the git tools to patch up the history and to remove redundancies, rather than the original plan to tidy them up "at the time of conversion". Mark Shinwell is working on loser.
<p>Sam continues to work on upgrading django from v1.7 . We are using python 2.7.17 and while we <a href="https://docs.djangoproject.com/en/1.10/topics/python3/">could upgrade</a> to Python v3 using the same version (1.7) of django, we would rather upgrade django as much as possible first before we tackle that. Old versions of django have unpatched security issues.
<p> "Django 1.11 is the last version to support Python 2.7. Support for Python 2.7 and Django 1.11 ends in 2020." see: <a href="https://docs.djangoproject.com/en/3.0/faq/install/">django versions</a>.
<p>Ubuntu 20.04 came out on 23rd April but it does not support python2 at all. So we cannot use it for software maintenance.
<p>
<p>For a table displaying the various versions of django and support expiry dates
see <a href="https://www.djangoproject.com/download/">the django download</a> page.
Django 1.7 expired in December 2015.
Django: <a href="">full deprecation timeline</a>.</p>
<p>Ideally we should upgrade from django 1.7 to django 1.11, then port from python2 to python3 on
the same version of django,
then upgrade to as recent a version of django as we can.
</p>
<p>
Enforced time at home is giving us a new impetus to writing and restructuring the documentation for everything.
<hr />
Go on to: <a href="trogmanual.html">Troggle maintenance manual</a><br />