mirror of
https://expo.survex.com/repositories/expoweb/.git/
synced 2024-11-22 07:11:55 +00:00
178 lines
10 KiB
HTML
178 lines
10 KiB
HTML
<!DOCTYPE html>
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
<title>Handbook Troggle - a kinder simpler troggle?</title>
|
|
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
|
|
</head>
|
|
<body><style>body { background: #fff url(/images/style/bg-system.png) repeat-x 0 0 }</style>
|
|
<h2 id="tophead">CUCC Expedition Handbook</h2>
|
|
<h1>Troggle - a kinder simpler troggle?</h1>
|
|
<p>Troggle runs much of the the cave survey data management, presents the data on the website and manages the Expo Handbook.
|
|
<p>This part of the handbook is intended for people
|
|
maintaining the troggle software.
|
|
Day to day cave recording and surveying tasks are documented
|
|
in the expo "survey handbook"
|
|
<h3>stroggle</h3>
|
|
<p>At one time Martin Green attempted to reimplement troggle as "stroggle" using <a href="https://www.fullstackpython.com/flask.html">flask</a> instead of Django at
|
|
<a href="https://en.wikipedia.org/wiki/Gitorious">git@gitorious.org:stroggle/stroggle.git</a> (but gitorious has been deleted).</p>
|
|
<p>A copy of this project is archived by Wookey on <a href="http://wookware.org/software/cavearchive/stroggle/">wookware.org/software/cavearchive/stroggle/</a>.
|
|
<p>There is also a copy of stroggle on the backed-up, read-only copy of gitorious on "<a href="https://gitorious.org/">gitorious valhalla</a>"<br />
|
|
<a href="https://gitorious.org/stroggle/stroggle.git/">stroggle code</a></br>
|
|
<a href="https://gitorious.org/stroggle/stroggle-gitorious-wiki.git/">stroggle-gitorious-wiki</a></br>
|
|
but note that this domain has an expired certificate so https:// complains.
|
|
<p>The schema for stroggle is <a href="http://wookware.org/software/cavearchive/stroggle/git/exampledata/expo/schema.json">a schema.json file</a>. See the comparable <a href="datamodel.html">troggle schema file</a> which is indeed horrendously bigger.
|
|
|
|
<h3>Radost's proposal</h3>
|
|
|
|
|
|
|
|
<div style="font-family: monospace; font-size: large">
|
|
<p>Radost Waszkiewicz (CUCC member 2016-2019) proposed a plan for superceding troggle</p>
|
|
|
|
<p>
|
|
Hey,<br>
|
|
|
|
on the design sesh we've looked a bit on the way data is organised in the
|
|
loser repo and how to access it via troggle.
|
|
<p>
|
|
A proposal arised that all this database shannaingans is essentially
|
|
unnecessary - we have about 200 caves, about 250 entrances, about 200
|
|
people and couple dozen expos. We don't need efficient lookups at all. We
|
|
can write something which will be 'slow' and make only things we actually
|
|
care about.
|
|
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
|
|
[What Rad has misunderstood here is that the database is not for speed. We use it mostly so that we can manage
|
|
'referential integrity' i.e. have all the different bits of information match up correctly.
|
|
While the total size of the data is small, the interrelationships and complexity is quite large.
|
|
From the justification for troggle:
|
|
<p></em>"A less obvious but more deeply rooted problem was the lack of relational information. One table named folk.csv stored
|
|
names of all expedition members, the years in which they were present, and a link to a biography page. This was great
|
|
for displaying a table of members by expedition year, but what if you wanted to display a list of people who wrote in the
|
|
logbook about a certain cave in a certain expedition year? Theoretically, all of the necessary information to produce that
|
|
list has been recorded in the logbook, but there is no way to access it because there is no connection between the
|
|
person's name in folk.csv and the entries he wrote in the logbook".
|
|
[<a href="/expofiles/documents/troggle/troggle_paper.pdf" download>Adrian Curtis</a>]<em>
|
|
<p>And for ensuring survey data does not get lost we need to coordinate people, trips, survex blocks,
|
|
survex files, drawing files (several formats), QMs, wallet-progress pages, rigging guides, entrance photos, GPS tracks, kataster boundaries, scans of sketches, scans of underground notes, and dates for all those - Philip Sargent]
|
|
</em>
|
|
</div>
|
|
|
|
<p>Similarly I see little gain from doing the html - python himera
|
|
template pages. These contain mainly nested for loops which could just as
|
|
well be written in e.g. python.
|
|
|
|
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
|
|
[He could indeed. But for most people producing HTML while writing in python is just unnecessarily difficult.
|
|
But it has to be said that the
|
|
<a href="https://docs.djangoproject.com/en/2.1/ref/templates/builtins/#std:templatefilter-dictsort">django
|
|
HTML templating mechanism</a> is sufficiently powerful that it does almost
|
|
amount to an additional language to learn.
|
|
<p>Troggle has 66 different url recognisersand there are 71 HTML django
|
|
template files which the recognisers direct to.
|
|
Not all page templates are currently used but still some kind of templating system would seem to be
|
|
probably necessary.</p>
|
|
<p>The django system is sufficiently well-thought-of
|
|
that it forms the basis for the framework-independent templaing engine
|
|
<a href="https://jinja.palletsprojects.com/en/2.11.x/">Jinja</a> - and that site has a good discussion
|
|
on whether templating is a good thing or not. There are about
|
|
<a href="https://wiki.python.org/moin/Templating">20 different python template engines</a>.]
|
|
</em>
|
|
</div>
|
|
|
|
I'd advocate following solution:
|
|
<ul>
|
|
<li> give up on fixing things in troggle
|
|
<li> have a small number of .py scripts which generate static webpages with
|
|
content we want
|
|
</ul>
|
|
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
|
|
[A reasonable proposal, but needs quantifying with all the things troggle does
|
|
which Rad was unaware of. This will not be a "small number" but it needs estimating. We don't need
|
|
everything troggle does for us of course, but that doesn't mean that removing django/troggle
|
|
will reduce the total amount of code. The input data parsers will be nearly the same size obviously.]
|
|
</em>
|
|
</div>
|
|
Why do this:
|
|
<ul>
|
|
<li> we don't have multiple intermediate layers all of which are difficult to
|
|
maintain
|
|
<li> anyone can run this on their machine without need for local vm's
|
|
<li> to change something there is one thing you change - script that generates
|
|
pages belonging to particular pattern. currently you need to take care of
|
|
parsers, view script, url script, and template
|
|
<li> it runs offline (just like that as pages are just blocks of literal text
|
|
+ some javascript)
|
|
<li> we never end up with no working copy during expo - run scripts once
|
|
before expedition and put results in /backup and just leave it there till
|
|
next year. Something goes wrong on the daily updated version - we have a
|
|
very easy fallback
|
|
<li> just one (or very close to it) programming language
|
|
<li> change to python3 to parse silly chars
|
|
<li> wayyyy fewer lines of code and substantially better code to comment ratio
|
|
(if this is us writing this now)
|
|
<li> compatible with all the already existing static pages
|
|
<li> we can host this on a potato hosting as we're not running anything fancy
|
|
anymore
|
|
<li> get's rid of the horrifying url rewrites that correspond to no files that
|
|
exist
|
|
</ul>
|
|
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
|
|
[This vastly underestimates the number of things that troggle does for us.
|
|
See "<a href="/expofiles/documents/troggle/troggle_paper.pdf" download>Troggle: a novel system for
|
|
cave exploration information management</a>". And a VM is not required to run and debug troggle.
|
|
Sam has produced a docker variant which he uses extensively.
|
|
<p>Troggle today has 8,200 lines of python (including comments and blank lines), plus 600 in imagekit and 200 in flatpages. 2,200 of those are in the parsers. Django itself has a lot more, including integration with TinyMCE in-browser HTML editor. </em>]
|
|
</div>
|
|
How much work would this actually take:
|
|
<ul>
|
|
<li> most likely one script per page type, so far page types that are
|
|
obviously needed:
|
|
<ul>
|
|
<li>cave index
|
|
<li>individual cave descriptions
|
|
<li>logbooks
|
|
</ul>
|
|
<li> more than half of the parsers are already rewriten by me and can be
|
|
changed to do this instead of modifying SQL database with minimal effort
|
|
<li> html/css side of things already exists, but it would be nice to go for a
|
|
more modern look there as well
|
|
</ul>
|
|
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
|
|
[The effort estimate is similarly a gross underestimate because (a) he assumes one script per page of output, forgetting all the core work to create a central consistent dataset, and (b) he is missing out most
|
|
of the functionality we use without realizing it because it is built into
|
|
django's SQL system, such as multi-user operations.
|
|
<p><span style="color:red">Eventually we will have to <a href="trogdesignx.html">migrate from django</a> of course</span>, as it will eventually
|
|
fail to keep up with the rest of the world. Right now we need to get ourselves onto python3
|
|
so that we can use an LTS release which has current security updates. This is
|
|
<a href="https://docs.djangoproject.com/en/3.0/internals/release-process/#supported-versions-policy">more urgent for django</a> than for Linux. In Ubuntu terms we are on 18.04 LTS (Debian 10) which has no free maintenance updates from 2023. <span style="color:red">We should plan to migrate troggle from django to another framework in about 2025. See stroggle below.</span>]
|
|
</em>
|
|
</div>
|
|
Things this [Rad's] solution doesn't solve:
|
|
<ul>
|
|
<li> no webpage-based 'wizzard' for creating caves and such (did people use it
|
|
much? do we care?) -> maybe 'send an email to this address' is the ultimate
|
|
solution to this
|
|
<li> uploading photos is still difficult (in the sense that they don't get
|
|
minified automatically)
|
|
</ul>
|
|
Rad
|
|
<div style="margin-left: 2em; margin-right:2em; color: blue; font-family: sanserif"><em>
|
|
[Creating a cave description for a new cave, and especially linking in images,
|
|
is currently so difficult that only a couple of people can do it. Fixing this is
|
|
a matter of urgency. No one should have to imagine where the path to a file will be but isn't now.
|
|
We need a file uploading system to put things in the right place; and this would help photos too.]
|
|
</em>
|
|
</div>
|
|
|
|
</div>
|
|
|
|
|
|
|
|
<hr />
|
|
Go on to: <a href="trogarch.html">Troggle architecture</a><br />
|
|
Return to: <a href="trogdesign.html">Troggle design and future implementations</a><br />
|
|
<hr />
|
|
</body>
|
|
</html>
|