From c744dfe733d736920a85c026e0775a7bd1d34e62 Mon Sep 17 00:00:00 2001 From: Philip Sargent Date: Sat, 11 Mar 2023 20:52:08 +0000 Subject: [PATCH] QM docum update --- handbook/troggle/datamodel.html | 4 +++ handbook/troggle/scriptsqms.html | 47 ++++++++++++++++++++++++-------- handbook/troggle/trogarch.html | 6 ++-- 3 files changed, 43 insertions(+), 14 deletions(-) diff --git a/handbook/troggle/datamodel.html b/handbook/troggle/datamodel.html index 86e3c209b..4c73c97d2 100644 --- a/handbook/troggle/datamodel.html +++ b/handbook/troggle/datamodel.html @@ -10,6 +10,10 @@

Troggle Data Model (python)

Troggle data architecture

+
+ +
Class Diagram
(Click to enlarge)
+
Auto-generated on 3 April 2020 with troggle$ python3 manage.py inspectdb. (Omitting all the 'Meta' sub-classes. )
Several classes have been edited out of this file between then and 30 January 2023 as they have been deleted as troggle has been tidied up.

All the classes below inherit from the django class (models.Model) and are thereby made persistent in the database. All persistent instance variables are defined with e.g. "models.BooleanField()" types which is why they look so strange to a normal python programmer. diff --git a/handbook/troggle/scriptsqms.html b/handbook/troggle/scriptsqms.html index cd7b176a4..623c4895e 100644 --- a/handbook/troggle/scriptsqms.html +++ b/handbook/troggle/scriptsqms.html @@ -64,16 +64,39 @@ It has been retired because the mapping software packages it used were terminall

QMs all use the same QM description conventions. +

QMs - monitoring progress

+

Each cave has a report listing all the extant and ticked-off QMs, e.g. +/cave/qms/1623-264/ or /cave/qms/1623-161/. The 161 report includes both old (spreadhseet-era) QMs and modern (2015 survex file) QMs. + +

QMs - recording progress

+

Since 2015 we have had no generally agreed way of recording whether a QM has been ticked-off or not. + +

In 2022 we developed a proposal (see "Ticking them off" below) to add an extra line to the original survex file, now that survex files can be edited easily on-line (and the version control happens invisibly and automatically). + +

QMs - how we first record them

+

Today we write the QM into the survex file: see documenting newly found QMs. +

We used to write them into a spreadsheet file (pre-2015). These old files are today still parsed by troggle to produce reports. +

troggle/parsers/survex.py

Troggle troggle/parsers/survex.py currently parses and stores all the QMs it finds in survex files. The tables where the data -is put are listed in the current data model. For ticking them off we have only protoype code: adding an extra line +is put are listed in the current data model. + + +

Ticking them off
+For ticking them off we have only protoype code: adding an extra line in the survex file of the format -
QMnn TICK date comment
 -e.g. -
QM15 TICK 2022-07-20 This is a dummy ticked QM
 -as illustrated in + + +QMnn TICK date comment + +e.g. +QM15 TICK 2022-07-20 This is a dummy ticked QM + +

as illustrated in 2015-15A -in 258/flashhard2.svx. +in 258/flashhard2.svx.
+The TICKed QM appears at the end of the report /cave/qms/1623-258 at the bottom of the page.

+

troggle/parsers/qms.py

Troggle currently reports QMs separately collated for three historic caves and also imports all the QMs inside survex files. @@ -101,6 +124,8 @@ These URL recognisers work:

Note that the hand-edited qm.csv for Balkonhohle was apparently abandoned unfinished as we transitioned to putting the QMs in the survex files instead. It contains QMs from 2014 and 2016:
/1623/264/qm.csv - unused
+

QM archeology

+

js/QM_helper.js

A relic.

This is referred to in core/admin.py and appears to help with the userinterface within the @@ -137,9 +162,9 @@ Fields in 264/qm.csv are:

There are also three versions of the QM list for cave 161 (Kaninchenhohle) apparently produced by this method but hand-edited:
-/1623/161/qmaven.html 1996 version
-/1623/161/qmtodo.html 1998 version
-/1623/161/qmdone.html 1999 (incomplete) version +/1623/161/qmaven.htm 1996 version
+/1623/161/qmtodo.htm 1998 version
+/1623/161/qmdone.htm 1999 (incomplete) version

In the /1623/204/ folder there is a script qmreader.pl which apparently does the inverse of tablize-qms.pl: it transforms a QMs' HTML file into a CSV file. @@ -182,7 +207,7 @@ This will work on all survex *.svx files even those which have not yet been run

Phil says (13 April 2020): "The generated files are not meant to be served by the webserver, it's a tool for people to run locally. Someone could modify it to create HTML output (or post-process the CSV output to do the same), but that is work still to be done." -

troggle archeology

+

Even older troggle archeology

Looking through urls.py and core/view_caves.py we see a lot of archaic code for providing new QM numbers, producing lists of QMs for a given cave and for downloading QM.csv files generated by the database. @@ -197,8 +222,8 @@ But none of it appears to be working today (5 July 2022). So someone was busy at one time. -

QMs - monitoring progress

+

QMs - monitoring progress - old script

find-dead-qms.py

This stand-alone script finds references to completed qms in the qm.csv files in the cave folders (/1623/ etc.) in the :expoweb: repository. It looks to see which QMs have been completed but where there is not yet a matching text in the cave description.

Quick and dirty Python script to find references to completed qms in the diff --git a/handbook/troggle/trogarch.html b/handbook/troggle/trogarch.html index 4ad4dc676..3a76034ea 100644 --- a/handbook/troggle/trogarch.html +++ b/handbook/troggle/trogarch.html @@ -41,12 +41,12 @@ The core of the troggle software is the data architecture: the set of tables int

Also there have been tables added to the core representation which are not included in this old diagram, e.g. Scannedimage, Survexdirectory, Survexscansfolder, Survexscansingle, Tunnelfile, TunnelfileSurvexscansfolders, Survey. See Troggle data model python code (15 May 2020) and click on the Class Diagram below on the right. -

-
+ +

Class Diagram
(Click to enlarge)
-
+

Purpose

The reasons why we have an online system at all are described in our website history.

There is an introductory article "Troggle: a revised system for cave data management".