From 463c861409b3d326954d20dcf6652e1aedf90459 Mon Sep 17 00:00:00 2001 From: Philip Sargent Date: Tue, 5 Jul 2022 17:42:09 +0300 Subject: [PATCH] Documn update for QMs --- handbook/troggle/scriptsqms.html | 68 +++++++++++++++++++------------- 1 file changed, 41 insertions(+), 27 deletions(-) diff --git a/handbook/troggle/scriptsqms.html b/handbook/troggle/scriptsqms.html index 31b382ec7..e0ab988b5 100644 --- a/handbook/troggle/scriptsqms.html +++ b/handbook/troggle/scriptsqms.html @@ -30,20 +30,46 @@ tl;dr - use svx2qm.py. Look at the output at:

QMs - the fourfold path

-

You will be familiar with documenting newly found QMs in the survex file when you type it in. But QMs are only useful if they can be easily scanned by people planning the next pushing trip. That's what we are discussing here. +

You will be familiar with documenting newly found QMs in the survex file when you type it in. But +QMs are only useful if they can be easily scanned by people planning the next pushing trip. That's what we are discussing here. -

There are four (and a half) ways we have used to manage QMs: +

There are four (roughly) ways we have used to manage QMs:

    -
  1. Perl script - Historically QMs were not in the survex file but typed up in a separate list qms.csv for each cave system. A perl script turned that into an HTML file for the website. But there appear to be 3 different formats for this. -
  2. Perl + troggle - One of troggle's input parsers "QM parser" is specifically designed to import the three HTML files produced from qms.csv but doesn't do anything with that data (yet). +
  3. Hand-edited lists of QMS - only exist for 1623-161 Kaninchenhöhle +
  4. Perl script - Historically QMs were not in the survex file but typed up in a separate list qms.csv for +each cave system. A perl script turned that into an HTML file for the website. But there appear to be 3 different formats for this. Not currently used. +
  5. Perl + troggle - One of troggle's input parsers "QM parser" is specifically designed to import the three HTML files +produced from qms.csv and produces reports by cave and individually, e.g. see the 161 QMs +(slow page), which is old compared with the hand-edited 1623-161 page which was derived from it.
  6. Python script - Phil Withnall's 2019 script svx2qm.py scans all the QMs in a single survex file. See below for how to run it on all survex files. -
  7. New troggle - Sam's recent addition to troggle's "survex parser" makes it recognise and store QMs when it parses the survex files. -
  8. The elderly Prospecting Guide - covers some of the same sort sof information as needed by someone wanting to -chase QMs. It is a troggle-generated document at expo.survex.com/prospecting_guide/. It is so old that "top camp" in the guide refers to the col camp and not the Stonebridge bivvy. Some updates were done in 2007. +
  9. The elderly Prospecting Guide - Used to cover some of the same sorts of information as needed by someone wanting to +chase QMs. It was a troggle-generated document at expo.survex.com/prospecting_guide/. +It has been retired because the mapping software packages it used were terminally outdated.
+

The plan is to write new parsers to extract the QM information from the survex files and to +integrate these QMs into the same reports as those parsed from the CSV files.

QMs all use the same QM description conventions. +

troggle/parsers/qms.py

+

Troggle currently reports QMs for only three historic caves and ignores all the QMs inside survex files. +

The parser troggle/parsers/qms.py currently imports the qm.csv files used by +the 2004 perl script tablize-qms.pl (see below) into troggle using a mixture of csv and html parsers: +

parseCaveQMs(cave='stein',inputFile=r"1623/204/qm.csv")
+parseCaveQMs(cave='hauch',inputFile=r"1623/234/qm.csv")
+parseCaveQMs(cave='kh', inputFile="1623/161/qmtodo.htm")
+#parseCaveQMs(cave='balkonhoehle',inputFile=r"1623/264/qm.csv")
+and reports these by cave and individually, e.g. see the 204 QMs (slow page). +

+ +These URL recognisers work: + +

Note that the hand-edited qm.csv for Balkonhohle was apparently abandoned unfinished as we transitioned to putting the QMs in the survex files instead. It contains QMs from 2014 and 2016:
+/1623/264/qm.csv - unused
+

js/QM_helper.js

A relic.

This is referred to in core/admin.py and appears to help with the userinterface within the @@ -107,16 +133,6 @@ with these details: then a short description here is OK." -

troggle/parsers/qms.py

-

The parser troggle/parsers/qms.py currently imports those same qm.csv files from the perl script into troggle using a mixture of csv and html parsers: -

parseCaveQMs(cave='stein',inputFile=r"1623/204/qm.csv")
-parseCaveQMs(cave='hauch',inputFile=r"1623/234/qm.csv")
-parseCaveQMs(cave='kh', inputFile="1623/161/qmtodo.htm")
-#parseCaveQMs(cave='balkonhoehle',inputFile=r"1623/264/qm.csv")
-but does not apparently have any output webpage to display them (yet). -

-

Note that the hand-edited qm.csv for Balkonhohle was apparently abandoned unfinished as we transitioned to putting the QMs in the survex files instead. It contains QMs from 2014 and 2016:
-/1623/264/qm.csv - unused

svx2qm.py

Philip Withnall's 2019 QM extractor svx2qm.py (in :loser:/qms/) can be used to generate a list of all the QMs in all the svx files in either text or CSV format. When run together with file and xargs it will produce a output listing all the QMs: @@ -135,27 +151,25 @@ This will work on all survex *.svx files even those which have not yet been run

Phil says (13 April 2020): "The generated files are not meant to be served by the webserver, it's a tool for people to run locally. Someone could modify it to create HTML output (or post-process the CSV output to do the same), but that is work still to be done."

troggle/parsers/survex.py

-

The QMs inside the survex files are parsed by troggle along with all the other information -inside survex files and stored in the database. But the webpages which display this data are very rudimentary and currently useless, e.g. /getQMs/1623-204 or /cave/qms/1623-204. -Looking through urls.py and core/view_caves.py we see a lot of code for providing new QM numbers, producing lists of QMs for a given cave and for downloading QM.csv files generated by the database. But none of it appears to be working today (14 May 2020), see below. +

Troggle troggle/parsers/survex.py currently parses and stores all the QMs it finds in survex files. The tables where the data +is put are listed in the current data model including structure for ticking them off. These QMs are stored in +the database. But the webpage which displays this data is currently broken, e.g. /getQMs/1623-204. -

Sam's parser additions

-

Troggle troggle/parsers/survex.py currently parses and stores all the QMs it finds in survex files. The tables where the data is put are listed in the current data model including structure for ticking them off. +

troggle archeology

+ +

Looking through urls.py and core/view_caves.py we see a lot +of archaic code for providing new QM numbers, producing lists of QMs for a given cave and for downloading QM.csv files generated by the database. +But none of it appears to be working today (5 July 2022). -

There is a troggle template file :troggle:/templates/qm.html which is intended to become a useful outstanding QM report in future. Though since it was last edited in 2009, this does not seem to be on anyone's urgent task list.

Troggle has archaic URL recognisers in :troggle:/urls.py for:

So someone was busy at one time. -

There is not yet a troggle report listing the QMs which works. -

QMs - monitoring progress

find-dead-qms.py