update wallets documn

handbook/survey/newwallet.html

@@ -122,12 +122,18 @@ should also fill in so that people can find your survey again.</p>-->
 <p>As well as looking through the other plastic wallets in this year's lever-arch binder, have a look online at 
 the <a href="/survey_scans/">list of all online wallets</a> and have a look at all the folders referred to
 in your cave in, e.g. here is the data for <a href="/survexfile/204">cave 204</a>: the wallet identifier is 
-in the right-most column titles "Scans". If you are doing an entirely new cave, then this won't exist yet but it
+in the right-most column titles "Scans". (But this is not all the wallets, only those from which a survex file has been created).
+If you are doing an entirely new cave, then this page won't exist yet but it
 will be where your new discovery is eventually published.
+
+<p>There are also lists of the wallets per year, per cave and per person (surveyor), 
+e.g. see <a href="/wallets/year/2018">all the wallets for 2018</a>.
+
 <p>As well as following through the links on those pages, have a look at the 
 <a href="/dwgfiles">drawings files page</a> which lists all the Tunnel and Therion files, and associated with each, the wallets and 
 their the scanned notes, plan and elevation scans ("Scan files" column). 
 
+
 <h3 id="onlinew">Scan the notes into the online wallet</h3>
 
 <style>figure {font-weight: bold; font-size: small; font-family: sans-serif;font-variant-caps: small-caps;}</style>

handbook/survey/onlinewallet.html

@@ -31,7 +31,7 @@
 </ol>
 
 <h2>You need to know this bit</h2>
-<p>All this information is in <span style="font-family:monospace">expo.survex.com/expofiles/</span> which is <font color=red><b>not under 
+<p>All this scanned information is in <span style="font-family:monospace">expo.survex.com/expofiles/</span> which is <font color=red><b>not under 
 version control</b></font>, 
 so the most recent person
 to change anything <font color=red><b>can overwrite everyone else's work</b></font>. 
@@ -96,17 +96,20 @@ Creating a new cave...</a> for the full list of steps.
 <p>We keep an index of how many of those steps have been completed in two places:
 <ul>
 <li>On paper, in the tick boxes in the contents list of this years' survey trips' lever-arch file in the potato hut.
-<li>Online in the <em><span style="font-family:monospace">contents.json</span></em> file which exists in each trip online wallet.
+<li>Online in the <em><span style="font-family:monospace">contents.json</span></em> file, a copy of which exists in each trip online wallet.
 </ul>
 
 <p>but the  <em><span style="font-family:monospace">contents.json</span></em> file has another,completely different function:
 it may be the <b>only online record</b> that connects the wallet number to the cave identifier. So if a future cave surveyor desperately needs
 to consult the original cave survey, it can be done by, e.g. <br>
 <span style="font-family:monospace">
-grep -rl "2018-dm-07" expofiles/surveyscans
+grep -rl "2001-12" drawings/walletjson
 </span><br>
-will find and list all the wallets which contain survey data for cave 2018-dm-07 (which is also known as "Homecoming Cave" and which will
+or<br>
-have a different Austrian Kataster number issued for it in due course).
+<span style="font-family:monospace">
+grep -rl "234" expofiles/surveyscans
+</span><br>
+will find and list all the wallets which contain survey data for cave 1623-234 .
 
 <p>The link between a .svx file and the wallet should also be recorded in the .svx file itself using the "*ref:" field, e.g. 
 <pre>
@@ -123,7 +126,9 @@ and so this information was not properly recorded. This will be fixed by hand-ed
 <li><a href="http://expo.survex.com/survey_scans/">List of all wallets</a> [SLOW to load] and the survex files produced from them (incomplete due to poor data entry especially since 2015)
 <li><a href="http://expo.survex.com/expedition/2018">List of all trips and survex files</a> (scroll down to the bottom of the page) in any one year - includes a link to the logbook fragment for the relevant day
 <li><a href="http://expo.survex.com/personexpedition/MichaelSargent/2018">List of trips and surveys</a> by a particular person in a particular year (nothing to do with wallets but added here for completeness).
-</ul>
+<li> <a href="/wallets/year/2019">Report on wallets status for 2019</a>
+<li><a href="/cave/scans/1623-264">Report on wallets status for cave 1623-264</a>
+</ul></ul>
 These troggle reports are invaluable for finding data entry errors or other mistakes.
 
 <p> The paper tick-list tracks the following steps for each online wallet:
@@ -180,138 +185,88 @@ looks like this:
 Yes, this is <a href="https://en.wikipedia.org/wiki/JSON">a programming format</a>
   (standardised in 2013) and every comma is critical.
 
-<p>When entering people's names it is important not to use any funny characters (such as "?") because
+<p>If you feel an urge to edit the file directly, instead of using the online form and checkboxes, 
+you should take great care with the format and also with also updating the version-controlled copy of 
+<em><span style="font-family:monospace">contents.json</span></em>
+which is committed in the drawings repository. [The drawings repo copy will beocme the master copy 
+sometime during 2022 and the <em><span style="font-family:monospace">contents.json</span></em> in the same
+folder as all the scans will be removed.]
+ <p>
+When entering people's names, using the form or with a text editor,  it is important not to use any funny characters (such as "?") because
 peoples names here are used by the software to construct filenames for the surveying to-do lists. And "?" (for instance) is illegal
 in filenames on Windows computers.
 
 <h3>"To do" lists for every caver</h3>
-<p>The folder containing all the wallets for the year, e.g.
+<p>You can get a summary of the outstanding 'tick list' for each individual caver at, e.g.
-<pre>
+<ul>
-/home/expo/expofiles/surveyscans/2018/
+<li><a href="/wallets/person/CorinDonne">/wallets/person/CorinDonne</a>
-</pre>
+</ul>
-will, after the appropriate magic has happened, contain a file 
-<p>
-<span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span>
-<p>
-which lists all the wallets which have uncompleted tasks, and lists all the people responsible for completing them. 
-You can see <a href="http://expo.survex.com/expofiles/surveyscans/2018/walletindex.html">the  walletindex.html for 2018</a> . Also there will be a linked file for each individual 
-for their personal to-do list, and each online wallet contains its own 
-<span style="font-family:monospace">index.html</span> file which  describes the survey production status for all the wallets.
 
-<p>The magic also creates walletindex.html files in each folder /2018/2018#nn/
+<p>
-and creates or updates a webpage for each person listed in any of the contents.json files
+which lists all the wallets which have uncompleted tasks in a table of coloured boxes showing the status as "red" or "green". 
-in the folder/2018/ e.g. <a href="http://expo.survex.com/expofiles/surveyscans/2018/Becka%20Lawson.html">Becka Lawson.html</a>.
 
 <p>If you want to see what the files are in each online wallet then look at <br>
 <a href="http://expo.survex.com/expofiles/surveyscans/2018/">expo.survex.com/expofiles/surveyscans/2018/</a> <br>
-and you can navigate to the individual wallet folders too and look directly at the files there.
+and you can navigate to the individual wallet folders too and look directly at the files there. Or you can navigate using 
-
+the wallet upload form, which similarly lists the files in each wallet but also reports 
-<p>All this magic is created by a script <span style="font-family:monospace; size=x-small; background-color: lightgray">wallets.py</span>. 
+on the progress data for the wallet.
 
 <h3>Setting up the online wallets</h3>
 <p>When, at the beginning of expo, you create the folder in
 <span style="font-family:monospace; size=x-small; background-color: lightgray">expofiles/surveyscans/</span> for the current year, e.g. 
-<span style="font-family:monospace; size=x-small; background-color: lightgray">/2019/</span>, you will copy <span style="font-family:monospace">wallets.py</span> 
+<span style="font-family:monospace; size=x-small; background-color: lightgray">/2019/</span>. You will do this on your own laptop or on the expo laptop.
-from the previous year's folder. You will do this on your own laptop or on the expo laptop.
+<p>You may also manually create a number of subfolders, e.g. 2019#01, 2019#02 etc. to be ready for the influx of
-<p>You will also manually create a number of subfolders, e.g. 2019#01, 2019#02 etc. to be ready for the influx of
+new trip surveys, though this is absolutely not necessary as the folders are created as-needed by the upload form.
-new trip surveys.
-<p>Next you will test that the magic works: open a terminal in 
-<span style="font-family:monospace; size=x-small; background-color: lightgray">expofiles/surveyscans/2018/</span> and run
-<p>
-<span style="font-family:monospace; size=x-small; background-color: lightgray">python wallets.py</span>
-<br>
-<p>This will create a default <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> and
-<span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span> in each online wallet subfolder and also a 
-<span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span> in the <span style="font-family:monospace; size=x-small; background-color: lightgray">/2018/</span> folder.
-
-<p>This python3 script works fine on Linux (Debian, Xubuntu, etc.) and also now works fine in the <a href="https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/">Windows 10 bash system</a>. 
 
 <h3>Maintaining the online wallets</h3>
 <p>Ideally the cavers who are scanning their notes and typing in the survey data will also be updating the 
-<span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> file in their wallet. In your dreams.
+<span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> data in their wallet using the tick-boxes on the upload form. In your dreams.
 
-<p>The first difficulty when editing a blank <span style="font-family:monospace">contents.json</span> 
+<p>The first difficulty when editing the data  
 for a newly-created wallet is finding out which cave the wallet describes. 
-The lable on the plastic wallet may say "radaghost to blitzkriek"
+The label on the plastic wallet may say "radaghost to blitzkriek"
 (or whatever) but without the name of the cave you can't find the .svx files 
-as you don't know that you need to look in e.g. loser/caves-1626/2018-dm-07/.
+as you don't know that you need to look in e.g. loser/caves-1626/2022-W-04/.
 Usually the cave number is written by hand on the label of the wallet. Sometimes it will just give the 
-informal name of the cave,e.g. "Homecoming",instead of the identifier "2018-dm-07" you want.
+informal name of the cave,e.g. "BrokenNeckHole",instead of the identifier "1623/2022-W-04" you want.
-<p>A regular task during expo is for a nerd to review the <span style="font-family:monospace">contents.json</span> files for
+<p>A regular task during expo is for a nerd to review the files and tick-list data for
 recently created wallets and to check that names, dates and cave numbers are correct.
 
-You will run 
-<p>
-<span style="font-family:monospace; size=x-small; background-color: lightgray">python wallets.py</span>
-<p>
-regularly, after every batch of survey data is entered or scanned. 
-<p>This will always overwrite all the <span style="font-family:monospace">walletindex.html</span> files but it will never touch
-the <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> files.
-<p>You will also regularly synchronise your laptop
-and the expo laptop with <br />
-<span style="font-family:monospace">expo.survex.com/expofiles/surveyscan/2018/</span>
-<br />and this is where it gets tricky.
-<p><span style="font-family:monospace">expo.survex.com/expofiles/</span> is <font color=red><b>not under version control</b></font>, 
-so the most recent person
-to upload the contents of <span style="font-family:monospace">/2018/</span> <font color=red><b>will overwrite everyone else's work</b></font>. 
-This does not matter for the autogenerated files, but it is vital that it does not overwrite all the painfully manually edited
-<span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> files. Which is very easy to do.
-This does mean that this is one of the cases where 
-it may be better to use <span style="font-family:monospace; size=x-small; background-color: lightgray">rsync</span> rather than an FTP client such as Filezilla.
 
 <h4>Naming the included files</h4>
-<p>The script detects if there are notesX.jpg planX.jpg and elevX.jpgfiles present, and 
+<p>Troggle detects if there are notesX.jpg planX.jpg and elevX.jpgfiles present, and 
 produces a reminder/warning if they are not,even if these have all been scanned 
 and given different names.
 <p>
 The job of the checker (perhaps on a second pass) if to rename files so that these
-warnings disappear. But if tunnel or therion files have already been produce don't rename anything.
+warnings disappear. But if tunnel or therion files have already been produced, then don't rename anything.
 
 
-<h4>Not under version control</h4>
-<p>
-As all this is not under version control the timestamps of the files are really quite important in figuring things out when someone makes an update mistake.
-<p>
-So script <span style="font-family:monospace; size=x-small; background-color: lightgray">wallets.py</span> has been fixed so that
-<ul>
-<li>the generated <span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span> file in each wallet folder is given  the same timestamp as the <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> file there, <em>not the time of when the script is run</em>. This is unusual but intentional and in practice very helpful.
-<li>the script no longer overwrites the <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> files every  time it runs. It only changes that file's timestamp if it actually changes anything in the contents.json file.
-</ul>
-
 <h4>Useful rsync scripts</h4>
 <p>A copy of useful rsync scripts is kept in a file such as 
 <span style="font-family:monospace; size=x-small; background-color: lightgray">expo.survex.com/expofiles/rsync2018toserver</span>. Always run it with the -n option first,
-to see what overwriting you will do.
+to see what overwriting you will do. These snippets update the server with your recent changes on your own laptop and vice versa, but these days (2022) it is probably easier
+to do everything online to the server, and occasionally make a copy to your local machine to keep it synchronised - if you have a laptop you want to use for this sort of thing.
 
-<h3>More <em><span style="font-family:monospace; size=x-small; background-color: lightgray">wallets.py</span></em> magic</h3>
+<h3>More Troggle magic</h3>
-<p>The python script does more than just re-format the <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> data into
+<p> Troggle also 
-different formats. It also 
 <ul>
 <li>checks whether the .svx files listed are actually present <br>in the <a href="../computing/repos.html">:loser: repository</a>
 <li>checks for the presence of notesXXX.jpg, planXXX.jpg and elevXXX.jpg files
-<li>checks for the presence of XXXnotes.jpg, XXXplan.jpg and XXXelev.jpg files
+<li>checks for the presence of XXXnotes.jpg, XXXplan.jpg, XXXelevation.jpg and XXXelev.jpg files
-<li>creates a template <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> in any wallet which does not have one.
 <li>creates helpful URL links to the existing online survey documentation for the cave being surveyed
 <li>creates helpful URL links to the working files you are managing on your own laptop
-<li>Inserts the date & time into each generated file so that anyone can see when it was last updated
 </ul>
 
 <p>Things it might do in future (if someone gets around to it) include: <br>
 - checking the cave number specified matches the folder for the .svx file,<br>
 - checking that the *ref: filed in the survex file is the same as the wallet name<br>
-- detecting whether there is a description or a list of QMs in the survex file,<br>
-- accepting a list of .svx files and not just one (a very common thing),<br>
-- checking the name of the cave against the cave number,<br>
-- checking whether the website page even exists for this cave,<br>
 - being more intelligent about .topo files and thus the lack of scan files,<br>
 - checking the date is in the recent past etc.<br><br>
 
-<h3>How <em><span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span></em> fields match 
-<em><span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span></em> reports</h3>
-<p>
-<em>to be written...</em>
 <hr>
 Old notes, being turned into real documentation...
+<br>Much of this just needs deleting..
 <pre>