mirror of
https://expo.survex.com/repositories/expoweb/.git/
synced 2024-11-22 07:11:55 +00:00
298 lines
15 KiB
HTML
298 lines
15 KiB
HTML
<!DOCTYPE html>
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
<title>Survey Handbook - online wallet</title>
|
|
<link rel="stylesheet" type="text/css" href="/css/main2.css" />
|
|
</head>
|
|
<body>
|
|
<h2 id="tophead">CUCC Expedition Handbook</h2>
|
|
<h1>Maintaining the online wallets</h1>
|
|
<p>If you are a newcomer to the system, read the <a href="newwallet.html#onlinew">beginner's introduction to online wallets</a> first.
|
|
|
|
|
|
<style>figure {font-weight: bold; font-size: small; font-family: sans-serif;font-variant-caps: small-caps;}</style>
|
|
<div class="onright">
|
|
<figure>
|
|
<a href="../i/qm-image.jpg">
|
|
<img src="../i/qm-image.jpg"></a>
|
|
<figcaption style="font-variant-caps: small-caps;">
|
|
<em>Original notes</em>
|
|
</figcaption>
|
|
</figure>
|
|
</div>
|
|
|
|
<h2>Why we have online wallets</h2>
|
|
<p>There are three quite different reasons:
|
|
<ol>
|
|
<li>The scans of the survey notebook pages are the ultimate original raw survey data and completely irreplaceable.
|
|
<li>The other files in the wallet are part of the process of producing a survey of the cave as a whole.
|
|
<li>Individual online to-do lists are produced automatically for each caver, each cave, and each year. These
|
|
have a colour-coded table showing what survey processing tasks they haven't finished yet.
|
|
</ol>
|
|
|
|
<h2>You need to know this bit</h2>
|
|
<p>All this scanned information is in <span style="font-family:monospace">expo.survex.com/expofiles/</span> which is <font color=red><b>not under
|
|
version control</b></font>,
|
|
so the most recent person
|
|
to change anything <font color=red><b>can overwrite everyone else's work</b></font>.
|
|
|
|
<h3>What's in the online wallet</h3>
|
|
<h4>scans in the expofiles/surveyscans/</h4>
|
|
<ul>
|
|
<li>The <a href="#scans">scanned images</a> of survey notes
|
|
|
|
</ul>
|
|
<h4>metadata in the version control system (drawings/walletjson/)</h4>
|
|
|
|
<ul>
|
|
<li><a href="#status">Progress status</a> of drawing up the survey
|
|
<li>Links to <a href="#derived">derived data</a>: survex files, survey drawings
|
|
</ul>
|
|
|
|
|
|
<h3 id="scans">The scanned images</h3>
|
|
<p>These are simply the scanned images (or digital photographs) of each page of the original survey notes.
|
|
They should be named <em><span style="font-family:monospace">notesXXX.jpg</span></em> where "XXX" can be
|
|
anything you like. In the past we have had scanned pages called notes1.jpg, notes2.jpg, notes3.jpg; but <em>don't
|
|
do this</em>, instead give descriptive names, e.g. <span style="font-family:monospace">notes-twentyseconddropdronesurvey.jpg</span> or whatever,
|
|
but do put "notes-.." at the beginning or "..-notes" at the end of the filename (or "plan-.." or "elev-.." as appropriate).
|
|
<p>It is important that you use use the .jpg (JPEG) file format, and definitely not PNG (very voluminous)
|
|
or PDF (very hard to re-use elsewhere). Set the scanner at 300 dpi and adjust the contrast of the image after scanning
|
|
by using photo-editing software to enhance the writing. Also please crop each image to just the area containing
|
|
the survey data.
|
|
<p>As soon as the notes have been scanned you should (a) copy them to a USB stick or email them to someone, (b) check that the entire online wallet is on the expo server at
|
|
<span style="font-family:monospace">expo.survex.com</span> using the <a href="/walletedit/2019:01">Upload Scans</a> form.
|
|
This is so that these precious files are backed-up as soon as possible.
|
|
<p>(If you want to move things around, then assuming you have done the
|
|
<a href="../computing/keyexchange.html">key-pair procedure</a>, do the upload by <a href="../uploading.html">sFTP</a>
|
|
as you do for photos, but to the directory described in the <a href="newwallet.html">introduction to online wallets</a>.)
|
|
|
|
<p>The <em><span style="font-family:monospace">notesXXX.jpg</span></em> files need to be at moderately high resolution but the plan and
|
|
elevation files are usually fine at 200 dpi. So if the caver has scanned these at high resolution you can reduce the size of these files
|
|
without damange.
|
|
|
|
<div class="onright">
|
|
<figure>
|
|
<a href="../i/scan-upload.jpg">
|
|
<img src="../t/scan-upload.jpg"></a>
|
|
<figcaption style="font-variant-caps: small-caps;">
|
|
<em>Wallet data on Scan Upload (click to enlarge)</em>
|
|
</figcaption>
|
|
</figure>
|
|
</div>
|
|
|
|
<h3 id="status">Progress status</h3>
|
|
<p>The illustration shows the lower half of the <a href="/walletedit/2019:01">Upload Scans</a> form.
|
|
This is the information about the wallet itself (this is called 'metadata').
|
|
<p>Once you are logged on as user 'expo', you can edit all the data. If you use the right runes for the survex files (as shown in the image),
|
|
a clickable-list of the survex files will appear on the upper-left which will take you directly to the survex files - which you can edit
|
|
there online too.
|
|
<p>There is only one absolutely required field: the date. Unless you specify a date the form will not save your data and the "Submit changes"
|
|
button won't work.
|
|
|
|
<h3 id="derived">Links to derived data</h3>
|
|
<p>These links are not actually in the wallet, they are in the survex files, Tunnel files and Therion files.
|
|
The links are simply <b>the name of the wallet</b>, e.g. <var>2019#01</var>. So once a wallet has been used, i.e. once you have typed up the first
|
|
survex file, you should not rename it or move the contents (unless you can find and change all the files referring to it).
|
|
|
|
<h2>How this all actually works</h2>
|
|
|
|
<h3>The other files and online index <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span></h3>
|
|
<p>All the other files are part of the multi-step process of producing the cave survey - see <a href="newcave.html">
|
|
Creating a new cave...</a> for the full list of steps.
|
|
|
|
<p>We keep an index of how many of those steps have been completed in two places:
|
|
<ul>
|
|
<li>On paper, in the tick boxes in the contents list of this years' survey trips' lever-arch file in the potato hut.
|
|
<li>Online in the <em><span style="font-family:monospace">contents.json</span></em> file, a copy of which exists in the version control system.
|
|
</ul>
|
|
|
|
<p>but the <em><span style="font-family:monospace">contents.json</span></em> file has another,completely different function:
|
|
it may be the <b>only online record</b> that connects the wallet number to the cave identifier. So if a future cave surveyor desperately needs
|
|
to consult the original cave survey, it can be done by, e.g. <br>
|
|
<a href="/cave/scans/1623-264">/cave/scans/1623-264</a><br>
|
|
will find and list all the wallets which contain survey data for cave 1623-264.
|
|
|
|
<p>The link between a .svx file and the wallet should also be recorded within the .svx file itself using the "*ref:" field, e.g.
|
|
<pre>
|
|
*ref 2018#06
|
|
; the #number is on the clear pocket containing the original notes
|
|
</pre>
|
|
But sometime in mid-Expo 2015 everyone stopped using the survex template file and copied from a friend instead,
|
|
and so this information was not properly recorded. This will be fixed by hand-editing in due course.
|
|
(Note that many old .svx files were processed with an older version of survex which did not support this feature and so a comment was used instead.)
|
|
|
|
<h2>Much more detail - more than you want</h2>
|
|
<p>Troggle produces very useful auto-generated reports of the status of the wallets and the survex files
|
|
<ul>
|
|
<li><a href="http://expo.survex.com/survey_scans/">List of all wallets</a> [SLOW to load] and the survex files produced from them (incomplete due to poor data entry especially since 2015)
|
|
<li><a href="http://expo.survex.com/expedition/2018">List of all trips and survex files</a> (scroll down to the bottom of the page) in any one year - includes a link to the logbook fragment for the relevant day
|
|
<li><a href="http://expo.survex.com/personexpedition/michael-sargent/2018">List of trips and surveys</a> by a particular person in a particular year (nothing to do with wallets but added here for completeness).
|
|
<li> <a href="/wallets/year/2022">Report on wallets status for 2022</a>
|
|
<li><a href="/cave/scans/1623-264">Report on wallets status for cave 1623-264</a>
|
|
</ul></ul>
|
|
These troggle reports are invaluable for finding data entry errors or other mistakes.
|
|
|
|
<p> The paper tick-list tracks the following steps for each online wallet:
|
|
<ul>
|
|
<li><u>Survex</u>
|
|
<ul>
|
|
<li>Data
|
|
<li>LRUD
|
|
<li>Description + QMs
|
|
</ul>
|
|
<li><u>Drawn</u>
|
|
<ul>
|
|
<li>Plan + Xsections
|
|
<li>Elevation
|
|
</ul>
|
|
<li><u>Scanned</u>
|
|
<ul>
|
|
<li>In cave notes
|
|
<li>Plan + Xsections
|
|
<li>Elevation
|
|
</ul>
|
|
|
|
<li>Tunnel
|
|
<li>Online guidebook updated
|
|
<li>json file edited
|
|
</ul>
|
|
<p>(where the "json file updated" step only refers to the initial editing of the json file to ensure
|
|
that it has the right people, date and cave identifier and name).
|
|
|
|
<p>A fully-populated and complete <em><span style="font-family:monospace">contents.json</span></em> file (this is the 'metadata')
|
|
looks like this:
|
|
<pre>
|
|
{
|
|
"description written": false,
|
|
"website updated": false,
|
|
"people": [
|
|
"Dickon Morris",
|
|
"Jon Arne Toft",
|
|
"Becka Lawson"],
|
|
"elev not required": false,
|
|
"cave": "1626-359",
|
|
"survex not required": false,
|
|
"qms written": true,
|
|
"plan not required": false,
|
|
"electronic survey": false,
|
|
"plan drawn": true,
|
|
"date": "2018-07-13",
|
|
"elev drawn": true,
|
|
"description url": "",
|
|
"survex file": "['caves-1626/2018-dm-07/2018-dm-07.svx', 'caves-1626/359/359.svx'] ",
|
|
"name": "Homecoming cave"
|
|
}
|
|
</pre>
|
|
Yes, this is <a href="https://en.wikipedia.org/wiki/JSON">a programming format</a>
|
|
(standardised in 2013) and every comma is critical.
|
|
(Though you can miss off the ".svx" of the survex filenames in the list and it will cope.)
|
|
|
|
<p>If you feel an urge to edit the file directly, instead of using the online form and checkboxes,
|
|
you should take great care with the format and also with also updating the version-controlled copy of
|
|
<em><span style="font-family:monospace">contents.json</span></em>
|
|
which is committed in the drawings repository. [The drawings repo copy is now, August 2022, the master and only copy.]
|
|
<p>
|
|
When entering people's names, using the form or with a text editor, it is important not to use any funny characters (such as "?") because
|
|
peoples names here are used by the software to construct filenames for the surveying to-do lists. And "?" (for instance) is illegal
|
|
in filenames on Windows computers.
|
|
|
|
<h3>"To do" lists for every caver</h3>
|
|
<p>You can get a summary of the outstanding 'tick list' for each individual caver at, e.g.
|
|
<ul>
|
|
<li><a href="/wallets/person/corin-donne">/wallets/person/CorinDonne</a>
|
|
</ul>
|
|
|
|
<p>
|
|
which lists all the wallets which have uncompleted tasks in a table of coloured boxes showing the status as "red" or "green".
|
|
|
|
<p>If you want to see what the files are in each online wallet then look at <br>
|
|
<a href="http://expo.survex.com/expofiles/surveyscans/2018/">expo.survex.com/expofiles/surveyscans/2018/</a> <br>
|
|
and you can navigate to the individual wallet folders too and look directly at the files there. Or you can navigate using
|
|
the wallet upload form, which similarly lists the files in each wallet but also reports
|
|
on the progress data for the wallet.
|
|
|
|
<h3>Setting up the online wallets</h3>
|
|
At the beginning of a new expo you do not need to do any setup for new wallets. The necessary folders etc. will be created automatically when any user creates a wallet with the 'Upload Scans' form. [In past years before 2022, a nerd would have needed to create all the wallets in advance.]
|
|
|
|
|
|
<h3>Maintaining the online wallets</h3>
|
|
<p>Ideally the cavers who are scanning their notes and typing in the survey data will also be updating the 'metadata' (in the
|
|
<span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> file) using the tick-boxes on the upload form. In your dreams.
|
|
|
|
<p>The first difficulty when editing the data
|
|
for a newly-created wallet is finding out which cave the wallet describes.
|
|
The label on the plastic wallet may say "radaghost to blitzkriek"
|
|
(or whatever) but without the name of the cave you can't find the .svx files
|
|
as you don't know that you need to look in e.g. loser/caves-1626/2022-W-04/.
|
|
Usually the cave number is written by hand on the label of the wallet. Sometimes it will just give the
|
|
informal name of the cave,e.g. "BrokenNeckHole",instead of the identifier "1623/2022-W-04" you want.
|
|
<p>A regular task during expo is for a nerd to review the files and tick-list data for
|
|
recently created wallets and to check that names, dates and cave numbers are correct.
|
|
|
|
|
|
<h4>Naming the included files</h4>
|
|
<p>Troggle detects if there are notesX.jpg planX.jpg and elevX.jpg files present, and
|
|
produces a reminder/warning if they are not,even if these have all been scanned
|
|
and given different names.
|
|
<p>
|
|
The job of the checker (perhaps on a second pass) if to rename files so that these
|
|
warnings disappear. But if tunnel or therion files have already been produced, then don't rename anything.
|
|
|
|
|
|
<h4>Useful rsync scripts</h4>
|
|
<p>A copy of useful rsync scripts is kept in a file such as
|
|
<span style="font-family:monospace; size=x-small; background-color: lightgray">expo.survex.com/expofiles/rsync2018toserver</span>. Always run it with the -n option first,
|
|
to see what overwriting you will do. These snippets update the server with your recent changes on your own laptop and vice versa, but these days (2022) it is probably easier
|
|
to do everything online to the server, and occasionally make a copy to your local machine to keep it synchronised - if you have a laptop you want to use for this sort of thing.
|
|
|
|
<h4>Surface trips</h4>
|
|
<p>For prospecting and surface surveying it is not clear whether the default folder
|
|
for the url link to a survex file should be repo :loser: surface/1623/allplateau.svx
|
|
[More documentation required here.]
|
|
|
|
<h3>More Troggle magic</h3>
|
|
<p> Troggle also
|
|
<ul>
|
|
<li>checks whether the .svx files listed are actually present in the <a href="../computing/repos.html">:loser: repository</a>
|
|
<li>checks for the presence of notesXXX.jpg, planXXX.jpg and elevXXX.jpg files
|
|
<li>checks for the presence of XXXnotes.jpg, XXXplan.jpg, XXXelevation.jpg and XXXelev.jpg files
|
|
<li>creates helpful URL links to the existing online survey documentation for the cave being surveyed
|
|
<li>creates links to other things happening on the same day as a wallet's date: logbook entries and other survex files
|
|
,!--<li>creates helpful URL links to the working files you are managing on your own laptop-->
|
|
</ul>
|
|
|
|
<p>Things it might do in future (if someone gets around to it) include: <br>
|
|
- checking the cave number specified matches the folder for the .svx file,<br>
|
|
- checking that the *ref: filed in the survex file is the same as the wallet name<br>
|
|
- being more intelligent about .topo files and thus the lack of scan files,<br>
|
|
- checking the date is in the recent past etc.<br><br>
|
|
|
|
<!--
|
|
The old wallets.py script runs without errors on earliers years before 1999 when there was not the same wallet system, but the results are less useful, e.g. see
|
|
http://expo.survex.com/expofiles/surveyscans/1999/
|
|
and compare with
|
|
http://expo.survex.com/wallets/year/1999
|
|
<p>
|
|
Old notes, being turned into real documentation...
|
|
<br>Much of this just needs deleting..
|
|
<pre>
|
|
# Update March 2019
|
|
|
|
For 2014 and earlier one needs to do a lot more data entry. The contents.json files for 2014 and earlier do not say who the
|
|
people were on the trip. So we would need to work from the svx files (where they contain the *ref: wallet ID),
|
|
original plastic wallets (and the scanned drawings and notes - which are incomplete) to enter that data.
|
|
This is made much easier by the troggle reports
|
|
http://expo.survex.com/survey_scans/
|
|
http://expo.survex.com/expedition/2014
|
|
http://expo.survex.com/survey/2018%2330
|
|
|
|
This is probably not worth doing except maybe for specific critical connections.
|
|
</pre>
|
|
-->
|
|
|
|
|
|
<hr /></body>
|
|
</html>
|