As well as looking through the other plastic wallets in this year's lever-arch binder, have a look online at the list of all online wallets and have a look at all the folders referred to in your cave in, e.g. here is the data for cave 204: the wallet identifier is -in the right-most column titles "Scans". If you are doing an entirely new cave, then this won't exist yet but it +in the right-most column titles "Scans". (But this is not all the wallets, only those from which a survex file has been created). +If you are doing an entirely new cave, then this page won't exist yet but it will be where your new discovery is eventually published. + +
There are also lists of the wallets per year, per cave and per person (surveyor), +e.g. see all the wallets for 2018. +
As well as following through the links on those pages, have a look at the drawings files page which lists all the Tunnel and Therion files, and associated with each, the wallets and their the scanned notes, plan and elevation scans ("Scan files" column). +
All this information is in expo.survex.com/expofiles/ which is not under
+ All this scanned information is in expo.survex.com/expofiles/ which is not under
version control,
so the most recent person
to change anything can overwrite everyone else's work.
@@ -96,17 +96,20 @@ Creating a new cave... for the full list of steps.
We keep an index of how many of those steps have been completed in two places:
but the contents.json file has another,completely different function:
it may be the only online record that connects the wallet number to the cave identifier. So if a future cave surveyor desperately needs
to consult the original cave survey, it can be done by, e.g. The link between a .svx file and the wallet should also be recorded in the .svx file itself using the "*ref:" field, e.g.
The paper tick-list tracks the following steps for each online wallet:
@@ -180,138 +185,88 @@ looks like this:
Yes, this is a programming format
(standardised in 2013) and every comma is critical.
- When entering people's names it is important not to use any funny characters (such as "?") because
+ If you feel an urge to edit the file directly, instead of using the online form and checkboxes,
+you should take great care with the format and also with also updating the version-controlled copy of
+contents.json
+which is committed in the drawings repository. [The drawings repo copy will beocme the master copy
+sometime during 2022 and the contents.json in the same
+folder as all the scans will be removed.]
+
+When entering people's names, using the form or with a text editor, it is important not to use any funny characters (such as "?") because
peoples names here are used by the software to construct filenames for the surveying to-do lists. And "?" (for instance) is illegal
in filenames on Windows computers.
The folder containing all the wallets for the year, e.g.
-
-walletindex.html
-
-which lists all the wallets which have uncompleted tasks, and lists all the people responsible for completing them.
-You can see the walletindex.html for 2018 . Also there will be a linked file for each individual
-for their personal to-do list, and each online wallet contains its own
-index.html file which describes the survey production status for all the wallets.
+ You can get a summary of the outstanding 'tick list' for each individual caver at, e.g.
+ The magic also creates walletindex.html files in each folder /2018/2018#nn/
-and creates or updates a webpage for each person listed in any of the contents.json files
-in the folder/2018/ e.g. Becka Lawson.html.
+
+which lists all the wallets which have uncompleted tasks in a table of coloured boxes showing the status as "red" or "green".
If you want to see what the files are in each online wallet then look at All this magic is created by a script wallets.py.
+and you can navigate to the individual wallet folders too and look directly at the files there. Or you can navigate using
+the wallet upload form, which similarly lists the files in each wallet but also reports
+on the progress data for the wallet.
When, at the beginning of expo, you create the folder in
expofiles/surveyscans/ for the current year, e.g.
-/2019/, you will copy wallets.py
-from the previous year's folder. You will do this on your own laptop or on the expo laptop.
- You will also manually create a number of subfolders, e.g. 2019#01, 2019#02 etc. to be ready for the influx of
-new trip surveys.
- Next you will test that the magic works: open a terminal in
-expofiles/surveyscans/2018/ and run
-
-python wallets.py
- This will create a default contents.json and
-walletindex.html in each online wallet subfolder and also a
-walletindex.html in the /2018/ folder.
-
- This python3 script works fine on Linux (Debian, Xubuntu, etc.) and also now works fine in the Windows 10 bash system.
+/2019/. You will do this on your own laptop or on the expo laptop.
+ You may also manually create a number of subfolders, e.g. 2019#01, 2019#02 etc. to be ready for the influx of
+new trip surveys, though this is absolutely not necessary as the folders are created as-needed by the upload form.
Ideally the cavers who are scanning their notes and typing in the survey data will also be updating the
-contents.json file in their wallet. In your dreams.
+contents.json data in their wallet using the tick-boxes on the upload form. In your dreams.
- The first difficulty when editing a blank contents.json
+ The first difficulty when editing the data
for a newly-created wallet is finding out which cave the wallet describes.
-The lable on the plastic wallet may say "radaghost to blitzkriek"
+The label on the plastic wallet may say "radaghost to blitzkriek"
(or whatever) but without the name of the cave you can't find the .svx files
-as you don't know that you need to look in e.g. loser/caves-1626/2018-dm-07/.
+as you don't know that you need to look in e.g. loser/caves-1626/2022-W-04/.
Usually the cave number is written by hand on the label of the wallet. Sometimes it will just give the
-informal name of the cave,e.g. "Homecoming",instead of the identifier "2018-dm-07" you want.
- A regular task during expo is for a nerd to review the contents.json files for
+informal name of the cave,e.g. "BrokenNeckHole",instead of the identifier "1623/2022-W-04" you want.
+ A regular task during expo is for a nerd to review the files and tick-list data for
recently created wallets and to check that names, dates and cave numbers are correct.
-You will run
-
-python wallets.py
-
-regularly, after every batch of survey data is entered or scanned.
- This will always overwrite all the walletindex.html files but it will never touch
-the contents.json files.
- You will also regularly synchronise your laptop
-and the expo laptop with expo.survex.com/expofiles/ is not under version control,
-so the most recent person
-to upload the contents of /2018/ will overwrite everyone else's work.
-This does not matter for the autogenerated files, but it is vital that it does not overwrite all the painfully manually edited
-contents.json files. Which is very easy to do.
-This does mean that this is one of the cases where
-it may be better to use rsync rather than an FTP client such as Filezilla.
The script detects if there are notesX.jpg planX.jpg and elevX.jpgfiles present, and
+ Troggle detects if there are notesX.jpg planX.jpg and elevX.jpgfiles present, and
produces a reminder/warning if they are not,even if these have all been scanned
and given different names.
The job of the checker (perhaps on a second pass) if to rename files so that these
-warnings disappear. But if tunnel or therion files have already been produce don't rename anything.
+warnings disappear. But if tunnel or therion files have already been produced, then don't rename anything.
-
-As all this is not under version control the timestamps of the files are really quite important in figuring things out when someone makes an update mistake.
-
-So script wallets.py has been fixed so that
- A copy of useful rsync scripts is kept in a file such as
expo.survex.com/expofiles/rsync2018toserver. Always run it with the -n option first,
-to see what overwriting you will do.
+to see what overwriting you will do. These snippets update the server with your recent changes on your own laptop and vice versa, but these days (2022) it is probably easier
+to do everything online to the server, and occasionally make a copy to your local machine to keep it synchronised - if you have a laptop you want to use for this sort of thing.
- The python script does more than just re-format the contents.json data into
-different formats. It also
+ Troggle also
Things it might do in future (if someone gets around to it) include:
-to be written...
-grep -rl "2018-dm-07" expofiles/surveyscans
+grep -rl "2001-12" drawings/walletjson
-will find and list all the wallets which contain survey data for cave 2018-dm-07 (which is also known as "Homecoming Cave" and which will
-have a different Austrian Kataster number issued for it in due course).
+or
+
+grep -rl "234" expofiles/surveyscans
+
+will find and list all the wallets which contain survey data for cave 1623-234 .
@@ -123,7 +126,9 @@ and so this information was not properly recorded. This will be fixed by hand-ed
"To do" lists for every caver
-
-/home/expo/expofiles/surveyscans/2018/
-
-will, after the appropriate magic has happened, contain a file
-
expo.survex.com/expofiles/surveyscans/2018/
-and you can navigate to the individual wallet folders too and look directly at the files there.
-
-Setting up the online wallets
-Maintaining the online wallets
-expo.survex.com/expofiles/surveyscan/2018/
-
and this is where it gets tricky.
-Naming the included files
-Not under version control
-
-
-
Useful rsync scripts
More wallets.py magic
-More Troggle magic
+
in the :loser: repository
- checking the cave number specified matches the folder for the .svx file,
- checking that the *ref: filed in the survex file is the same as the wallet name
-- detecting whether there is a description or a list of QMs in the survex file,
-- accepting a list of .svx files and not just one (a very common thing),
-- checking the name of the cave against the cave number,
-- checking whether the website page even exists for this cave,
- being more intelligent about .topo files and thus the lack of scan files,
- checking the date is in the recent past etc.
-How contents.json fields match
-walletindex.html reports
-
Old notes, being turned into real documentation...
+
Much of this just needs deleting..