repo usage tutorial with pictures

2024-11-22 15:21:55 +00:00 · 2019-03-26 14:45:33 +00:00 · 2019-03-26 14:45:33 +00:00 · 3a84a38b1c
commit 3a84a38b1c
parent 8e18a94fa8
13 changed files with 825 additions and 0 deletions
--- a/handbook/computing/datamgt.html
+++ b/handbook/computing/datamgt.html
@ -0,0 +1,58 @@
 <html>
 <head>
 <title>CUCC Expo Handbook - Data Management</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook - Data Management</h2>
 <h1>Why cavers need effective data management</h1>
 <div style="text-align:left">
 <!-- Comment 
 Justified text is hard to read:
 https://designshack.net/articles/mobile/the-importance-of-designing-for-readability/
 https://designforhackers.com/blog/justify-text-html-css/
 -->
 <p>
 Cave exploration is more data-intensive than any other sport. The only way to "win" at this
 sport is to bring back large quantities of interesting survey, and possibly photos or scientific 
 data. Aside from the data collection requirements of the game itself, setting up a game (an 
 expedition) of cave exploration often involves collection of personal information ranging from 
 dates available to medical information to the desire to purchase an expedition t-shirt.
 <p>
 If an expedition will only happen once, low-tech methods are usually adequate to record 
 information. Any events that need to be recorded can go in a logbook. Survey notes must be 
 turned into finished cave sketches, without undue concern for the future expansion of those sketches.
 <p>
 However, many caving expeditions are recurring, and managing their data is a more challenging 
 task. For example, let us discuss annual expeditions. Every year, for each cave explored, a list 
 of unfinished leads (which will be called "Question Marks" or "QMs" here) must be maintained to 
 record what has and has not been investigated. Each QM must have a unique id, and information 
 stored about it must be easily accessible to future explorers of the same area. Similarly, on 
 the surface, a "prospecting map" showing which entrances have been investigated needs to be 
 produced and updated at least after every expedition, if not more frequently.
 <p>
 These are only the minimum requirements for systematic cave exploration on an annual expedition. 
 There is no limit to the set of data that would be "nice" to have collected and organized 
 centrally. An expedition might collect descriptions of every cave and every passage within every 
 cave. Digital photos of cave entrances could be useful for re-finding those entrances. Scans of 
 notes and sketches provide good backup references in case a question arises about a finished 
 survey product, and recording who did which survey work when can greatly assist the workflow, 
 for example enabling the production of a list of unfinished work for each expedition member. The 
 expedition might keep an inventory of their equipment or a catalog of their library. Entering 
 the realm of the frivolous, an expedition might store mugshots and biographies of its members, 
 or even useful recipes for locally available food. The more of this information the expedition 
 wishes to keep, the more valuable an effective and user-friendly system of data management.
 </div>
 <p><em>From "<a href="../../troggle/docsEtc/troggle_paper.odt" download>
 Troggle: a novel system for cave exploration information management</a>", by Aaron Curtis, CUCC.</em>
 <hr />
 </body>
 </html>
--- a/handbook/computing/essentials.html
+++ b/handbook/computing/essentials.html
@ -0,0 +1,98 @@
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
 <title>Essential GPS information</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook</h2>
 <h1>Essential GPS information</h1>
 <p>We have a regularly-updated file of all the cave entrances for the entire Schvartzmoosk&ouml;gel system (SMK). 
 This file also includes the tracks of the paths we regularly take: from Loser Alm car park to the col and to top camp, 
 and from top camp to Fisch Gesicht H&ouml;hle and to Tunnocks's, Balkonh&ouml;hle and Organh&ouml;hle.
 <h3><a href="#down">Download the data from the expo server</a></h3>
 <h3><a href="#up">Upload the data to your device</a></h3>
 <h2 id="down">Download the GPS essentials file from the expo server</h2>
 <p>
 We regularly create a new version of this essential data as the expo progresses and as expoers discover new entrances and 
 devise new routes to reach them.
 <p>
 To get the most recent version you will need to ask someone who is competent in using the version control system 
 (it's in loser/gpx/ and is generated from the survex data by a script). A fairly recent copy (17 July 2018)
 can be downloaded from here: <a href="essentials.gpx" download>essentials.gpx</a> (190K).
 Phones have a problem with a simple link like that. So with a phone you may need to try this link 
 <a href="https://www.wikiloc.com/outdoor-trails/carpark-stonebridge-27029006&utm_source=social&utm_medium=twitter.com&utm_campaign=badge">
 Wikilocs</a>
 from where you can download a short version of the file (track and one waypoint at StoneBridge only) or a Google Earth trail (you need to create a Wikiloc account first <em>and be logged-in</em> when you click on this). Or here is another <a href="https://www.wikiloc.com/wikiloc/download.do?id=27029006">direct download link</a>.
 <h2 id="up">Upload the GPS essentials file to your device</h2>
 <p>
 This is where it gets tricky because every device and phone app does this differently.
 <h3>GPS phone apps</h3>
 <p>
 This should work the same way whether you have an iPhone or an Android phone.
 <p>We do not have a recommended app as there is nothing we have found which quite does quite what we need. We are using
 <a href="http://www.gpsessentials.com/">www.gpsessentials.com</a> and <a href="https://osmand.net/">OsmAnd</a> so try one of these first. If you discover a good app, tell everyone about it.
 <p>
 Visit the <a href="http://www.gpsessentials.com/">www.gpsessentials.com</a> website and read the manual (top left, on the menu bar: "Manual") for how to do this. 
 Except that the manual doesn't tell you.
 <p>The OsmAnd documentation says:
 <ul>
 <li>"The simplest way to view a track you've downloaded is to tap on it in your device's file manager and choose to open it in OsmAnd. After that, you'll see the track in My places - My tracks or in the Dashboard - My tracks."
 </ul>
 <p>
 More documentation on this to follow...
 <h3>Modern Garmin handheld GPS devices</h3>
 <p>
 Connect the GPS device to your laptop (or the expo laptop) using the USB cable. A folder will open on the laptop showing the contents
 of the device. 
 You will see a subfolder called "GARMIN". Open the folder "GARMIN" and copy the file essentials.gpx which you 
 downloaded into that folder.
 <h3>Old Garmin handheld GPS devices</h3>
 <p>
 These need the Garmin communication protocol to import cave entrance locations (waypoints) and paths (tracks). You can't do it by simply copyingfiles.
 This means that you need special software on your laptop in addition to a USB cable that connects your laptop to the Garmin device.
 <p>
 <figure><a href="https://www.cablestogo.com/learning/connector-guides/usb">
    <img src="t/usb-minib-5pin-m-c.jpg"' 
 		alt="mini-USB socket" 
 	/></a>
    <figcaption><em>mini-USB b socket</em></figcaption>
 </figure>
 If your Garmin has a <b>mini</b>-USB socket, rather than the usual micro-USB found in phones, then you might have an "old" Garmin handheld, but some modern handhelds still use this old socket.
 <p>Once you have the right cable and connected your handheld to your laptop:
 <ul>
 <li>On a Windows machine, use "GPSbabel for Windows" which has an easy to use graphical user interface: 
 <a href="https://www.gpsbabel.org/download.html">download GPS Babel</a>
 <li>On a Linux machine the core gpsbabel command line utility is probably already installed; 
 but there is no simple, easy to use graphical interface. 
 Instead you must use other software such as QGIS or Viking (download using your usual Linux software installer) 
 which uses gpsbabel to talk to your device. 
 If this doesn't work then there are no useful error messages from Viking.
 </ul>
 <hr />
 <ul id="links">
 <li><a href="index.htm">Expedition Handbook</a>
 <ul>
 	<li><a href="survey/index.htm">Surveying guide</a> - Overview</li>
 	<li><a href="look4.htm">Prospecting guide</a> &ndash; Overview</li>
 	<li><a href="rescue.htm">Rescue guide</a></li>
 </ul></li>
 <li><a href="../index.htm">Back to Expedition Intro page</a></li>
 <li><a href="../../index.htm">Back to CUCC Home page</a></li>
 </ul>
 </body>
 </html>
--- a/handbook/computing/exposerver.html
+++ b/handbook/computing/exposerver.html
@ -0,0 +1,31 @@
 <html>
 <head>
 <title>Expo server</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook - Expo server in Cambridge</h2>
 <h1>Expo server in Cambridge</h1>
 <p>The server hosts 
 <a href="http://expo.survex.com">expo.survex.com</a>.
 <p>It lives (2018) under Sam Wenham's desk. Sam works for the University Computing Service (UCS), University of Cambridge, 
 so this is a secure place.
 <p>We interact with it using:
 <ul>
 <li>TortoiseHg - communicating with hg running on the server - for the DVCS 
 <a href="../../repositories">repositories</a> expoweb, loser, tunnelx, tunneldata.
 <li>FTP - mostly for uploading to /expofiles/
 <li>ssh - occasionally, by experts,to fix things when something goes wrong or for major site reconfiguration.
 </ul>
 Now readthe <a href="manual.html#manual">Expo data management systems manual</a>.
 <hr />
 <ul id="links">
 <li><a href="index.htm">Expedition Handbook</a>
 </ul>
 </body>
 </html>
--- a/handbook/computing/getsurvex.html
+++ b/handbook/computing/getsurvex.html
@ -0,0 +1,68 @@
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
 <title>CUCC Expedition Handboo - Survey software</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook</h2>
 <h1>Installing surveying tools</h1>
 <h2>Survex</h2>
 <p>The main software we use to process cave data and surface surveys is <b>survex</b>
 which has been written over several decades by CUCC cavers. 
 The first version was written during the 1990 Expo in Austria in the (old) potato hut.
 A <a href="survexhistory96.htm">history of survex</a> article was published in Cambridge Underground 1996. It covers the period 1988-1996.
 <p>Download the survex package here: <a href="https://survex.com/">www.survex.com</a> and install it.
 <p>You will discover that the application installed is actually called "aven" but do not be concerned. 
 This is what you will use to visualise .svx files as beautiful cave centre-line surveys.
 <p>If you are entering new survey data from a new cave, you will also need either 
 <a href="https://bitbucket.org/goatchurch/tunnelx/wiki/Home">TunnelX</a> or 
 <a href="https://therion.speleo.sk/">Therion</a> to 
 convert your sketches into actual plan and elevation presentation-quality surveys.
 <h2>TunnelX</h2>
 <p>Tunnel was written by <a href="http://expo.survex.com/folk/l/jtodd.htm">Julian Todd</a> (18 Austrian expos since 1989). It allows the generation of full 3D models of cave passages which can be viewed using a VRML browser.
 <p>
 <a href="https://bitbucket.org/goatchurch/tunnelx/wiki/Home">TunnelX intro</a>.
 <p>
 <a href="https://bitbucket.org/goatchurch/tunnelx/wiki/Installing_and_Running">TunnelX installation and running</a>.
 <p>
 <a href="http://expo.survex.com/expofiles/tunnelwiki/wiki/pages/Tunnel.html">TunnelX detailed wiki documentation (old)</a>.
 <h2>Therion</h2>
 <p>
 <a href="https://therion.speleo.sk/">Therion</a>
 <h2>GPS stuff</h2>
 <p>GPS is increasingly important for all the surface work.
 <p>
 <a href="https://wiki.openstreetmap.org/wiki/GpsPrune">GPS Prune</a> is a vitally useful utility.
 <h2>Other old stuff</h2>
 <p>
 <a href="https://code.google.com/archive/p/loch/">Loch</a> This is fork off of Therion's loch program. 
 The goal is to model Cave systems in 3d and have a high degree 
 of interaction between the user and the information. 
 There is currently no working version and is under very heavy development.
 <p>
 <a href="https://code.google.com/archive/p/topolinux/">TopoLinux</a> 
 consists of cave surveying applications for Linux PC and Android devices. 
 TopoDroid is an Open Source Android app to make cave surveys with the DistoX.
 <p>
 <a href="https://code.google.com/archive/p/psurvex/">psurvex</a>  
 My task  was to write a new program which could read the data from an external file, 
 ideally one which resembled a .svx file as much as possible.
 Psurvex supports 
 only a subset of the .svx file format but the same file should process in Survex 
 without any problems. You need to write a .svx file and save it somewhere on your machine. 
 This program will then process the file and create a second file containing a list of 
 points for plotting on graph paper and some statistics about your survey. 
 <hr />
 </body>
 </html>
--- a/handbook/computing/gpxupload.html
+++ b/handbook/computing/gpxupload.html
@ -0,0 +1,103 @@
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <title>CUCC Expedition Handbook: Uploading files/photos</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook</h2>
 <h1>Uploading GPS tracks and locations</h1>
 <h2>The end-result you are trying to achieve</h2>
 What you are trying to do is to get your recorded locations (waypoints) and wanderings (tracks) 
 (a) are recorded somewhere, (b) eventually	appear properly in the cave survey database and. You have to  upload 
 the tracks and waypoints in a GPX file to the right place. 
 <p>If you are really lazy (or really a beginner) you can use the simple upload method, but there are some 
 unavoidable complexities in getting the GPX file out of your device.
 <p><em>(If you are looking for how to upload some photos instead, those instructions are 
 <a href="uploading.html">here</a>)</em>.
 <h3>Instructions: contents</h3>
 <ol>
 <li><a href="#getgpx">Get the GPX file</a> that holds your locations and wanderings from your phone.
 <li><a href="#uploadgpx">Upload the GPX file</a> to the proper place.
 </ol>
 <h2 id="getgpx">Getting the GPX data out of your phone or device</h2>
 <ol>
 <li>Use the "Wikilocs" app (or another app with GPS tracking function) to record your track as you walk across the plateau. 
 <li>When you have finished your walk and are back on the internet, publish your track using the app.
 <li>In the app,there is an option to "share" your track by email:
 <ul>
 <li>Share it with yourself i.e. email it to your own email address.
 <li>Share it with someone who knows how to do the GPX thing and upload it properly.
 </ul>
 <li>On your laptop (or possibly on your phone) look at the email and visit the web page by clicking on the link.
 <li>The web page has a "Download" button: click on it.
 <li>It may give you options such as "Garmin", or "File". Choose "File". 
 <li>It will ask for a filename to use. Pick something like "northplat-asmith-2018-07-29" (if your name is Aaron Smith)
 <li>A GPX file "northplat-asmith-2018-07-29.gpx" will be downloaded to the Downloads folder on your laptop.
 </ol>
 <p>
 Congratulations. You now have your track recorded using GPS as a GPX file.
 <h2 id="uploadgpx">Upload instructions</h2>
 <h3>Simple upload instructions</h3>
 <ol>
 <li>Email the public link from the app to someone who knows how to do it. <br>
 <li>Email the  GPX file to someone who knows how to do it. <br>
 GPX files are small enough for email systems, so don't be shy of adding them as attachments.
 </ol>
 <p>If you can't find someone who knows how to do it, find the most extreme nerd you can find and point them at the 
 <a href="#expert">Expert instructions</a> below.
 <h3>Slightly less simple upload instructions</h3>
 <p>Using your own laptop on expo, or after you return from expo, 
 use the "more complex" instructions for <a href="uploading.html">uploading photos</a> to /uploads/, 
 but upload your GPX files instead.
 <h2 id="expert">Experts only</h2>
 <p>
 GPX data is stored in two places. 
 <ul>
 <li>initially in <em>expofiles/gpslogs/...</em>
 <li>some key selected tracks are later stored in the cave survey repository <em>::loser::</em>
 </ul>
 <p>
 GPS tracks are voluminous and we also get a lot of repetition 
 as people tend to follow the same routes for part of their walks. So the initial raw data is kept in 
 <pre>
 expofiles/gpslogs/&lt;year&gt;/&lt;MyName&gt;/
 </pre>
 e.g.
 <pre>
 expofiles/gpslogs/2018/PhilipSargent/
 </pre>
 <p>and you can create sub-folders for raw data and edited data, or for different parts of the plateau. You should always
 keep the raw, untouched data as well as any hand-edited data.
 <p>The process for uploading the GPX files to a specific folder <em>expofiles/gpslogs/...</em> is exactly the 
 same as for uploading photographs, so go to <a href="uploading.html#morecomplex">these "more complex" instructions</a>
 to learn how to do it.
 <p>Note the  naming convention for this folder created by Philip Sargent in 2018. 
 Human names in folders in expofiles are written in CamelCase; not lower-case letters. 
 This is for consistency with the naming for
 <a href="uploading.html#experienced">uploading photos</a>.
 <p>
 If you have edited GPS tracks and waypoints with no extraneous data 
 then, after agreeing this with other people as to its qualityand appropriateness, 
 it will go into the <a href="http://expo.survex.com/repositories/home/expo/loser/graph/">Loser DVCS repository</a> 
 in folder <em>/gpx/&lt;year&gt;/</em> e.g. <br>
 <em>::loser::/2018/stone-bridge-to-fischgesicht_aday-2018-07-12.gpx</em><br>
 <p>Note the  naming convention for this file created by Anthony Day on July 12th 2018. 
 Everything in any repository is always named using lower-case letters.
 <hr />
 </body>
 </html>
--- a/handbook/computing/logbooks.html
+++ b/handbook/computing/logbooks.html
@ -0,0 +1,125 @@
 <html>
 <head>
 <title>CUCC Expedition Handbook: The Website</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook</h2>
 <h1>Logbooks</h1>
 <p>As soon as possible after a trip finishes, a <b>hand-written write-up</b> of the trip is made in the nearest logbook: 
 the base camp logbook or the top camp logbook. All these logbook entries are then typed into a laptop (often the expo laptop) 
 which is then synchronised the version control system.
 <p>The logbook writeup is the oldest and most basic way of recording your trip but it must not be neglected. This is also where you put 
 your speculations and ideas for what looks promising  and what is obvious but doesn't go: things that are vital to future expoers. And please, please
 do lots of sketches in the logbook.
 <p>If you are at basecamp, then it is an excellent idea to 
 <b>type your logbook trip report</b> instead of writing it by hand - see <a href=#type">below</a>.
 <p>The contents of both the topcamp logbook and the basecamp logbook are typed into the same  
 "logbook.html" file for archiving. The drawings are scanned and stored in the same place, and hand-edited
 into the logbook.html file after expo finished.
 <p>If this is all new to you, please now read <a href="datamgt.html">Cave data management</a>, 
 and <a href="survey/why.htm">why we make surveys</a> and then the 
 <a href="survey/index.htm">Survey Handbook</a>
 <h3 =id="type">Typing just your trip report</h3>
 <p>If you are at basecamp, then it is an excellent idea to type up your logbook trip report.
 You can then print this and stick it in the logbook, adding any sketches by hand. 
 This will save someone (probably you) deciphering your handwriting and typing it up later.
 <p>These instructions assume that (a) you are sitting at the <em>expo laptop</em> and that someone who knows 
 the password has logged in for you (as user "expo"), and (b) that you know nothing about the software
 systems used by expo.
 <ul>
 <li>You will type your trip report as plain text using a text editor.
 <li>You will be typing into a file called something like "logbook-mynewtrip.txt" in the folder "Downloads"
 <li>You will be asking someone nerdy to take this trip report and to edit it into the proper place later.
 </ul>
 <p>The first challenge is to find how to start up the text editor. The <em>expo laptop</em> is running debian Linux
 with the Gnome 3.2 desktop manager, so click on "Activities" in the top left corner. 
 This will bring down a vertical menu of icons down the left hand side of the screen. Hovering 
 over the icons brings up a label, and the one you want is at (or near) the bottom with the label "Text editor". Click on it.
 <p>If you are lucky this will bring up an empty window for a new file. 
 <br>If you are unlucky it will bring up the previous person's file.
 <p>If it is a new file, save it to the Downloads folder (/home/expo/Downloads) using the "File->Save" menu 
 item and give it a sensible name such as "logbook-mynewtrip.txt".
 <p>If it was someone else's file, save it using the "File->Save" menu. Then close the text editor ("File->Close"). 
 Then start it up again from the vertical icon menu as before.
 <p>Now type in your trip report using whatever format you like, but please leave a blank line between paragraphs.
 <h3>Adding your trip to the logbook online file</h3>
 <p>If you are using the <em>expo laptop</em> just edit this file:
 <pre>
 /home/expoweb/years/2018/logbook.html
 </pre>
 copy the format you can see other people have used; 
 and other people will take care of synchronising it with the version control system.
 <p>
 <b>DO NOT</b> take a copy of the logbook.html file from the expo laptop, 
 copy it by email or USB stick to another laptop, edit it there and then copy it back. That will 
 <em>delete other people's work</em>.
 <p>If you are using your own laptop then you will need to either:
 <ul>
 <li>Just type up your trip as a separate file e.g. "logbook-mynewtrip.txt", or just write it in an email, and send it to someone nerdish, or
 <li><a href="onlinesystems.html#manual">install and learn how to use</a> the version control software. 
 And you will need to synchronise regularly (every day) to 
 ensure that the updates from all the people entering trip data are OK and don't get overwritten by ignorant use of this software.
 </ul>
 <p>Logbooks are typed up and kept in the [expoweb]/years/[nnnn]/ directory as 'logbook.html'.</p>
 <h3>Recommended procedure</a></h3>
 <p>
 Rather than editing logbook.html when you type up your trip, it is a much better 
 idea to type up your trip(s) in a separate file, e.g. "logbook-mynewtrip.txt", and store it in the same 
 place on the <em>expo laptop</em>, i.e.
 <pre>
 /home/expoweb/years/2018/
 </pre>
 <p>or email it to a nerd if you are sitting at a different laptop.
 <h3>Format of the online logbooks</a></h3>
 <p>Do whatever you like to try and represent the logbook in html. The only rigid structure is the markup to allow troggle to parse the files into 'trips':</p>
 <pre>
 &lt;div class="tripdate" id="t2007-07-12B"&gt;2007-07-12&lt;/div&gt;
 &lt;div class="trippeople"&gt;&lt;u&gt;Jenny Black&lt;/u&gt;, Olly Betts&lt;/div&gt;
 &lt;div class="triptitle"&gt;Top Camp - Setting up 76 bivi&lt;/div&gt;
 &lt;div class="timeug"&gt;T/U 10 mins&lt;/div&gt;
 </pre>
 <p>Note that the ID's must be unique, so are generated from 't' plus the trip date plus a,b,c etc. 
 when there is more than one trip on a day.</p>
 <p>T/U stands for "Time Underground" in hours (6 minutes would be "0.1 hours").
 <hr />
 <h3>Historical logbooks</h3>
 <p>Older logbooks (prior to 2007) were stored as logbook.txt with just a bit of consistent markup to allow troggle parsing.</p>
 <p>The formatting was largely freeform, with a bit of markup ('===' around header, bars separating date, <place> - <description>, and who) which allows the troggle import script to read it correctly. The underlines show who wrote the entry. There is also a format for time-underground info so it can be automagically tabulated.</p>
 <p>So the format should be:</p>
 <pre>
 ===2009-07-21|204 - Rigging entrance series| Becka Lawson, Emma Wilson, Jess Stirrups, Tony Rooke===
 &lt;Text of logbook entry&gt;
 T/U: Jess 1 hr, Emma 0.5 hr
 </pre>
 <hr />
 <ul id="links">
 <li><a href="index.htm">Expedition Handbook</a>
 </ul>
 </body>
 </html>
--- a/handbook/computing/nerd.html
+++ b/handbook/computing/nerd.html
@ -0,0 +1,32 @@
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
 <title>Handbook placeholder page</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook</h2>
 <h1>Placeholder</h1>
 <p>This is not the page you are looking for.
 <p>This will be replaced with the information you want as soon as someone gets around to writing it. Why not find out how to do this yourself ?
 <hr />
 <div id="menu">
 <ul id="links">
 <li><a href="index.htm">Expedition Handbook</a>
 <ul>
 	<li><a href="rig/rigit.html">Rigging guide</a></li>
 	<li><a href="survey/index.htm">Surveying guide</a></li>
 	<li><a href="look4.htm">Prospecting guide</a></li>
 	<li><a href="rescue.htm">Rescue guide</a></li>
 </ul></li>
 <li><a href="../infodx.htm">Main index</a></li>
 <li><a href="../index.htm">Expo Home</a></li>
 <li><a href="../../index.htm">CUCC Home</a></li>
 </ul>
 </div>
 </body>
 </html>
--- a/handbook/computing/onlinesystems.html
+++ b/handbook/computing/onlinesystems.html
@ -0,0 +1,147 @@
 <html>
 <head>
 <title>CUCC Expedition Handbook: Online system overview</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook - Online systems</h2>
 <h1>Expo Online Systems Overview</h1>
 <p>The online data  and web publishing system (i.e. "the website") is now large and complicated with a lot of aspects. 
 This handbook section contains info at various levels: 
 simple 'How to add stuff' information for the typical expoer, 
 more detailed info for cloning it onto your own machine for more significant edits, 
 and structural info on how it's all put together for people who want/need to change things. 
 <p>We have <a href="http://wookware.org/talks/expocomputer/#/">an Overview Presentation</a> (many parts out of date)
 on how the cave data, 
 handbook and public website are constructed and managed. 
 It contains material which will be merged into this online systems manual.
 <p>These pages listed below have been reviewed recently (2018), and a 
 fuller list of "How do I..." instruction pages are on <a href="index.htm">the handbook opening page</a>.
 <ul>
 <li><a href="uploading.html">Uploading your photos</a></li>
 <li><a href="logbooks.html">Uploading typed logbooks</a></li>
 <li><a href="gpxupload.html">Uploading GPS tracks</a></li>
 <li><a href="survey/newcave.html">Recording a new cave discovery</a></li>
 <li><a href="survey/status.html">Monitoring the status cave survey workflow during and after expo</a></li>
 </ul>
 <p>But the systems Manual is still being actively edited to extract and simplify documentation. At the moment 
 it is the only documentation we have for:
 <ul>
 <li><a href="manual.html#update">Manual: Creating a new 'year' in the system</a></li>
 <li><a href="manual.html#update">Manual: Updating the cave guidebook descriptions</a></li>
 <li><a href="manual.html#manual">Manual: Expo software and server maintenance manual</a></li>
 </ul>
 <h2><a id="update">Updating the online systems - overview</a></h2>
 <h3>Experts short cut</h3>
 <p>Short <a href="checkin.htm">command-line instructions</a> for updating the
 data on the server
 (using the <em>expo laptop</em>). This is a memory jog for experts, not beginners.</p>
 <h3>Autogenerated pages</h3>
 <p>Some key sections of the online webpages  are autogenerated by scripts or by
 <a href="#troggle">troggle</a>, and are not static files,
 so you have to edit the base data, not the generated file (e.g cave
 pages, QM (question mark) lists, expo members list, prospecting pages). All
 autogenerated files say 'This file is autogenerated - do not edit' at
 the top - so check for that before wasting time on changes that will
 just be overwritten</p>
 <h3 id="edithandbook">Editing this handbook and historic expo documentation</h3>
 <p>The primary and recommended way of editing this handbook (and the website generally) is to use 
 a laptop which has the <a href="#mercurial">Distributed Version Control System</a> software installed. The 
 person editing  needs to know how to use this software, and also needs to know how to edit raw HTML files
 using a text editor.
 <p>The <em>Expo laptop</em> has the software installed, so it is best to learn how to do this
 when sitting at that laptop.
 <p>It's important to understand that the pages you can edit by this method 
 are stored in a distributed version control system (see below). This stops us losing data and
 makes it very hard for you to screw anything up permanently, so don't
 worry about making changes - they can always be reverted if there is a
 problem. It also means that several people can work on the site on
 different computers at once and normally merge their changes
 easily.
 <p>The recommended editing workflow is to (a) use the DVCM to synchronise your local laptop copy of the 
 website files with that on the server; (b) edit a set of .html files on your laptop so that all links between them are consistent, 
 save the files locally, and "commit" them locally; 
 (c) "push" the collection of changes to the expo online server as a single action.
 <p>See the <a href="manual.html#manual">Expo data management systems manual</a> for a fuller description of the DVCM
 repositories and how to install and use the software.
 <h3 id="editthispage">Using "Edit this page"</h3>
 <p>You can update a single webpage
 online via a browser. This is best used for urgent edits to a single page, e.g. 
 if the emergency phone at top-camp has to use a new SIM with a different phone number. 
 If you are a logged-on user you will see "Edit this page" on the menu on the left of this page. It appears on
 nearly all pages in this website. If you click on it you will be able to edit the raw HTML of the page - so you need 
 to know how to do that.
 <p>After doing the page editing and saving your work, you need to ask a nerd to finish the process fairly soon as the "Edit this page"
 mechanism does not tidy-up after itself properly. 
 See <a href="manual.html#editthispage">these instructions for this tidy-up</a>
 <h3 id="mercurial">DVCS - version control</a></h3>
 <p>We use a distributed revision control system (DVCS) for all the important data.  
 This means that many people can edit and merge their changes with the expo 
 server in Cambridge at the same time: inlcuding people still on expo in the Tatty Hut 
 and those who have returned to the UK.  Also anyone who is up 
 to date can take their laptop somewhere and enter data even if they have no internet access, 
 and the updates will be merged when they get back to civilization.
 </p>
 <p>In principle, survey notes can be typed into a laptop up on the plateau which would 
 then get synchronised when it next gets internet access.
 </p>
 <p>A DVCS is inefficient for scanned survey notes, which are large files that 
 do not get modified, so they are kept as a plain directory of files 'expofiles'. 
 The same goes for holiday photographs and GPS logs.</p>
 <h3 id="troggle">Troggle - what it is</a></h3>
 <p>
 Troggle is the software collection (not really a "package") based on <a href="https://www.djangoproject.com/">Django</a>  
 originally intended to manage all expo data in a logical and accessible way 
 and publish it on the web.
 <p>Only a small part of troggle's original plan was fully implemented and deployed. 
 Many of the things it was intended to replace are still operating as a motley collection written by many different people in
 several languages (but mostly perl and python; we won't talk about the person who likes to use OCamL).
 Today troggle is used for only three things:
 <ol>
 <li>Reformatting all the visible webpages such that they have a coherent style and have a contents list at the top-left
 hand corner. This is particularly true of the handbook you are reading now and the historic records of past expeditions.
 <li>Publishing the "guidebook descriptions" of caves. The user who is creating a new guidebook description
 can do this by filling-in some online forms.
 <li>Providing a secondary way of editing individual pages of the handbook and historic records pages 
 for very quick and urgent changes. 
 This is the "Edit this page" capability; see <a href="#editthispage">above for 
 how to use it</a> and how to tidy up afterwards.
 </ol> 
 <p>See the <a href="troggle-ish.html">notes on troggle</a>  page 
 for how and why it was developed and what needs to be done.
 <hr />
 </body>
 </html>
--- a/handbook/computing/repousage.html
+++ b/handbook/computing/repousage.html
@ -0,0 +1,42 @@
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <title>CUCC Expedition Handbook: Uploading files/photos</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook</h2>
 <p>There are essentially two ways of using expo server: one is just FTP, second using version controll.
    <strong>If you can handle version controll use that!</strong>. Currently we're using HG wich is like GIT but a litte retarded.
 <h1>Using expo server via version controll software connection</h1>
 <h2>Configure on Windows</h2>
 <ul>
    <li>Get <a href="https://tortoisehg.bitbucket.io/">TortoiseHG</a> application to handle mercurial repo:<br>
        <img src="./tutorialimgs/tortoisehg.png" width="300"></li>
    <li>Install and open the app. The click 'clone':<br>
        <img src="./tutorialimgs/hg_win_clone.png" width ="300">
    </li>
    <li>Configure as:<br>
        <b>source:</b><i>ssh://expo@expo.survex.com/<b>expoweb</b></i> 
        (you can replace 'expoweb' with one of four repos we have on the server c.f. <a href="http://expo.survex.com/repositories">repositories</a>)<br>
        <b>destination:</b> pick location on the local machine<br>
        <img src="./tutorialimgs/hg_win_setup.png" width="300">
    </li>
    <li>Press 'clone'</li>
    <li>You should now have a copy of the repo on your machine. Use commands 'commit','pull','push' as in any version controll system. (This page is not meant as version controll tutorial, pls just websearch something)</li>
 <hr>
 <h2>Configure on Linux</h2>
 <ul>
    <li>Instal mercurial:<br>
    <pre>sudo apt-get install mercurial</pre></li>
    <li>Clone database:<br>
    <pre>hg clone ssh://expo@expo.survex.com/expoweb ~/localdirofyourchoice</pre></li>
    <li>You should now have a copy of the repo on your machine. Use commands 'commit','pull','push' as in any version controll system. (This page is not meant as version controll tutorial, pls just websearch something)</li>
 </ul>
 </body>
 </html>
--- a/handbook/computing/tutorialimgs/hg_win_clone.png
+++ b/handbook/computing/tutorialimgs/hg_win_clone.png
--- a/handbook/computing/tutorialimgs/hg_win_setup.png
+++ b/handbook/computing/tutorialimgs/hg_win_setup.png
--- a/handbook/computing/tutorialimgs/tortoisehg.png
+++ b/handbook/computing/tutorialimgs/tortoisehg.png
--- a/handbook/computing/uploading.html
+++ b/handbook/computing/uploading.html
@ -0,0 +1,121 @@
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <title>CUCC Expedition Handbook: Uploading files/photos</title>
 <link rel="stylesheet" type="text/css" href="../css/main2.css" />
 </head>
 <body>
 <h2 id="tophead">CUCC Expedition Handbook</h2>
 <h1>Uploading Photos</h1>
 <h2>The end-result you are trying to achieve</h2>
 What you are trying to do is to get your happy holiday snaps appear properly indexed with all the others from the previous decades of expo history. You can see them all here: <br /> <a
 href="http://expo.survex.com/photos/">http://expo.survex.com/photos/</a><br />
 which is the end result. But all you have to do is to upload the photos to the right place. 
 A hidden script does the hard work to make it all look nice.
 <p>If you are really lazy (or really a beginner) you can use the initial simple method (using /uploads/ )
 for the photos  you have taken of cave entrances for cave survey and prospecting purposes. But please
 rename the filenames of the photos intelligently, e.g. "big-hole-near-path-to-fgh.jpg", or
 "2018-ad-07-entrance3.jpg" (rather than "DSC31415926.jpg"), and explain to an admin/nerd what you have done. 
 Please use lower-case for all filenames.
 <p>If you are looking for how to upload a GPS track, those instructions have <a href="gpxupload.html">been moved to here</a>.
 <h2>Simple instructions</h2>
 <ol>
 <li>Email a photo or two to someone who knows how to do it. <br>
 (If you are doing more than a few photos, email will be clunky, so use another method).
 <li>Use the Expo laptop in the tatty hut. Get someone to show you how to do it.
 </ol>
 <p>That's it. There used to be other ways of doing it using browser extensions but these either don't work anymore [since 2017] or the instructions to install them properly have become too complicated.
 <p>Beginners should always put all their files into the folder <b>/home/expo/expofiles/uploads/</b> and ask an admin to move them to the right place.
 <p>Now go to <a href="#init">using Filezilla initially</a>  - still using the Expo laptop.
 <h2 id="morecomplex">More complex instructions</h2>
 <p>Using your own laptop on expo, or after you return from expo:
 <ol>
 <li>Download and install <b>Filezilla</b>.
 </ol>
 <p>You do need to know the expo password.</p>
 <h3 id="install">Installing Filezilla</h3>
 <p>This software works identically on both Windows and Linux.
 <p>Filezilla is an "FTP client". This means that it connects to servers using a venerable service called "file transfer protocol" i.e. FTP. It looks a bit like copying files from one folder to another on your desktop but it works between different machines.
 <ul>
 <li> Download the software from here <a href="https://filezilla-project.org/download.php?show_all=1">Filezilla Downloads</a>. ( Obviously Linux users will use their usual package management system instead of doing this download.)
 <li>Now install the software following <a href="https://wiki.filezilla-project.org/Client_Installation">the instructions here</a>. 
 <li>Now configure it to connect to the expo server using the instructions <a href="fzconfig.html">on this expo handbook page</a>
 </ul>
 <h2 id="init">Using Filezilla initially</h2>
 <p>The expo website has a big section under 'home/expo/expofiles/' that is <b>not under version control</b>. This is dangerous as there is no backup. If you overwrite some important files with your holiday snaps then we are in big trouble. This is where we store big files that we don't want to keep multiple versions of which is why it is not under verson control.
 <p>So beginners should always put all their files into the folder 
 <pre>/home/expo/expofiles/uploads/</pre> and then ask an admin to move them to the right place. The configuration which you just did (if you followed <a href="fzconfig.html">the instructions</a>) will set you up pointing at the correct folder automatically.
 <p>
 To make the admin's life easier, create your own folder in <pre>/home/expo/expofiles/uploads/</pre> with your name like this: <pre>/home/expo/expofiles/uploads/YourName/</pre> and put your files into that folder. ("YourName" should have no spaces, hyphens or underline characters.) So that the admin knpows who is responsible. And for goodness sake please don't upload lots of duplicate photos: cull them first to just the good ones.
 <h2 id="experienced">Experienced users</h2>
 <p>OK the admin/nerd you have been asking to move the files to the right place is getting fed up and now wants you to put the uploaded photos in the right place yourself. This is where they go:
 <pre>
 directory: /home/expo/expofiles/<b>photos</b>/2018/YourName/
 </pre>
 or, for GPS logs (GPX files):
 <pre>
 directory: /home/expo/expofiles/<b>gpslogs</b>/2018/YourName/
 </pre>
 <p>Obviously replace 'YourName' with your actual name (no spaces!). 
 It is important that you get this right as this specific way of writing 
 your name is standardised across the website 
 (this is known as "<a href="https://en.wikipedia.org/wiki/Camel_case">CamelCase</a>"). 
 <p style="margin-left:20px">This is the only place in the whole system
 that you should use some capital letters. Absolutely everywhere else you should use
 only lower-case letters, no spaces, and hyphens (not underlines) when creating filenames, folder names or variable names of any sort.
 <p>
 Use the year that the photo was taken or the GPStrack logged.</p>
 <p>Note that uploading photos does not automatically update the view
 at <a href="http://expo.survex.com/photos/">http://expo.survex.com/photos/</a> immediately. An update script needs to be run. This should run automatically once/day around midnight UTC (2017 and earlier) or a couple of minutes after you do the upload to the right place (2018 if Wookey gets this sorted out in time) but may be broken. Prod a web admin if nothing is updated by the next morning..</p>
 <h2>Experts only</h2>
 <p>If you have been using FTP since the last century or are particularly keen on doing everything using the command-line, read on.
 <h3>Using scp</h3>
 <p>Works on Windows (using winscp), Linux (using scp), and no doubt
 mac and android with other tools. If you have Windows 10 and <a href="https://msdn.microsoft.com/en-gb/commandline/wsl/about">have installed bash</a>, then you can use scp.</p>
 <p>If you don't have winscp installed you can get it from here:
 <a href="http://winscp.net/eng/index.php">http://winscp.net/eng/</a>.</p>
 <p>quick start guide:
 <a href="http://winscp.net/eng/docs/getting_started">http://winscp.net/eng/docs/getting_started</a></p>
 <p>screenshots:
 <a href="http://winscp.net/eng/docs/screenshots">http://winscp.net/eng/docs/screenshots</a></p>
 <!-- WedDAV no longer works because the effective user is "apache" and the permissions
 for the folder /uploads/ no longer allow it to write anything because we had to change it to allow Filezilla 
 to work.Philip & Wookey 11 August 2018
 <h3>Using WebDAV - cadaver</h3>
 <p>Command-line people can use the 'cadaver' client which is even
 available for windows too:
 <a href="http://www.phtagr.org/2009/04/01/cadaver-for-windows/">www.phtagr.org/2009/04/01/cadaver-for-windows/</a></p>
 -->
 <p>scp gives you an 'explorer-like' interface (although winscp can
 give you a norton-commander-style 2-pane UI as well).</p>
 <h3>Using WebDAV</h3>
 This no longer works as we had to change the folder permissions for /uploads/. Sorry.
 <h3>Using rsync</h3>
 <p>No, don't use rsync for this. Really don't. It's too liable to delete everything or to overwrite files which are not changed at all because of the incompatibilities between Linux and Windows filename conventions (uppercase and lowercase are automagically converted and rsync gets it wrong).
 <hr />
 </body>
 </html>