expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2025-12-09 07:14:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

Basic/Survey/Bulk laptops

Browse Source
This commit is contained in:
Philip Sargent
2022-11-26 01:08:07 +00:00
parent b4353d2c0b
commit 98546eb0a3
11 changed files with 396 additions and 312 deletions
Download Patch File Download Diff File Expand all files Collapse all files

121
handbook/computing/basiclaptop.html
View File

@@ -11,15 +11,45 @@
<h1>Setting up a basic Expo laptop</h1>
<h2 id="any">What you can do from any laptop</h2>
<p>There is a lot you can do without installing any software on your own machine. Using a browser, you can logon to the Expo online system ("the website", also known as "troggle") as user 'expo' at <a href="/accounts/login/">the Troggle User Login page</a>. (Ask another expoer for the 'cavey:beery' password.) You can:
<p>There is a lot you can do without installing any software on your own machine. Anything with a web browser is an 'Expo Basic laptop'.
Using the browser, you can logon to the Expo online system ("the website", also known as "troggle") as user 'expo' at <a href="/accounts/login/">the Troggle User Login page</a>. (Ask another expoer for the 'cavey:beery' password.) You can:
<ul>
<li>Edit any HTML page in the online handbook: correct errors, update phone numbers
<li>Edit existing Survex files in our complete Loser cave data repository
<li>Create new Survex files - a template is provided - for new cave passages
<li>Edit the cave description and entrance description text for existing caves
<li>Create entirely new caves in the system by filling out online forms
<li>Create entirely new caves in the system by filling out online forms<br>&nbsp;
<li>Upload many photograph images at once
<li>Upload a single photo and rename it as you do so
</ul>
<p>Documentation on how to actually do these things are in the <a href="manual.html">data maintenance manual</a>.
<p>Documentation on how to actually do these things are in the <a href="manual.html">data maintenance manual</a>.
<br>Or use the "Search" box below the menu on the left-hand side of the page.
<br>Or go to any of the troggle report pages and explore the menu of items across the top of the page:
<code style="font-size: 10">
<a href="/survexfile/264">264</a> |
<a href="/survexfile/290">290</a> |
<a href="/survexfile/359">359</a> |
<a href="/survexfile/">Survex</a> |
<a href="/survexfile/caves/">All Survex</a> |
<a href="/survey_scans/">Scans</a> |
<a href="/scanupload/2022:01">Upload Scans</a> |
<a href="/dwgfiles/">Drawings</a> |
<a href="/dwgupload/">Upload Drawings</a> |
<a href="/photoupload/">Upload Photos</a> |
<a href="/1623/290/290">290 (FGH)</a> |
<a href="/1623/264/264.html">264 (Balkonh&ouml;hle)</a> |
<a href="/dataissues">Data Issues</a> |
<a href="/handbook/computing/todo-data.html">tasks to do </a> |
<a id="cavesLink" href="/indxal.htm">caves</a> |
<a id="qmsLink" href="/cave/qms/1623-290">QMs</a> |
<a id="entsLink" href="/eastings">ents</a> |
<a id="folklink" href="/folk">expoers</a> |
<a id="caversLink" href="/people">survey lengths</a> |
<a href="/stats">statistics</a> |
<a href="/wallets/year/2022">Wallets(2022)</a> |
<a href="/expedition/2022">Expo(2022)</a> |
</code>
<p>And using email to send the results to an expo nerd, you can:
<ul>
<li>Type up your logbook entry for any trip you do (surface or underground), but please use <a href="../logbooks.html">our standard format</a>
@@ -31,9 +61,7 @@
</ul>
<p>and of course using your phone or laptop you can update entries on expo antics on public forums such as ukcaving.
<p>If you also have Survex and Therion installed on the laptop, you can do nearly everything for initial cave survey data entry. See the
<a href="getsurvex.html">Survex, Tunnel and Therion</a> installation instructions and the <a href="surveylaptop.html">expo survey laptop</a> installation instructions.
(These will be moved to a different page in the handbook soon).
<p>If you also have Survex and Therion installed on the laptop (which makes it an "Expo Survey Laptop"), you can do nearly everything for initial cave survey data entry. See the <a href="surveylaptop.html">expo survey laptop</a> installation instructions.
<p>We are actively working on increasing the number of expo activities that can be done with just a browser and no, or minimal, installed software.
@@ -43,83 +71,36 @@ browser. Many of these capabilities are not new, but they weren't documented and
href="manual.html">these capabilities are documented</a>, though writing better documentation is an unending job, and we have a <a
href="manual.html">data maintenance manual</a>.
<ul>
<li><a href="#any">Any laptop</a> with a broswer and email.
<li><a href="#any">A Survey laptop</a>: a basic laptop with Survex and Therion installed - for cave surveying data entry.
<li>An <a href="#basic">expo basic laptop</a>
<li>A <a href="winlaptop.html">Windows expo basic laptop</a>
<li>A full <a href="surveylaptop.html">expo survey computer</a> and Android phone config
<li>A <a href="#any">Basic Laptop</a> any laptop with a broswer and email.
<li>A <a href="surveylaptop.html">Survey laptop</a>: a basic laptop with Survex and Tunnel/Therion installed - for cave surveying data entry.
<li>An <a href="bulkupdatelaptop.html">expo Bulk Data laptop</a>
<li>A <a href="winlaptop.html">Windows expo Bulk Data laptop</a>
<li>An <a href="../troggle/troglaptop.html">expo software development computer</a> for troggle programming
</ul>
<p>See <a href="../troggle/trognotes.html#devtree">the expertise sequence</a> which lists what you can do at each stage.
<h2 id="basic">Your own basic laptop</h2>
<p>If you are new to expo and can't do what you want with just a browser and email, then please use the <em>expo laptop</em> in the potato hut first. You don't <em>need</em> to use your own laptop - which can take several hours to configure completely.
<p>To set up your own basic laptop for bulk cave data maintenance you need to do this:</p>
<ol>
<li>Register an SSH key</a> with an expo nerd i.e 'get a login'. (see "Key Configuration" below)</li>
<li>Install <a href="#software">git version control software</a> to download ("clone"), view and edit caving data.</li>
<li>Clone two <a href="../computing/repos.html">expo repositories</a> <var>loser</var> and <var>drawings</var> so you have the files on your machine. (Use the <a href="qstart-git.html">git reminder</a> for how to do this, e.g. <em>git clone ssh://expo@expo.survex.com:/home/expo/expoweb</em> .</li>
<li>Install survex, and therion or tunnel for editing cave data.
<li>Install image editing software such as Irfanview or gimp.
<li>If you are also planning on extensive work rewriting parts of the handbook, then you will also need the <a href="../computing/repos.html">expo repository</a> <var>expoweb</var>.
</ol>
<p>The <em>expo laptop</em> is a basic laptop configuration. It has everything for editing and testing survey files (survex, aven, cavern), drawings (tunnel, therion), scanned images of sketches and centre-lines, and photographs. The <em>expo laptop</em> in the potato hut is also physically connected to a flatbed scanner but you can use your phone camera instead and email the images to yourself on your laptop.
<h2>Your own Survey laptop</h2>
<p>If you are new to expo and can't do what you want with just a browser and email, then please use the <em>expo laptop</em> in the potato hut first. You don't <em>need</em> to use your own laptop - which can take a while to configure with survex, tunnel and therion.
<p>You may also need to install image editing software such as Irfanview or gimp.
<h3>The expo laptop</h3>
<p>The <em>expo laptop</em> is a bulk update laptop configuration. It has everything for editing and testing survey files (survex, aven, cavern), drawings (tunnel, therion), scanned images of sketches and centre-lines, and photographs. The <em>expo laptop</em> in the potato hut is also physically connected to a flatbed scanner but you can use your phone camera instead and email the images to yourself on your laptop.
<p>The <em>expo laptop</em> may also have some software for managing vector images (such as rigging guides), <a href="https://paperless.bheeb.ch/">PocketTopo</a> files, <a href="https://en.wikipedia.org/wiki/Geographic_information_system">GIS digital maps</a> and <a href="https://activityworkshop.net/software/gpsprune/development.html">GPS tracks</a>. See the <a href="surveylaptop.html">full survey laptop configuration</a> for details.
<p>Managing large sets of photographs and scanned images, and managing several folders of these on your laptop and on <var>expofiles</var> on the server is finicky and time-consuming. Many programmers use rsync to help them with this, but if you have never used rsync, now is <em>not</em> the time to learn. Use filezilla and FTP. It is at this point that if you are using a Windows machine, you <em>really</em> need to <a href="winlaptop.html#problems">read about how expo uses hard and soft links and filenames on Windows</a>. If things get screwed up badly, it will need someone on a Linux machine to sort it out.
<h4>Bulk Updates</h4>
<p>Managing large sets of photographs and scanned images, and managing several folders of these on your laptop and on <var>expofiles</var> on the server is finicky and time-consuming. Many programmers use rsync to help them with this, but if you have never used rsync, now is <em>not</em> the time to learn. Use filezilla on the expo laptop. It is at this point that if you are using a Windows machine, you <em>really</em> need to <a href="winlaptop.html#problems">read about how expo uses hard and soft links and filenames on Windows</a>. If things get screwed up badly, it will need someone on a Linux machine to sort it out.
<p>Once you have got all this working, and if it doesn't do what you want or you don't understand how to use it,
look at the <a href="surveylaptop.html">full survey laptop configuration</a> for everything else.
And please <b>write some documentation for the next person</b> in your situation.
<h3>Cheat lists and quick reminders</h3>
<ul>
<li>Quick <a href="qstart-git.html">reminders for using git</a> at the command line.
<li>Quick <a href="qstart-rsync.html">reminders for using rsync</a> at the command line.
</ul>
<h2 id="software">Software</h2>
<p>If you are just typing up logbook entries then you don't need any other software. If you are working with survey data download this software (short list):
<ul>
<li><a href="fzconfig.html">Filezilla</a> - ftp GUI software with a configuration file to get to the expo server</li>
<li><a href="https://git-scm.com/">git</a> - version control system</li>
<li><a href="getsurvex.html">Installing surveying software</a> - survex, tunnel, therion - specifically for expo use
<li><a href="https://survex.com/download.html">Survex</a>, including the Aven visualisation tool.
<li><a href="https://github.com/CaveSurveying/tunnelx">Tunnel</a>: 2.5D cave drawing program based on Survex-compatible data which can also read PocketTopo files. (Generally called 'tunnel' even though the project and executable is actually 'tunnelx'.)
<li><a href="https://therion.speleo.sk/">Therion</a> - Therion processes survey data and generates maps or 3D models of caves.
</ul>
<h2 id="configuration">Configuration</h2>
<p>Follow this link to <a href="keyexchange.html">register a key with the expo server</a> to get upload (i.e. read/write) access.
Do this first, Without it none of git, scp, ftp or rsync will work.
<p>On a Windows machine you will need to configure pageant (the putty authentication agent)
to <a href="https://blog.shvetsov.com/2010/03/making-pageant-automatically-load-keys.html">run at startup to load your key</a>.
Note that you are loading your <em>private</em> key, the .ppk file, into pageant and that this key never leaves your laptop.</p>
<ul>
<li><a href="../putty/putty.html">Installing PuTTy</a>.</li>
<li><a href="winlaptop.html#hard">A Windows laptop for expo</a> - PuTTy and WSL here.</li>
</ul>
<p>When using Windows please, please be <a href="http://expo.survex.com/handbook/survey/getin.htm#filenames">excessively
careful when naming files and survex names</a> and be <a href="manual.html#quickstart">exceptionally careful when using rsync</a>.
<h3>Learning how to use this software</h3>
<ul>
<li>Using Survex, Tunnel and Therion, see the <a href="http://expo.survex.com/handbook/survey/">Expo Handbook - Surveying section</a>.
<li>For more detailed configuration, explanations and help see the <a href="surveylaptop.html#configuration">survey laptop</a> instructions
</ul>
<h3>FTP</h3>
<p>It is necessary to use scp or sftp to manage large collections of files in 'expofiles'
<p>It is necessary to use Filezilla scp or sftp to manage large collections of files in 'expofiles'. Before you attempt this, <b>talk
to a more experienced expoer</b>.
<p>
See <a href="upload-expert.html">Experts: Uploading files</a>,
<a href="uploading.html">Uploading files</a> and <a href="gpxupload.html">Uploading GPS tracks</a>.
Only machines which have done the key-pair setup process can do scp, sftp or rsync.
Only Bulk Data machines which have done the key-pair setup process can do scp, sftp or rsync or run Filezilla.
</p>

134
handbook/computing/bulkupdatelaptop.html Normal file
View File

@@ -0,0 +1,134 @@
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>CUCC Expedition Handbook: Bulk Update laptop</title>
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
</head>
<body>
<h2 id="tophead">CUCC Expedition Handbook - Bulk Update laptop</h2>
<h1>Cave data - Bulk Updates</h1>
<p>You will already have configured your laptop to do all the basic stuff using the
<a href="basiclaptop.html">basic Expo laptop</a> guide,
and you probably will also have installed the survey tools survex, tunnel, therion etc.
in configuring a <a href="surveylaptop.html">survey laptop</a>.
<p><code>This is an attempt at describing how to configure your laptop to do bulk file transfers to and from the server. If you are only managing one or two files at a time, you don't need this.
<p>If you want to do software development instead, go to <a href="../troggle/troglaptop.html">configuring a troggle development machine</a>.
</code></p>
<h2>Windows and Linux</h2>
<ul>
<li>You should be able to use any OS (Linux, Windows, Mac), but it's a bit simpler to get set up using Linux.
<li>Windows machines can do almost everything needed directly in Windows itself, but some useful software has no Windows version and you will
need to find your own equivalents. There are also some 'gotchas' to look out for due to filesystem differences (e.g letter case). If you have
the choice, use Linux. Windows/Linux WSL is fine.</li>
<li><a href="winlaptop.html">A Windows laptop for expo</a> - a page of more detailed instructions for those who are definitely going to be
using a Windows PC or laptop.</li>
<li>The <i>expo laptop</i> uses <a href="https://www.debian.org/intro/about">Debian</a> with the <a
href="https://en.wikipedia.org/wiki/Cinnamon_(desktop_environment)">Cinnamon</a> interface, but pretty much any Linux
system works fine. Old, slow machines without much memory can be very effective with <a href="https://xubuntu.org/">Xubuntu/xfce</a>.
</ul>
<h2>Chromebooks and Macs</h2>
<ul>
<li>If using a Mac or Red-Hat-based linux you'll need to work it all out for yourself.</li>
<li>You can't
yet do everything on a phone, but you can do some things. See <a href="phone.html">Android phone page</a>.</li>
<li>Chromebooks work just like a phone, but with a keyboard, so far as Expo is concerned (mostly). So without some tricky configuration you won't be able to do bulk updates. See our <a href="chromebook.html">chromebooks</a> page.
</ul>
</ul>
<h2><a name="software">Software</a></h2>
<p>You need three kinds of software
<ol>
<li>Encrypted keys to access the server (ssh). None of the rest works without this.
<li>Version control software (<a href="https://git-scm.com/">git</a>) for the loser survex data files, drawings (tunnel, therion) files, website and handbook (expoweb) files.
<li>File-transfer tools scp, rsync and sftp (<a href="fzconfig.html">Filezilla</a>) for the files not under version control: photos, documents, scanned logbooks, scanned
raw survey notes (surveyscans), scanned cave centre-line plots and everything else. [Never use these on the version-controlled folders.]
</ol>
<p>If you are only managing photos, GPS tracks or documents then you don't need the version control stuff.
<p>If you are only managing survey data then you probably don't need the file-transfer stuff.
<h4>QMs and scripts</h4>
<p>You may not need a full troggle software development machine if you are only fixing a small script. If you are, you need
<ul>
<li>python - maybe - but only if you are working on the <a href="../troggle/scriptsother.html">scripts in expoweb/noinfo/</a> such as <var>make-folklist.py</var> or some of the <a href="../troggle/scriptsqms.html">standalone QM scripts</a> - but not on troggle itself - then please use the as close to the same setup as you can to <a href="../troggle/troglaptop.html">the setup we use for troggle</a>. <img src="https://www.python.org/favicon.ico" width=64 hspace="20" align="right">
</ul>
<h3>Your own Bulk Update laptop</h3>
<p>To set up your own laptop for bulk cave data maintenance you need to do this:</p>
<ol>
<li>Register an SSH key</a> with an expo nerd i.e 'get a login'. (see "Key Configuration" below)</li>
<li>Install <a href="#software">git version control software</a> to download ("clone"), view and edit caving data.</li>
<li>Clone two <a href="../computing/repos.html">expo repositories</a> <var>loser</var> and <var>drawings</var> so you have the files on your machine. (Use the <a href="qstart-git.html">git reminder</a> for how to do this, e.g. <em>git clone ssh://expo@expo.survex.com:/home/expo/expoweb</em> .</li>
<li>If you are also planning on extensive work rewriting parts of the handbook, then you will also need the <a href="../computing/repos.html">expo repository</a> <var>expoweb</var>.
</ol>
<h4>For Linux users only:</h4>
<ul>
<li><a href="https://git-scm.com/download/gui/linux">GUI for git</a> - There is a choice on Linux but many people just use the command line.
<li>Your favourite text editor. gedit is installed on the <em>expo laptop</em> and has syntax highlighting for .svx files too. This works with <a href="https://survex.com/changes.html">all GtkSourceView-based editors</a> (thanks to Phil Withnall). Download a local copy of the syntax colourizer <a download href="/site_media/survex.lang">survex.lang</a>.
</ul>
<p>Note that on a Debian or Ubuntu machine you should normally install the versions that come with the specific distro (e.g. v11 Bulleye for Debian, or Jammy Jellyfish 22.04 for Ubuntu) i.e. install using 'sudo apt install xxx', not by downloading things from the above sites. So installing everything you need should be as simple as:
<ul>
<li><tt>apt install openssh-client survex-aven tunnelx therion git sftp</tt></li>
<li><tt>ssh-keygen</tt> (to generate a key if you haven't already)</li>
<li><tt>apt install gpsprune qgis firefox-esr gedit tig gitg meld rsync</tt> (you may prefer other tools - e.g: elpa-magit is amazing but only if you know how to drive emacs)</li>
</ul>
</p>
<h4 id="win">For Windows users only:</h4>
<p></p>None of this works until you set up the key-pair setup using PuTty/Pageant.
<p><img src="https://wiki.filezilla-project.org/favicon.ico" width=64 style="float:right; margin: 10px">To install and configure Filezilla on your machine see <a href="fzconfig.html">FileZilla install instructions</a> which will set you up pointing at the correct folder automatically.
<p>Read the <a href="winlaptop.html">Detailed Windows Configuration Instructions</a> for foconfiguring a Windows Bulk Update machine.
<h2 id="git">Configuring git</h2>
<p>On a new machine you need to configure your git identity:
<pre><code>git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git config --global pull.rebase true
</pre></code>
<h3>Cheat lists and quick reminders</h3>
<ul>
<li>Quick <a href="qstart-git.html">reminders for using git</a> at the command line.
<li>Quick <a href="qstart-rsync.html">reminders for using rsync</a> at the command line.
</ul>
<h3>Complementary tools</h3>
<p>When maintaining the HTML files in the expo handbook (the :expoweb: git repository) a link-checker is useful to report bad URLs (links to external sites go bad regularly) and to find orphaned pages with no in-links. The website has <a href="https://www.klebos.net/subdomains/keldos/LinkScans/TUNNEL-wiki-link-report.html">about 2,000 internal URLs</a> in just the Tunnel wiki section alone.
<ul>
<li><a href="https://www.brokenlinkcheck.com/">Online brokenlinkcheck tool</a> - does not install on your laptop. Limited to 3,000 pages.
<li><a href="https://www.deadlinkchecker.com/website-dead-link-checker.asp">Another online brokenlinkcheck tool</a> - does not install on your laptop. Limited to 2,000 pages but breadth-first rather than depth-first, so it covers a different chunk of the expo website.
<li><a href="https://www.drlinkcheck.com/account/subscriptions/1/projects/1/overview">Yet another link-checker</a>, only 1,500 pages checked but oh-so-stylish results presentation. Very informative.
<li><a href="http://home.snafu.de/tilman/xenulink.html">Xenu Link Sleuth</a> - install on Windows to do comprehensive link check.
<li><a href="https://www.digitalocean.com/community/tutorials/how-to-find-broken-links-on-your-website-using-wget-on-debian-7">Linkchecking with wget</a> - command line methods on Linux
</ul>
<hr />
Go back to: <a href="basiclaptop.html">Basic laptop</a><br />
Go back to: <a href="surveylaptop.html">Survey laptop</a><br />
Go on to: <a href="winlaptop.html">Windows Bulk Update laptop</a><br />
<hr /></body>
</html>

11
handbook/computing/contribute.html
View File

@@ -136,12 +136,15 @@ of the things we are already thinking about. And before you do anything, please
a page or two on what you think would be a good idea (using the handbook editing
<a href="hbmanual1.html">"Edit this page"</a> capability to
add a new design document).
<h3>Your laptop</h3>
<p>
Before you do anything substantial you will probably need to
<a href="basiclaptop.html">set up a basic expo machine</a> and
then <a href="surveylaptop.html">a survey data management configuration</a>. This
can take several hours, but you don't need to install everything before
Your own laptop + web-browser is already a <a href="basiclaptop.html">basic expo machine</a>.
You will need to install cave survey software to make it a
then <a href="surveylaptop.html">a survey laptop</a>. This
can take an hour or so, but you don't need to install everything before
you can contribute to fixing things in the <a href="todo.html">Expo <em>Systems</em> To-Do List</a>.
Once you have got a bit of experience with the how expo works, you can upgrade your laptop
to a <a href="bulupdatelaptop.html">bulk update machine</a>.
If you really want to get stuck into the code you will need a <a href="../troggle/troglaptop.html">troggle software development configuration</a>.
<p>Finally, if that all sounds like too much hassle for you, were are in desparate need

30
handbook/computing/keyexchange.html
View File

@@ -10,7 +10,7 @@
<h1>Key-Pair Setup</h1>
<p><var>You only need to do this if you need to do bulk updates of a lot of files to the server.</var> <br> - Copying files <em>from</em> the server does not need this.<br>
- Only people updating or rearranging files on the server need this, e.g.
- Only people updating or rearranging files on the server with a <a href="bulkupdatelaptop.html">Bulk Update Laptop</a>, or programmers doing <a href="../troggle/troglaptop.html">troggle software development</a> need this, e.g.
<ul>
<li>Editing lots of survex files in the Loser repository
<li>Moving photographs or Drawing files to the final archive locations
@@ -125,12 +125,13 @@ documentation</a>, start from scratch, and do it all again but this time using a
<h3>Chromebook</h3>
<li>See initial instructions in <a href="basiclaptop.html#chromebook">Chromebooks</a> then follow the instructions for Linux here.</li>
<li>See initial instructions in <a href="chromebook.html">Chromebooks</a> then follow the instructions for Linux here.</li>
<h3>Android</h3>
<ul>
<li>Install the <a href="https://play.google.com/store/apps/details?id=com.server.auditor.ssh.client&hl=en">Termius app</a>
which is a ssh client. The free version does ssh but not sFTP. Follow the <a href="https://docs.termius.com/">Termius documentation</a> and in-app help to generate a key pair and then use the same process to upload the public key to the expo server as for Linux machines, i.e. email it to an admin.
<li>See the expo page on <a href="phone.html">phones</a>.
</li>
</ul>
<h3>iOS</h3>
@@ -141,7 +142,7 @@ documentation</a>, start from scratch, and do it all again but this time using a
<h2>Your second machine</h2>
<h2 id="secondmachine">Your second machine</h2>
<p>OK, you have an uploaded and usable key and you can ssh into the expo server. If you want to connect from another machine (e.g. your phone) it's better to make another key than try to re-use the first one. You don't need a nerd admin now, you can do this yourself. But be <b>extremely careful</b> to follow this exactly. If you innocently rename the files to something that appears more sensible it won't work.
<ol>
<li>On your new machine, generate a key-pair. Since you are probably using a different operating system on your second machine, read the instructions above for the relevant OS. Yes you will be generating a new key. Do not re-use the key you had already.
@@ -151,7 +152,22 @@ documentation</a>, start from scratch, and do it all again but this time using a
<li>
If you accepted the defaults, the public key will be called <var>~/.ssh/id_rsa.pub</var> - check that this is the case.
<li>Now copy the public key file to your <em>first machine</em>, the one that is already set up with a working key-pair setup with the expo server. Using email to yourself is easiest.
<li>Now upload the public key using sFTP (Filezilla configured to use sFTP, which uses Pageant by default) to
<li>Now upload the public key. There are 2 ways to do it:
<ol>
<li>Using ssh-copy-id
<li>Using a complex collection of file transfers and file copying and appending
</ol>
</ul>
<h4>Using ssh-copy-id</h4>
<ul>
<li>On the first machine which already has a key installed, download the second key from email and rename it as, e.g. <var>pulsifer_id_rsa.pub</var>
<li>do this command:<br>
<code>ssh-copy-id -i pulsifer_id_rsa.pub expo@expo.survex.com</code>
<li>That's it.
</ul>
<h4>Using Filezilla and file copying</h4>
<ul>
<li>using sFTP (Filezilla configured to use sFTP, which uses Pageant by default) to
<var>expo.survex.com/home/expo/.ssh/keys/</var>
<li>Now login to the expo server on your first machine and do these commands:<br />
Be <b>extremely careful</b> to type ">>" and not ">" in the fourth line below otherwise you will delete <i>everyone's</i> logins.
@@ -162,13 +178,15 @@ Be <b>extremely careful</b> to type ">>" and not ">" in the fourth line below ot
<br /><font color=blue">expo@expo:~/.ssh$</font> tail -n 1 authorized_keys
</code>
This adds your key on to the end of the authorized keys list and prints the last line - which should be your key that you just added.
<p>Note that by using sFTP like this we avoid having to use a text editor over ssh. If you know what you are doing you can do this of course, but the above process is less likely to cause problems for a Windows user setting up their phone as a second device where they are not experienced with vi or nano.
</ul>
<li>Now your public key is installed for your second machine. This will enable ssh login instantly. You check that it works by logging into the expo server using ssh from your second machine: <var>ssh expo@expo.survex.com </var>.
</p>
<p>Note that by using sFTP like this we avoid having to use a text editor over ssh. If you know what you are doing you can do this of course, but the above process is less likely to cause problems for a Windows user setting up their phone as a second device where they are not experienced with vi or nano.
</ol>
<hr />
Return to <a href="basiclaptop.html">Setting up a basic laptop</a></body>
Return to <a href="bulkupdatelaptop.html">Setting up a bulk update laptop</a></body>
</html>

44
handbook/computing/manual.html
View File

@@ -25,7 +25,9 @@
'<a href="../survey/newcave.html">complete process for recording cave data</a>',
possibly in one of the side-pages which you skipped through when you read it.
<p>You already have the <a href="#password">password</a>, so just using troggle web pages you can already
<h3>With a Basic Laptop</h3>
<p>You already have the <a href="#password">password</a>, so just using troggle web pages and a
<a href="basiclaptop.html">basic laptop</a> you can already do these things which have been "trogglized":
<ul>
<li>Edit any existing survex files, save them, and run survex on them <em>on the server</em>
@@ -40,6 +42,30 @@ possibly in one of the side-pages which you skipped through when you read it.
<li>Edit any HTML page in the online handbook: correct errors, update phone numbers
<li>Edit the HTML of any online expo logbook
</ul>
<p>See <a href="basiclaptop.html">configuring a Basic laptop</a>.
<h3>With a Survey Laptop</h3>
<p>A survey laptop has Tunnel and Therion, and you use it for cave survey production from raw data. Generally it is not what you use to fix things that have gone wrong - unless what is wrong is simply that something is missing because a job has not been done.
<p>See <a href="surveylaptop.html">configuring a Survey laptop</a>.
<h3>With a Bulk Update Laptop</h3>
<p>There are several standard data maintenance jobs that haven't yet got trogglized. So if:
<ul>
<li>We get a new Austrian kataster number for a cave, so we need to move and rename a lot of files
<li>A cave has got so big that we need to create sudirectories for the long and complicated cave descriptions in separate HTML files
<li>Some scanned notes have been uploaded into the wrong online wallet
</ul>
<p>See <a href="bulkupdatelaptop.html">configuring a Bulk Update laptop</a>.
we will need to use the things* installed on the <em>expo laptop</em> or on <a href="basiclaptop.html">your own laptop</a> if you are not in the potato hut on expo. <a href="fzconfig.html">Filezilla</a> will do most of what you need for moving files.
We also have <a href="uploading.html#android">andftp Android instructions</a> for file manipulation.
<p>While Filezilla (including the digital key) is enough for moving files in <a href="/expofiles">expofiles</a>, moving files in the version-controlled <a href="repos.html">repositories</a> means you need to find someone who knows git (see <a href="qstart-git.html">git cheatsheet</a>) to clean up everything after you have finished.
<p>* footnote: the 'things' include the <a href="keyexchange.html">digital key</a> that allows the laptop to be trusted by the server, as well as various installed software.
<h3>Data maintenance task list</h3>
<p>We have an online list of outstanding data maintenance tasks. See the <a href="contribute.html#surveydata">'Survey Data' to-do</a> list
@@ -96,22 +122,6 @@ possibly in one of the side-pages which you skipped through when you read it.
<p>This password is all you need to log in to troggle. There is also an a systems account 'expoadmin' with a different password which enables the <a href="/controlpanel">import/export control panel</a> for re-importing <em>all</em> the input data files.
<h2>Beyond the easy stuff</h2>
<p>There are several standard data maintenance jobs that haven't yet got trogglized. So if:
<ul>
<li>We get a new Austrian kataster number for a cave, so we need to move and rename a lot of files
<li>A cave has got so big that we need to create sudirectories for the long and complicated cave descriptions in separate HTML files
<li>Some scanned notes have been uploaded into the wrong online wallet
</ul>
we will need to use the things* installed on the <em>expo laptop</em> or on <a href="basiclaptop.html">your own laptop</a> if you are not in the potato hut on expo. <a href="fzconfig.html">Filezilla</a> will do most of what you need for moving files.
We also have <a href="uploading.html#android">andftp Android instructions</a> for file manipulation.
<p>While Filezilla (including the digital key) is enough for moving files in <a href="/expofiles">expofiles</a>, moving files in the version-controlled <a href="repos.html">repositories</a> means you need to find someone who knows git (see <a href="qstart-git.html">git cheatsheet</a>) to clean up everything after you have finished.
<p>* footnote: the 'things' include the <a href="keyexchange.html">digital key</a> that allows the laptop to be trusted by the server, as well as various installed software.
<hr />
Return to:
<a href="../computing/onlinesystems.html">expo online systems overview</a><br />

46
handbook/computing/phone.html
View File

@@ -34,6 +34,52 @@ with the expo survey data workflow.
<li><a href="ftpusage.html">FTP using Cx_File_Explorer</a> (this is probably out of date)
</ul>
<h2 id="android">Bulk Data Phone: Using an FTP app </h2>
<p>If you want to do it yourself and the USB cable trick does not work then you will need to install a file manager and FTP app. Currently (on Expo 2019) the best seems to be the free <a href="http://www.lysesoft.com/products/andftp/">andftp</a> app. Assuming you can find where your phone camera has put the photo files on your phone, you can use your phone to upload photos directly to the /uploads/ folder on the expo server. However renaming them to something sensible and putting them in your own <span style="font-family: monospace">/YourName/</span> folder (see above for file naming guidelines) is fiddly on a phone.
<p>To use the FTP app you will need:<a href="http://www.lysesoft.com/products/andftp/"><img src="andftpsplash.jpg" width=128 style="float:right; margin: 10px"></a>
<ol>
<li>The hostname of the server: <span style="font-family: monospace">expo.survex.com</span>
<li>the username: <span style="font-family: monospace">expo</span>
<li>the password: (the usual cavey:beery password which you can get verbally from another expoer)
<li>the target folder: <span style="font-family: monospace">/uploads/</span>, or it may appear as <span style="font-family: monospace">/expofiles/uploads/</span>
<li>the port number: 21 (if you leave this blank it will probably work)
</ol>
<p>But <em>none of this will work</em> until you have also done the <a href="keyexchange.html">key-pair setup</a> procedure. On a phone this means that you will also need to install a terminal (command line) app. See <a href="phone.html">Android</a> instructions.
<ol>
<li>Install the <a href="http://www.lysesoft.com/products/andftp/">andftp</a> app onto your Android phone using the Google Play Store and allow it to access your files and photos on your phone (just click on the popup to do this).
<li>Click on the icon that looks like a "plus" synbol in a circle
<li>This opens a window with 3 tabs: General, Advanced and Sync
<li>On the General tab, type in the hostname, username and password as listed above.
<li>Also tick the checkbox "Enable resume support"
<li>Click OK to the two popups that appear
<li>You will now see an icon of a networked file folder (light blue) with the label "expo.survex.com". Click on it.
<li>Wait a while as it loads...
<li>Keep waiting. Various messages will scroll by at the bottom of the screen in tiny font.
<li>Keep waiting..
<li>The tiny text at the bottom will say "226 Transfer complete" and the main part of the window will show a listing of the files in the /uploads/ folder on the server.
<li>Now click on the little icon of a phone near the middle of the top row of icons.
<li>This will show a list of folders on your phone. Initially it will show "alt_autocycle, Android and DCIM".
<li>Click on ""DCIM", it will show a folder "Camera", click on that
<li>Now you can see a list of all your photo files by name and with the size shown on the right. Typically they will be 1.3MB or so in size.
<li>Before you upload, you need to create your own folder.
<li>Now click on the icon of a little cloud near the middle of the icon bar. This will show the files on the server.
<li>Now you need to create a folder <span style="font-family: monospace">/YourName/</span> and move those files into it. Click on the 3-dots icon on the right-hand end of the icon bar.
<li>Select the Create Folder option.
<li>A popup appears and you type in YourName. The folder is created.
<li>Click on the <span style="font-family: monospace">/YourName/</span> folder. This will makeyour folder the destination for uploaded files.
<li>Now click on the little icon of a phone near the middle of the top row of icons.
<li>This shows you the list of your photo files you saw before.
<li>Clicking on each photo filename will cause a green tick to appear on the left. This is selecting files for FTP upload.
<li>Now click on the "upload icon" on the top row: it looks like a short horizontal line with an up-arrow on it.
<li>A popup will appear asking if you wish to proceed. Click OK.
<li>The upload happens and you can see progress bars for each file.
<li>When it finishes it presents a popup with an OK button. Click on it.
<li>Now you will see the files you have just uploaded. And they will be in your folder. Success.
<li>Now you need to rename the files to something descriptive. This is very time-consuming on a phone.
<li>Click on one file to give it a green tick. Then select the 3-dot icon and then select the "Rename" option in the drop-down menu.
<li>Of course you will have earlier noted down the names of all your photos and made a note of their contents (which you see using the phone's Gallery app) so that you can do the renaming intelligently.
</ol>
<hr />
Return to <a href="chromebook.html">Setting up a Chromebook</a></body><br>
Return to <a href="basiclaptop.html">Setting up a basic laptop</a></body><br>

177
handbook/computing/surveylaptop.html
View File

@@ -2,54 +2,26 @@
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>CUCC Expedition Handbook: Your laptop</title>
<title>CUCC Expedition Handbook: Your Survey laptop</title>
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
</head>
<body>
<h2 id="tophead">CUCC Expedition Handbook - Your laptop</h2>
<h2 id="tophead">CUCC Expedition Handbook - Your Survey laptop</h2>
<h1>Cave survey software - beyond the basics</h1>
<h1>Cave survey software</h1>
<h2>Phones and laptops</h2>
<h3>Your phone</h3>
<p>See <a href="phone.html">Android phone page</a>.
<h3>Your laptop</h3>
<h3>Your Survey laptop</h3>
<p>You will already have configured your laptop to do all the basic stuff using the
<a href="basiclaptop.html">basic Expo laptop</a> guide.
<p><code>This is an attempt at a <em>complete set of optional software</em> for using survex, tunnel, therion, photos and GPS tracks to document our caves: using the existing data archive and processing new survey data.
<p>If you want to do software development instead, go to <a href="../troggle/troglaptop.html">configuring a troggle development machine</a>.
</code></p>
<p>This page documents what else you might find useful if the basic laptop setup does not do what you need.
<h2>Windows, Macs, Chromebooks and Linux</h2>
<ul>
<li>You should be able to use any OS (Linux, Windows, Mac) to interact with expo, but it's a bit simpler to get set up using Linux. You can't
yet do everything on a phone, but you can do some things.</li>
<li>The <i>expo laptop</i> uses <a href="https://www.debian.org/intro/about">Debian</a> with the <a
href="https://en.wikipedia.org/wiki/Cinnamon_(desktop_environment)">Cinnamon</a> interface, but pretty much any Linux
system works fine. Old, slow machines without much memory can be very effective with <a href="https://xubuntu.org/">Xubuntu/xfce</a>.
<li>If using a Mac or Red-Hat-based linux you'll need to work it all out for yourself.</li>
<li>Chromebooks work just like a phone, but with a keyboard, so far as Expo is concerned (mostly). See our <a href="chromebook.html">chromebooks</a> page.
<li>Windows machines can do almost everything needed directly in Windows itself, but some useful software has no Windows version and you will
need to find your own equivalents. There are also some 'gotchas' to look out for due to filesystem differences (e.g letter case). If you have
the choice, use Linux. Windows/Linux WSL is fine.</li>
<li><a href="winlaptop.html">A Windows laptop for expo</a> - a page of more detailed instructions for those who are definitely going to be
using a Windows PC or laptop.</li>
</ul>
<h2><a name="software">Software</a></h2>
<a href="https://survex.com/screenshots.html"><img src="https://survex.com/img/aven-1.2.37-by-date.png" hspace="20" align="right" width="300"></a>
<p></a>Long-standing Expo policy is to use open tools and protocols so we can retain control of our own data over the long term. And not to require expo-goers to sign up to external services or spend money on software.
@@ -59,9 +31,13 @@ but all the recommended software here is open source (and please don't install p
<h3>Installing the software</h3>
<p><em>For installing Survex, Tunnel etc. see <a href="getsurvex.html">this page</a> which may be merged in here eventually.</em>
<p>The list of software:
<ul>
<li><a href="getsurvex.html">Installing surveying software</a> - survex, tunnel, therion - specifically for expo use
<li><a href="https://survex.com/download.html">Survex</a>, including the Aven visualisation tool.
<li><a href="https://survex.com/related.html">Survex related tools</a> Carto, CaveScript (used by CaveView the 3D browser visualiser).
<li><a href="https://github.com/CaveSurveying/tunnelx">Tunnel</a>: 2.5D cave drawing program based on Survex-compatible data which can also read PocketTopo files. (Generally called 'tunnel' even though the project and executable is actually 'tunnelx'.)
@@ -77,116 +53,22 @@ but all the recommended software here is open source (and please don't install p
<li><a href="https://github.com/patrickbwarren/inkscape-survex-export">inkscape-survex</a> - Patrick's plugin to export a traced inkscape line drawing to a survex file.
<li><a href="http://www.thomas-holder.de/projects/inkscape-speleo/extensions/">inkscape-speleo</a> - plugin to import/export survex, therion and pockettopo files to the drawing package inkscape.
<li><a href="https://imagemagick.org/script/">Imagemagick convert</a> for re-scaling photographs and scanned images and converting from PDF to JPG. (NB you may need to fix this <a href="https://stackoverflow.com/questions/52998331/imagemagick-security-policy-pdf-blocking-conversion">security issue</a>)
<li><a href="https://git-scm.com/">git</a> - distributed version control system - already installed on Linux probably</li>
<li><a href="fzconfig.html">Filezilla</a> - ftp GUI software with a configuration file to get to the expo server (since October 2021 you can't use ftp:// in a Chrome browser anymore)</li>
<li>python, if you are working on the <a href="../troggle/scriptsother.html">scripts in expoweb/noinfo/</a> such as <var>make-folklist.py</var> or <var>wallets.py</var>- but not on troggle itself - then please use the as close to the same setup as you can to <a href="../troggle/troglaptop.html">the setup we use for troggle</a>. <img src="https://www.python.org/favicon.ico" width=64 hspace="20" align="right">
</ul>
<p>Nearly all our Austrian surveys have beeen produced using Tunnel (or were hand-drawn) but many smaller caves and some areas of SMKsystem are done with Therion because Therion does elevations and Tunnel doesn't. Expo has a policy decision on which to use: if it is an entirely new disconnected cave, then use Therion. If it is a passage in a cave where previously we used Tunnel, then use Tunnel. See also <a href="/expofiles/tunnelwiki/wiki/pages/Other_Cave_Software.html">Comparison of Tunnel to Other Cave Software</a>.</p>
<a href="https://therion.speleo.sk/2D.php"><img src="https://therion.speleo.sk/img/head.jpg" hspace="20"></a>
<h4>For Linux users only:</h4>
<ul>
<li><a href="https://git-scm.com/download/gui/linux">GUI for git</a> - There is a choice on Linux but many people just use the command line.
<li>Your favourite text editor. gedit is installed on the <em>expo laptop</em> and has syntax highlighting for .svx files too. This works with <a href="https://survex.com/changes.html">all GtkSourceView-based editors</a> (thanks to Phil Withnall). Download a local copy of the syntax colourizer <a download href="/site_media/survex.lang">survex.lang</a>.
</ul>
<p>Note that on a Debian/Ubuntu machine you should normally install the versions that come with the distro (i.e. install using 'apt install xxx', not be downloading things from the above sites. So installing everything you need should be as simple as:
<ul>
<li><tt>apt install openssh-client survex-aven tunnelx therion git sftp</tt></li>
<li><tt>ssh-keygen</tt> (to generate a key if you haven't already)</li>
<li><tt>apt install gpsprune qgis firefox-esr gedit tig gitg meld rsync</tt> (you may prefer other tools - e.g: elpa-magit is amazing but only if you know how to drive emacs)</li>
</ul>
</p>
<h4 id="win">For Windows users only:</h4>
<p></p>None of this works until you set up the key-pair setup using PuTty/Pageant.
<ul>
<li><a href="../putty/putty.html">PuTTY</a>.You need this to generate and to use ssh keys on Windows. Otherwise none of git, scp, ftp or rsync will work. It includes command line tools ssh, scp
(pscp) and sFTP (psftp). The <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html">PuTTY</a> installation includes puttygen and pagent which you also need. Version 0.77 was released on 2022-05-27.
<li><a href="https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview">OpenSSH for Windows</a> is new since 2018. It is a
set of command line tools which are an alternative to PuTTY. It also includes scp and sftp. It integrates well with Github for Windows and enables you to
use rsync over ssh. NB use this Microsoft documented install method which puts the executables in <var>windows\system32\openssh\</var>.
[This is so new that none of the other troggle documentation refers to it: all those other pages need to be rewritten to use this
instead of or as an alternative to PuTty/Pageant.]
<li><a href="https://desktop.github.com/">GitHub Desktop for Windows</a> - yes this works with the expo server git even though we don't use GitHub itself.
<li><a href="https://code.visualstudio.com/">VS Code</a> is a free (but not FOSS) editor with in-built git capability and plug-ins ("Git History",
"GitLens")which render git branches graphically. Also available for Linux.
<li><a href="https://gitforwindows.org/">Git for Windows</a> - not as simple to use as it looks
<li><a href="https://www.gitkraken.com/">GitKraken</a> - Very pretty GUI interface to git, also Linux version.
Times-out for our server unless you buy it but free for university people.
<li><a href="https://winscp.net/eng/download.php">WinScp</a> can be used as an alternative to Filezilla if you like (like Filezilla, it uses PuTTY ssh keys)
<li><a href="https://www.java.com/en/">java</a> - needed for GPSprune and CaveConverter. Has to be installed separately on a Windows machine.
<li><a href="https://notepad-plus-plus.org/">Notepad++</a> or any other syntax-highlighting code editor for HTML and python. We have a syntax-highlighter
to colourize .svx files, download it: <a href="/site_media/survex.xml">survex.xml</a> (no dark-mode yet though).
</ul>
<h4>Visual Studio Code editor</h4>
<p>If you use VS Code here are some relevant extras.
Not really for beginners but here are <a href="https://docs.microsoft.com/en-gb/learn/modules/python-install-vscode/">instructions for
configuring it for python</a>.
On Windows you run VS Code as a Windows app but it communicates directly ("remotely") with the WSL Linux environment.
<p>You will definitely want the
<a href="https://marketplace.visualstudio.com/items?itemName=donjayamanne.githistory">"Git History" extension</a>
and probably the "GitLens" extension too.
<p>It is entirely feasible (and on a slow machine, recommended) to use VS Code only for git management, and using a faster Windows-native editor,
such as Notepad++, for actually editing code and text.
<h2><a name="configuration">Configuration</a></h2>
<p>You will already have configured your machine as a 'basic laptop'.
If you have not yet done this, <a href="basiclaptop.html#configuration">do that now</a>, before continuing with these instructions.
<p>If you are using a Windows machine, go and skim read the <a href="winlaptop.html">Windows advanced configuration</a> section before you actually start
any of this simpler stuff.
<p><a href="../../documents/Idiots guide to accessing expo git.pdf">Idiots guide to setting up git for expo</a>
- PDF - Brendan's guide. Uses PuTTy and GitKraken.
<p>You need to <a href="keyexchange.html">register a key with the expo server</a> to get upload (i.e. read/write) access. Do this first, Without it none of git, scp, ftp or rsync will work.
You can do this entirely on your own if you have access to the <i>expo laptop</i> to upload and install the public key generated by your laptop.</p>
<p>On a Windows machine you will need to configure pageant (the putty authentication agent)
to <a href="https://blog.shvetsov.com/2010/03/making-pageant-automatically-load-keys.html">run at startup to load your key</a>. Note that you are loading your <em>private</em> key, the .ppk file, into pageant and that this key never leaves your laptop.</p>
<p>Full illustrated instructions:<img src="https://wiki.filezilla-project.org/favicon.ico" width=64 hspace="20" align="right"></p>
<ul>
<li><a href="../putty/putty.html">Installing PuTTy on Windows</a>.</li>
<li><a href="fzconfig.html">Installing Filezilla</a>.</li>
</ul>
<p>The above gets the command-line PuTTY tools (ssd, sftp, pscp) running, but doesn't get rsync working. You might like to try <a href="https://stackoverflow.com/questions/23517023/rsync-from-windows-to-linux-using-puttys-pagent-authentication">this</a> (untested).</p>
<p>When using Windows please, please be <a href="http://expo.survex.com/handbook/survey/getin.htm#filenames">excessively careful when naming files and survex names</a> and be <a href="manual.html#quickstart">exceptionally careful when using rsync</a>.
<h3>Learning how to use this software</h3>
<ul>
<li>Using Survex, Tunnel and Therion, see the <a href="http://expo.survex.com/handbook/survey/">Expo Handbook - Surveying section</a>.
<li><p><a href="/expofiles/documents/therion/therion-cp33.pdf"><em>Drawing Surveys with Therion</em></a> (Wookey, Compass Points, March 2004)
<li><p><a href="/expofiles/documents/therion/tharticle-vercors-2008.pdf"><em>Therion - Digital Cave Maps </em></a> Martin Budaj, Stacho Mudr&aacute;k, 2008)
<li><p><a href="/expofiles/documents/therion/thbook-v6.0.4.pdf"><em>The Therion Book</em></a> v6.0.4 (2012-11-28)
<li><em>For installing Survex, Tunnel etc. see <a href="getsurvex.html">this page</a> which may be merged in here eventually.</em>
<li><a href="../../documents/tunnel-loefflerCP35-only.pdf">Introduction to using Tunnel</a> - article by Dave Loeffler. This is really good and should be read first.
@@ -196,23 +78,15 @@ to <a href="https://blog.shvetsov.com/2010/03/making-pageant-automatically-load-
</ul>
<img src="tunnelbanner204.jpg" > <br />
<h3>Cheat lists and quick reminders</h3>
<ul>
<li>Quick <a href="qstart-git.html">reminders for using git</a> at the command line.
<li>Quick <a href="qstart-rsync.html">reminders for using rsync</a> at the command line.
</ul>
<h3>Logins to external systems</h3>
<p>
<a href="https://web.libera.chat/#expo">#expo</a> - public. An open-access <a href="https://en.wikipedia.org/wiki/Internet_Relay_Chat">
open-access IRC channel</a>
(ephemeral, not archived) for real-time discussions about everything but mostly software people. If you are having trouble using the software try here first.
<p>The <a href="https://app.element.io/#/room/#LoserExpoSurveying:matrix.org">Surveying</a> and <a href="https://app.element.io/#/room/#LoserExpoDataEntry:matrix.org">Data entry</a> discussion areas on the Loser Expo Matrix chatroom are directly relevant. When we are drawing up the surveys after each expo we discuss things online here. Ask Wookey for an invitation.
</p>
<ul>
<li><a href="https://github.com/join">Github</a> - You'll need an account if you want to use this.
<li><a href="https://github.com/CaveSurveying/CUCCexposurveyissues/issues">Expo issues list</a> - Issues with data reduction for recent expos and software updates for tunnel
<li><a href="https://bitbucket.org/product/">Bitbucket</a> - create an account to help develop tunnel. <img src="https://tortoisehg.bitbucket.io/img/bitbucket-icon.png" width=64 hspace="20" align="right">
<li> Expo <a href="https://app.element.io/#/room/#LoserExpoSurveying:matrix.org">Surveying</a> chat
<li> Expo <a href="https://app.element.io/#/room/#LoserExpoDataEntry:matrix.org">Data entry</a> chat
<li><a href="https://github.com/CaveSurveying/CUCCexposurveyissues/issues">Expo issues list</a> - Issues with data reduction for recent expos and software updates for tunnel (GitHub logon account required)
<li><a href="https://trac.survex.com/wiki">Trac issue tracker and wiki</a> for developing survex. You'll need a login to file bugs there.
</ul>
@@ -225,18 +99,11 @@ issue list so you will need to subscribe to the Cave Surveying Group on github t
a wiki on github discussing workflows to generate centerlines, GEOTIFF and QGIS integration.
</ul>
<h3>Complementary tools</h3>
<p>When maintaining the HTML files in the expo handbook a link-checker is useful to report bad URLs (links to external sites go bad regularly) and to find orphaned pages with no in-links. The website has <a href="https://www.klebos.net/subdomains/keldos/LinkScans/TUNNEL-wiki-link-report.html">about 2,000 internal URLs</a> in just the Tunnel wiki section alone.
<ul>
<li><a href="https://www.brokenlinkcheck.com/">Online brokenlinkcheck tool</a> - does not install on your laptop. Limited to 3,000 pages.
<li><a href="https://www.deadlinkchecker.com/website-dead-link-checker.asp">Another online brokenlinkcheck tool</a> - does not install on your laptop. Limited to 2,000 pages but breadth-first rather than depth-first, so it covers a different chunk of the expo website.
<li><a href="https://www.drlinkcheck.com/account/subscriptions/1/projects/1/overview">Yet another link-checker</a>, only 1,500 pages checked but oh-so-stylish results presentation. Very informative.
<li><a href="http://home.snafu.de/tilman/xenulink.html">Xenu Link Sleuth</a> - install on Windows to do comprehensive link check.
<li><a href="https://www.digitalocean.com/community/tutorials/how-to-find-broken-links-on-your-website-using-wget-on-debian-7">Linkchecking with wget</a> - command line methods on Linux
</ul>
<a href="http://wscc.darkgem.com/caveconverter/"><img src="CaveConverterWorkflow.png" width=600 hspace="20"></a>
<hr />
Go back to: <a href="basiclaptop.html">Basic laptop</a><br />
Go on to: <a href="bulkupdatelaptop.html">Bulk Update laptop</a><br />
<hr /></body>
<hr />
</body>
</html>

54
handbook/computing/uploading.html
View File

@@ -119,11 +119,10 @@ place (2023 maybe.. if Wookey gets this sorted out in time) but may be broken. P
<h2 id="morecomplex">Using your own laptop</h2>
<p>To use your own laptop on expo, or after you return from expo, you need need to use FTP. So become an <a href="#experienced">experienced user</a> first.
<p>To use your own laptop on expo, or after you return from expo, it needs to be an <a href="bulkupdatelaptop.html">expo Bulk data laptop</a>, which takes quite a lot of effort to set up.
You need need to use FTP. So become an <a href="#experienced">experienced user</a> first.
<p>You will need to know the expo password but <em>none of this will work</em> until you have also done the <a href="keyexchange.html">key-pair setup</a> procedure.</p>
<p><img src="https://wiki.filezilla-project.org/favicon.ico" width=64 style="float:right; margin: 10px">To install and configure Filezilla on your machine see <a href="fzconfig.html">FileZilla install instructions</a> which will set you up pointing at the correct folder automatically.
<p>You will need to know the expo password but <em>none of this will work</em> until you have also done the <a href="keyexchange.html">key-pair setup</a> procedure and configured your Bulk Data laptop</p>
<h2 id="android">Getting photo files from phones</h2>
@@ -140,53 +139,8 @@ for your caving holiday snaps and cave entrance location photos and share them w
place on the server. Be careful that you are sharing the full original resolution of the photos and not some cut-down compressed
bastardized "enhanced" version of the photos.
<p>See the <a href="phone.html">Bulk Data phone</a> page for how to do bulk data transfer directly from your phone.
<h2 id="android">Using an FTP app </h2>
<p>If you want to do it yourself and the USB cable trick does not work then you will need to install a file manager and FTP app. Currently (on Expo 2019) the best seems to be the free <a href="http://www.lysesoft.com/products/andftp/">andftp</a> app. Assuming you can find where your phone camera has put the photo files on your phone, you can use your phone to upload photos directly to the /uploads/ folder on the expo server. However renaming them to something sensible and putting them in your own <span style="font-family: monospace">/YourName/</span> folder (see above for file naming guidelines) is fiddly on a phone.
<p>To use the FTP app you will need:<a href="http://www.lysesoft.com/products/andftp/"><img src="andftpsplash.jpg" width=128 style="float:right; margin: 10px"></a>
<ol>
<li>The hostname of the server: <span style="font-family: monospace">expo.survex.com</span>
<li>the username: <span style="font-family: monospace">expo</span>
<li>the password: (the usual cavey:beery password which you can get verbally from another expoer)
<li>the target folder: <span style="font-family: monospace">/uploads/</span>, or it may appear as <span style="font-family: monospace">/expofiles/uploads/</span>
<li>the port number: 21 (if you leave this blank it will probably work)
</ol>
<p>But <em>none of this will work</em> until you have also done the <a href="keyexchange.html">key-pair setup</a> procedure. On a phone this means that you will also need to install a terminal (command line) app. See <a href="phone.html">Android</a> instructions.
<ol>
<li>Install the <a href="http://www.lysesoft.com/products/andftp/">andftp</a> app onto your Android phone using the Google Play Store and allow it to access your files and photos on your phone (just click on the popup to do this).
<li>Click on the icon that looks like a "plus" synbol in a circle
<li>This opens a window with 3 tabs: General, Advanced and Sync
<li>On the General tab, type in the hostname, username and password as listed above.
<li>Also tick the checkbox "Enable resume support"
<li>Click OK to the two popups that appear
<li>You will now see an icon of a networked file folder (light blue) with the label "expo.survex.com". Click on it.
<li>Wait a while as it loads...
<li>Keep waiting. Various messages will scroll by at the bottom of the screen in tiny font.
<li>Keep waiting..
<li>The tiny text at the bottom will say "226 Transfer complete" and the main part of the window will show a listing of the files in the /uploads/ folder on the server.
<li>Now click on the little icon of a phone near the middle of the top row of icons.
<li>This will show a list of folders on your phone. Initially it will show "alt_autocycle, Android and DCIM".
<li>Click on ""DCIM", it will show a folder "Camera", click on that
<li>Now you can see a list of all your photo files by name and with the size shown on the right. Typically they will be 1.3MB or so in size.
<li>Before you upload, you need to create your own folder.
<li>Now click on the icon of a little cloud near the middle of the icon bar. This will show the files on the server.
<li>Now you need to create a folder <span style="font-family: monospace">/YourName/</span> and move those files into it. Click on the 3-dots icon on the right-hand end of the icon bar.
<li>Select the Create Folder option.
<li>A popup appears and you type in YourName. The folder is created.
<li>Click on the <span style="font-family: monospace">/YourName/</span> folder. This will makeyour folder the destination for uploaded files.
<li>Now click on the little icon of a phone near the middle of the top row of icons.
<li>This shows you the list of your photo files you saw before.
<li>Clicking on each photo filename will cause a green tick to appear on the left. This is selecting files for FTP upload.
<li>Now click on the "upload icon" on the top row: it looks like a short horizontal line with an up-arrow on it.
<li>A popup will appear asking if you wish to proceed. Click OK.
<li>The upload happens and you can see progress bars for each file.
<li>When it finishes it presents a popup with an OK button. Click on it.
<li>Now you will see the files you have just uploaded. And they will be in your folder. Success.
<li>Now you need to rename the files to something descriptive. This is very time-consuming on a phone.
<li>Click on one file to give it a green tick. Then select the 3-dot icon and then select the "Rename" option in the drop-down menu.
<li>Of course you will have earlier noted down the names of all your photos and made a note of their contents (which you see using the phone's Gallery app) so that you can do the renaming intelligently.
</ol>
<hr />
</body>
</html>

81
handbook/computing/winlaptop.html
View File

@@ -6,9 +6,9 @@
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
</head>
<body>
<h2 id="tophead">CUCC Expedition Handbook - Computers</h2>
<h2 id="tophead">CUCC Expedition Handbook - Windows Bulk Updates</h2>
<h1>Setting up a Windows expo laptop</h1>
<h1>A Windows Bulk Update laptop</h1>
<p>First read the generic instructions for all the software installations you will need:
<br>
@@ -17,8 +17,6 @@
<a href="surveylaptop.html">Setting up a machine for Expo survey processing</a>
</p>
<p>It used to be quite simple: software equivalents existed for Windows for everything we needed. Since Spring 2018, we unfortunately have to use ssh keys. This has simply added an extra step for most things, but has made one thing (rsync) really quite hard to get working.
<p>What makes this all much more tiresome is that Microsoft are introducing some cool new ways of interacting with Linux systems but these are new and need some effort to get configured correctly. See the end of this page for more details.
<ul>
<li><a href="#well">Things that already work well with a Windows laptop</a><br />
- Editing the handbook webpages, typing up SVX files and transcribing the logbook, <br />
@@ -32,9 +30,75 @@
- well not "hard" exactly, but complicated with lot of steps that are easy to get wrong and with poor feedback as to whether you have done each step correctly.
<li><a href="#bold">The easier way to do it</a><br />
- using WSL: Windows System for Linux
<li><a href="surveylaptop.html#win">Possibly an even easier way to do it</a><br />
<li>See our own expo page about <a href="../computing/wsllaptop.html">setting up a WSL Expo Laptop</a>
<li><a href="bulkupdatelaptop.html#win">Possibly an even easier way to do it</a><br />
- using OpenSSH for Windows</ul>
<h3>Software</h3>
<ul>
<li><a href="../putty/putty.html">PuTTY</a>.You need this to generate and to use ssh keys on Windows. Otherwise none of git, scp, ftp or rsync will work. It includes command line tools ssh, scp
(pscp) and sFTP (psftp). The <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html">PuTTY</a> installation includes puttygen and pagent which you also need. Version 0.77 was released on 2022-05-27.
<li><a href="https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview">OpenSSH for Windows</a> is new since 2018. It is a
set of command line tools which are an alternative to PuTTY. It also includes scp and sftp. It integrates well with Github for Windows and enables you to
use rsync over ssh. NB use this Microsoft documented install method which puts the executables in <var>windows\system32\openssh\</var>.
[This is so new that none of the other troggle documentation refers to it: all those other pages need to be rewritten to use this
instead of or as an alternative to PuTty/Pageant.]
<li><a href="https://desktop.github.com/">GitHub Desktop for Windows</a> - yes this works with the expo server git even though we don't use GitHub itself.
<li><a href="https://code.visualstudio.com/">VS Code</a> is a free (but not FOSS) editor with in-built git capability and plug-ins ("Git History",
"GitLens")which render git branches graphically. Also available for Linux.
<li><a href="https://gitforwindows.org/">Git for Windows</a> - not as simple to use as it looks
<li><a href="https://www.gitkraken.com/">GitKraken</a> - Very pretty GUI interface to git, also Linux version.
Times-out for our server unless you buy it but free for university people.
<li><a href="https://winscp.net/eng/download.php">WinScp</a> can be used as an alternative to Filezilla if you like (like Filezilla, it uses PuTTY ssh keys)
<li><a href="https://www.java.com/en/">java</a> - needed for GPSprune and CaveConverter. Has to be installed separately on a Windows machine.
<li><a href="https://notepad-plus-plus.org/">Notepad++</a> or any other syntax-highlighting code editor for HTML and python. We have a syntax-highlighter
to colourize .svx files, download it: <a href="/site_media/survex.xml">survex.xml</a> (no dark-mode yet though).
</ul>
<h4>Visual Studio Code editor</h4>
<p>If you use VS Code here are some relevant extras.
Not really for beginners but here are <a href="https://docs.microsoft.com/en-gb/learn/modules/python-install-vscode/">instructions for
configuring it for python</a>.
On Windows you run VS Code as a Windows app but it communicates directly ("remotely") with the WSL Linux environment.
<p>You will definitely want the
<a href="https://marketplace.visualstudio.com/items?itemName=donjayamanne.githistory">"Git History" extension</a>
and probably the "GitLens" extension too.
<p>It is entirely feasible (and on a slow machine, recommended) to use VS Code only for git management, and using a faster Windows-native editor,
such as Notepad++, for actually editing code and text.
<h2><a name="configuration">Configuration</a></h2>
<p><a href="/expofiles/documents/Idiots guide to accessing expo git.pdf">Idiots guide to setting up git for expo</a>
- PDF - Brendan's guide. Uses PuTTy and GitKraken on WIndows.
<p>You need to <a href="keyexchange.html">register a key with the expo server</a> to get upload (i.e. read/write) access. Do this first, Without it none of git, scp, ftp or rsync will work.
You can do this entirely on your own if you have access to the <i>expo laptop</i> to upload and install the public key generated by your laptop.</p>
<p>On a Windows machine you will need to configure pageant (the putty authentication agent)
to <a href="https://blog.shvetsov.com/2010/03/making-pageant-automatically-load-keys.html">run at startup to load your key</a>. Note that you are loading your <em>private</em> key, the .ppk file, into pageant and that this key never leaves your laptop.</p>
<p>Full illustrated instructions:<img src="https://wiki.filezilla-project.org/favicon.ico" width=64 hspace="20" align="right"></p>
<ul>
<li><a href="../putty/putty.html">Installing PuTTy on Windows</a>.</li>
<li><a href="fzconfig.html">Installing Filezilla</a>.</li>
</ul>
<p>The above gets the command-line PuTTY tools (ssd, sftp, pscp) running, but doesn't get rsync working. You might like to try <a href="https://stackoverflow.com/questions/23517023/rsync-from-windows-to-linux-using-puttys-pagent-authentication">this</a> (untested).</p>
<p>When using Windows please, please be <a href="http://expo.survex.com/handbook/survey/getin.htm#filenames">excessively careful when naming files and survex names</a> and be <a href="qstart-rsync.html">exceptionally careful when using rsync</a>.
<h3 id="well">Things that already work well</h3>
<p>Anything where the file upload and download is done via the verson control client software works really well.
@@ -139,7 +203,7 @@ $
</tt>
</pre>
The generated key is in the current directory and you need to move them to ~/.ssh/ as is standard on Linux (which is not at all the same place that PuTTy uses to keep keys on Windows).
<p>Now you have to complete the <a href="keyexchange.html">key-pair setup</a> with the new key "id_ras_wsl.pub". But you don't need anyone else's help this time as you can use PuTTy to ssh into the server and copy your key to the right place yourself.
<p>Now you have to complete the <a href="keyexchange.html">key-pair setup</a> with the new key "id_ras_wsl.pub". But you don't need anyone else's help this time as you can use PuTTy to ssh into the server and copy your key to the right place yourself: it is a <a href="keyexchange.html#secondmachine">"Second Machine"</a>.
<p>
Now finally you can use all the usual command line tools at yor wsl command line to communicate with the server with ssh, scp, rsync, such as:
<pre>
@@ -214,8 +278,9 @@ it shares with WSL2).
</ul>
<h3>Installing and Configuring the rest of the software you need on Windows</h3>
<p>Now return to <a href="surveylaptop.html">the survey laptop</a> page to configure all the rest of the software you need.
<hr />
Go back to: <a href="basiclaptop.html">Basic laptop</a><br />
Go back to: <a href="surveylaptop.html">Survey laptop</a><br />
<hr /></body>
</html>

9
handbook/computing/wsllaptop.html
View File

@@ -14,9 +14,11 @@
<br>
<a href="basiclaptop.html">Setting up a basic Expo laptop</a>
<br>
<a href="surveylaptop.html">Setting up a machine for survey data maintenance</a>
<a href="surveylaptop.html">Setting up a Survey machine for cave survey production</a>
<br>
<a href="winlaptop.html">Setting up a Windows machine for Expo data maintenance</a>
<a href="bulkupdatelaptop.html">Setting up a machine for expo bulk data rearrangements</a>
<br>
<a href="winlaptop.html">Setting up a Windows machine for expo bulk data rearrangements</a>
</p>
<h3>WSL1 and WSL2</h3>
@@ -74,6 +76,9 @@ in the \\wsl$\ ext4 filesystem (WSL2 only, not WSL1).
<h3>Installing and Configuring the rest of the software you need on Windows</h3>
<p>Now return to &#8251;<a href="winlaptop.html">the Windows data maintenance laptop</a> page to configure all the rest of the software you need.&#8258;
<hr />
Go back to: <a href="basiclaptop.html">Basic laptop</a><br />
Go back to: <a href="surveylaptop.html">Survey laptop</a><br />
<hr /></body>
</html>

1
handbook/troggle/troglaptop.html
View File

@@ -42,6 +42,7 @@ http://expo.survex.com/repositories/troggle/.git/tree/README.txt
<ul>
<li>[On 23 November 2022 a new way of installing and using WSL was announced, see <a href="https://www.theregister.com/2022/11/23/wsl_microsoft_store_default_version/">
Windows Subsystem for Linux now packaged as a Microsoft Store app</a> so all documentation on this expo site needs to be revised.]
<li>See our own expo page about <a href="../computing/wsllaptop.html">setting up a WSL Expo Laptop</a>
<li>Install in <a href="https://docs.microsoft.com/en-us/windows/wsl/install">WSL on Windows 10/11</a>
<li>Install <a href="https://docs.microsoft.com/en-us/windows/wsl/install">Ubuntu 20.04 LTS</a> - can be done directly from wsl command line <var>wsl --install -d Ubuntu</var>.
<li>Install <a href="https://linuxconfig.org/ubuntu-22-04-on-wsl-windows-subsystem-for-linux">Ubuntu 22.04 LTS</a> - must be done using MS App Store.