CUCC Expedition Handbook - Your laptop
Cave survey software - beyond the basics
Phones and laptops
Your Android phone
Everyone has a phone pretty much, and everyone should have GPS enabled for safety. See the expo GPS configuration pages.
There are several cave survey apps which run on a phone. We don't yet have a well documented way of using these
with the expo survey data workflow.
- OsmAnd - a GPS app.
- GPSessentials - another GPS app.
- Ape@Map - another GPS app, compatible with Kompass commerical map sand used by Austrian cavers
- TopoDroid makes cave surveys with the DistoX.
- Cave3D is a Therion 3D viewer.
- ThManager organizes single surveys, exported by TopoDroid, into Therion projects encoded by Therion "thconfig" files.
You can also use a phone to upload files and manage version control on the server. This is not the place to start, but if you are already doing this on your own laptop then these can be useful:
- Termius - command line to create a registratkey for your phone, to ssh (and mosh) login to the expo server. To get sftp you have to buy it.
- andftp FTP app - needs an ssh key installed on your phone
- FTP using Cx_File_Explorer (this is probably out of date)
Your laptop
You will already have configured your laptop to do all the basic stuff using the
basic Expo laptop guide.
This is an attempt at a complete set of optional software for using survex, tunnel, therion, photos and GPS tracks to document our caves: using the existing data archive and processing new survey data.
If you want to do software development instead, go to configuring a troggle development machine.
This page documents what else you might find useful if the basic laptop setup does not do what you need.
Windows, Macs, Chromebooks and Linux
- You should be able to use any OS (Linux, Windows, Mac) to interact with expo, but it's a bit simpler to get set up using Linux. You can't
yet do everything on a phone, but you can do some things.
- The expo laptop uses Debian with the Cinnamon interface, but pretty much any Linux
system works fine. Old, slow machines without much memory can be very effective with Xubuntu/xfce.
- If using a Mac or Red-Hat-based linux you'll need to work it all out for yourself.
- Chromebooks, since 2019, allow you to enable the underlying debian system in a container
using a tool called Crostini. This should work just fine, but no one has tried it yet.
It should be even better than WSL on Windows as you can run GUI applications on Debian. Or you could install a full Ubuntu system with chroot using
Crouton.
- Windows machines can do almost everything needed directly in Windows itself, but some useful software has no Windows version and you will
need to find your own equivalents. There are also some 'gotchas' to look out for due to filesystem differences (e.g letter case). If you have
the choice, use Linux. Windows/Linux WSL is fine.
- A Windows laptop for expo - a page of more detailed instructions for those who are definitely going to be
using a Windows PC or laptop.
Long-standing Expo policy is to use open tools and protocols so we can retain control of our own data over the long term. And not to require expo-goers to sign up to external services or spend money on software.
So we use FOSS software.
You can use other software on your own machine if it is format-compatible and exports data in the formats we want,
but all the recommended software here is open source (and please don't install proprietary software on the 'expo laptop').
The list of software:
- Survex, including the Aven visualisation tool.
- Survex related tools Carto, CaveScript (used by CaveView the 3D browser visualiser).
- Tunnel: 2.5D cave drawing program based on Survex-compatible data which can also read PocketTopo files. (Generally called 'tunnel' even though the project and executable is actually 'tunnelx'.)
- Therion - Therion processes survey data and generates maps or 3D models of caves.
- TopParser - to convert PocketTopo .top files from a DistoX into Therion.
- Footleg Cave Converter - to convert PocketTopo .top files from a DistoX into survex (Java)
- QGIS - A Geographic Information System
- GPSprune - to edit GPS tracks in GPX files (Java).
- Viking - an alternative to GPSprune. The code is on github.
- qgis3-survex-import - Patrick's plugin to import/export survex .3d files
- Vectr svg - quick and easy sketcher (browser-based, Chrome only) for making rigging topo diagrams: "free forever" committment, but not open source. Please export as svg not jpg or png.
- inkscape-survex - Patrick's plugin to export a traced inkscape line drawing to a survex file.
- inkscape-speleo - plugin to import/export survex, therion and pockettopo files to the drawing package inkscape.
- Imagemagick convert for re-scaling photographs and scanned images and converting from PDF to JPG. (NB you may need to fix this security issue)
- git - distributed version control system - already installed on Linux probably
- Filezilla - ftp GUI software with a configuration file to get to the expo server (since October 2021 you can't use ftp:// in a Chrome browser anymore)
- python, if you are working on the scripts in expoweb/noinfo/ such as make-folklist.py or wallets.py- but not on troggle itself - then please use the as close to the same setup as you can to the setup we use for troggle.
Nearly all our Austrian surveys have beeen produced using Tunnel (or were hand-drawn) but many smaller caves and some areas of SMKsystem are done with Therion because Therion does elevations and Tunnel doesn't. Expo has a policy decision on which to use: if it is an entirely new disconnected cave, then use Therion. If it is a passage in a cave where previously we used Tunnel, then use Tunnel. See also Comparison of Tunnel to Other Cave Software.
For Linux users only:
- GUI for git - There is a choice on Linux but many people just use the command line.
- Your favourite text editor. gedit is installed on the expo laptop and has syntax highlighting for .svx files too. This works with all GtkSourceView-based editors (thanks to Phil Withnall). Download a local copy of the syntax colourizer survex.lang.
Note that on a Debian/Ubuntu machine you should normally install the versions that come with the distro (i.e. install using 'apt install xxx', not be downloading things from the above sites. So installing everything you need should be as simple as:
- apt install openssh-client survex-aven tunnelx therion git sftp
- ssh-keygen (to generate a key if you haven't already)
- apt install gpsprune qgis firefox-esr gedit tig gitg meld rsync (you may prefer other tools - e.g: elpa-magit is amazing but only if you know how to drive emacs)
For Windows users only:
None of this works until you set up the key-pair setup using PuTty/Pageant.
- PuTTY.You need this to generate and to use ssh keys on Windows. Otherwise none of git, scp, ftp or rsync will work. It includes command line tools ssh, scp
(pscp) and sFTP (psftp). The PuTTY installation includes puttygen and pagent which you also need. Version 0.77 was released on 2022-05-27.
- OpenSSH for Windows is new since 2018. It is a
set of command line tools which are an alternative to PuTTY. It also includes scp and sftp. It integrates well with Github for Windows and enables you to
use rsync over ssh. NB use this Microsoft documented install method which puts the executables in windows\system32\openssh\.
[This is so new that none of the other troggle documentation refers to it: all those other pages need to be rewritten to use this
instead of or as an alternative to PuTty/Pageant.]
- GitHub Desktop for Windows - yes this works with the expo server git even though we don't use GitHub itself.
- VS Code is a free (but not FOSS) editor with in-built git capability and plug-ins ("Git History",
"GitLens")which render git branches graphically. Also available for Linux.
- Git for Windows - not as simple to use as it looks
- GitKraken - Very pretty GUI interface to git, also Linux version.
Times-out for our server unless you buy it but free for university people.
- WinScp can be used as an alternative to Filezilla if you like (like Filezilla, it uses PuTTY ssh keys)
- java - needed for GPSprune and CaveConverter. Has to be installed separately on a Windows machine.
- Notepad++ or any other syntax-highlighting code editor for HTML and python. We have a syntax-highlighter
to colourize .svx files, download it: survex.xml (no dark-mode yet though).
Visual Studio Code editor
If you use VS Code here are some relevant extras.
Not really for beginners but here are instructions for
configuring it for python.
On Windows you run VS Code as a Windows app but it communicates directly ("remotely") with the WSL Linux environment.
You will definitely want the
"Git History" extension
and probably the "GitLens" extension too.
It is entirely feasible (and on a slow machine, recommended) to use VS Code only for git management, and using a faster Windows-native editor,
such as Notepad++, for actually editing code and text.
You will already have configured your machine as a 'basic laptop'.
If you have not yet done this, do that now, before continuing with these instructions.
If you are using a Windows machine, go and skim read the Windows advanced configuration section before you actually start
any of this simpler stuff.
Idiots guide to setting up git for expo
- PDF - Brendan's guide. Uses PuTTy and GitKraken.
You need to register a key with the expo server to get upload (i.e. read/write) access. Do this first, Without it none of git, scp, ftp or rsync will work.
You can do this entirely on your own if you have access to the expo laptop to upload and install the public key generated by your laptop.
On a Windows machine you will need to configure pageant (the putty authentication agent)
to run at startup to load your key. Note that you are loading your private key, the .ppk file, into pageant and that this key never leaves your laptop.
Full illustrated instructions:
The above gets the command-line PuTTY tools (ssd, sftp, pscp) running, but doesn't get rsync working. You might like to try this (untested).
When using Windows please, please be excessively careful when naming files and survex names and be exceptionally careful when using rsync.
Learning how to use this software
Cheat lists and quick reminders
Logins to external systems
#expo - public. An open-access
open-access IRC channel
(ephemeral, not archived) for real-time discussions about everything but mostly software people. If you are having trouble using the software try here first.
- Github - You'll need an account if you want to use this.
- Expo issues list - Issues with data reduction for recent expos and software updates for tunnel
- Bitbucket - create an account to help develop tunnel.
- Trac issue tracker and wiki for developing survex. You'll need a login to file bugs there.
Some expo survey stuff is tracked externally:
Complementary tools
When maintaining the HTML files in the expo handbook a link-checker is useful to report bad URLs (links to external sites go bad regularly) and to find orphaned pages with no in-links. The website has about 2,000 internal URLs in just the Tunnel wiki section alone.