CUCC Expedition Handbook - Your laptop
Setting up a machine to access Expo data
Overview
The main things you need to do to get set up are:
- Install software to access version control and edit/view caving data.
- Register an SSH key with the expo server (i.e 'get a login').
- Download the expo repositories so you have the files on your machine.
Operating Systems
- You should be able to use any OS to interact with expo, but it's a bit simpler to get set up using Linux or MacOS, rather than Windows, just because the core stuff will already be installed. You can't yet do everything on a phone, but you can do some things.
- The expo laptop uses Debian with the Cinnamon interface, but pretty much any Linux system works fine. This handbook does assume that you are using apt - the Debian package manager - which is good for all Debian-derived Linuxes such as Ubuntu. Old, slow machines without much memory can be very effective with Xubuntu/xfce. If using a Red-Hat-based linux you'll need to work out the equivalent rpm/yum commands.
- Windows machines can do almost everything needed (rsync is a problem), but some useful software has no Windows version and you will need to find your own equivalents. There are also some 'gotchas' to look out for due to filesystem differences (e.g letter case). If you have the choice, use Linux.
- A Windows laptop for expo - a page of more detailed instructions for those who are definitely going to be using a Windows PC or laptop.
- Mac users will need to use the Linux documentation as a guide and work it all out for themselves (and then hopefully update this handbook).
- Android phone apps can be invaluable too.
Long-standing Expo policy is to use open tools and protocols so we can retain control of our own data over the long term. And not to require expo-goers to sign up to external services or spend money on software.
So we use FOSS software.
You can use other software on your own machine if it is format-compatible and exports data in the formats we want,
but all the recommended software here is open source (and please don't install proprietary software on the 'expo laptop').
The list of software:
- git - distributed version control system - already installed on Linux probably
- TortoiseHg - GUI to mercurial (to be deprecated during 2020)
- mercurial - distributed version control system (to be deprecated during 2020)
- Filezilla - ftp GUI software with a configuration file to get to the expo server
- Survex, including the Aven visualisation tool.
- Tunnel: 2.5D cave drawing program based on Survex-compatible data which can also read PocketTopo files. (Generally called 'tunnel' even though the project and executable is actually 'tunnelx'.)
- Therion - Therion processes survey data and generates maps or 3D models of caves.
- TopParser - to convert PocketTopo .top files from a DistoX into Therion.
- Footleg Cave Converter - to convert PocketTopo .top files from a DistoX into survex (Java)
- QGIS - A Geographic Information System
- GPSprune - to edit GPS tracks in GPX files (Java).
- Viking - an alternative to GPSprune. The code is on github.
- python, if you are extending troggle or the scripts in expoweb/noinfo/ - already installed on Linux
- python packages you will want to install: [not yet documented]
- qgis3-survex-import - plugin to import/export survex .3d files
- inkscape-speleo - plugin to import/export survex, therion and pockettopo files to the drawing package inkscape.
Nearly all our Austrian surveys have beeen produced using Tunnel (or were hand-drawn) but many smaller caves and some areas of SMKsystem are done with Therion because Therion does elevations and Tunnel doesn't.
Note that on a Debian/Ubuntu machine you should normally install the versions that come with the distro, not be getting things fromthe above sites
For Linux users only:
- GUI for git - There is a choice on Linux but many people just use the command line.
- Your favourite text editor. gedit is installed on the expo laptop and has syntax highlighting for .svx files too. This works with all GtkSourceView-based editors (thanks to Phil Withnall).
For Windows users only:
- Read our instructions for setting up TortoiseHg in Tortoise-on-Windows. (But this won't work at all until you set up the key-pair setup using PuTty/Pageant.)
- PuTTY including pagent. Version 0.73 was released on 2019-09-29. You need this to generate and to use ssh keys on Windows. Otherwise none of git, mercurial, scp, ftp or rsync will work. It includes command line tools ssh, scp (pscp) and sFTP (psftp).
- Git for Windows
- TortoiseGit - GUI interface to git
- python for windows. We are using python 2.7 not 3.8.
- WinScp can be used as an alternative to Filezilla if you like.
- java - needed for GPSprune and CaveConverter. Has to be installed separately on a Windows machine.
- Notepad++ or any other syntax-highlighting code editor for HTML and python such as Aptana Studio. Configuring these to syntax-highlight .svx files has yet to be done.
- Cygwin rsync - not installed as standard on Windows so you need to get Cygwin or use WSL. But getting WSL to work with ssh keys has been tricky and pagent-managed ssh keys are apparently not visible to ssh and rsync in WSL. If you find a good solution for Windows please edit this documentation.
For Android phones:
You need to register a key with the expo server to get upload (i.e. read/write) access. Do this first, Without it none of git, mercurial, scp, ftp or rsync will work.
You can do this entirely on your own if you have access to the expo laptop to upload and install the public key generated by your laptop.
On a Windows machine you will need to configure pageant (the putty authentication agent)
to run at startup to load your key. Note that you are loading your private key, the .ppk file, into pageant and that this key never leaves your laptop.
The above gets TortoiseHg and the command-line PuTTY tools (ssd, sftp, pscp) running, but doesn't get Cygwin rsync working. You might like to try this (untested).
Full illustrated instructions:
When using Windows please, please be excessively careful when naming files and survex names and be exceptionally careful when using rsync.
The handbook has documents where it is necessary to use scp or sftp to manage large files in 'expofiles'. See Experts: Uploading files, Uploading files and Uploading GPS tracks. Only machines which have done the key-pair setup process can do scp, sftp or rsync.
Learning how to use this software
Cheat lists and quick reminders
Logins to external systems
Some expo survey stuff is tracked externally:
Complementary tools
When maintaining the HTML files in the expo handbook a link-checker is useful to report bad URLs (links to external sites go bad regularly) and to find orphaned pages with no in-links. The website has about 2,000 internal URLs in just the Tunnel wiki section alone.
