CUCC Expedition Handbook - Bulk Update laptop

Cave data - Bulk Update Laptop

You will already have explored what your laptop can do in the basic Expo laptop guide, and you probably will also have installed the survey tools survex, tunnel, therion etc. in configuring a survey laptop.

This is an attempt at describing how to configure your laptop to do bulk file transfers to and from the server. If you are only managing one or two files at a time, you don't need this.

If you want to do software development instead, go to configuring a troggle development machine.

Windows and Linux

• You should be able to use any OS (Linux, Windows, Mac), but it's a bit simpler to get set up using Linux.
• Windows machines can do almost everything needed directly in Windows itself, but some useful software has no Windows version and you will need to find your own equivalents. There are also some 'gotchas' to look out for due to filesystem differences (e.g letter case). If you have the choice, use Linux. Windows/Linux WSL is fine.
• A Windows laptop for expo - a page of more detailed instructions for those who are definitely going to be using a Windows PC or laptop.
• The expo laptop uses Debian with the Cinnamon interface, but pretty much any Linux system works fine.

Chromebooks and Macs

• If using a Mac or Red-Hat-based linux you'll need to work it all out for yourself.
• You can't yet do everything on a phone, but you can do some things. See Android phone page.
• Chromebooks work just like a phone, but with a keyboard, so far as Expo is concerned (mostly). So without some tricky configuration you won't be able to do bulk updates. See our chromebooks page.

Software

You need three kinds of software

1. Encrypted keys to access the server (ssh). None of the rest works without this.
2. Version control software (git) for the loser survex data files, drawings (tunnel, therion) files, website and handbook (expoweb) files.
3. File-transfer tools scp, rsync and sftp (Filezilla) for the files not under version control: photos, documents, scanned logbooks, scanned raw survey notes (surveyscans), scanned cave centre-line plots and everything else. [Never use these on the version-controlled folders.]

If you are only managing photos, GPS tracks or documents then you don't need the version control stuff.

If you are only managing survey data then you probably don't need the file-transfer stuff.

You need to register a key with the expo server to get upload (i.e. read/write) access. Do this first, Without it none of git, scp, ftp or rsync will work. You can do this entirely on your own if you have access to the expo laptop to upload and install the public key generated by your laptop.

Your own Bulk Update laptop

To set up your own laptop for bulk cave data maintenance you need to do this:

1. Register an SSH key with an expo nerd i.e 'get a login'. (see "Key Configuration" below)
2. Install git version control software to download ("clone"), view and edit caving data.
3. Clone two expo repositories loser and drawings so you have the files on your machine. (Use the git reminder for how to do this, e.g. git clone ssh://expo@expo.survex.com:/home/expo/expoweb .
4. If you are also planning on extensive work rewriting parts of the handbook, then you will also need the expo repository expoweb.

For Linux users only:

• GUI for git - There is a choice on Linux but many people just use the command line.
• Your favourite text editor. gedit is installed on the expo laptop and has syntax highlighting for .svx files too. This works with all GtkSourceView-based editors (thanks to Phil Withnall). Download a local copy of the syntax colourizer survex.lang.

Note that on a Debian or Ubuntu machine you should normally install the versions that come with the specific distro (e.g. v11 Bulleye for Debian, or Jammy Jellyfish 22.04 for Ubuntu) i.e. install using 'sudo apt install xxx', not by downloading things from the above sites. So installing everything you need should be as simple as:

• apt install openssh-client survex-aven tunnelx therion git sftp
• ssh-keygen (to generate a key if you haven't already)
• apt install gpsprune qgis firefox-esr gedit tig gitg meld rsync (you may prefer other tools - e.g: elpa-magit is amazing but only if you know how to drive emacs)

For Windows users only:

To install and configure Filezilla on your machine see FileZilla install instructions which will set you up pointing at the correct folder automatically.

Read the Detailed Windows Configuration Instructions for foconfiguring a Windows Bulk Update machine.

QMs and scripts

You may not need a full troggle software development machine if you are only fixing a small script. If you are, you need

• python - maybe - but only if you are working on the scripts in expoweb/noinfo/ such as make-folklist.py or some of the standalone QM scripts - but not on troggle itself - then please use the as close to the same setup as you can to the setup we use for troggle.

The Expo Laptop(s)

[These notes are to remind nerds when configuring or updating a replacement expo laptop.]

The expo laptop is configured as a "Bulk Update" laptop (including al the "Survey laptop" tools), but has a few extras.

• Create a user 'expo'
• Add user 'expo' to the sudoers list or alternatively use the gui to do the same thing.
• Log in as user 'expo' again (so that ~/ = /home/expo/)
• Set up the expo Gmail account in Firefox
• Install Firezilla FTP software and download zilla-uploads.xml to configure it
• Copy ~/.ssh/* from the previous install or from the other expo laptop
• Copy ~/expofiles from from the previous install or from the other expo laptop
• Run rsync to ensure /expofiles is correctly identical to the server
• Configure git as user 'expo laptop' (or 'aziraphale' or 'crowley') - see below, with expo Gmail address
• git clone the four repositories
• Check that you have a GPS trck editor installed, e.g. GPSprune

Configuring git

On a new machine you need to configure your git identity:

git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git config --global pull.rebase true

Cheat lists and quick reminders

• Quick reminders for using git at the command line.
• Quick reminders for using rsync at the command line.

Complementary tools

When maintaining the HTML files in the expo handbook (the :expoweb: git repository) a link-checker is useful to report bad URLs (links to external sites go bad regularly) and to find orphaned pages with no in-links. The website currently (Dec.2024) has 87,493 URLS including all the generated pages.

• Xenu Link Sleuth - install on Windows to do comprehensive link check. Can run this against a locally-running troggle server. You need this 30-thread system running locally on a fast machine to check for bad links if you want to check all ~80,000 links.
• Online brokenlinkcheck tool - online, does not install on your laptop. Limited to 3,000 pages.
• Another online brokenlinkcheck tool - does not install on your laptop. Limited to 2,000 pages but breadth-first rather than depth-first, so it covers a different chunk of the expo website.
• Yet another link-checker, only 1,500 pages checked but oh-so-stylish results presentation. Very informative.
• Linkchecking with wget - command line methods on Linux if you are feeling hardcore.