mirror of
https://expo.survex.com/repositories/expoweb/.git/
synced 2024-11-25 08:41:54 +00:00
238 lines
18 KiB
HTML
238 lines
18 KiB
HTML
<!DOCTYPE html>
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
<title>CUCC Expedition Handbook: Programmers manual</title>
|
|
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
|
|
</head>
|
|
<body>
|
|
<h2 id="tophead">CUCC Expedition Handbook - Your laptop</h2>
|
|
|
|
<h1>Setting up a machine for Expo</h1>
|
|
|
|
<h2>Overview</h2>
|
|
|
|
<p>The main things you need to do to get set up are:</p>
|
|
<ul>
|
|
<li>Install <a href="#software">software</a> to access version control and edit/view caving data.</li>
|
|
<li><a href="#configuration">Register an SSH key</a> with the expo server (i.e 'get a login').</li>
|
|
<li>Download the expo <a href="../computing/repos.html">repositories</a> so you have the files on your machine.</li>
|
|
</ul>
|
|
<p>For a quicker, shorter guide to only the most basic setup, see the
|
|
<ul>
|
|
<li><a href="basiclaptop.html">minimal Expo laptop</a> guide.</li>
|
|
</ul>
|
|
first.
|
|
</p>
|
|
<p><code>This is a <em>complete list</em> for doing software development of survex and tunnel
|
|
(but not troggle) as well as accessing expo data and doing survey data reduction.
|
|
This <b>page needs to be edited</b> so that the software development stuff is separated
|
|
out into separate pages. <br /><br />
|
|
So that the people who only want to do cave data stuff
|
|
don't see a lot of complex stuff they don't need.</code></p>
|
|
|
|
<h2>Operating Systems</h2>
|
|
<ul>
|
|
<li>You should be able to use any OS to interact with expo, but it's a bit simpler to get set up using Linux or MacOS, rather than Windows, just because the core stuff will already be installed. You can't yet do everything on a phone, but you can do some things.</li>
|
|
|
|
<li>The <i>expo laptop</i> uses <a href="https://www.debian.org/intro/about">Debian</a> with the <a href="https://computingforgeeks.com/how-to-install-cinnamon-desktop-environment-on-debian/">Cinnamon</a> interface, but pretty much any Linux system works fine. This handbook does assume that you are using apt - the Debian package manager - which is good for all Debian-derived Linuxes such as <a href="https://ubuntu.com/">Ubuntu</a>. Old, slow machines without much memory can be very effective with <a href="https://xubuntu.org/">Xubuntu/xfce</a>. If using a Red-Hat-based linux you'll need to work out the equivalent rpm/yum commands.</li>
|
|
|
|
<li>Windows machines can do almost everything needed (rsync is a problem), but some useful software has no Windows version and you will need to find your own equivalents. There are also some 'gotchas' to look out for due to filesystem differences (e.g letter case). If you have the choice, use Linux.</li>
|
|
|
|
<li><a href="winlaptop.html">A Windows laptop for expo</a> - a page of more detailed instructions for those who are definitely going to be using a Windows PC or laptop.</li>
|
|
|
|
<li>Mac users will need to use the Linux documentation as a guide and work it all out for themselves (and then hopefully update this handbook).</li>
|
|
|
|
<li>Android phone apps can be invaluable too.</li>
|
|
</ul>
|
|
|
|
<h2><a name="software">Software</a></h2><a href="https://survex.com/screenshots.html"><img src="https://survex.com/img/aven-1.2.37-by-date.png" hspace="20" align="right" width="300"></a>
|
|
<p></a>Long-standing Expo policy is to use open tools and protocols so we can retain control of our own data over the long term. And not to require expo-goers to sign up to external services or spend money on software.
|
|
So we use <a href="https://en.wikipedia.org/wiki/Free_and_open-source_software">FOSS software</a>.
|
|
You can use other software on your own machine if it is format-compatible and exports data in the formats we want,
|
|
but all the recommended software here is open source (and please don't install proprietary software on the '<em>expo laptop</em>').</p>
|
|
|
|
|
|
|
|
<p>The list of software:
|
|
<ul>
|
|
<li><a href="https://git-scm.com/">git</a> - distributed version control system - already installed on Linux probably</li>
|
|
<li><a href="fzconfig.html">Filezilla</a> - ftp GUI software with a configuration file to get to the expo server</li>
|
|
<li><a href="https://survex.com/download.html">Survex</a>, including the Aven visualisation tool.
|
|
<li><a href="https://github.com/CaveSurveying/tunnelx">Tunnel</a>: 2.5D cave drawing program based on Survex-compatible data which can also read PocketTopo files. (Generally called 'tunnel' even though the project and executable is actually 'tunnelx'.)
|
|
<li><a href="https://therion.speleo.sk/">Therion</a> - Therion processes survey data and generates maps or 3D models of caves.
|
|
<li><a href="https://bitbucket.org/AndrewA/topparser/src/default/">TopParser</a> - to convert PocketTopo .top files from a DistoX into Therion.
|
|
<li><a href="http://wscc.darkgem.com/caveconverter/">Footleg Cave Converter</a> - to convert PocketTopo .top files from a DistoX into survex (Java)
|
|
<li><a href="https://www.qgis.org/en/site/">QGIS</a> - A Geographic Information System
|
|
<li><a href="https://activityworkshop.net/software/gpsprune/development.html">GPSprune</a> - to edit GPS tracks in GPX files (Java).
|
|
<li><a href="https://sourceforge.net/projects/viking/">Viking</a> - an alternative to GPSprune. The <a href="https://github.com/viking-gps/viking">code</a> is on github.
|
|
<li>python, if you are extending troggle or the scripts in expoweb/noinfo/ - already installed on Linux <img src="https://www.python.org/favicon.ico" width=64 hspace="20" align="right">
|
|
<li>python packages you will want to install: [<em>not yet documented</em>]
|
|
<li><a href="https://github.com/patrickbwarren/qgis3-survex-import">qgis3-survex-import</a> - plugin to import/export survex .3d files</li>
|
|
<li><a href="http://www.thomas-holder.de/projects/inkscape-speleo/extensions/">inkscape-speleo</a> - plugin to import/export survex, therion and pockettopo files to the drawing package inkscape.
|
|
<li><a href="https://imagemagick.org/script/">Imagemagick convert</a> for re-scaling photographs and scanned images and converting from PDF to JPG. (NB you may need to fix this <a href="https://stackoverflow.com/questions/52998331/imagemagick-security-policy-pdf-blocking-conversion">security issue</a>)
|
|
</ul>
|
|
<p>Nearly all our Austrian surveys have beeen produced using Tunnel (or were hand-drawn) but many smaller caves and some areas of SMKsystem are done with Therion because Therion does elevations and Tunnel doesn't.</p>
|
|
|
|
<a href="https://therion.speleo.sk/2D.php"><img src="https://therion.speleo.sk/img/head.jpg" hspace="20"></a>
|
|
|
|
<p>Note that on a Debian/Ubuntu machine you should normally install the versions that come with the distro, not be getting things from the above sites</p>
|
|
|
|
<h4>For Linux users only:</h4>
|
|
<ul>
|
|
<li><a href="https://git-scm.com/download/gui/linux">GUI for git</a> - There is a choice on Linux but many people just use the command line.
|
|
<li>Your favourite text editor. gedit is installed on the <em>expo laptop</em> and has syntax highlighting for .svx files too. This works with <a href="https://survex.com/changes.html">all GtkSourceView-based editors</a> (thanks to Phil Withnall).
|
|
</ul>
|
|
<h4>For Linux and WSL (text mode only) Windows users only:</h4>
|
|
<ul>
|
|
<li>Survex is a package on debian and ubuntu but is not up to date. You need to attach to a ppa to get the latest survex:
|
|
<a href="https://survex.com/download.html?platform=debian">ubuntu and debian</a> download. This is also the version to use on WSL for doing 'cavern' command-line things in WSL (such as doing troggle imports of all the svx data)
|
|
</ul>
|
|
|
|
|
|
<h4>For Windows users only:</h4>
|
|
<p></p>None of this works until you set up the key-pair setup using PuTty/Pageant.
|
|
|
|
<ul>
|
|
<li><a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html">PuTTY</a> including pagent. Version 0.73 was released on 2019-09-29. You need this to generate and to use ssh keys on Windows. Otherwise none of git, scp, ftp or rsync will work. It includes command line tools ssh, scp (pscp) and sFTP (psftp).
|
|
<li><a href="https://desktop.github.com/">GitHub Desktop for Windows</a> - yes this works with the expo server
|
|
<li><a href="">VS Code</a> is a free (but not FOSS) editor with in-built git capability and plug-ins which render git branches graphically. Also available for Linux.
|
|
<li><a href="https://gitforwindows.org/">Git for Windows</a> - not as simple to use as it looks
|
|
<li><a href="https://www.gitkraken.com/">GitKraken</a> - Very pretty GUI interface to git, also Linux version.
|
|
Times-out for our server unless you buy it but free for university people.
|
|
<li><a href="https://www.python.org/downloads/windows/">python for windows</a>. We are moving from python 2.7 to python 3.8 during June 2020.
|
|
<li><a href="https://winscp.net/eng/download.php">WinScp</a> can be used as an alternative to Filezilla if you like.
|
|
<li><a href="https://www.java.com/en/">java</a> - needed for GPSprune and CaveConverter. Has to be installed separately on a Windows machine.
|
|
<li><a href="https://notepad-plus-plus.org/">Notepad++</a> or any other syntax-highlighting code editor for HTML and python such as <a href="https://www.aptana.com/">Aptana Studio</a>. Configuring these to syntax-highlight .svx files has yet to be done.
|
|
</ul>
|
|
|
|
<p>A short note about the phenomenon of VS code is in order. In case you didn't know, by 2018 over <a href="https://insights.stackoverflow.com/survey/2018">a third</a> of software developers used this editor for their Linux and Windows work and it is undoubtedly more now. Not for beginners.
|
|
|
|
<h4>For Android phones:</h4>
|
|
<ul>
|
|
<li><a href="https://www.termius.com/">Termius</a> - command line to create a key for your phone, to ssh (and mosh) login to the expo server. To get sftp you have to buy it.
|
|
<li><a href="https://osmand.net/">OsmAnd</a> - a GPS app. See also <a href="../essentials.html">the expo GPS configuration pages</a>
|
|
<li><a href="http://www.gpsessentials.com/">GPSessentials</a> - another GPS app.
|
|
<li><a href="http://apemap.com/">Ape@Map</a> - another GPS app, compatible with Kompass commerical map sand used by Austrian cavers
|
|
<li><a href="../uploading.html#android">andftp</a> FTP app - needs an <a href="keyexchange.html">ssh key</a> installed on your phone
|
|
<li><a href="ftpusage.html">FTP using Cx_File_Explorer</a> (this is probably out of date)
|
|
<li><a href="https://sites.google.com/site/speleoapps/home">TopoDroid</a> makes cave surveys with the DistoX.
|
|
<li><a href="https://sites.google.com/site/speleoapps/home">Cave3D</a> is a Therion 3D viewer.
|
|
<li><a href="https://sites.google.com/site/speleoapps/home">ThManager</a> organizes single surveys, exported by TopoDroid, into Therion projects encoded by Therion "thconfig" files.
|
|
</ul>
|
|
|
|
|
|
<h2><a name="configuration">Configuration</a></h2>
|
|
|
|
<p>You need to <a href="keyexchange.html">register a key with the expo server</a> to get upload (i.e. read/write) access. Do this first, Without it none of git, scp, ftp or rsync will work.
|
|
You can do this entirely on your own if you have access to the <i>expo laptop</i> to upload and install the public key generated by your laptop.</p>
|
|
|
|
<p>On a Windows machine you will need to configure pageant (the putty authentication agent)
|
|
to <a href="https://blog.shvetsov.com/2010/03/making-pageant-automatically-load-keys.html">run at startup to load your key</a>. Note that you are loading your <em>private</em> key, the .ppk file, into pageant and that this key never leaves your laptop.</p>
|
|
|
|
<p><a href="../../documents/Idiots guide to accessing expo git.pdf">Idiots guide to setting up git for expo</a>
|
|
- PDF - Brendan's guide. Uses PuTTy and GitKraken.
|
|
|
|
<p>The above gets the command-line PuTTY tools (ssd, sftp, pscp) running, but doesn't get rsync working. You might like to try <a href="https://stackoverflow.com/questions/23517023/rsync-from-windows-to-linux-using-puttys-pagent-authentication">this</a> (untested).</p>
|
|
|
|
<p>Full illustrated instructions:<img src="https://wiki.filezilla-project.org/favicon.ico" width=64 hspace="20" align="right"></p>
|
|
|
|
<ul>
|
|
|
|
<li><a href="../putty/putty.html">Installing PuTTy on Windows</a>.</li>
|
|
|
|
<li><a href="fzconfig.html">Installing Filezilla</a>.</li>
|
|
|
|
</ul>
|
|
<p>When using Windows please, please be <a href="http://expo.survex.com/handbook/survey/getin.htm#filenames">excessively careful when naming files and survex names</a> and be <a href="manual.html#quickstart">exceptionally careful when using rsync</a>.
|
|
|
|
<p>The handbook has documents where it is necessary to use scp or sftp to manage large files in 'expofiles'.
|
|
See <a href="../upload-expert.html">Experts: Uploading files</a>,
|
|
<a href="../uploading.html">Uploading files</a> and <a href="../gpxupload.html">Uploading GPS tracks</a>.
|
|
Only machines which have done the key-pair setup process can do scp, sftp or rsync.
|
|
</p>
|
|
|
|
<h4>Cheap hack for Linux</h4>
|
|
<p>On a Linux machine, it will not be necessary once you have everything set-up correctly, but
|
|
a useful initial short-cut is to create 'mimic' symbolic links to directories to match those on the
|
|
server and the <em>expo laptop</em>.
|
|
(You should not rely on these as other people will have set up their machines differently.)
|
|
So, assuming that your machine has
|
|
no user <var>expo</var>:<pre><code>cd /home
|
|
sudo mkdir expo
|
|
cd expo
|
|
sudo ln -s /mnt/f/expofiles expofiles
|
|
sudo ln -s /mnt/d/CUCC-Expo/expoweb expoweb
|
|
sudo ln -s /mnt/d/CUCC-Expo/troggle troggle
|
|
sudo ln -s /mnt/d/CUCC-Expo/loser loser
|
|
ls -tlA
|
|
</code></pre>
|
|
where obviously you will be using your own locations for expofiles, expoweb, loser etc. instead of <var>/mnt/d/CUCC...</var> etc. This creates valid directory paths for, e.g. <var>/home/expo/loser</var> etc.
|
|
</p>
|
|
|
|
|
|
|
|
|
|
|
|
<h3>Learning how to use this software</h3>
|
|
|
|
<ul>
|
|
<li>Using Survex, Tunnel and Therion, see the <a href="http://expo.survex.com/handbook/survey/">Expo Handbook - Surveying section</a>.
|
|
|
|
<li><em>For installing Survex, Tunnel etc. see <a href="getsurvex.html">this page</a> which may be merged in here eventually.</em>
|
|
|
|
<li><a href="../../documents/tunnel-loefflerCP35-only.pdf">Introduction to using Tunnel</a> - article by Dave Loeffler. This is really good and should be read first.
|
|
|
|
<li><a href="http://expo.survex.com/expofiles/tunnelwiki/wiki/pages/Tunnel.html">The Tunnel tutorial</a> - installation notes and a wiki of examples and tutorials
|
|
<li><a href="https://github.com/CaveSurveying/tunnelx">Tunnel - master copy since August 2019</a>. This is where the software is actively developed and is the most up to date copy, but the documentation is not as complete as in the other locations.
|
|
<li><a href="https://bitbucket.org/goatchurch/tunnelx/src/default/">bitbucket.org/goatchurch/tunnelx</a> - documentation and old source code in the bitbucket repository system.
|
|
</ul>
|
|
<img src="tunnelbanner204.jpg" > <br />
|
|
|
|
<h3>Cheat lists and quick reminders</h3>
|
|
<ul>
|
|
<li>Quick <a href="qstart-git.html">reminders for using git</a> at the command line.
|
|
<li>Quick <a href="qstart-rsync.html">reminders for using rsync</a> at the command line.
|
|
</ul>
|
|
|
|
<h3>Logins to external systems</h3>
|
|
|
|
<p>
|
|
<a href="https://webchat.freenode.net/#expo">#expo</a> - public. An open-access <a href="https://en.wikipedia.org/wiki/Internet_Relay_Chat">
|
|
open-access IRC channel</a>
|
|
(ephemeral, not archived) for real-time discussions about everything but mostly software people. If you are having trouble using the software try here first.
|
|
</p>
|
|
<ul>
|
|
<li><a href="https://github.com/join">Github</a> - You'll need an account if you want to use this.
|
|
<li><a href="https://github.com/CaveSurveying/CUCCexposurveyissues/issues">Expo issues list</a> - Issues with data reduction for recent expos and software updates for tunnel
|
|
<li><a href="https://bitbucket.org/product/">Bitbucket</a> - create an account to help develop tunnel. <img src="https://tortoisehg.bitbucket.io/img/bitbucket-icon.png" width=64 hspace="20" align="right">
|
|
<li><a href="https://trac.survex.com/wiki">Trac issue tracker and wiki</a> for developing survex. You'll need a login to file bugs there.
|
|
</ul>
|
|
|
|
<p>Some expo survey stuff is tracked externally:
|
|
<ul>
|
|
<li><a href="https://github.com/CaveSurveying/CUCCexposurveyissues/issues">CUCC CaveSurveying Bug lists and open issues discussed on github</a> -
|
|
issue list so you will need to subscribe to the Cave Surveying Group on github to participate.
|
|
|
|
<li><a href="https://github.com/CaveSurveying/CUCCexposurveyissues/wiki/Expo-tunnel-workflows">Expo-tunnel-workflows</a> -
|
|
a wiki on github discussing workflows to generate centerlines, GEOTIFF and QGIS integration.
|
|
</ul>
|
|
|
|
|
|
<h3>Complementary tools</h3>
|
|
<p>When maintaining the HTML files in the expo handbook a link-checker is useful to report bad URLs (links to external sites go bad regularly) and to find orphaned pages with no in-links. The website has <a href="https://www.klebos.net/subdomains/keldos/LinkScans/TUNNEL-wiki-link-report.html">about 2,000 internal URLs</a> in just the Tunnel wiki section alone.
|
|
<ul>
|
|
<li><a href="https://www.brokenlinkcheck.com/">Online brokenlinkcheck tool</a> - does not install on your laptop. Limited to 3,000 pages.
|
|
<li><a href="https://www.deadlinkchecker.com/website-dead-link-checker.asp">Another online brokenlinkcheck tool</a> - does not install on your laptop. Limited to 2,000 pages but breadth-first rather than depth-first, so it covers a different chunk of the expo website.
|
|
<li><a href="https://www.drlinkcheck.com/account/subscriptions/1/projects/1/overview">Yet another link-checker</a>, only 1,500 pages checked but oh-so-stylish results presentation. Very informative.
|
|
<li><a href="http://home.snafu.de/tilman/xenulink.html">Xenu Link Sleuth</a> - install on Windows to do comprehensive link check.
|
|
<li><a href="https://www.digitalocean.com/community/tutorials/how-to-find-broken-links-on-your-website-using-wget-on-debian-7">Linkchecking with wget</a> - command line methods on Linux
|
|
</ul>
|
|
|
|
<a href="http://wscc.darkgem.com/caveconverter/"><img src="CaveConverterWorkflow.png" width=600 hspace="20"></a>
|
|
|
|
<hr />
|
|
|
|
</body>
|
|
</html>
|