expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2024-11-22 07:11:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

Brendan's PDF guides (original PPTX files - some - on expofiles/training-info)

Browse Source
This commit is contained in:
Philip Sargent 2020-03-01 15:47:10 +00:00
parent 4ff398c330
commit 7bdc1f5435
7 changed files with 104 additions and 59 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
handbook/computing/logbooks-parsing.html
View File

@ -9,6 +9,9 @@
<h2 id="tophead">CUCC Expedition Handbook</h2>
<h1>Logbooks Import</h1>
<!-- Yes we need some proper context-marking here, breadcrumb trails or something.
Maybe a colour scheme for just this sequence of pages
-->
<h3 id="import">Importing the logbook into troggle</a></h3>

59
handbook/computing/qstart-rsync.html Normal file
View File

@ -0,0 +1,59 @@
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Handbook - Quick rsync</title>
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
</head>
<body>
<h2 id="tophead">CUCC Expedition Handbook - rsync Quick</h2>
<h1>Quick Reminder - rsync</h1>
<h2><em>Manual</em> Version Control using rsync</h2>
<p>This is NOT a tutorial. This is a set of reminders for people who already know all this stuff.
<dl>
<dt>expofiles (all the big files and documents)</dt>
<p>Photos, scans (logbooks, drawn-up cave segments) (This was about
40GB of stuff in 2019 which you probably don't actually need locally).
<p>If you don't need an entire copy of all of it, then it is probably best to use Filezilla/ftp to
copy just a small part of the filesystem to your own machine and to upload the bits you add to or edit.
Instructions for installing and using Filezilla are found in the expo user instructions for
uploading photographs: <a href="../uploading.html">uploading.html</a>.
<p> To sync all
the files from the server to your local expofiles directory on your laptop:</p>
<p><tt>rsync -nazv --delete-after --prune-empty-dirs expo@expo.survex.com:expofiles/ /home/expo/expofiles</tt></p>
<p>To sync the local expofiles directory back to the server after you have edited updates (e.g. scanned some hand-drawn surveys into expofiles/surveyscans/ (but only if your machine runs Linux):</p>
<p><tt>rsync -nazv /home/expo/expofiles/surveyscans/2019/ expo@expo.survex.com/expofiles/surveyscans/2019</tt></p>
then CHECK that the list of files it produces matches the ones you absolutely intend to delete forever! ONLY THEN do it without the "-n" option. "-n" is the same as "--dry-run" which shows you the overwriting changes but doesn't actually do them.
<p>Always
<ul>
<li>do a dry-run of rsync from the server to your laptop immediately before you do an upload to the server
<li>use --delete-after --prune-empty-dirs when downloading, but never when uploading
<li>work at the minimum scope of folders you need, e.g. within expofiles/photos/ or expofiles/surveyscans/ not for the whole of expofiles all at once.
<li>take exagerated care with the placement of the final slash in directory parameters to the rsync. Get it wrong and you duplicate things instead of updating them and it takes ages to sort out.
</ul>
<p>(do be <b>incredibly</b> careful not to delete piles of stuff then rsync back, or to get the directory level of the command wrong - as it'll all get deleted on the server too, and we may not have backups!). It's <b>absolutely vital</b> to use rsync --dry-run --delete-after first to check what would be deleted.
<p>If you are using rsync from a Windows machine you will <em>not</em> get all the files as some filenames are incompatible with Windows. What will happen is that rsync will invisibly change the names as it downloads them from the Linux expo server to your Windows machine, but then it forgets what it has done and tries to re-upload all the renamed files to the server even if you have touched none of them. Now there won't be any problems with simple filenames using all lowercase letters and no funny characters, but we have nothing in place to stop anyone creating such a filename somewhere in that 40GB or of detecting the problem at the time. So don't do it. If you have a Windows machine use Filezilla not rsync.
<p>(We may also have an issue with rsync not using the appropriate user:group attributes for files pushed back to the server. This may not cause any problems, but watch out for it.)</p>
</dl>
<hr />
</ul>
</div>
</body>
</html>

31
handbook/computing/yourlaptop.html
View File

@ -18,8 +18,12 @@
<li><a href="#configuration">Register an SSH key</a> with the expo server (i.e 'get a login').</li>
<li>Download the expo repositories so you have the files on your machine.</li>
</ul>
<p>This is a <em>complete list</em> for doing software development of survex and tunnel (but not troggle) as well as accessing expo data and doing survey data reduction.
The software development stuff should be separated out into separate pages.</p>
<p><code>This is a <em>complete list</em> for doing software development of survex and tunnel
(but not troggle) as well as accessing expo data and doing survey data reduction.
This <b>page needs to be edited</b> so that the software development stuff is separated
out into separate pages. <br /><br />
So that the people who only want to do cave data stuff
don't see a lot of complex stuff they don't need.</code></p>
<h2>Operating Systems</h2>
<ul>
@ -76,17 +80,21 @@ but all the recommended software here is open source (and please don't install p
</ul>
<p>For Windows users only:
<p></p>None of this works until you set up the key-pair setup using PuTty/Pageant.
<ul>
<li>Read our instructions for setting up TortoiseHg in <a href="../tortoise/tortoise-win.htm">Tortoise-on-Windows</a>. (But this won't work at all until you set up the key-pair setup using PuTty/Pageant.)
<li><a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html">PuTTY</a> including pagent. Version 0.73 was released on 2019-09-29. You need this to generate and to use ssh keys on Windows. Otherwise none of git, mercurial, scp, ftp or rsync will work. It includes command line tools ssh, scp (pscp) and sFTP (psftp).
<li><a href="https://gitforwindows.org/">Git for Windows</a>
<li><a href="https://tortoisegit.org/support/faq/#prerequisites">TortoiseGit</a> - GUI interface to git
<li><a href="https://desktop.github.com/">GitHub Desktop for Windows</a> - yes this works with the expo server
<li><a href="https://gitforwindows.org/">Git for Windows</a> - not as simple to use as it looks
<li><a href="https://www.gitkraken.com/">GitKraken</a> - Very pretty GUI interface to git, also Linux version.
Times-out for our server unless you buy it but free for university people.
<li><a href="https://www.python.org/downloads/windows/">python for windows</a>. We are using python 2.7 not 3.8.
<li><a href="https://winscp.net/eng/download.php">WinScp</a> can be used as an alternative to Filezilla if you like.
<li><a href="https://www.java.com/en/">java</a> - needed for GPSprune and CaveConverter. Has to be installed separately on a Windows machine.
<li><a href="https://notepad-plus-plus.org/">Notepad++</a> or any other syntax-highlighting code editor for HTML and python such as <a href="https://www.aptana.com/">Aptana Studio</a>. Configuring these to syntax-highlight .svx files has yet to be done.
<li><a href="https://www.cygwin.com/">Cygwin rsync</a> - not installed as standard on Windows so you need to get Cygwin or <a href="https://serverfault.com/questions/878887/rsync-server-using-windows-subsystem-for-linux">use WSL</a>. But getting WSL to work with ssh keys has been <a href="https://heejune.me/2018/08/02/setup-rsync-server-over-ssh-on-windows-server-2012-easy-way/">tricky</a> and pagent-managed ssh keys are apparently not visible to ssh and rsync in WSL. If you find a good solution for Windows please edit this documentation.
</ul>
<li>Read our instructions for setting up TortoiseHg in
<a href="../tortoise/tortoise-win.htm">Tortoise-on-Windows</a>. To be replaced by git in Sprin 2020.
</ul>
<p>For Android phones:
<ul>
@ -110,6 +118,9 @@ You can do this entirely on your own if you have access to the <i>expo laptop</i
<p>On a Windows machine you will need to configure pageant (the putty authentication agent)
to <a href="https://blog.shvetsov.com/2010/03/making-pageant-automatically-load-keys.html">run at startup to load your key</a>. Note that you are loading your <em>private</em> key, the .ppk file, into pageant and that this key never leaves your laptop.</p>
<p><a href="../../documents/Idiots guide to accessing expo git.pdf">Idiots guide to setting up git for expo</a>
- PDF - Brendan's guide. Uses PuTTy and GitKraken.
<p>The above gets TortoiseHg and the command-line PuTTY tools (ssd, sftp, pscp) running, but doesn't get Cygwin rsync working. You might like to try <a href="https://stackoverflow.com/questions/23517023/rsync-from-windows-to-linux-using-puttys-pagent-authentication">this</a> (untested).</p>
<p>Full illustrated instructions:<img src="https://wiki.filezilla-project.org/favicon.ico" width=64 hspace="20" align="right"></p>
@ -146,10 +157,12 @@ to <a href="https://blog.shvetsov.com/2010/03/making-pageant-automatically-load-
<li><a href="https://bitbucket.org/goatchurch/tunnelx/src/default/">bitbucket.org/goatchurch/tunnelx</a> - documentation and old source code in the bitbucket repository system.
</ul>
<img src="tunnelbanner204.jpg" > <br />
<h3>Cheat lists and quick reminders</h3>
<ul>
<li>Quick <a href="qstart-hg.html">reminders for using mercurial</a> at the command line.
<li>Quick <a href="../manual.html#quickstart">reminders for using rsync and mercurial</a> at the command line.
<li>Quick <a href="qstart-git.html">reminders for using git</a> at the command line.
<li>Quick <a href="qstart-hg.html">reminders for using mercurial</a> at the command line. (to be removed during 2020)
<li>Quick <a href="qstart-rsync.html">reminders for using rsync</a> at the command line.
</ul>
<h3>Logins to external systems</h3>

54
handbook/manual.html
View File

@ -156,64 +156,22 @@ which submits the obligatory comment witht he commit operation.
</ul>
<p>Again, we hope that this issue will go away when we migrate the expoweb repo from mercurial to git before the 2020 Expo.
<h3><a id="quickstart">Quick start</a></h3>
<p>If you know what you are doing here is the basic info on what's where:<br>
(if you don't know what you're doing, skip to <a href="#editingthedata management system">Editing the data management system</a> below.)
<h3><a id="../manual/#editingthedata management system">Editing the data management system</a></h3>
<p>This section is all about how to use mercurial. Since we are changing to git it has been
removed to <a href="computing/qstart-hg.html">a separate place</a>.
<dl>
<dt>expofiles (all the big files and documents)</dt>
<p>Photos, scans (logbooks, drawn-up cave segments) (This was about
40GB of stuff in 2019 which you probably don't actually need locally).
<p>If you don't need an entire copy of all of it, then it is probably best to use Filezilla/ftp to
copy just a small part of the filesystem to your own machine and to upload the bits you add to or edit.
Instructions for installing and using Filezilla are found in the expo user instructions for
uploading photographs: <a href="uploading.html">uploading.html</a>.
<p> To sync all
the files from the server to your local expofiles directory on your laptop:</p>
<p><tt>rsync -nazv --delete-after --prune-empty-dirs expo@expo.survex.com:expofiles/ /home/expo/expofiles</tt></p>
<p>To sync the local expofiles directory back to the server after you have edited updates (e.g. scanned some hand-drawn surveys into expofiles/surveyscans/ (but only if your machine runs Linux):</p>
<p><tt>rsync -nazv /home/expo/expofiles/surveyscans/2019/ expo@expo.survex.com/expofiles/surveyscans/2019</tt></p>
then CHECK that the list of files it produces matches the ones you absolutely intend to delete forever! ONLY THEN do it without the "-n" option. "-n" is the same as "--dry-run" which shows you the overwriting changes but doesn't actually do them.
<p>Always
<ul>
<li>do a dry-run of rsync from the server to your laptop immediately before you do an upload to the server
<li>use --delete-after --prune-empty-dirs when downloading, but never when uploading
<li>work at the minimum scope of folders you need, e.g. within expofiles/photos/ or expofiles/surveyscans/ not for the whole of expofiles all at once.
<li>take exagerated care with the placement of the final slash in directory parameters to the rsync. Get it wrong and you duplicate things instead of updating them and it takes ages to sort out.
</ul>
<p>(do be <b>incredibly</b> careful not to delete piles of stuff then rsync back, or to get the directory level of the command wrong - as it'll all get deleted on the server too, and we may not have backups!). It's <b>absolutely vital</b> to use rsync --dry-run --delete-after first to check what would be deleted.
<p>If you are using rsync from a Windows machine you will <em>not</em> get all the files as some filenames are incompatible with Windows. What will happen is that rsync will invisibly change the names as it downloads them from the Linux expo server to your Windows machine, but then it forgets what it has done and tries to re-upload all the renamed files to the server even if you have touched none of them. Now there won't be any problems with simple filenames using all lowercase letters and no funny characters, but we have nothing in place to stop anyone creating such a filename somewhere in that 40GB or of detecting the problem at the time. So don't do it. If you have a Windows machine use Filezilla not rsync.
<p>(We may also have an issue with rsync not using the appropriate user:group attributes for files pushed back to the server. This may not cause any problems, but watch out for it.)</p>
</dl>
<h3><a id="editingthedata management system">Editing the data management system</a></h3>
<p>To edit the data management system fully, you need to use the disributed version control system
(DVCM) software which is currently mercurial/TortoiseHg.
<p>To edit the data management system fully, you need to use the version control system
software which is currently git and mercurial.
Some (static text) pages can be edited directly on-line using the 'edit this page link' which you'll
see if you are logged into troggle. In general the dynamically-generated pages, such as those describing
caves which are generated from the cave survey data, can not be edited in this way, but forms are provided
for some types of these like 'caves'.</p>
<h3>Cheat lists and quick reminders</h3>
<p>If you know what you are doing here is the basic info on what's where:<br>
(if you don't know what you're doing, skip to <a href="#editingthedata management system">Editing the data management system</a> below.)
<ul>
<li>Quick <a href="computing/qstart-git.html">reminders for using git</a> at the command line.
<li>Quick <a href="computing/qstart-hg.html">reminders for using mercurial</a> at the command line.
<li>Quick <a href="computing/qstart-hg.html">reminders for using mercurial</a> at the command line. (to be removed)
<li>Quick <a href="../manual.html#quickstart">reminders for using rsync</a> at the command line.
</ul>

3
handbook/survey/newcave.html
View File

@ -9,6 +9,9 @@
<body>
<h2 id="tophead">CUCC Expedition Handbook - New Cave</h2>
<h1>Creating a new cave in the online system</h1>
<!-- Yes we need some proper context-marking here, breadcrumb trails or something.
Maybe a colour scheme for just this sequence of pages
-->
<h2>Great, I have discovered a new cave...</h2>
<p>So, you have staggered off the plateau with a fist-full of notes and surveys,

9
handbook/survey/newsurvex.html
View File

@ -14,6 +14,9 @@
<p>If you have not come to this page from the sequence starting at <a href="newcave.html">Starting a New Cave"</a> then go and read that first.
<li>This page outlines the next step of the process. Each part of it is documented separately.
<!-- Yes we need some proper context-marking here, breadcrumb trails or something.
Maybe a colour scheme for just this sequence of pages
-->
</ul>
<h2>Process</h2>
<p>
@ -34,7 +37,7 @@ and tell someone nerdy when you have finished and they will
ensure that it is <em>saved, committed, </em>and<em> pushed</em> appropriately.
<p>If you have several parts of the cave surveyed on one trip, create several distinct .svx files.
<ul>
<li><a href="how_to_make_a_survex_file.pdf">How to make a survex file</a> - PDF
<li><a href="../../documents/survex-guide.pdf">How to create a survex file</a> - PDF - Brendan;s guide.
<li><a href="getin.htm">From muddy book to survex plot</a> - the survex file format (to be revised)
<li><a href="qmentry.html">How to add QM data and cave descriptions</a> - and why this is vital
<li><font color=red>[survex software docm.]</font> <a href="https://survex.com/docs/manual/svxhowto.htm">Contents of .svx files</a> - How do I?
@ -74,7 +77,9 @@ as the scanned notes, i.e. (for wallet #19) you would put them in:
<h3 id="therion">Using tunnel or therion for final survey production</h3>
<p><em>to be documented</em>
<ul>
<li><a href="/expofiles/tunnelwiki/wiki/pages/Tunnel.html">Tunnel tutorial</a> - a wiki of examples and tutorials
<li><a href="../../documents/tunnel-guide.pdf">How to use Tunnel</a> - PDF - Brendan's guide.
<li><a href="../../documents/tunnel-loefflerCP35-only.pdf">Guide to using Tunnel</a> - PDF - David Loeffler's documentation.
<li><a href="/expofiles/tunnelwiki/wiki/pages/Tunnel.html">Tunnel tutorial</a> - a wiki of examples and tutorials
</ul>
<p>Tunnel only produces plan surveys, but they are very pretty.

4
handbook/survey/newwallet.html
View File

@ -15,6 +15,10 @@
<li>This page outlines the next step of the process. Each part of it is documented separately.
</ul>
<!-- Yes we need some proper context-marking here, breadcrumb trails or something.
Maybe a colour scheme for just this sequence of pages
-->
<h2>Process</h2>
<p>