From 458a9b15ea83de3cfbdcde2ade75a16fa08cd6da Mon Sep 17 00:00:00 2001
From: Philip Sargent <philip.sargent@gmail.com>
Date: Tue, 10 Dec 2019 19:52:49 +0100
Subject: [PATCH] updating rsync commands in manual

---
 handbook/manual.html | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/handbook/manual.html b/handbook/manual.html
index 8ae0be73f..667205ad5 100644
--- a/handbook/manual.html
+++ b/handbook/manual.html
@@ -104,7 +104,7 @@ below for details on that.</p>
 <p>Examples of troggle-generated pages from data:
 <ul>
 <li><a href="http://expo.survex.com/caves">expo.survex.com/caves</a> - list of caves surveyed and links to guidebook descriptions
-<li>< href="http://expo.survex.com/pubs.htm">http://expo.survex.com/pubs.htm</a> -  reports, accounts and logbooks
+<li><a href="http://expo.survex.com/pubs.htm">http://expo.survex.com/pubs.htm</a> -  reports, accounts and logbooks
 <li><a href="http://expo.survex.com/expedition/2018">expo.survex.com/expedition/2018</a> - Members on expo 2018: . Scroll down for a list of all the data typed in from survey trips.
 <li><a href="http://expo.survex.com/survexfile/caves/">expo.survex.com/survexfile/caves/</a> - List of caves with all the surveys done for each.
 <li><a href="http://expo.survex.com/survexfile/caves-1623/115/cucc/futility.svx">expo.survex.com/survexfile/caves-1623/115/cucc/futility.svx</a> - Cave survey data from 1983 in Schnellzughohle. 
@@ -165,25 +165,35 @@ removed to <a href="computing/qstart-hg.html">a separate place</a>.
     <dt>expofiles (all the big files and documents)</dt>
 
 <p>Photos, scans (logbooks, drawn-up cave segments) (This was about
-40GB of stuff in 2017 which you probably don't actually need locally).
+40GB of stuff in 2019 which you probably don't actually need locally).
 <p>If you don't need an entire copy of all of it, then it is probably best to use Filezilla/ftp to 
 copy just a small part of the filesystem to your own machine and to upload the bits you add to or edit.
 Instructions for installing and using Filezilla are found in the expo user instructions for 
 uploading photographs: <a href="uploading.html">uploading.html</a>.
 
 <p> To sync all
-the files from the server to local expofiles directory:</p>
+the files from the server to your local expofiles directory on your laptop:</p>
 
-<p><tt>rsync -av expo@expo.survex.com:expofiles /home/expo</tt></p>
+<p><tt>rsync -nazv --delete-after --prune-empty-dirs expo@expo.survex.com:expofiles/ /home/expo/expofiles</tt></p>
 
-<p>To sync the local expofiles directory back to the server (but only if your machine runs Linux):</p>
-<p><tt>rsync --dry-run --delete-after -a /home/expo/expofiles expo@expo.survex.com</tt></p>
-then CHECK that the list of files it produces matches the ones you absolutely intend to delete forever! ONLY THEN do:
-<p><tt>rsync -av /home/expo/expofiles expo@expo.survex.com:</tt></p>
+<p>To sync the local expofiles directory back to the server after you have edited updates (e.g. scanned some hand-drawn surveys into expofiles/surveyscans/ (but only if your machine runs Linux):</p>
 
-<p>(do be <b>incredibly</b> careful not to delete piles of stuff then rsync back, or to get the directory level of the command wrong - as it'll all get deleted on the server too, and we may not have backups!). It's <b>absolutely vital</b>Use rsync --dry-run --delete-after -a first to check what would be deleted. 
+<p><tt>rsync -nazv  /home/expo/expofiles/surveyscans/2019/ expo@expo.survex.com/expofiles/surveyscans/2019</tt></p>
 
-<p>If you are using rsync from a Windows machine you will <em>not</em> get all the files as some filenames are incompatible with Windows. What will happen is that rsync will invisibly change the names as it downloads them from the Linux expo server to your Windows machine, but then it forgets what it has done and tries to re-upload all the renamed files to the server even if you have touched none of them. Now there won't be any problems with simple filenames using all lowercase letters and no funny characters, but we have nothing in place to stop anyone creating such a filename somewhere in that 60GB or of detecting the problem at the time. So don't do it. If you have a Windows machine use Filezilla not rsync.
+then CHECK that the list of files it produces matches the ones you absolutely intend to delete forever! ONLY THEN do it without the "-n" option. "-n" is the same as "--dry-run" which shows you the overwriting changes but doesn't actually do them.
+
+<p>Always
+<ul>
+
+<li>do a dry-run of rsync from the server to your laptop immediately before you do an upload to the server
+<li>use --delete-after --prune-empty-dirs when downloading, but never when uploading
+<li>work at the minimum scope of folders you need, e.g. within expofiles/photos/ or expofiles/surveyscans/ not for the whole of expofiles all at once.
+<li>take exagerated care with the placement of the final slash in directory parameters to the rsync. Get it wrong and you duplicate things instead of updating them and it takes ages to sort out.
+</ul>
+
+<p>(do be <b>incredibly</b> careful not to delete piles of stuff then rsync back, or to get the directory level of the command wrong - as it'll all get deleted on the server too, and we may not have backups!). It's <b>absolutely vital</b> to use rsync --dry-run --delete-after  first to check what would be deleted. 
+
+<p>If you are using rsync from a Windows machine you will <em>not</em> get all the files as some filenames are incompatible with Windows. What will happen is that rsync will invisibly change the names as it downloads them from the Linux expo server to your Windows machine, but then it forgets what it has done and tries to re-upload all the renamed files to the server even if you have touched none of them. Now there won't be any problems with simple filenames using all lowercase letters and no funny characters, but we have nothing in place to stop anyone creating such a filename somewhere in that 40GB or of detecting the problem at the time. So don't do it. If you have a Windows machine use Filezilla not rsync.
 
 <p>(We may also have an issue with rsync not using the appropriate user:group  attributes for files pushed back to the server. This may not cause any problems, but watch out for it.)</p>
 </dl>