expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2024-11-25 08:41:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fixing FTP instructions as sfpt and scp no longer work for generic users

Browse Source
This commit is contained in:
Philip Sargent 2019-07-11 16:25:19 +02:00
parent 9148437530
commit 595125ea3c
5 changed files with 218 additions and 74 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
handbook/fzconfig.html
View File

@ -7,6 +7,17 @@
<body> <body>
<h2 id="tophead">CUCC Expedition Handbook</h2> <h2 id="tophead">CUCC Expedition Handbook</h2>
<h1>Configuring Filezilla to upload photos etc.</h1> <h1>Configuring Filezilla to upload photos etc.</h1>
<h3 id="install">Installing Filezilla</h3>
<p>This software works identically on both Windows and Linux.
<p>Filezilla is an "FTP client". This means that it connects to servers using a venerable service called "file transfer protocol" i.e. FTP. It looks a bit like copying files from one folder to another on your desktop but it works between different machines.
<ul>
<li> Download the software from here <a href="https://filezilla-project.org/download.php?show_all=1">Filezilla Downloads</a>. ( Obviously Linux users will use their usual package management system instead of doing this download.)
<li>Now install the software following <a href="https://wiki.filezilla-project.org/Client_Installation">the instructions here</a>.
<li>Now configure it to connect to the expo server using the instructions below.
</ul>
<p>Before continuing, make sure that you have the "cavey:beery" password for the expo server. You will need it. <p>Before continuing, make sure that you have the "cavey:beery" password for the expo server. You will need it.
<p>Having installed Filezilla using the <a href="uploading.html#install">installation instructions</a>, you now need to import a configuration file to set it up to use with the public expo server. <p>Having installed Filezilla using the <a href="uploading.html#install">installation instructions</a>, you now need to import a configuration file to set it up to use with the public expo server.
@ -36,7 +47,7 @@ may try to delete things on the server which you don't have on your laptop, i.e.
This setting should be fixed automatically in <b>zilla-uploads.xml</b> but we have seen occasions where it This setting should be fixed automatically in <b>zilla-uploads.xml</b> but we have seen occasions where it
seems to have been re-set somehow. seems to have been re-set somehow.
</ul> </ul>
<li>Now you are in and can copy and move files anywhere. But please don't: stick to copying files from your machine (the left window) to the server (the right window):<br><br> <li>Now you are in and can copy and move files anywhere. But please stick to copying files from your machine (the left window) to the server (the right window) into the /uploads/, /photos/ or /gpslogs/ folders only:<br><br>
<a href="fzexample.png" border=2><img src="fzexample.png" align=center width=65%></a><br>(Click to enlarge). <a href="fzexample.png" border=2><img src="fzexample.png" align=center width=65%></a><br>(Click to enlarge).
</ul> </ul>

32
handbook/keyexchange.html Normal file
View File

@ -0,0 +1,32 @@
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>Handbook placeholder page</title>
<link rel="stylesheet" type="text/css" href="../css/main2.css" />
</head>
<body>
<h2 id="tophead">CUCC Expedition Handbook</h2>
<h1>Placeholder</h1>
<p>This is not the page you are looking for.
<p>This will be replaced with the information you want as soon as someone gets around to writing it. Why not find out how to do this yourself ?
<hr />
<div id="menu">
<ul id="links">
<li><a href="index.htm">Expedition Handbook</a>
<ul>
<li><a href="rig/rigit.html">Rigging guide</a></li>
<li><a href="survey/index.htm">Surveying guide</a></li>
<li><a href="look4.htm">Prospecting guide</a></li>
<li><a href="rescue.htm">Rescue guide</a></li>
</ul></li>
<li><a href="../infodx.htm">Main index</a></li>
<li><a href="../index.htm">Expo Home</a></li>
<li><a href="../../index.htm">CUCC Home</a></li>
</ul>
</div>
</body>
</html>

80
handbook/upload-expert.html Normal file
View File

@ -0,0 +1,80 @@
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>CUCC Expedition Handbook: Uploading files/photos</title>
<link rel="stylesheet" type="text/css" href="../css/main2.css" />
</head>
<body>
<h2 id="tophead">CUCC Expedition Handbook</h2>
<h1>Uploading Photos and files - Expert Use</h1>
<h2>Experts only</h2>
<p>If you have been using FTP since the last century or are particularly keen on doing everything using the command-line, read on.
<p>With the new server in 2019 it is harder to configure scp or sftp (or ssh or rsync) because the server needs a key exchange
setting up first. So while scp (once working) is now
more secure than it was, we are forced to allow ordinary ftp to allow experienced but non-key-exchanged expoers' laptops or phones to upload photos
to the server. This is worse than it was as the password is now sent in clear.
<p>So it is important that the ordinary ftp access is restricted to the /uploads/ folder tree.
See <a href="keyexchange.html">key exchange instructions</a> for how to arrange with an admin for this to be done for your device.
<h3>Using scp - requires key exchange for the device</h3>
<p>Works on Windows (using winscp), Linux (using scp), and no doubt
mac and android with other tools. If you have Windows 10 and <a href="https://msdn.microsoft.com/en-gb/commandline/wsl/about">have installed bash</a>, then you can use scp.</p>
<p>If you don't have winscp installed you can get it from here:
<a href="http://winscp.net/eng/index.php">http://winscp.net/eng/</a>.</p>
<p>quick start guide:
<a href="http://winscp.net/eng/docs/getting_started">http://winscp.net/eng/docs/getting_started</a></p>
<p>screenshots:
<a href="http://winscp.net/eng/docs/screenshots">http://winscp.net/eng/docs/screenshots</a></p>
<!-- WedDAV no longer works because the effective user is "apache" and the permissions
for the folder /uploads/ no longer allow it to write anything because we had to change it to allow Filezilla
to work.Philip & Wookey 11 August 2018
<h3>Using WebDAV - cadaver</h3>
<p>Command-line people can use the 'cadaver' client which is even
available for windows too:
<a href="http://www.phtagr.org/2009/04/01/cadaver-for-windows/">www.phtagr.org/2009/04/01/cadaver-for-windows/</a></p>
-->
<p>scp gives you an 'explorer-like' interface (although winscp can
give you a norton-commander-style 2-pane UI as well).</p>
<p>rewite this bit...
<ol>
<li>Download and install <b>Filezilla</b>.
<li>Or learn to use <b><a href="https://www.computerhope.com/unix/scp.htm">scp</a></b>.
<li>Or on a Windows machine try Winscp.
<li>Or if you are using an Android mobile phone, follow <a href="computing/ftpusage.html">Radost's instructions</a> which also cover Winscp briefly.
</ol>
<h3>Expo laptops</h3>
<p>The <i>expo laptop</i> has got the key exchange set up on it so it is configured for Filezilla to use sftp not ftp but this is invisible to normal use.
<h3>Using WebDAV</h3>
This no longer works as we had to change the folder permissions for /uploads/. Sorry.
<h3>Using rsync</h3>
<p>No, don't use rsync for this. Really don't. It's too liable to delete everything or to overwrite files which are not changed at all because of the incompatibilities between Linux and Windows filename conventions (uppercase and lowercase are automagically converted and rsync gets it wrong).
<h3>Regenerating photos albums</h3>
<p>ssh in to the server, cd to /expofiles/ and
<pre>
make photos
</pre>
will regenerate it all using the installation of bins on the server
using the config in /home/expo/config/bins/binsrc
<hr />
</body>
</html>

154
handbook/uploading.html
View File

@ -9,66 +9,62 @@
<h1>Uploading Photos</h1> <h1>Uploading Photos</h1>
<h2>The end-result you are trying to achieve</h2> <h2>The end-result you are trying to achieve</h2>
What you are trying to do is to get your happy holiday snaps appear properly indexed with all the others from the previous decades of expo history. You can see them all here: <br /> <a What you are trying to do is to get your happy holiday snaps appear <i>properly indexed</i> with all the others from the previous decades of expo history. You can see them all here: <br /> <a
href="http://expo.survex.com/photos/">http://expo.survex.com/photos/</a><br /> href="http://expo.survex.com/photos/">http://expo.survex.com/photos/</a><br />
which is the end result. But all you have to do is to upload the photos to the right place. which is the end result. But all you have to do is to upload the photos to the right place.
A hidden script does the hard work to make it all look nice after the upload. A hidden script does the hard work to make it all look nice after the upload.
<p>If you are a beginner you can use the initial simple method below (using /uploads/ ) <p>If you are a beginner you can use the initial simple method below (using /uploads/ )
for the photos you have taken of cave entrances for cave survey and prospecting purposes. But please for the photos you have taken of cave entrances for cave survey and prospecting purposes.
rename the filenames of the photos intelligently, e.g. "big-hole-near-path-to-fgh.jpg", or
"2018-ad-07-entrance3.jpg" (rather than "DSC31415926.jpg"), and explain to an admin/nerd what you have done.
Please use lower-case for all filenames.
<p>If you are looking for how to upload a GPS track, those instructions have <a href="gpxupload.html">been moved to here</a>. <p>If you are looking for how to upload a GPS track, those instructions have <a href="gpxupload.html">been moved to here</a>.
<h3>Names for your photo files and folders</h3>
<p>Please use lower-case for all filenames.
<p>Please
rename the filenames of the photos intelligently to e.g. "big-hole-near-path-to-fgh.jpg", or
"2018-ad-07-entrance3.jpg" rather than "DSC31415926.jpg" or whatever your camera app produces. Please explain by email to an admin/nerd what you have done, where you have put them and how you have renamed files.
<p>Please don't upload lots of near-duplicate photos: cull them first to just the good ones. We don't want any that are out of focus either.
<p>When copying many files put them into a new folder which identifies you personally.
When you copy the files create your own folder with your name like this: <pre>/YourName/</pre> and put your files into that folder. "YourName" should have no spaces, hyphens or underline characters (CamelCase).
<h2>Simple instructions</h2> <h2>Simple instructions</h2>
<ol> <ol>
<li>Email a photo or two to someone who knows how to do it. <br> <li>If you only have a photo or two just email them to someone who knows how to do it. <br>
(If you are doing more than a few photos, email will be clunky, so use another method). If you are doing more than a few photos, email will be clunky, so do this:
<li>Use the Expo laptop in the tatty hut. Get someone to show you how to use FTP. <li>Copy the photo files from your phone to the <i>expo laptop</i> in the tatty hut using a USB cable.<br> See <a href="#android">phone app</a> if you have problems.
<li>Rename the filenames of the photos to something descriptive (see para above).
<li>After copying your files to the <i>expo laptop</i> into the /uploads/ folder tell a nerd that you have done it.
</ol> </ol>
<p>Beginners should always put all their files into the folder <b>/home/expo/expofiles/uploads/</b> and ask an admin to move them to the right place. <p><b>Complete beginners</b> should always put all their files into their own folder within the folder <b>/home/expo/expofiles/uploads/</b> on the <a href="computer.html"><i>expo laptop</i></a> and ask an experienced user to move them to the right place later.
<p>Now go to <a href="#init">using Filezilla initially</a> - still using the Expo laptop.
<h2 id="morecomplex">More complex instructions</h2> <p>If you want to use your photos in a blog (see below) tell an experienced user which and they will copy a lower-resolution version to the appropriate place.
<p>Using your own laptop on expo, or after you return from expo:
<ol>
<li>Download and install <b>Filezilla</b>.
<li>Or learn to use <b><a href="https://www.computerhope.com/unix/scp.htm">scp</a></b>.
<li>Or on a Windows machine try Winscp.
<li>Or if you are using an Android mobile phone, follow <a href="computing/ftpusage.html">Radost's instructions</a> which also cover Winscp briefly.
</ol>
<p>You do need to know the expo password.</p> <p>That's it, you're done.
<h3 id="install">Installing Filezilla</h3> <h2>Using uploaded photos in blogs</h2>
<p>This software works identically on both Windows and Linux. <p>The <a href="https://ukcaving.com/board/index.php?topic=25195.msg310908#msg310908">UK Caving blog</a> requires that
<p>Filezilla is an "FTP client". This means that it connects to servers using a venerable service called "file transfer protocol" i.e. FTP. It looks a bit like copying files from one folder to another on your desktop but it works between different machines. photos in it are actually hosted elsewhere. This needs to be a permanent place so
<ul> that the blogs can be read in years to come. Imgur, Flikr or Instagram are OK though not ideal (expo has its own Instagram account) but
<li> Download the software from here <a href="https://filezilla-project.org/download.php?show_all=1">Filezilla Downloads</a>. ( Obviously Linux users will use their usual package management system instead of doing this download.) your personal GoogleDocs or GooglePhotos stash definitely is not.
<li>Now install the software following <a href="https://wiki.filezilla-project.org/Client_Installation">the instructions here</a>. Preferably
<li>Now configure it to connect to the expo server using the instructions <a href="fzconfig.html">on this expo handbook page</a> they should be put on the expo server e.g. 2018 blog photos are permanently archived on <a href="http://expo.survex.com/expofiles/photos/2018/BlogPostPhotos/">expo.survex.com<b>/expofiles/</b>photos/2018/BlogPostPhotos/</a> in the published resolution.
</ul>
<h2 id="init">Using Filezilla initially</h2>
<p>The expo website has a big section under 'home/expo/expofiles/' that is <b>not under version control</b>. This is dangerous as there is no backup. If you overwrite some important files with your holiday snaps then we are in big trouble. This is where we store big files that we don't want to keep multiple versions of which is why it is not under verson control.
<p>So beginners should always put all their files into the folder
<pre>/home/expo/expofiles/uploads/</pre> and then ask an admin to move them to the right place. The configuration which you just did (if you followed <a href="fzconfig.html">the instructions</a>) will set you up pointing at the correct folder automatically.
<p>
To make the admin's life easier, create your own folder in <pre>/home/expo/expofiles/uploads/</pre> with your name like this: <pre>/home/expo/expofiles/uploads/YourName/</pre> and put your files into that folder. ("YourName" should have no spaces, hyphens or underline characters.) So that the admin knpows who is responsible. And for goodness sake please don't upload lots of duplicate photos: cull them first to just the good ones.
<h2 id="experienced">Experienced users</h2> <h2 id="experienced">Experienced users</h2>
<p>OK the admin/nerd you have been asking to move the files to the right place is getting fed up and now wants you to put the uploaded photos in the right place yourself. This is where they go: <p>OK the admin/nerd you have been asking to move the files to the right place on the <i>expo laptop</i> is getting fed up and now wants you to put the uploaded photos in the right place yourself and not just in /uploads/. This is where they go:
<pre> <pre>
directory: /home/expo/expofiles/<b>photos</b>/2018/YourName/ directory: /home/expo/expofiles/<b>photos</b>/2019/YourName/
</pre> </pre>
or, for GPS logs (GPX files): or, for GPS logs (GPX files):
<pre> <pre>
directory: /home/expo/expofiles/<b>gpslogs</b>/2018/YourName/ directory: /home/expo/expofiles/<b>gpslogs</b>/2019/YourName/
</pre> </pre>
<p>Obviously replace /2019/ with the current year. This is the expo year the photos are taken, so if you are uploading a previous years' photos which you never got around to uploading you will use /2018/ or whatever.
<p>Obviously replace 'YourName' with your actual name (no spaces!). <p>Obviously replace 'YourName' with your actual name (no spaces!).
It is important that you get this right as this specific way of writing It is important that you get this right as this specific way of writing
your name is standardised across the website your name is standardised across the website
@ -79,45 +75,69 @@ only lower-case letters, no spaces, and hyphens (not underlines) when creating f
<p> <p>
Use the year that the photo was taken or the GPStrack logged.</p> Use the year that the photo was taken or the GPStrack logged.</p>
<h3 id="init">Uploading files from the <i>expo laptop</i></h3>
<p>Always learn how to <i>use</i> the system first on the <i>expo laptop</i> before trying to do it with your own machine.
<p>OK the files have been put into the correct place on the laptop by an experienced user and now you need to copy them to the server. Don't worry about deleting them from the <i>expo laptop</i> after copying as we can do that clean-up at the end of expo. In fact <i>don't</i> delete them from /uploads/ on the <i>expo laptop</i>.
<p>The expo server has a big section under 'home/expo/expofiles/' that is <b>not under version control</b>. This is dangerous as there is no backup. If you overwrite some important files with holiday snaps then we are in big trouble.
<p>This is where we store big files that we don't want to keep multiple versions of which is why it is not under verson control.
<p>This is why we do not want beginners directly uploading photos to the server but to just put them on the <i>expo laptop</i> instead.
<p>So assuming that you have the photos on the laptop in directory:
<pre>
/home/expo/expofiles/photos/2019/YourName/
</pre>
you are going to be copying them to directory:
<pre>
/home/expo/expofiles/photos/2019/YourName/
</pre>
on the server.
<p>
<ul>
<li>Start up Filezilla - click on red "Fz" logo in the application favourites bar on the laptop desktop.
<li>Click on the "Bookmarks" menu item:
<ul>
<li>At the bottom of the "Bookmarks" drop-down menu you may see "expo-uploads";
if so, click on it.
<li>If you can't see "expo-uploads" in that menu
(because this is temporarily broken in version 3.28 of Filezilla),
click on the "File" menu item and select "Site Manager...".
This will display a tree-structured menu in a sub-window and one of the items will be "expo-uploads". Highlight it
and then click on the "Connect" button at the bottom of the sub-window.
</ul>
<li>If this is the first time it is used this expo you will now be prompted for the password for the expo server. This is the "cavey:beery" one which we never write down or write in emails. Get it verbally or by phone or secure text message from another expoer.
</ul>
<p>Now you are in and can copy and move files anywhere. But please stick to copying files from your machine (the left window) to the server (the right window) into the /uploads/, /photos/ or /gpslogs/ folders only:<br><br>
<a href="fzexample.png" border=2><img src="fzexample.png" align=center width=65%></a><br>(Click to enlarge).
<p>For installing and pre-configuring Filezilla on a new machine see <a href="fzconfig.html">FileZilla install instructions</a> which will set you up pointing at the correct folder automatically.
<h3>Automated magic</h3>
<p>Note that uploading photos does not automatically update the view <p>Note that uploading photos does not automatically update the view
at <a href="http://expo.survex.com/photos/">http://expo.survex.com/photos/</a> immediately. An update script needs to be run. This should run automatically once/day around midnight UTC (2017 and earlier) or a couple of minutes after you do the upload to the right place (2018 if Wookey gets this sorted out in time) but may be broken. Prod a web admin if nothing is updated by the next morning..</p> at <a href="http://expo.survex.com/photos/">http://expo.survex.com/photos/</a> immediately. An update script needs to be run. This should run automatically once/day around midnight UTC (2017 and earlier) or a couple of minutes after you do the upload to the right place (2018 if Wookey gets this sorted out in time) but may be broken. Prod a web admin if nothing is updated by the next morning..</p>
<h2>Experts only</h2>
<p>If you have been using FTP since the last century or are particularly keen on doing everything using the command-line, read on. <h2>Expert instructions</h2>
<p>These have been moved to a separate page: <a href="upload-expert.html">expert upload instructions</a>.
<h3>Using scp</h3> <h2 id="morecomplex">Using your own laptop</h2>
<p>To use your own laptop on expo, or after you return from expo, you need need to use FTP. So become an <a href="#experienced">experienced user</a> first.
<p>Works on Windows (using winscp), Linux (using scp), and no doubt <p>You will need to know the expo password.</p>
mac and android with other tools. If you have Windows 10 and <a href="https://msdn.microsoft.com/en-gb/commandline/wsl/about">have installed bash</a>, then you can use scp.</p>
<p>If you don't have winscp installed you can get it from here: <p>To installand configure Filezilla on your machine see <a href="fzconfig.html">FileZilla install instructions</a> which will set you up pointing at the correct folder automatically.
<a href="http://winscp.net/eng/index.php">http://winscp.net/eng/</a>.</p>
<p>quick start guide:
<a href="http://winscp.net/eng/docs/getting_started">http://winscp.net/eng/docs/getting_started</a></p>
<p>screenshots: <h3 id="android">Getting files from phones</h3>
<a href="http://winscp.net/eng/docs/screenshots">http://winscp.net/eng/docs/screenshots</a></p> <p>to be written. test <a href="http://www.lysesoft.com/products/andftp/">andftp app</a> and document properly...
<!-- WedDAV no longer works because the effective user is "apache" and the permissions
for the folder /uploads/ no longer allow it to write anything because we had to change it to allow Filezilla
to work.Philip & Wookey 11 August 2018
<h3>Using WebDAV - cadaver</h3>
<p>Command-line people can use the 'cadaver' client which is even
available for windows too:
<a href="http://www.phtagr.org/2009/04/01/cadaver-for-windows/">www.phtagr.org/2009/04/01/cadaver-for-windows/</a></p>
-->
<p>scp gives you an 'explorer-like' interface (although winscp can
give you a norton-commander-style 2-pane UI as well).</p>
<h3>Using WebDAV</h3>
This no longer works as we had to change the folder permissions for /uploads/. Sorry.
<h3>Using rsync</h3>
<p>No, don't use rsync for this. Really don't. It's too liable to delete everything or to overwrite files which are not changed at all because of the incompatibilities between Linux and Windows filename conventions (uppercase and lowercase are automagically converted and rsync gets it wrong).
<hr /> <hr />
</body> </body>
</html> </html>

13
handbook/zilla-uploads.xml
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<FileZilla3 version="3.28.0" platform="*nix"> <FileZilla3 version="3.41.2" platform="*nix">
<Servers> <Servers>
<Server> <Server>
<Host>expo.survex.com</Host> <Host>expo.survex.com</Host>
@ -14,17 +14,18 @@
<MaximumMultipleConnections>0</MaximumMultipleConnections> <MaximumMultipleConnections>0</MaximumMultipleConnections>
<EncodingType>Auto</EncodingType> <EncodingType>Auto</EncodingType>
<BypassProxy>0</BypassProxy> <BypassProxy>0</BypassProxy>
<Name>Expo-uploads</Name> <Name>expo-uploads</Name>
<Comments /> <Comments />
<Colour>0</Colour> <Colour>0</Colour>
<LocalDir>/home/</LocalDir> <LocalDir>/home/</LocalDir>
<RemoteDir>1 0 4 home 4 expo 9 expofiles 7 uploads</RemoteDir> <RemoteDir>1 0 4 home 4 expo 9 expofiles 7 uploads</RemoteDir>
<SyncBrowsing>0</SyncBrowsing> <SyncBrowsing>0</SyncBrowsing>
<DirectoryComparison>1</DirectoryComparison> <DirectoryComparison>1</DirectoryComparison>
<Name>Expo-uploads</Name>Expo-uploads<Bookmark> <Bookmark>
<Name>Expo-uploads</Name> <Name>expo-uploads</Name>
<LocalDir>/home/</LocalDir> <LocalDir>~</LocalDir>
<RemoteDir>1 0 4 home 4 expo 9 expofiles 7 uploads</RemoteDir> <RemoteDir>1 0 7 uploads</RemoteDir>
<DirectoryComparison>1</DirectoryComparison>
</Bookmark> </Bookmark>
</Server> </Server>
</Servers> </Servers>