expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2025-12-08 14:54:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

Added Bank of Expo, Creating a SUrvex file, rearranged website manual a bit.

Browse Source
This commit is contained in:
Philip Sargent
2018-06-30 04:03:12 +01:00
parent 0d3399dfd3
commit 9f594cc3a9
10 changed files with 129 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files

114
handbook/update.htm
View File

@@ -8,36 +8,33 @@
<h1>Expo Website Manual</h1>
<p>The website is now large and complicated with a lot of aspects. This handbook section contains info at various levels: simple 'How to add stuff' information for the typical expoer, more detailed info for cloning it onto your own machine for more significant edits, and structural info on how it's all put together for people who want/need to change things. [This manual is now so big that it is being restructured and split up. Much of it is obsolete.]</p>
<p>We have <a href="http://wookware.org/talks/expocomputer/#/"an Overview Presentation</a> on how the cave data, handbook and website are constructed and managed. It contains material which will be merged into this website manual.
<p>We have <a href="http://wookware.org/talks/expocomputer/#/">an Overview Presentation</a> on how the cave data, handbook and website are constructed and managed. It contains material which will be merged into this website manual.
<ul>
<li><a href="#photos">Uploading your photos and GPS tracks</a></li>
<li><a href="#logbooks">Uploading typed logbooks</a></li>
<li><a href="#update">Updating the website</a></li>
<li><a href="#manual">Expo Website manual</a></li>
<li><a href="expodata.html">Expo website developer info</a></li>
</ul>
<h2><a id="update">Updating the website - overview</a></h2>
<p>Short <a href="checkin.htm">command-line instructions</a> for updating the website
(on the expo machine). Thius is a memory job for experts, not beginners.</p>
(on the expo machine). This is a memory jog for experts, not beginners.</p>
<p>You can update the site via the troggle pages, by editing pages
online via a browser ("Edit this page" on the menu on the left), by
editing them locally on disk, or by checking out the relevant part to
editing them on the server remotely, or by checking out the relevant part to
your computer and editing it there. Which is best depends on your
knowledge and what you want to do. For simple addition of cave or
survey data troggle is recommended. For other edits it's best if you
can edit the files directly rather than using the 'edit this page'
button, but that means you either need to be on expo with the expo
computer, or be able to check out a local copy. If neither of these
survey data troggle ("edit this page") is recommended. For other edits it's best if you
can edit the files directly but that means you either need to be on expo with the expo
computer, or be able to check out a local copy using the version control system. If neither of these
apply then using the 'edit this page' button is fine.</p>
<p>It's important to understand that everything on the site (except things under
'expofiles') is stored in a distributed version control system (DVCS)
(called '<a href=#mercurial>Mercurial</a>' and accessed by most people
using software called 'TortoiseHg'), which means that every edited
which means that every edited
file needs to be 'checked in' after editing. The Expo website manual
goes into more detail about this, below. This stops us losing data and
makes it very hard for you to screw anything up permanently, so don't
@@ -46,13 +43,20 @@ problem. It also means that several people can work on the site on
different computers at once and normally merge their changes
easily.</p>
<p>Increasing amounts of the site are autogenerated, not static files,
<p>Increasing amounts of the site are autogenerated by troggle, and are not static files,
so you have to edit the base data, not the generated file (e.g cave
pages, QM lists, expo members list, prospecting pages). All
pages, QM (queston mark) lists, expo members list, prospecting pages). All
autogenerated files say 'This file is autogenerated - do not edit' at
the top - so check for that before wasting time on changes that will
just be overwritten</p>
<h3 id="mercurial">DVCS - version control</a></h3>
<p>We use a distributed revision control system (DVCS) for all the important data. On expo this means that many people can edit and merge their changes with the expo server in the Tatty Hut even if there is no internet access. Also anyone who is up to date with the Tatty Hut can take their laptop somewhere where there is internet access and update expo.survex.com - which will then get all the updates done by everyone on expo.
</p>
<p>In principle, survey notes can be typed into a laptop up on the plateau which is then synchronised with the Tatty Hut on returning to base.
</p>
<p>A DVCS is inefficient for scanned survey notes, which are large files that do not get modified, so they are kept as a plain directory of files 'expofiles'. The same goes for holiday photographs and GPS logs.</p>
<h2><a id="manual">Expo website manual</a></h2>
<p>Editing the expo website is an adventure. Until 2007, there was no
@@ -73,7 +77,7 @@ processes that a maintainer would want to do.</p>
<li><a href="#howitworks">How the website works</a></li>
<li><a href="#quickstart">Quick start</a></li>
<li><a href="#editingthewebsite">Editing the website</a></li>
<li><a href="#Mercurialinwindows">Using Mercurial/TortoiseHg in Windows</a></li>
<li><a href="#Mercurialinwindows">Using version control software in Windows</a></li>
<li><a href="#expowebupdate">The expoweb-update script</a></li>
<li><a href="#cavepages">Updating cave pages</a></li>
<li><a href="#updatingyears">Updating expo year pages</a></li>
@@ -85,7 +89,7 @@ processes that a maintainer would want to do.</p>
</ol>
Appendices:
<ul>
<li><a href="website-history">History of the website</a></li>
<li><a href="website-history.html">History of the website</a></li>
</ul>
<h3><a id="usernamepassword">Getting a username and password</a></h3>
@@ -99,16 +103,16 @@ Appendices:
<h3><a id="repositories">The repositories</a></h3>
<p>All the expo data is contained in 4 Mercurial repositories at
expo.survex.com. This is currently hosted on a server at the university. Mercurial* is a distributed version control system which allows collaborative editing and keeps track of all changes so we can roll back and have branches if needed.</p>
<p>All the expo data is contained in 4 "repositories" at
expo.survex.com. This is currently hosted on a server at the university. We use a distributed version control system (DVCS) to manage these repositories because this allows simultaneous collaborative editing and keeps track of all changes so we can roll back and have branches if needed.</p>
<p>The site has been split into four parts:</p>
<ul>
<li>expoweb - the website itself, including generation scripts</li>
<li>troggle - the database-driven part of the website</li>
<li>loser - the survex survey data</li>
<li>tunneldata - the tunnel (and therion) data and drawings</li>
<li><a href="http://expo.survex.com/repositories/home/expo/expoweb/graph">expoweb</a> - the website itself, including generation scripts</li>
<li><a href="http://expo.survex.com/repositories/home/expo/troggle/graph/">troggle</a> - the database-driven part of the website</li>
<li><a href="http://expo.survex.com/repositories/home/expo/loser/graph/">loser</a> - the survex survey data</li>
<li><a href="http://expo.survex.com/repositories/home/expo/tunneldata/graph/">tunneldata</a> - the tunnel (and therion) data and drawings</li>
</ul>
@@ -157,23 +161,29 @@ below for details on that.</p>
<dt>expofiles (all the big files and documents)</dt>
<p>Photos, scans (logbooks, drawn-up cave segments) (This was about
60GB of stuff in 2017 which you probably don't actually need locally).<p> To sync
60GB of stuff in 2017 which you probably don't actually need locally).
<p>If you don't need an entire copy of all 60GB, then it is probably best to use Filezilla to copy just a small part of the filesystem to your own machine and to upload the bits you add to or edit.
Instructions for installing and using Filezilla are found in the expo user instructions for uploading photographs: <a href="uploading.html">uploading.html</a>.
<p> To sync all
the files from the server to local expofiles directory:</p>
<p><tt>rsync -av expo@expo.survex.com:expofiles /home/expo</tt></p>
<p>To sync the local expofiles directory back to the server:</p>
<p>To sync the local expofiles directory back to the server (but only if your machine runs Linux):</p>
<p><tt>rsync --dry-run --delete-after -a /home/expo/expofiles expo@expo.survex.com</tt></p>
then CHECK that the list of files it produces matches the ones you absolutely intend to delete forever! ONLY THEN do:
<p><tt>rsync -av /home/expo/expofiles expo@expo.survex.com:</tt></p>
<p>(do be <b>incredibly</b> careful not to delete piles of stuff then rsync back, or to get the directory level of the command wrong - as it'll all get deleted on the server too, and we may not have backups!). It's <b>absolutely vital</b>Use rsync --dry-run --delete-after -a first to check what would be deleted. If you are using rsync from a Windows machine you will <em>not</em> get all the files as some filenames are incompatible with Windows: see more detail under <a href="Mercurialinwindows">Using Mercurial/TortoiseHg in Windows</a> below.</p>
<p>(do be <b>incredibly</b> careful not to delete piles of stuff then rsync back, or to get the directory level of the command wrong - as it'll all get deleted on the server too, and we may not have backups!). It's <b>absolutely vital</b>Use rsync --dry-run --delete-after -a first to check what would be deleted.
<p>(We have an issue with rsync not using the appropriate user:group attributes for files pushed back to the server. This may not cause any problems, but watch out for it.)</p>
<p>If you are using rsync from a Windows machine you will <em>not</em> get all the files as some filenames are incompatible with Windows. What will happen is that rsync will invisibly change the names as it downloads them from the Linux expo server to your Windows machine, but then it forgets what it has done and tries to re-upload all the renamed files to the server even if you have touched none of them. Now there won't be any problems with simple filenames using all lowercase letters and no funny characters, but we have nothing in place to stop anyone creating such a filename somewhere in that 60GB or of detecting the problem at the time. So don't do it. If you have a Windows machine use Filezilla not rsync.
<p>(We may also have an issue with rsync not using the appropriate user:group attributes for files pushed back to the server. This may not cause any problems, but watch out for it.)</p>
</dl>
<h3><a id="editingthewebsite">Editing the website</a></h3>
<p>To edit the website fully, you need a Mercurial client such as TortoiseHg. Some (static text) pages can be edited directly on-line using the 'edit this page link' which you'll see if you are logged into troggle. In general dynamically-generated pages can not be edited in this way, but forms are provided for some page-types like 'caves'.</p>
<p>To edit the website fully, you need to use the disributed version control system (DVCM) software which is currently mercurial/TortoiseHg. Some (static text) pages can be edited directly on-line using the 'edit this page link' which you'll see if you are logged into troggle. In general the dynamically-generated pages, such as those describing caves which are generated from the cave survey data, can not be edited in this way, but forms are provided for some types of these like 'caves'.</p>
<p>What follows is for Linux. If you are running Windows then see below <a href="#Mercurialinwindows">Using Mercurial/TortoiseHg in Windows</a>.
@@ -261,6 +271,7 @@ editing these files.</p>
<p>(If you remember something about CAVETAB2.CSV for editing caves, that was
superseded in 2012).</p>
<p>This may be a useful reminder of what is in a survex file <a href="survey/how_to_make_a_survex_file.pdf">how to create a survex file</a>.
<h3><a id="updatingyears">Updating expo year pages</a></h3>
@@ -302,26 +313,18 @@ superseded in 2012).</p>
T/U: Jess 1 hr, Emma 0.5 hr
</pre>
<hr>
<h3>Adding a new year</h3>
<p>Edit noinfo/folk.csv, adding the new year to the end of the header
line, a new column, with just a comma (blank
cell) for people who weren't there, a 1 for people who were there, and
a -1 for people who were there but didn't go caving. Add new lines for
new people, with the right number of columns.</p>
<h3><a id="photos">Uploading photos and GPS tracks</a></h3>
<p>Photos are stored in the general file area of the site under <br /><a
href="http://expo.survex.com/expofiles/photos/">http://expo.survex.com/expofiles/photos/</a></p>
<p>GPS tracks over the surface of the plateau (GPX files from your handheld GPS or phone) are stored in the general file area of the site under <br /><a
href="http://expo.survex.com/expofiles/gpslogs/">http://expo.survex.com/expofiles/gpslogs/</a></p>
<p>
They are each organised by year, and by photographer (walker). </p>
<p>They are much more conveniently viewed at <br /> <a
href="http://expo.survex.com/photos/">http://expo.survex.com/photos/</a><br />
or<br />
<a href="http://expo.survex.com/gpslogs/">http://expo.survex.com/gpslogs/</a><br />
because the website scripts create a much friendlier interface
</p>
The directory names are like this: "2014/YourName/" (i.e no spaces, CamelCase for names).
<p>See <a href="uploading.html">Photo/GPS/File Upload Instructions</a> for instructions on intalling and using software to do this uploading - for all types of operating system.</p>
<p>This proces is tedious and error-prone and ripe for improvement.
Adding a list of people, fro the bier book, and their aliases would be
a lot better, but some way to make sure that names match with previous
years would be
good.</p>
<h3><a id="tickingoff">Ticking off QMs</a></h3>
@@ -371,13 +374,24 @@ http://expo.survex.com/expo/surveys/surveytable.html http://expo.survex.com/surv
<li>Other people's work - the noinfo hierarchy.</li>
<li>Style guide for writing cave descriptions: correct use of boldface (<em>once</em> for each passage name, at the primary definition thereof; other uses of the name should be links to this, and certainly should not be bold.) </li>
</ul>
<hr />
<h2><a id="mercurial">Mercurial and TortoiseHG</a></h2>
<p>Mercurial is a distributed revision control system. On expo this means that many people can edit and merge their changes with the Mercurial server in the Tatty Hut even if there is no Internet access. Also anyone who is up to date with the Tatty Hut can take their laptop somewhere where there is Internet access and update expo.survex.com - which will then get all the updates done by everyone on expo.
<h3><a id="photos">Uploading photos and GPS tracks</a></h3>
<p>Photos are stored in the general file area of the site under <br /><a
href="http://expo.survex.com/expofiles/photos/">http://expo.survex.com/expofiles/photos/</a></p>
<p>GPS tracks over the surface of the plateau (GPX files from your handheld GPS or phone) are stored in the general file area of the site under <br /><a
href="http://expo.survex.com/expofiles/gpslogs/">http://expo.survex.com/expofiles/gpslogs/</a></p>
<p>
They are each organised by year, and by photographer (walker). </p>
<p>They are much more conveniently viewed at <br /> <a
href="http://expo.survex.com/photos/">http://expo.survex.com/photos/</a><br />
because the website script creates a much friendlier interface
</p>
<p>In principle, survey notes can be typed into a laptop up on the plateau which is then synchronised with the Tatty Hut on returning to base.
</p>
<p>Mercurial is inefficient for scanned survey notes, which are large files that do not get modified, so they are kept as a plain directory of files 'expofiles'.</p>
The directory names are like this: "2014/YourName/" (i.e no spaces, CamelCase for names).
<p>See <a href="uploading.html">Photo/GPS/File Upload Instructions</a> for ordeinary expo user instructions on how to do this .</p>
<hr />
</body>
</html>