expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2025-12-08 14:54:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

Documentation fixes for ssh key needs

Browse Source
This commit is contained in:
Philip Sargent
2019-07-17 16:07:52 +02:00
parent 23f01ce82e
commit 3d53810df4
3 changed files with 164 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files

96
handbook/computing/qstart-hg.html Normal file
View File

@@ -0,0 +1,96 @@
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>Handbook placeholder page</title>
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
</head>
<body>
<h2 id="tophead">CUCC Expedition Handbook - Mercurial Quick</h2>
<h1>Quick Reminder - Mercurial</h1>
<h2>Version Control Software Reminders</h2>
<p>This is NOT a tutorial. This is a set of reminders for people who already know all this stuff.
<p>Since 2019 all use of version control software requires that you
have <a href="computing/keyexchange.html">cyptographic key exchnage</a> already set up before any of this will work on your own machine.
<p>You can, however, do all this on the <i>expo laptop</i> as this has already been configured with the right keys.
<dl>
<dt>expoweb (The data management system)</dt>
<dd>
<tt>hg clone ssh://expo@expo.survex.com/expoweb</tt> (read/write)<br />
<tt>hg clone http://expo.survex.com/repositories/home/expo/expoweb/</tt> (read-only checkout)
</dd>
<dt>troggle (The data management system backend)</dt>
<dd>
<tt>hg clone ssh://expo@expo.survex.com/troggle</tt> (read/write)<br />
<tt>hg clone http://expo.survex.com/repositories/home/expo/troggle/</tt> (read-only checkout)
</dd>
<dt>loser (The survey data)</dt>
<dd>
<tt>hg clone ssh://expo@expo.survex.com/loser</tt> (read/write)<br />
<tt>hg clone http://expo.survex.com/repositories/home/expo/loser/</tt> (read-only)
</dd>
<dt>tunneldata (The Tunnel drawings)</dt>
<dd>
<tt>hg clone ssh://expo@expo.survex.com/tunneldata</tt> (read/write)<br />
<tt>hg clone http://expo.survex.com/repositories/home/expo/expoweb/</tt> (read-only)
</dd>
</dl>
<h3><a id="Mercurialinwindows">Using Mercurial/TortoiseHg in Windows</a></h3>
<p>Read the instructions for setting up TortoiseHG in <a href="../tortoise/tortoise-win.htm">Tortoise-on-Windows</a>. But this won't work at all until you set up the key exchange using PuTty/Pageant.
<p>In Windows: install Mercurial and TortoiseHg of the relevant flavour from <a href="https://tortoisehg.bitbucket.io/">https://tortoisehg.bitbucket.io/</a> (ignoring antivirus/Windows warnings). This will install a submenu in your Programs menu)</p>
<p>To start cloning a repository: first create the folders you need for the repositories you are going to use, e.g. D:\CUCC-Expo\loser and D:\CUCC-Expo\expoweb. Then start TortoiseHg Workbench from your Programs menu, click File -> Clone repository, a dialogue box will appear. In the Source box type</p>
<p>What follows is for Linux. If you are running Windows then see below <a href="#Mercurialinwindows">Using Mercurial/TortoiseHg in Windows</a>.
<p>Mercurial can be used from the command line, but if you prefer a GUI, TourtoiseHg is highly recommended on all OSes.</p>
<p>Linux: Install mercurial and tortoisehg-nautilus from synaptic,
then restart nautilus <tt>nautilus -q</tt>. If it works, you'll be able to see the menus of tortoise within your Nautilus windows. </p>
<p>Once you've downloaded and installed a client, the first step is to create what is called a checkout of the data management system. This creates a copy on your machine which you can edit to your heart's content. The command to initially check out ('clone') the entire expo data management system is:</p>
<p><tt>hg clone ssh://expo@expo.survex.com/expoweb</tt></p>
<p>for subsequent updates</p>
<p><tt>hg update</tt></p>
<p>will generally do the trick.</p>
<p>In TortoiseHg, merely right-click on a folder you want to check out to, choose "Mercurial checkout," and enter</p>
<p><tt>ssh://expo@expo.survex.com/expoweb</tt></p>
<p>After you've made a change, commit it to you local copy with:</p>
<p><tt>hg commit</tt> (you can specify filenames to be specific)</p>
<p>or right clicking on the folder and going to commit in TortoiseHg. Mercurial can't always work out who you are. If you see a message like "abort: no username supplied" it was probably not set up to deduce that from your environment. It's easiest to give it the info in a config file at ~/.hgrc (create it if it doesn't exist, or add these lines if it already does) containing something like</p>
<p><tt>ssh://expo@expo.survex.com/expoweb</tt></p>
<p>for expoweb (or similar for the other repositories). In the Destination box type whatever destination you want your local copies to live in on your laptop e.g. D:\CUCC-Expo\expoweb. Hit Clone, and it should hopefully prompt you for the usual beery password.
<p>The first time you do this it will probably not work as it does not recognise the server. Fix this by running putty (downloading it from <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/">https://www.chiark.greenend.org.uk/~sgtatham/putty/</a>), and connecting to the server 'expo@expo.survex.com' (on port 22). Confirm that this is the right server. If you succeed in getting a shell prompt then ssh connection are working and TortoiseHg should be able to clone the repo, and send changes back.</p>
<hr />
<div id="menu">
<ul id="links">
<li><a href="index.htm">Expedition Handbook</a>
</li>
<li><a href="../infodx.htm">Main index</a></li>
</ul>
</div>
</body>
</html>

149
handbook/manual.html
View File

@@ -1,13 +1,20 @@
<html>
<head>
<title>CUCC Expedition Handbook: Online system manual</title>
<title>CUCC Expedition Handbook: Programmers manual</title>
<link rel="stylesheet" type="text/css" href="../css/main2.css" />
</head>
<body>
<h2 id="tophead">CUCC Expedition Handbook - Online systems</h2>
<h1>Expo Online Systems Manual</h1>
<h1>Expo Data Maintenance Manual</h1>
<h2><a id="manual">Expo data management systems manual</a></h2>
<h2><a id="manual">Expo data management programmers' manual</a></h2>
<ul>
<li>This page is <i>not</i> for cavers wanting to know how to record their cave survey data.
<li>This page is <i>not</i> for cavers wanting to know how to type in logbooks or upload photographs.
<li>This page <i>is for programmers</i> who are modifying the software or helping cavers do their thing.
</ul>
<p>Editing the expo data management system is an adventure. Until 2007, there was no
guide which explained the whole thing as a functioning system. Learning
@@ -20,28 +27,32 @@ rather than referring to specific elements of the data management system, refer
processes that a maintainer would want to do.</p>
<p>Note that to display the survey data you will need a copy of the <a href="getsurvex.html">survex</a> software.
<h3>Contents</h3>
<p>Go elsewhere if this is what you want to know:
<ul>
<li><a href="uploading.html">How to upload photos</a></li>
<li><a href="logbooks.html">Typing in logbook entries</a></li>
</ul>
<h3>Contents of this manual</h3>
<ol>
<li><a href="#usernamepassword">Getting a username and password</a></li>
<li><a href="#repositories">The repositories</a></li>
<li><a href="#howitworks">How the data management system works</a></li>
<li><a href="#quickstart">Quick start</a></li>
<li><a href="#editingthedata management system">Editing the data management system</a></li>
<li><a href="#Mercurialinwindows">Using version control software in Windows</a></li>
<li><a href="#editingthedata management system">Modifying the data management system</a></li>
<li><a href="#expowebupdate">The expoweb-update script</a></li>
<li><a href="#cavepages">Updating cave pages</a></li>
<li><a href="#updatingyears">Updating expo year pages</a></li>
<li><a href="logbooks.html">Adding typed logbooks</a></li>
<li><a href="uploading.html">Uploading photos</a></li>
<li><a href="#tickingoff">Ticking off QMs</a></li>
<li><a href="#surveystatus">Maintaining the survey status table</a></li>
<li><a href="#automation">Automation</a></li>
<li><a href="#arch">Archived updates</a></li>
</ol>
Appendices:
<ul>
<li><a href="c21bs.html">Taking Expo Bullshit into the 21st Century</a> - a history of the data management system up to 1996</li>
<li><a href="website-history.html">Website history</a> - a history of the data management system up to 2019</li>
<li><a href="c21bs.html">Taking Expo Bullshit into the 21st Century</a> - initial report from 1996</li>
</ul>
<h3><a id="usernamepassword">Getting a username and password</a></h3>
@@ -50,23 +61,33 @@ Appendices:
with a cavey:beery password. Ask someone if this isn't enough clue for you.
<b>This password is important for security</b>. The whole site <strong>will</strong> get hacked by spammers or worse if you are not careful with it. Use a secure method for passing it on to others that need to know (i.e not unencrypted email), don't publish it anywhere, don't check it in to the data management system by accident. A lot of people use it and changing it is a pain for everyone so do take a bit of care.
</p>
<p>Note that you don't need a password to view most things, but you will need one to change them</p>
<p>This password is all you need to log in to troggle and to use the control panel. But if you want to edit the software itself, or update webpages, then
you will also need to get a cryptographic key and register it with the server. See <a href="computing/keyexchange.html">key exchange</a> for details.
<p>Unfortunately, pushing cave data to ::loser:: and ::drawings:: also needs a key. So cavers entering their cave survey data
currently have to use a machine on which this already set up. These machines are
the <i>expo laptop</i> and the laptop '<i>aziaphale</i>' which live in the potato hut during expo.
<h3><a id="repositories">The repositories</a></h3>
<p>All the expo data is contained in 4 "repositories" at
expo.survex.com. This is currently hosted on a server at the university. We use a distributed version control system (DVCS) to manage these repositories because this allows simultaneous collaborative editing and keeps track of all changes so we can roll back and have branches if needed.</p>
expo.survex.com. This is currently hosted on a free virtual server we have blagged on a server farm.
We use a distributed version control system (DVCS) to manage these repositories because this allows simultaneous collaborative
editing and keeps track of all changes so we can roll back and have branches if needed.</p>
<p>The site has been split into four parts:</p>
<ul>
<li><a href="http://expo.survex.com/repositories/home/expo/expoweb/graph">expoweb</a> - the data management system itself, including generation scripts</li>
<li><a href="http://expo.survex.com/repositories/home/expo/troggle/graph/">troggle</a> - the database-driven part of the data management system - see <a href="troggle-ish.html">notes on troggle</a> for further explanation</li>
<li><a href="http://expo.survex.com/repositories/home/expo/loser/graph/">loser</a> - the survex survey data</li>
<li><a href="http://expo.survex.com/repositories/home/expo/tunneldata/graph/">tunneldata</a> - the tunnel (and therion) data and drawings</li>
<li><a href="http://expo.survex.com/repositories/home/expo/loser/graph/">loser</a> - the survex cave survey data</li>
<li><a href="http://expo.survex.com/repositories/home/expo/drawings">drawings</a> - the tunnel and therion cave data and drawings</li>
<li><a href="http://expo.survex.com/repositories/home/expo/expoweb/graph">expoweb</a> - the website pages, handbook, generation scripts</li>
<li><a href="http://expo.survex.com/repositories/home/expo/troggle/graph/">troggle</a> - the database/software part of the survey data management system - see <a href="troggle-ish.html">notes on troggle</a> for further explanation</li>
</ul>
<p>In 2019 we are in the process of migrating from mercurial to git for the repos.
<p>All the scans, photos, presentations, fat documents and videos are
stored just as files (not in version control) in 'expofiles'. See
@@ -74,14 +95,21 @@ below for details on that.</p>
<h3><a id="howitworks">How the data management system works</a></h3>
<p>Part of the data management system is static HTML, but quite a lot is generated by scripts. So anything you check in which affects cave data or descriptions won't appear on the site until the data management system update scripts are run. This happens automatically every 30 mins, but you can also kick off a manual update. See 'The expoweb-update script' below for details.</p>
<p>Part of the data management system is static HTML, but quite a lot is generated by scripts.
So anything you check in which affects cave data or descriptions won't appear on the site until
the data management system update scripts are run.
This happens automatically every 30 mins, but you can also kick off a manual update.
See 'The expoweb-update script' below for details.</p>
<p>Also note that the data management system you see is its own Mercurial checkout (just like your local one) so that has to be 'pulled' from the server before your changes are reflected.</p>
<p>Also note that the ::expoweb:: web pages and cave data reports you see on the visible website
are not the same as the version-controlled "master" expoweb repo.
So in order that your committed and pushed changes become visible on the website,
they have to be 'pulled' from the repo onto the webserver before your changes are reflected.</p>
<h3><a id="editthispage">Using 'Edit This Page'</a></h3>
<p>This edits the file served by the webserver on
the expo server in Cambridge but it does not update the copy of the file in the
repository in expo.survex.com. To properly finish the job you need to
the expo server but it does not update the copy of the file in the
repository. To properly finish the job you need to
<ul>
<li>
ssh into expo@expo.survex.com (use putty on a Windows machine)
@@ -103,39 +131,19 @@ which submits the obligatory comment witht he commit operation.
<p>If you know what you are doing here is the basic info on what's where:<br>
(if you don't know what you're doing, skip to <a href="#editingthedata management system">Editing the data management system</a> below.)
<p>This section is all about how to use mercurial. Since we are changing to git it has been
removed to <a href="computing/qstart-hg.html">a separate place</a>.
<dl>
<dt>expoweb (The data management system)</dt>
<dd>
<tt>hg clone ssh://expo@expo.survex.com/expoweb</tt> (read/write)<br />
<tt>hg clone http://expo.survex.com/repositories/home/expo/expoweb/</tt> (read-only checkout)
</dd>
<dt>troggle (The data management system backend)</dt>
<dd>
<tt>hg clone ssh://expo@expo.survex.com/troggle</tt> (read/write)<br />
<tt>hg clone http://expo.survex.com/repositories/home/expo/troggle/</tt> (read-only checkout)
</dd>
<dt>loser (The survey data)</dt>
<dd>
<tt>hg clone ssh://expo@expo.survex.com/loser</tt> (read/write)<br />
<tt>hg clone http://expo.survex.com/repositories/home/expo/loser/</tt> (read-only)
</dd>
<dt>tunneldata (The Tunnel drawings)</dt>
<dd>
<tt>hg clone ssh://expo@expo.survex.com/tunneldata</tt> (read/write)<br />
<tt>hg clone http://expo.survex.com/repositories/home/expo/expoweb/</tt> (read-only)
</dd>
</dl>
<dl>
<dt>expofiles (all the big files and documents)</dt>
<p>Photos, scans (logbooks, drawn-up cave segments) (This was about
60GB of stuff in 2017 which you probably don't actually need locally).
<p>If you don't need an entire copy of all 60GB, then it is probably best to use Filezilla to copy just a small part of the filesystem to your own machine and to upload the bits you add to or edit.
Instructions for installing and using Filezilla are found in the expo user instructions for uploading photographs: <a href="uploading.html">uploading.html</a>.
40GB of stuff in 2017 which you probably don't actually need locally).
<p>If you don't need an entire copy of all of it, then it is probably best to use Filezilla/ftp to
copy just a small part of the filesystem to your own machine and to upload the bits you add to or edit.
Instructions for installing and using Filezilla are found in the expo user instructions for
uploading photographs: <a href="uploading.html">uploading.html</a>.
<p> To sync all
the files from the server to local expofiles directory:</p>
@@ -155,34 +163,12 @@ then CHECK that the list of files it produces matches the ones you absolutely in
</dl>
<h3><a id="editingthedata management system">Editing the data management system</a></h3>
<p>To edit the data management system fully, you need to use the disributed version control system (DVCM) software which is currently mercurial/TortoiseHg. Some (static text) pages can be edited directly on-line using the 'edit this page link' which you'll see if you are logged into troggle. In general the dynamically-generated pages, such as those describing caves which are generated from the cave survey data, can not be edited in this way, but forms are provided for some types of these like 'caves'.</p>
<p>What follows is for Linux. If you are running Windows then see below <a href="#Mercurialinwindows">Using Mercurial/TortoiseHg in Windows</a>.
<p>Mercurial can be used from the command line, but if you prefer a GUI, TourtoiseHg is highly recommended on all OSes.</p>
<p>Linux: Install mercurial and tortoisehg-nautilus from synaptic,
then restart nautilus <tt>nautilus -q</tt>. If it works, you'll be able to see the menus of tortoise within your Nautilus windows. </p>
<p>Once you've downloaded and installed a client, the first step is to create what is called a checkout of the data management system. This creates a copy on your machine which you can edit to your heart's content. The command to initially check out ('clone') the entire expo data management system is:</p>
<p><tt>hg clone ssh://expo@expo.survex.com/expoweb</tt></p>
<p>for subsequent updates</p>
<p><tt>hg update</tt></p>
<p>will generally do the trick.</p>
<p>In TortoiseHg, merely right-click on a folder you want to check out to, choose "Mercurial checkout," and enter</p>
<p><tt>ssh://expo@expo.survex.com/expoweb</tt></p>
<p>After you've made a change, commit it to you local copy with:</p>
<p><tt>hg commit</tt> (you can specify filenames to be specific)</p>
<p>or right clicking on the folder and going to commit in TortoiseHg. Mercurial can't always work out who you are. If you see a message like "abort: no username supplied" it was probably not set up to deduce that from your environment. It's easiest to give it the info in a config file at ~/.hgrc (create it if it doesn't exist, or add these lines if it already does) containing something like</p>
<p>To edit the data management system fully, you need to use the disributed version control system
(DVCM) software which is currently mercurial/TortoiseHg.
Some (static text) pages can be edited directly on-line using the 'edit this page link' which you'll
see if you are logged into troggle. In general the dynamically-generated pages, such as those describing
caves which are generated from the cave survey data, can not be edited in this way, but forms are provided
for some types of these like 'caves'.</p>
<p><tt>
[ui]<br/>username = Firstname Lastname &lt;myemail@example.com&gt;
@@ -203,19 +189,6 @@ then restart nautilus <tt>nautilus -q</tt>. If it works, you'll be able to see t
<h3><a id="Mercurialinwindows">Using Mercurial/TortoiseHg in Windows</a></h3>
<p>Read the instructions for setting up TortoiseHG in <a href="tortoise/tortoise-win.htm">Tortoise-on-Windows</a>.
<p>In Windows: install Mercurial and TortoiseHg of the relevant flavour from <a href="https://tortoisehg.bitbucket.io/">https://tortoisehg.bitbucket.io/</a> (ignoring antivirus/Windows warnings). This will install a submenu in your Programs menu)</p>
<p>To start cloning a repository: first create the folders you need for the repositories you are going to use, e.g. D:\CUCC-Expo\loser and D:\CUCC-Expo\expoweb. Then start TortoiseHg Workbench from your Programs menu, click File -> Clone repository, a dialogue box will appear. In the Source box type</p>
<p><tt>ssh://expo@expo.survex.com/expoweb</tt></p>
<p>for expoweb (or similar for the other repositories). In the Destination box type whatever destination you want your local copies to live in on your laptop e.g. D:\CUCC-Expo\expoweb. Hit Clone, and it should hopefully prompt you for the usual beery password.
<p>The first time you do this it will probably not work as it does not recognise the server. Fix this by running putty (downloading it from <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/">https://www.chiark.greenend.org.uk/~sgtatham/putty/</a>), and connecting to the server 'expo@expo.survex.com' (on port 22). Confirm that this is the right server. If you succeed in getting a shell prompt then ssh connection are working and TortoiseHg should be able to clone the repo, and send changes back.</p>
<h3><a id="expowebupdate">The expoweb-update script</a></h3>

8
handbook/tortoise/tortoise-win.htm
View File

@@ -10,9 +10,15 @@
<body>
<h1>Windows: TortoiseHg and PuTTy</h1>
<p>Struggling to get TortoiseHg to work on Windows (v7 or v10)? Here’s a quick guide which I hope explains how to sort it all
<p>Struggling to get TortoiseHg to work on Windows (v10)? Here’s a quick guide which I hope explains how to sort it all
out.</p>
<p>First you need to set up <a href="../computing/keyexchange.html">keys</a> so that you can use ssh.
Until you can do that there is no point in doing anything else. This is <i>particularly difficult</i>
on Windows which requires <a href="https://rperki.blogspot.com/2015/06/install-puttypageant-on-win-10.html">extra software to be installed and configured</a>.
Allow an afternoon at least to get this bit to work as it is poorly documented.
To get the keys set up on the server you will have to wait for an already-authenticated nerd to do it for you so this is not a quick job.
<p>First download and install both <a href="http://tortoisehg.bitbucket.io/">TortoiseHg</a>
and <a href="http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html">PuTTy</a>. Then run PuTTy;</p>