From beeca2cc2507c331287f18e1af5fcf278830bf29 Mon Sep 17 00:00:00 2001 From: "Philip Sargent (muscogee)" Date: Thu, 9 Apr 2020 19:13:18 +0100 Subject: [PATCH] Moving online systems overview and computing manual pages --- folk/author.htm | 2 +- handbook/computer.html | 2 +- handbook/computing/basiclaptop.html | 6 +- handbook/computing/exposerver.html | 2 +- handbook/computing/newyear.html | 2 +- handbook/{ => computing}/onlinesystems.html | 22 +- handbook/computing/regular.html | 2 +- handbook/computing/yourlaptop.html | 2 +- handbook/manual.html | 222 -------------------- handbook/troggle/trogintro.html | 2 +- handbook/troggle/trogmanual.html | 2 +- handbook/troggle/trognotes.html | 2 +- handbook/website-history.html | 6 +- infodx.htm | 2 +- 14 files changed, 27 insertions(+), 249 deletions(-) rename handbook/{ => computing}/onlinesystems.html (86%) delete mode 100644 handbook/manual.html diff --git a/folk/author.htm b/folk/author.htm index febd9295a..bfe56f858 100644 --- a/folk/author.htm +++ b/folk/author.htm @@ -89,7 +89,7 @@

The large mass of cave descriptions have been taken from guidebook descriptions or exploration reports by the above authors, or translated from publications of groups listed under Other Groups. We'd like to thank the librarian of the Fédération Belgique de Spéléologie for supplying photocopies of articles, and Jill Gates (ULSA and NPC) for help with translation.

Certain parts of the Expedition Handbook are derived from earlier documents used by other clubs, most especially the Oxford University Caving Club. This particularly includes the Rescue guide, which owes much to Gavin Lowe.

The editors would like to thank the discoverers of Kaninchenhöhle for finding a cave so complex that hypertext seemed the only way to build a usable guidebook description. Without the start that this gave, the website might never have come about. Andy Waddington started the major rewrite of the description in an effort to understand the cave without having to make the sacrifice of going underground. The endless questions that this provoked stimulated Wookey and others to join the work. HTML provides the means to keep linking in new bits of material interminably and Wookey provided the stimulus to make it more cohesive by finding the space on a real web site. We'd also like to thank everyone who has browsed these pages and made constructive comments or reported bugs.

-

Finally, both exploration and documentation continue. The editors welcome your feedback on the existing site or contributions of writing or photographs of the area. In recent years maintenance of the website has been to some extent devolved, so that members of the expedition have responsibility for documenting their own finds; see the online systems manual. Nonetheless all the kudos for creating the edifice should go to the two original editors, both of whom are still working on the site:

+

Finally, both exploration and documentation continue. The editors welcome your feedback on the existing site or contributions of writing or photographs of the area. In recent years maintenance of the website has been to some extent devolved, so that members of the expedition have responsibility for documenting their own finds; see the online systems manual. Nonetheless all the kudos for creating the edifice should go to the two original editors, both of whom are still working on the site:

Andy Waddington
mail: <expo (at) pennine.ddns.me.uk>
Andy's own blog pages

Wookey
mail: <Wookey (at) wookware.org>
Wookey's own homepage


diff --git a/handbook/computer.html b/handbook/computer.html index 7c9da0e4c..e786731b1 100644 --- a/handbook/computer.html +++ b/handbook/computer.html @@ -52,7 +52,7 @@ are advised to use the Expo laptop first to see how it all works.

Either the Expo laptop or your own laptop will use the version control system to synchronise cave data. It's easier to use the Expo laptop as the software is already set up. But if you set up your own computer then you will get a more familiar environment. See -the Expo Online Systems Manual for info on how to do that.

+the Expo Online Systems Manual for info on how to do that.

Through the miracle of the distributed version control system, everyone can edit the data on multiple laptops at the same time and it should all get merged.

diff --git a/handbook/computing/basiclaptop.html b/handbook/computing/basiclaptop.html index c947c2ecd..b37ac2951 100644 --- a/handbook/computing/basiclaptop.html +++ b/handbook/computing/basiclaptop.html @@ -16,7 +16,7 @@
  1. Register an SSH key with an expo nerd i.e 'get a login'. (see "Key Configuration" below)
  2. Install git version control software to download ("clone"), view and edit caving data.
  3. -
  4. Clone three expo repositories so you have the files on your machine.
  5. +
  6. Clone three expo repositories so you have the files on your machine.
  7. Install survex, and therion or tunnel for editing cave data.
@@ -27,7 +27,7 @@ And please write some documentation for the next person in your situation

Cheat lists and quick reminders

Software

@@ -55,7 +55,7 @@ Note that you are loading your private key, the .ppk file, into pageant
  • A Windows laptop for expo - PuTTy and WSL here.
  • When using Windows please, please be excessively -careful when naming files and survex names and be exceptionally careful when using rsync. +careful when naming files and survex names and be exceptionally careful when using rsync.

    Learning how to use this software

    diff --git a/handbook/computing/exposerver.html b/handbook/computing/exposerver.html index 7e1d1824b..25c3f92c5 100644 --- a/handbook/computing/exposerver.html +++ b/handbook/computing/exposerver.html @@ -20,7 +20,7 @@
  • ssh - occasionally, by experts,to fix things when something goes wrong or for major site reconfiguration. -Now readthe Expo data management systems manual. +Now readthe Expo data management systems manual.
    diff --git a/handbook/computing/newyear.html b/handbook/computing/newyear.html index 5eb4d82fd..0e151fede 100644 --- a/handbook/computing/newyear.html +++ b/handbook/computing/newyear.html @@ -41,7 +41,7 @@

    Edit folk/folk.csv according to the instructions in the Folk Update process

    -Return to the main online systems manual. +Return to the main online systems manual.
    diff --git a/handbook/onlinesystems.html b/handbook/computing/onlinesystems.html similarity index 86% rename from handbook/onlinesystems.html rename to handbook/computing/onlinesystems.html index c8d9aad76..407b3e69f 100644 --- a/handbook/onlinesystems.html +++ b/handbook/computing/onlinesystems.html @@ -3,7 +3,7 @@ CUCC Expedition Handbook: Online system overview - +

    CUCC Expedition Handbook - Online systems

    @@ -21,30 +21,30 @@ handbook and public website are constructed and managed. It contains material which will be merged into this online systems manual.

    These pages listed below have been reviewed recently (2019), and a -fuller list of "How do I..." instruction pages are on the handbook opening page. +fuller list of "How do I..." instruction pages are on the handbook opening page.

    The systems manual is being actively edited to extract and simplify documentation.

    Updating the online systems - overview

    Troggle - what it is

    -

    Troggle runs the expo cave survey data management, presents the data on the website and manages the Expo Handbook. See the troggle intro. +

    Troggle runs the expo cave survey data management, presents the data on the website and manages the Expo Handbook. See the troggle intro.

    Autogenerated pages

    Some key sections of the online webpages are autogenerated by scripts or by -troggle, and are not static files, +troggle, and are not static files, so you have to edit the base data, not the generated file (e.g cave pages, QM (question mark) lists, expo members list, prospecting pages). All autogenerated files say 'This file is autogenerated - do not edit' at @@ -55,7 +55,7 @@ just be overwritten

    The primary and recommended way of editing this handbook (and the website generally) is to use a laptop which has the version control software installed. The person editing needs to know how to use this software, and also needs to know how to edit raw HTML files -using a text editor. The public expo server is on a machine far, far away that we only access remotely. +using a text editor. The public expo server is on a machine far, far away that we only access remotely.

    The Expo laptop has the software installed, so it is best to learn how to do this when sitting at that laptop. diff --git a/handbook/computing/regular.html b/handbook/computing/regular.html index 83802aee7..51521af15 100644 --- a/handbook/computing/regular.html +++ b/handbook/computing/regular.html @@ -29,7 +29,7 @@ find . -not -type d -exec file "{}" ";" | grep CRLF >crlf.txt


    Annual tasks New expo year jobs. -Return to the main online systems manual. +Return to the main online systems manual.
    diff --git a/handbook/computing/yourlaptop.html b/handbook/computing/yourlaptop.html index 3a0a27aac..f4c231f75 100644 --- a/handbook/computing/yourlaptop.html +++ b/handbook/computing/yourlaptop.html @@ -135,7 +135,7 @@ to Quick Start guide to TortoiseHg. -

    When using Windows please, please be excessively careful when naming files and survex names and be exceptionally careful when using rsync. +

    When using Windows please, please be excessively careful when naming files and survex names and be exceptionally careful when using rsync.

    The handbook has documents where it is necessary to use scp or sftp to manage large files in 'expofiles'. See Experts: Uploading files, diff --git a/handbook/manual.html b/handbook/manual.html deleted file mode 100644 index 557c84454..000000000 --- a/handbook/manual.html +++ /dev/null @@ -1,222 +0,0 @@ - - - - -CUCC Expedition Handbook: Programmers manual - - - -

    CUCC Expedition Handbook - Online systems

    -

    Expo Data Maintenance Manual

    - -

    Expo data management programmers' manual

    - - - -

    Editing the expo data management system is an adventure. Learning -it by trial and error is non-trivial. There are lots of things we -could improve about the system, and anyone with some computer nous is -very welcome to muck in. It is slowly getting better organised.

    - -

    This manual is organized in a how-to sort of style. The categories, -rather than referring to specific elements of the data management system, refer to -processes that a maintainer would want to do.

    -

    Note that to display the survey data you will need a copy of the survex software. - -

    Follow these links if you have reached this page by accident and this is what you want to know: -

    - -

    Contents of this manual

    - -
      -
    1. Getting a username, password and key
    2. -
    3. The repositories
    4. -
    5. How the data management system works
    6. -
    7. Your own laptop
    8. -
    9. Quick start
    10. -
    11. Modifying the data management system
    12. -
    13. The expoweb-update script
    14. -
    15. Updating cave pages
    16. -
    17. Updating expo year pages
    18. -
    19. Maintaining the survey status table
    20. -
    21. Cave pages and handbook menu system
    22. - -
    -Appendices: - - -

    Getting a username, password and key

    - -

    You don't need a password to view most things, but you will need one to change them.

    - -

    Use these credentials for access to the troggle site. The user is 'expo', - with a cavey:beery password. Ask someone if this isn't enough clue for you. - This password is important for security. The whole site will get hacked by spammers or worse if you are not careful with it. Use a secure method for passing it on to others that need to know (i.e not unencrypted email), don't publish it anywhere, don't check it in to the data management system by accident. A lot of people use it and changing it is a pain for everyone so do take a bit of care. -

    - -

    This password is all you need to log in to troggle and to use the troggle control panel (very few people need to do this). But if you want to update webpages (a much more common requirement) or to edit the software itself (very rare), then -you will also need to get a login (register a key with the server). See key-pair setup for details. - -

    Pushing cave data to the ::loser:: and ::drawings:: repositories also needs a key. So cavers entering their cave survey data have to use a machine on which this already set up. These machines are -the expo laptop and the laptop 'aziraphale' which live in the potato hut during expo. If you want to use your own laptop then -see below. - - -

    The repositories

    - -

    All the expo data is contained in 4 "repositories" at -expo.survex.com. This is currently hosted on a free virtual server we have blagged on a server farm. -We use a distributed version control system (DVCS) to manage these repositories because this allows simultaneous collaborative -editing and keeps track of all changes so we can roll back and have branches if needed.

    - -

    The site has been split into four parts:

    - - -

    We have migrated two of these to git but the other two still use mercurial. - -

    Mercurial Website Hack 2019

    -

    Currently (December 2019) after commiting and pushing your changes to expoweb to the mercurial server, you will need to -login to expo.survex.com using ssh, cd to /expoweb/ and issue a "hg update" command to make your changes noticed by the webserver. This problem will go away before Expo 2020 - we hope - when we finish migrating from mercurial to git. - -

    All the scans, photos, presentations, fat documents and videos are -stored just as files (not in version control) in 'expofiles'. See -below for details on that.

    - -

    How the data management system works

    - -

    Troggle runs the expo cave survey data management, presents the data on the website and manages the Expo Handbook. See the troggle intro. - -

    Anything you check in which affects cave data or descriptions won't appear on the site until -the data management system update scripts are run. -This should happen automatically every 30 mins (not since 2017), but you can also kick off a manual update. -See 'The expoweb-update script' below for details.

    - -

    Also note that the ::expoweb:: web pages and cave data reports you see on the visible website -are not the same as the version-controlled "master" expoweb repo. -So in order that your committed and pushed changes become visible on the website, -they have to be 'pulled' from the repo (on teh server machine) onto the webserver (another place on the same server machine) before your changes are reflected.

    - -

    Your own laptop

    -

    Setting your own laptop so that it can do everything the expo laptop can do is quite a -complicated process. At a minimum you will be an experienced software nerd already and will have git, mercurial and a text editor installed and you will know how to use them. -You will have done the -key-pair setup process - which you can only do entirely on your own if -you have access to the expo laptop. -

    See setting up a minimal laptop for a short list of software. This assumes you know how to use it all. -
    See setting up your own laptop for the full list of software we use and where to get it. -

    Note that the instructions are primarily for people using Linux with some help for those using Windows. If you are a Mac user then you are on your own. - - -

    Using 'Edit This Page'

    -

    This can be used to edit web pages without installing any software or doing any key-pair setup. It even works if your laptop is a Mac. -

    This is the capability that you can see in the top-left-hand menu on any website page if you log in to troggle using the cavey:beery password. -

    'Edit This Page' is a troggle capability edits the file served by the webserver but it does not update the copy of the file in the -repository (the invese of the problem described above as 'Mercurial Website Hack'). To properly finish the job you need to -

    -

    Again, we hope that this issue will go away when we migrate the expoweb repo from mercurial to git before the 2020 Expo. - - -

    Editing the data management system

    - -

    To edit the data management system fully, you need to use the version control system - software which is currently git and mercurial. -Some (static text) pages can be edited directly on-line using the 'edit this page link' which you'll -see if you are logged into troggle. In general the dynamically-generated pages, such as those describing -caves which are generated from the cave survey data, can not be edited in this way, but forms are provided -for some types of these like 'caves'.

    - -

    Cheat lists and quick reminders

    -

    If you know what you are doing here is the basic info on what's where:
    -(if you don't know what you're doing, skip to Editing the data management system below.) -

    - -

    Simple changes to static HTML files will take effect immediately (or as soon as the hg update hack is done, but this will disappear when we move entirely to git), -but changes to dynamically-generated files - cave descriptions, QM lists etc. - -will not take effect, until a nerd runs the expoweb-update script on the server.

    - -

    The expoweb & drawings import scripts

    - -

    The import scripts for the cave data are currently (Feb.2020) run manually by a nerd. So if you enter cave data,logbooks or survey scans you won't see the result -until a nerd has been placated.

    - - -

    Updating cave pages

    - -

    Cave description pages are automatically generated from a set of -cave files in noinfo/cave_data/ and noinfo/entrance_data/. These files -are named -.html (where area is 1623 or 1626). These -files are processed by troggle. Use python databaseReset.py -caves in /expofiles/troggle/ to update the site/database after -editing these files. -Use python databaseReset.py - on its own to get a list of command options and brief explanations.

    - -

    Clicking on 'New cave' (at the bottom of the cave index) lets you enter a new cave. Info on how to enter new caves has been split into its own page.

    - -

    (If you remember something about CAVETAB2.CSV for editing caves, that was -superseded in 2012).

    -

    This may be a useful reminder of what is in a survex file how to create a survex file. - - -

    Updating expo year pages

    - -

    Each year's expo is recorded in the folder

    - -/expoweb/years/ - -

    which contains a number of files used to manage and record that year's expo. Have a look at -expoweb/years/2018/ for a recent well-documented expo (the weather was good). -Files are added and edited using the version control system for the expoweb repository.

    - -

    To create a new 'year' for next year's expo see adding a new year. - - - -

    Maintaining the survey status table

    -

    See the documentation on updating the online surveyscans folders using the lever-arch file of plastic wallets. - -

    Cave pages and handbook menu design

    -

    See the menu design history and proposals -page on where we are and what we might do to improveand fix menus. - - -


    - - - diff --git a/handbook/troggle/trogintro.html b/handbook/troggle/trogintro.html index c2f164ece..4e51c28b8 100644 --- a/handbook/troggle/trogintro.html +++ b/handbook/troggle/trogintro.html @@ -46,7 +46,7 @@ All the data of all kinds is stored in files. When troggle starts up it imports Go on to: troggle status
    Return to: -expo online systems overview
    +expo online systems overview

    diff --git a/handbook/troggle/trogmanual.html b/handbook/troggle/trogmanual.html index 7efb3d25d..9cfeb2bc2 100644 --- a/handbook/troggle/trogmanual.html +++ b/handbook/troggle/trogmanual.html @@ -38,7 +38,7 @@ hand corner. These are the handbook you are reading now and the historic records
  • Provides a way of editing individual pages of the handbook for very quick and urgent changes. -[This is the "Edit this page" capability; see for +[This is the "Edit this page" capability; see for how to use it and how to tidy up afterwards].
    diff --git a/handbook/troggle/trognotes.html b/handbook/troggle/trognotes.html index 9340bcaf9..e42aedaad 100644 --- a/handbook/troggle/trognotes.html +++ b/handbook/troggle/trognotes.html @@ -32,7 +32,7 @@ can do this by filling-in some online forms. (And managing all the cave suvey da
  • Providing a secondary way of editing individual pages of the handbook and historic records pages for very quick and urgent changes. -This is the "Edit this page" capability; see for +This is the "Edit this page" capability; see for how to use it and how to tidy up afterwards. diff --git a/handbook/website-history.html b/handbook/website-history.html index 26549bb22..aa51ccbeb 100644 --- a/handbook/website-history.html +++ b/handbook/website-history.html @@ -63,7 +63,7 @@ more straightforward.

    Version control

    Another important element of this system was version control. The entire data structure was stored initially in a Concurrent Version System repository, and later migrated to -Subversion [now using a DVCS in 2019]. +Subversion [now using a DVCS in 2019]. Any edits to the spreadsheets which caused the scripts to fail, breaking the website, could be easily reversed. @@ -118,7 +118,7 @@ This reads in all the logbooks and surveys and provides a nice way to access the It was separate for a while until Martin Green added code to merge the old static pages and new troggle dynamic pages into the same site. This is now the live system running everything (in 2019). Work on developing Troggle further still continues (see Troggle intro).

    -

    After Expo 2009 the version control system was updated to a DVCS (Mercurial, aka 'hg'), +

    After Expo 2009 the version control system was updated to a DVCS (Mercurial, aka 'hg'), because a distributed version control system makes a great deal of sense for expo (where it goes offline for a month or two and nearly all the year's edits happen).

    @@ -151,7 +151,7 @@ For the current situation see expo systems sta
    Return to
    -
    expo online systems overbiew
    +expo online systems overbiew

    diff --git a/infodx.htm b/infodx.htm index 4ed65cf64..0029f3e83 100644 --- a/infodx.htm +++ b/infodx.htm @@ -102,7 +102,7 @@ and data-management procedures, the respository software system and manual proce
  • Full-size surveys
  • Rigging topos
  • Terrain models
  • -
  • Updating data on the server - online systems +
  • Updating data on the server - online systems (including most of the website and handbook pages). Experts only.
  • Tunnel Wiki (documentation)