mirror of
https://expo.survex.com/repositories/expoweb/.git/
synced 2024-11-22 23:31:56 +00:00
463 lines
25 KiB
HTML
463 lines
25 KiB
HTML
<!DOCTYPE html>
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
<title>Handbook - Expo Server</title>
|
|
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
|
|
</head>
|
|
<body>
|
|
<style>body { background: #fff url(/images/style/bg-system.png) repeat-x 0 0 }</style>
|
|
<h2 id="tophead">CUCC Expedition Handbook: Server</h2>
|
|
<h1>The Expo server</h1>
|
|
<p>Troggle is the software which runs the the expo cave survey data management and website. It is not the only thing running on the expo server.
|
|
|
|
<h3 id="what">Server configuration: directory structure</a></h3>
|
|
<p>This is the folder structure of the repo <var>:expoweb:</var> which is also the root
|
|
of the website. Note that the webserver (apache) presents many more apparent folders, such as <var>expofiles</var>, than there are folders because that's what it and troggle do.
|
|
<p>In the list below only the <var>handbook</var> folder has been expanded. The <var>years</var> folder includes
|
|
42 subfolders from <var>1976</var> to <var>2022</var>.
|
|
<code><textarea id="folders" rows="30" cols="30">
|
|
expoweb (repo)
|
|
├── 1623
|
|
├── 1626
|
|
├── cave_data
|
|
├── css
|
|
├── documents
|
|
├── entrance_data
|
|
├── folk
|
|
├── gallery
|
|
├── handbook
|
|
│ ├── computing
|
|
│ ├── festering
|
|
│ ├── i
|
|
│ ├── l
|
|
│ ├── putty
|
|
│ ├── rig
|
|
│ ├── survey
|
|
│ ├── t
|
|
│ └── troggle
|
|
├── images
|
|
├── map
|
|
├── maps
|
|
├── noinfo
|
|
├── others
|
|
├── pending
|
|
├── photos
|
|
├── piclinks
|
|
├── scripts
|
|
├── signup
|
|
├── templates
|
|
├── tinypix
|
|
├── topos
|
|
└── years</textarea></code>
|
|
|
|
<p>The handbook illustrates the i/t/l idiom whereby an image file (i) is displayed with a paragraph of text as an HTML file (l) and there is a thumbnail image (t) which is included in another document, e.g. see <a href="/guidebook/walkin.htm">the photographic guide to the walk from the toll road car park</a>.
|
|
|
|
<h4 id="css">CSS and HTML</a></h4>
|
|
|
|
<p>The <var>expoweb</var> repo contains all the HTML files and basic CSS files. The <var>troggle</var> repo contains the HTML templates, CSS for the troggle report pages generated dynamically and a copy of the CSS, thumbnail images and javascript for the Django admin package which has to be available under the url '/site_media/' (which resolves to expo/troggle/media/) for the admin system to work.
|
|
<p>
|
|
See the live report on which urls resolve to which actual folders at <a href="/pathsreport">pathsreport</a>.
|
|
|
|
|
|
<h3 id="what">Server configuration: software</a></h3>
|
|
|
|
<ul>
|
|
<li><b>apache webserver *</b>
|
|
<li><b>expofiles *</b>
|
|
<li><b>MySQL/MariaDB database *</b> <var>sudo apt install mariadb-server -y </var>
|
|
<li><a href="#xapian">xapian</a> (search function) - <a href="https://xapian.org/">xapian.org</a>. Wookey did this in 2020.
|
|
<li><a href="#kanboard">kanboard</a> (task planning) - <a href="https://kanboard.org/">kanboard.org</a>. Wookey did this in 2022.
|
|
<li><a href="#boe">boe</a> (bank of expo - not troggle)
|
|
<li><a href="#cgit">cgit</a> (live webpage showing git status of repos) - <a href="https://git.zx2c4.com/cgit/about/">qit.zx2c4.com/cgit</a>
|
|
<li>bins (used to regenerate the photo albums)
|
|
<li>packaged javascript
|
|
<li>cron
|
|
<li>Makefile (obsolete?)
|
|
<li><a href="#runserver">Running a test Troggle server</a>
|
|
</ul>
|
|
<b> * These are essential</b> to make troggle work at all.
|
|
<p>You will also need everythingto run Django, as documented in <a href="/handbook/troggle/troglaptop.html">troggle laptop</a> including all the python modules listed there and installed using pip.
|
|
|
|
<hr />
|
|
<p><b>Now, if you don't know</b> your <em>wsgi</em> from your <em>asgi</em>, read <a href=
|
|
"https://james.walters.click/what-django-deployment-is-really-about.html">this article
|
|
</a> first on how to generically configure a Django server in the simplest possible
|
|
way.
|
|
|
|
<hr />
|
|
|
|
<p>The server configuration scripts are in the file <var>troggle/_deploy/debian-server/wookey-exposerver-recipe.txt</var> and are also
|
|
documented with notes in <var>troggle/README.txt</var>. It is intended that the full documentation will
|
|
be moved here in due course.
|
|
<p>It is hoped that we will develop fully automated server setup scripts (such as are used by CUYC for their Django system)
|
|
<ul>
|
|
<li>to create a development server running locally on a personal machine
|
|
<li>to install the master system following a new operating system install on the public server
|
|
</ul>
|
|
|
|
<p>Apache needs to run as user 'expo', not 'www-data' as standard. This is due to a basic incompatibility in permissions between apache and git: git does not honour existing permissions exactly. See
|
|
<a href="https://nim-labs.com/run-apache-alternate-user/">How to run apache as an alternate user</a>.
|
|
|
|
<p>Although troggle will appear to work with sqlite database, it needs a proper concurrrent access database to manage multiple users. sqlite is
|
|
single-user (effectively a separate instance of django is created for each page access, so even one person looking at several pages at once is
|
|
"multi-user").
|
|
<p>The folder structure on the server is as shown below. It is all
|
|
in the user folder for the user <var>expo</var> i.e. in
|
|
<br>/home/user/expo:
|
|
<code><textarea id="folders" rows="30" cols="40">
|
|
expo
|
|
├── boe
|
|
│ └── ...
|
|
├── cuccfiles
|
|
│ └── ...
|
|
├── loser (repo)
|
|
│ ├── caves
|
|
│ ├── caves-1623
|
|
│ ├── caves-1624
|
|
│ ├── caves-1626
|
|
│ ├── caves-1627
|
|
│ ├── docs
|
|
│ ├── fixedpts
|
|
│ ├── gpx
|
|
│ ├── kataster
|
|
│ ├── qms
|
|
│ ├── routes
|
|
│ ├── surface
|
|
│ └── template
|
|
├── drawings (repo)
|
|
│ ├── 107
|
|
│ ├── 161
|
|
│ ├── 2010-03
|
|
│ ├── 2011-01
|
|
│ ├── 2014-ms-14
|
|
│ ├── 2014-neo-01
|
|
│ ├── 2014-sd-01
|
|
│ ├── 2015-mf-06 Purple Lupin Cave
|
|
│ ├── 2017-cucc-23
|
|
│ ├── 2017-cucc-24
|
|
│ ├── 2017-cucc-28
|
|
│ ├── 2017-gmh
|
|
│ ├── 2017-pw-01
|
|
│ ├── 204
|
|
│ ├── 234
|
|
│ ├── 264-and-258
|
|
│ ├── minorCavesNear204
|
|
│ ├── oldframes
|
|
│ ├── oldframes2016
|
|
│ ├── styles
|
|
│ └── therion
|
|
├── expoweb (repo)
|
|
│ └── [see above]
|
|
├── troggle (repo)
|
|
│ ├── core
|
|
│ ├── debian
|
|
│ ├── docker
|
|
│ ├── docsEtc
|
|
│ ├── export
|
|
│ ├── flatpages
|
|
│ ├── helper
|
|
│ ├── imagekit
|
|
│ ├── media
|
|
│ ├── parsers
|
|
│ ├── profiles
|
|
│ ├── templates
|
|
│ └── wiki
|
|
├── static
|
|
│ └── ...
|
|
├── repositories
|
|
│ ├── cvs
|
|
│ ├── git
|
|
│ └── hg
|
|
├── expofiles
|
|
│ └── ...
|
|
├── expofiles-private
|
|
│ └── ...
|
|
└── webphotos
|
|
└── ...</textarea></code>
|
|
|
|
<h4 id="expofiles">expofiles</a></h3>
|
|
|
|
<p><var>expofiles</var> contains ~40GB of files which are published by the webserver but which are not parsed by troggle. 28GB of these are photographs in <var>/expofiles/photos/</var> and there are over 4GB of scanned images of surveys in <var>/expofiles/surveyscans/</var>. There is a cleaned, complete copy of the documentation for the <var>tunnelX</var> cave plan drawing package in <var>/expofiles/tunnelwiki/</var>.
|
|
<p>
|
|
|
|
<code><textarea id="folders" rows="30" cols="40">
|
|
expofiles
|
|
├── accounts
|
|
│ ├── 2016
|
|
│ ├── 2017
|
|
│ ├── misery
|
|
│ └── policy
|
|
├── aerial_photos
|
|
├── defaults
|
|
├── documents
|
|
│ ├── 2014kataster_applications
|
|
│ ├── 2020-expo-planning
|
|
│ ├── DistoX
|
|
│ ├── GPF
|
|
│ ├── SAP
|
|
│ ├── austria-1984
|
|
│ ├── bierbook
|
|
│ ├── cave_science_articles
|
|
│ ├── excursions2018
|
|
│ ├── grants
|
|
│ ├── hardware
|
|
│ ├── proc.ubss
|
|
│ ├── surveying
|
|
│ ├── troggle
|
|
│ ├── tshirts
|
|
│ └── website-link-checks
|
|
├── geotiffsurveys
|
|
│ └── qgis-examples
|
|
├── geotiffsurveys_backup
|
|
│ └── qgis-examples
|
|
├── gpslogs
|
|
│ ├── 2017
|
|
│ ├── 2018
|
|
│ ├── 2019
|
|
│ ├── OpenStreetMap
|
|
│ └── essentials
|
|
├── location_maps
|
|
├── photos
|
|
│ ├── 1978
|
|
│ ├── 1981
|
|
│ ├── 1982
|
|
│ ├── 1983
|
|
│ ├── 1997
|
|
│ ├── 1999
|
|
│ ├── 2004
|
|
│ ├── 2005
|
|
│ ├── 2007
|
|
│ ├── 2008
|
|
│ ├── 2009
|
|
│ ├── 2010
|
|
│ ├── 2011
|
|
│ ├── 2012
|
|
│ ├── 2013
|
|
│ ├── 2014
|
|
│ ├── 2015
|
|
│ ├── 2016
|
|
│ ├── 2017
|
|
│ ├── 2018
|
|
│ ├── 2019
|
|
│ ├── OriginalPhotoCD-png
|
|
│ ├── WinklerChapterPhotos
|
|
│ └── kletteratlas
|
|
├── presentations
|
|
│ ├── CroatiaTrip2019
|
|
│ ├── HiddenEarth
|
|
│ ├── SpeleoAustria
|
|
│ ├── location_maps
|
|
│ ├── misc_2018-2019
|
|
│ └── test3js
|
|
├── qgis_projects
|
|
├── qgis_resources
|
|
│ ├── centreline-data
|
|
│ ├── country_borders
|
|
│ └── terrain
|
|
├── rigging_topos
|
|
│ ├── 107
|
|
│ ├── 204
|
|
│ ├── 258
|
|
│ ├── 264
|
|
│ ├── 76
|
|
│ ├── rigging-guides-found-in-hut-2019
|
|
│ └── rigging_topos2017-18
|
|
├── surveys
|
|
│ ├── 107
|
|
│ ├── 148
|
|
│ ├── 161
|
|
│ ├── 1987-02
|
|
│ ├── 2004-01
|
|
│ ├── 2004-03
|
|
│ ├── 2007-71
|
|
│ ├── 204
|
|
│ ├── 234
|
|
│ ├── 258
|
|
│ ├── 258_and_264
|
|
│ ├── 264
|
|
│ ├── 267
|
|
│ ├── 271
|
|
│ ├── 40
|
|
│ ├── 76
|
|
│ ├── drafts
|
|
│ ├── junk
|
|
│ ├── minisurveys2012
|
|
│ ├── minisurveys2014
|
|
│ ├── minisurveys2016
|
|
│ ├── minisurveys2017
|
|
│ ├── minisurveys2018
|
|
│ ├── overview
|
|
│ ├── smkhs
|
|
│ └── tunnocks
|
|
├── surveyscans
|
|
│ ├── 1975
|
|
│ ├── 1983
|
|
│ ├── 1984
|
|
│ ├── 1989LUSS
|
|
│ ├── 1989surveybook
|
|
│ ├── 1991surveybook
|
|
│ ├── 1992-94Surveybookkh
|
|
│ ├── 1994
|
|
│ ├── 1995-96kh
|
|
│ ├── 1996-1999NotKHbook
|
|
│ ├── 1997-99kh
|
|
│ ├── 1999
|
|
│ ├── 2000
|
|
│ ├── 2001
|
|
│ ├── 2002
|
|
│ ├── 2003
|
|
│ ├── 2004
|
|
│ ├── 2005
|
|
│ ├── 2006
|
|
│ ├── 2007
|
|
│ ├── 2008
|
|
│ ├── 2009
|
|
│ ├── 2010
|
|
│ ├── 2011
|
|
│ ├── 2012
|
|
│ ├── 2013
|
|
│ ├── 2014
|
|
│ ├── 2015
|
|
│ ├── 2016
|
|
│ ├── 2017
|
|
│ ├── 2018
|
|
│ ├── 2019
|
|
│ ├── 2021
|
|
│ └── 90Surveybookkh
|
|
├── terrain
|
|
├── training-info
|
|
├── tunnelwiki
|
|
│ ├── pdf
|
|
│ ├── skins
|
|
│ ├── training-info
|
|
│ ├── uploads
|
|
│ └── wiki
|
|
├── uploads
|
|
├── video
|
|
│ ├── 2004
|
|
│ ├── 2007
|
|
│ ├── 2009
|
|
│ ├── 2010
|
|
│ ├── 2011
|
|
│ ├── 2012
|
|
│ ├── 2013
|
|
│ ├── 2014
|
|
│ ├── 2016
|
|
│ ├── 2017
|
|
│ ├── 2018
|
|
│ └── 2019
|
|
└── writeups
|
|
├── 1992
|
|
├── 1993
|
|
├── 1994
|
|
├── 1999
|
|
├── 2000
|
|
├── 2003
|
|
├── 2007
|
|
├── 2008
|
|
├── 2009
|
|
├── 2010
|
|
├── 2011
|
|
├── 2012
|
|
├── 2013
|
|
├── 2014
|
|
├── 2015
|
|
├── 2016
|
|
├── 2017
|
|
├── 2018
|
|
├── 2019
|
|
└── 2021</textarea></code>
|
|
|
|
<h3 id="js">Javascript</a></h3>
|
|
There are javascript packages installed directly on the operating system on the server, i.e. in<br>
|
|
<code>/usr/share/javascript</code>
|
|
which are used by some <var>expoweb</var> pages:
|
|
<ul>
|
|
<li><var>CaveScript</var> for showing 3D views of cave centreline data,
|
|
<li><var>CodeMirror</var> for syntax-colouring survex files
|
|
<li><var>openlayers</var> for geographic mapping integration, for out slippy map showing paths and entrances at <a href="/map/map.html">/map/map.html</a>
|
|
<li><var>leaflets</var> and experimental mapping test (only on /map/index-leaflet.html )
|
|
</ul>
|
|
These javascript packages are not documented but some are mentioned in <a href="troglaptop.html">the soft dev laptop</a> configuration.
|
|
|
|
<p>Currently installed on the server (October 2021) are the following javascript packages which are <em>not used at all by troggle or any of the expo website</em>:
|
|
<ul>
|
|
<li>jquery
|
|
<li>jquery-ui
|
|
<li>mootools
|
|
<li>sphinxdoc
|
|
<li>underscore
|
|
</ul>
|
|
<p>Presumably these are used by something else hosted on the server ? Anyway, if you are setting up a new troggle sever you don't need them.
|
|
|
|
|
|
|
|
|
|
<h3 id="xapian">Search: xapian</h3>
|
|
<p>Installed independently of troggle simply with <var>apt install xapian-omega</var> and
|
|
then configured into the troggle-generated menus in <var>css/main2.css</var>.
|
|
You can see it at the bottom of the top-left menu on this page and on nearly all pages of the handbook.
|
|
The function is
|
|
connected with an apache configuration
|
|
<code>ScriptAlias /search /usr/lib/cgi-bin/omega/omega</code> in <var>~expo/config/apache/expo.conf</var>.
|
|
Installed by Wookey in May 2020.
|
|
|
|
<h3 id="kanboard">Planning: kanboard</h3>
|
|
<p>This is installed on the server and accessed at <a href="/kanboard"><var>/kanboard</var></a> It is an open source
|
|
equivalent of the Trello kanban card task planning system. The 2022 expo uses
|
|
<a href="https://trello.com/b/WddB62HI/expo-jobs">Trello itself</a> (separate login required) but we intend to move to our own kanboard
|
|
from 2023.
|
|
|
|
<h3 id="boe">boe</h3>
|
|
<p>This is a perl script, and served by the webserver using the url apache configuration<pre><code>#bank of expo
|
|
#current expedition
|
|
ScriptAlias /boe /home/expo/boe/boc/boc.pl
|
|
<Directory /home/expo/boe/boc>
|
|
AddHandler cgi-script .pl
|
|
SetHandler cgi-script
|
|
Options +ExecCGI
|
|
Require all granted
|
|
</Directory></code></pre> in <var>~expo/config/apache/expo.conf</var>.
|
|
<p>Handbook documentation for its use is at <a href="../bankofexpo.html">The Bank of Expo</a>.
|
|
|
|
<h3 id="cgit">cgit - web interface to server git repos</h3>
|
|
<p>This is a compiled executable written in C which, like boe, is installed as an Apache CGI redirection.
|
|
The installation instructions are at <a href="https://git.zx2c4.com/cgit/tree/README">https://git.zx2c4.com/cgit/tree/README</a>
|
|
but we use the Debian package <a href="https://packages.debian.org/stable/cgit">https://packages.debian.org/stable/cgit</a>.
|
|
|
|
<h3>cron - automatically run tasks to schedule</h3>
|
|
<p>This is currently disabled in Feb. 2022. If you need anything that would be done frequently (e.g. bins) you currently have to run it manually.
|
|
<p>The server runs it's hourly, daily and weekly scripts using the anacron system. In
|
|
<var>~expo/config/cron/</var> on the server, there are <var>expo.hourly</var>
|
|
and <var>expo.daily</var> scripts and these are (or should be) launched at the appropriate times by
|
|
the server root from <var>/etc/crontab</var>. This is not obviously working on the server at present.
|
|
|
|
<h3 id="runserver">Running a test Troggle server</h3>
|
|
<p>
|
|
For high volume use, Troggle should be run using a web server like apache.
|
|
However, a quick way to get started is to use the development server built into Django.
|
|
This is limited though: directory redirection needs apache so CodeMirror and CaveView won't work as the url /javascript/ will not exist.
|
|
<p>
|
|
To do this, run
|
|
<code>
|
|
$ python manage.py runserver 8000 -v 3
|
|
</code>
|
|
from the troggle directory. This runs it on port 8000 so you see the website
|
|
at http://localhost:8000/
|
|
<p>
|
|
gunicorn also works. This runs with 9 workers (suitable for a 4-core processor,
|
|
-w takes n+1 where n is the number of cores of your processor):
|
|
<code>
|
|
$ gunicorn --reload -w 9 -b :8000 wsgi
|
|
</code>
|
|
|
|
|
|
<hr />
|
|
Return to: <a href="trogdesign.html">Troggle design and future implementations</a><br />
|
|
Return to: <a href="trogintro.html">Troggle intro</a><br />
|
|
Troggle index:
|
|
<a href="trogindex.html">Index of all troggle documents</a><br />
|
|
<hr /></body>
|
|
</html>
|