2020-04-09 21:58:40 +01:00
<!DOCTYPE html>
< html >
< head >
< meta http-equiv = "Content-Type" content = "text/html; charset=utf-8" / >
< title > Handbook - Expo Server< / title >
< link rel = "stylesheet" type = "text/css" href = "../../css/main2.css" / >
< / head >
2020-04-22 19:37:10 +01:00
< body > < style > body { background : #fff url ( /images/style/bg-system.png ) repeat-x 0 0 } < / style >
2020-05-04 00:49:40 +01:00
< h2 id = "tophead" > CUCC Expedition Handbook: Server< / h2 >
2020-04-09 21:58:40 +01:00
< h1 > The Expo server< / h1 >
< p > Troggle is the software which runs the the expo cave survey data management and website. It is not the only thing running on the expo server.
2020-05-04 00:49:40 +01:00
< h3 id = "what" > Server configuration: directory structure< / a > < / h3 >
< p > This is the folder structure of the repo < var > :expoweb:< / var > which is also the root
of the website. Note that the webserver (apache) presents many more apparent folders, such as < var > expofiles< / var > , than there are folders because that's what it and troggle do.
< p > In the list below only the < var > handbook< / var > folder has been expanded. The < var > years< / var > folder includes
42 subfolders from < var > 1976< / var > to < var > 2019< / var > .
< code > < textarea id = "folders" rows = "30" cols = "30" >
2020-05-19 19:08:53 +01:00
expoweb (repo)
2020-05-04 00:49:40 +01:00
├── 1623
├── 1626
├── cave_data
├── css
├── documents
├── entrance_data
├── folk
├── gallery
├── handbook
│ ├── computing
│ ├── festering
│ ├── i
│ ├── l
│ ├── putty
│ ├── rig
│ ├── survey
│ ├── t
│ └── troggle
├── images
├── map
├── maps
├── noinfo
├── others
├── pending
├── photos
├── piclinks
├── scripts
├── signup
├── templates
├── tinypix
├── topos
└── years< / textarea > < / code >
< p > The handbook illustrates the i/t/l idiom whereby an image file (i) is displayed with a paragraph of text as an HTML file (l) and there is a thumbnail image (t) which is included in another document, e.g. see < a href = "../../1623/walkin.htm" > the photographic guide to the walk from the toll road car park< / a > .
2021-10-29 20:16:55 +01:00
< h4 id = "css" > CSS and HTML< / a > < / h4 >
2021-03-21 21:46:21 +00:00
2021-10-29 20:16:55 +01:00
< p > The < var > expoweb< / var > repo contains all the HTML files and basic CSS files. The < var > troggle< / var > repo contains the HTML templates, CSS for the troggle report pages generated dynamically and a copy of the CSS, thumbnail images and javascript for the Django admin package which has to be available under the url '/site_media/' (which resolves to expo/troggle/media/) for the admin system to work.
< p >
See the live report on which urls resolve to which actual folders at < a href = "/pathsreport" > pathsreport< / a > .
2021-03-21 21:46:21 +00:00
2022-02-27 22:17:18 +00:00
2020-05-04 00:49:40 +01:00
< h3 id = "what" > Server configuration: software< / a > < / h3 >
< ul >
2022-02-27 22:17:18 +00:00
< li > < b > apache webserver *< / b >
< li > < b > expofiles *< / b >
< li > < b > MySQL/MariaDB database *< / b >
< li > < a href = "#xapian" > xapian< / a > (search function) - < a href = "https://xapian.org/" > xapian.org< / a > . Wookey did this in 2020.
< li > < a href = "#kanboard" > kanboard< / a > (task planning) - < a href = "https://kanboard.org/" > kanboard.org< / a > . Wookey did this in 2022.
< li > < a href = "#boe" > boe< / a > (bank of expo - not troggle)
< li > < a href = "#cgit" > cgit< / a > (live webpage showing git status of repos) - < a href = "https://git.zx2c4.com/cgit/about/" > qit.zx2c4.com/cgit< / a >
2021-10-29 20:16:55 +01:00
< li > bins (used to regenerate the photo albums)
2022-02-27 22:17:18 +00:00
< li > packaged javascript
2020-05-04 00:49:40 +01:00
< li > cron
2022-02-27 22:17:18 +00:00
< li > Makefile (obsolete?)
< li > < a href = "#runserver" > Running a test Troggle server< / a >
2020-05-04 00:49:40 +01:00
< / ul >
2022-02-27 22:17:18 +00:00
< b > * These are essential< / b > to make troggle work at all.
2020-04-09 21:58:40 +01:00
2021-10-29 20:16:55 +01:00
< p > The server configuration scripts are in the file < var > troggle/debian/serversetup< / var > and are also
2021-03-21 21:46:21 +00:00
documented with notes in < var > troggle/README.txt< / var > . It is intended that the full documentation will
2021-10-29 20:16:55 +01:00
be copied here in due course.
< p > It is hoped that we will develop fully automated server setup scripts (such as are used by CUYC for their Django system)
2021-03-21 21:46:21 +00:00
< ul >
< li > to create a development server running locally on a personal machine
< li > to install the master system following a new operating system install on the public server
< / ul >
2020-05-19 19:08:53 +01:00
< p > The folder structure on the server is as shown below. It is all
in the user folder for the user < var > expo< / var > i.e. in
< br > /home/user/expo:
< code > < textarea id = "folders" rows = "30" cols = "40" >
expo
├── boe
│ └── ...
├── cuccfiles
│ └── ...
├── loser (repo)
│ ├── caves
│ ├── caves-1623
│ ├── caves-1624
│ ├── caves-1626
│ ├── caves-1627
│ ├── docs
│ ├── fixedpts
│ ├── gpx
│ ├── kataster
│ ├── qms
│ ├── routes
│ ├── surface
│ └── template
├── drawings (repo)
│ ├── 107
│ ├── 161
│ ├── 2010-03
│ ├── 2011-01
│ ├── 2014-ms-14
│ ├── 2014-neo-01
│ ├── 2014-sd-01
│ ├── 2015-mf-06 Purple Lupin Cave
│ ├── 2017-cucc-23
│ ├── 2017-cucc-24
│ ├── 2017-cucc-28
│ ├── 2017-gmh
│ ├── 2017-pw-01
│ ├── 204
│ ├── 234
│ ├── 264-and-258
│ ├── minorCavesNear204
│ ├── oldframes
│ ├── oldframes2016
│ ├── styles
│ └── therion
├── expoweb (repo)
│ └── [see above]
├── troggle (repo)
│ ├── core
│ ├── debian
│ ├── docker
│ ├── docsEtc
│ ├── export
│ ├── flatpages
│ ├── helper
│ ├── imagekit
│ ├── media
│ ├── parsers
│ ├── profiles
│ ├── templates
│ └── wiki
├── static
│ └── ...
├── repositories
│ ├── cvs
│ ├── git
│ └── hg
├── expofiles
│ └── ...
├── expofiles-private
│ └── ...
└── webphotos
└── ...< / textarea > < / code >
2021-03-21 21:46:21 +00:00
< h4 id = "expofiles" > expofiles< / a > < / h3 >
< p > < var > expofiles< / var > contains ~40GB of files which are published by the webserver but which are not parsed by troggle. 28GB of these are photographs in < var > /expofiles/photos/< / var > and there are over 4GB of scanned images of surveys in < var > /expofiles/surveyscans/< / var > . There is a cleaned, complete copy of the documentation for the < var > tunnelX< / var > cave plan drawing package in < var > /expofiles/tunnelwiki/< / var > .
< p >
2020-05-19 19:08:53 +01:00
< code > < textarea id = "folders" rows = "30" cols = "40" >
expofiles
├── accounts
│ ├── 2016
│ ├── 2017
│ ├── misery
│ └── policy
├── aerial_photos
├── defaults
├── documents
│ ├── 2014kataster_applications
│ ├── 2020-expo-planning
│ ├── DistoX
│ ├── GPF
│ ├── SAP
│ ├── austria-1984
│ ├── bierbook
│ ├── cave_science_articles
│ ├── excursions2018
│ ├── grants
│ ├── hardware
│ ├── proc.ubss
│ ├── surveying
│ ├── troggle
│ ├── tshirts
│ └── website-link-checks
├── geotiffsurveys
│ └── qgis-examples
├── geotiffsurveys_backup
│ └── qgis-examples
├── gpslogs
│ ├── 2017
│ ├── 2018
│ ├── 2019
│ ├── OpenStreetMap
│ └── essentials
├── location_maps
├── photos
│ ├── 1978
│ ├── 1981
│ ├── 1982
│ ├── 1983
│ ├── 1997
│ ├── 1999
│ ├── 2004
│ ├── 2005
│ ├── 2007
│ ├── 2008
│ ├── 2009
│ ├── 2010
│ ├── 2011
│ ├── 2012
│ ├── 2013
│ ├── 2014
│ ├── 2015
│ ├── 2016
│ ├── 2017
│ ├── 2018
│ ├── 2019
│ ├── OriginalPhotoCD-png
│ ├── WinklerChapterPhotos
│ └── kletteratlas
├── presentations
│ ├── CroatiaTrip2019
│ ├── HiddenEarth
│ ├── SpeleoAustria
│ ├── location_maps
│ ├── misc_2018-2019
│ └── test3js
├── qgis_projects
├── qgis_resources
│ ├── centreline-data
│ ├── country_borders
│ └── terrain
├── rigging_topos
│ ├── 107
│ ├── 204
│ ├── 258
│ ├── 264
│ ├── 76
│ ├── rigging-guides-found-in-hut-2019
│ └── rigging_topos2017-18
├── surveys
│ ├── 107
│ ├── 148
│ ├── 161
│ ├── 1987-02
│ ├── 2004-01
│ ├── 2004-03
│ ├── 2007-71
│ ├── 204
│ ├── 234
│ ├── 258
│ ├── 258_and_264
│ ├── 264
│ ├── 267
│ ├── 271
│ ├── 40
│ ├── 76
│ ├── drafts
│ ├── junk
│ ├── minisurveys2012
│ ├── minisurveys2014
│ ├── minisurveys2016
│ ├── minisurveys2017
│ ├── minisurveys2018
│ ├── overview
│ ├── smkhs
│ └── tunnocks
├── surveyscans
│ ├── 1975
│ ├── 1983
│ ├── 1984
│ ├── 1989LUSS
│ ├── 1989surveybook
│ ├── 1991surveybook
│ ├── 1992-94Surveybookkh
│ ├── 1994
│ ├── 1995-96kh
│ ├── 1996-1999NotKHbook
│ ├── 1997-99kh
│ ├── 1999
│ ├── 2000
│ ├── 2001
│ ├── 2002
│ ├── 2003
│ ├── 2004
│ ├── 2005
│ ├── 2006
│ ├── 2007
│ ├── 2008
│ ├── 2009
│ ├── 2010
│ ├── 2011
│ ├── 2012
│ ├── 2013
│ ├── 2014
│ ├── 2015
│ ├── 2016
│ ├── 2017
│ ├── 2018
│ ├── 2019
2021-03-21 21:46:21 +00:00
│ ├── 2021
2020-05-19 19:08:53 +01:00
│ └── 90Surveybookkh
├── terrain
├── training-info
├── tunnelwiki
│ ├── pdf
│ ├── skins
│ ├── training-info
│ ├── uploads
│ └── wiki
├── uploads
├── video
│ ├── 2004
│ ├── 2007
│ ├── 2009
│ ├── 2010
│ ├── 2011
│ ├── 2012
│ ├── 2013
│ ├── 2014
│ ├── 2016
│ ├── 2017
│ ├── 2018
│ └── 2019
└── writeups
├── 1992
├── 1993
├── 1994
├── 1999
├── 2000
├── 2003
├── 2007
├── 2008
├── 2009
├── 2010
├── 2011
├── 2012
├── 2013
├── 2014
├── 2015
├── 2016
├── 2017
├── 2018
├── 2019
2021-03-21 21:46:21 +00:00
└── 2021< / textarea > < / code >
2020-05-04 00:49:40 +01:00
2022-02-27 22:17:18 +00:00
< h3 id = "js" > Javascript< / a > < / h3 >
There are javascript packages installed directly on the operating system on the server, i.e. in< br >
< code > /usr/share/javascript< / code >
which are used by some < var > expoweb< / var > pages:
< ul >
< li > < var > CaveScript< / var > for showing 3D views of cave centreline data,
< li > < var > CodeMirror< / var > for syntax-colouring survex files
< li > < var > openlayers< / var > for geographic mapping integration, for out slippy map showing paths and entrances at < a href = "/map/map.html" > /map/map.html< / a >
< li > < var > leaflets< / var > and experimental mapping test (only on /map/index-leaflet.html )
< / ul >
These javascript packages are not documented but some are mentioned in < a href = "troglaptop.html" > the soft dev laptop< / a > configuration.
< p > Currently installed on the server (October 2021) are the following javascript packages which are < em > not used at all by troggle or any of the expo website< / em > :
< ul >
< li > jquery
< li > jquery-ui
< li > mootools
< li > sphinxdoc
< li > underscore
< / ul >
< p > Presumably these are used by something else hosted on the server ? Anyway, if you are setting up a new troggle sever you don't need them.
2020-05-04 00:49:40 +01:00
2022-02-27 22:17:18 +00:00
< h3 id = "xapian" > Search: xapian< / h3 >
2020-05-04 00:49:40 +01:00
< p > Installed independently of troggle simply with < var > apt install xapian-omega< / var > and
2022-02-27 22:17:18 +00:00
then configured into the troggle-generated menus in < var > css/main2.css< / var > .
You can see it at the bottom of the top-left menu on this page and on nearly all pages of the handbook.
The function is
2020-05-04 00:49:40 +01:00
connected with an apache configuration
< code > ScriptAlias /search /usr/lib/cgi-bin/omega/omega< / code > in < var > ~expo/config/apache/expo.conf< / var > .
Installed by Wookey in May 2020.
2022-02-27 22:17:18 +00:00
< h3 id = "kanboard" > Planning: kanboard< / h3 >
< p > This is installed on the server and accessed at < a href = "/kanboard" > < var > /kanboard< / var > < / a > It is an open source
equivalent of the Trello kanban card task planning system. The 2022 expo uses
< a href = "https://trello.com/b/WddB62HI/expo-jobs" > Trello itself< / a > (separate login required) but we intend to move to our own kanboard
from 2023.
< h3 id = "boe" > boe< / h3 >
< p > This is a perl script, and served by the webserver using the url apache configuration< pre > < code > #bank of expo
#current expedition
ScriptAlias /boe /home/expo/boe/boc/boc.pl
< Directory /home/expo/boe/boc>
AddHandler cgi-script .pl
SetHandler cgi-script
Options +ExecCGI
Require all granted
< /Directory> < / code > < / pre > in < var > ~expo/config/apache/expo.conf< / var > .
< p > Handbook documentation for its use is at < a href = "../bankofexpo.html" > The Bank of Expo< / a > .
< h3 id = "cgit" > cgit - web interface to server git repos< / h3 >
< p > This is a compiled executable written in C which, like boe, is installed as an Apache CGI redirection.
The installation instructions are at < a href = "https://git.zx2c4.com/cgit/tree/README" > https://git.zx2c4.com/cgit/tree/README< / a >
but we use the Debian package < a href = "https://packages.debian.org/stable/cgit" > https://packages.debian.org/stable/cgit< / a > .
< h3 > cron - automatically run tasks to schedule< / h3 >
< p > This is currently disabled in Feb. 2022. If you need anything that would be done frequently (e.g. bins) you currently have to run it manually.
< p > The server runs it's hourly, daily and weekly scripts using the anacron system. In
< var > ~expo/config/cron/< / var > on the server, there are < var > expo.hourly< / var >
and < var > expo.daily< / var > scripts and these are (or should be) launched at the appropriate times by
the server root from < var > /etc/crontab< / var > . This is not obviously working on the server at present.
< h3 id = "runserver" > Running a test Troggle server< / h3 >
2021-10-29 20:16:55 +01:00
< p >
For high volume use, Troggle should be run using a web server like apache.
However, a quick way to get started is to use the development server built into Django.
This is limited though: directory redirection needs apache so CodeMirror and CaveView won't work as the url /javascript/ will not exist.
< p >
To do this, run
< code >
$ python manage.py runserver 8000 -v 3
< / code >
from the troggle directory. This runs it on port 8000 so you see the website
at http://localhost:8000/
< p >
gunicorn also works. This runs with 9 workers (suitable for a 4-core processor,
-w takes n+1 where n is the number of cores of your processor):
< code >
$ gunicorn --reload -w 9 -b :8000 wsgi
< / code >
2020-04-09 21:58:40 +01:00
< hr / >
2020-07-27 01:42:09 +01:00
Return to: < a href = "trogdesign.html" > Troggle design and future implementations< / a > < br / >
Return to: < a href = "trogintro.html" > Troggle intro< / a > < br / >
Troggle index:
< a href = "trogindex.html" > Index of all troggle documents< / a > < br / >
2020-04-09 21:58:40 +01:00
< hr / >
< / body >
< / html >
