Updated 2 May 2023 Troggle is an application for caving expedition data management, originally created for use on Cambridge University Caving Club (CUCC)expeditions and licensed under the GNU Lesser General Public License. Troggle has been forked into two projects. The original one is maintained by Aaron Curtis and was used for Erebus caves in Antarctica. The CUCC variant uses files as the definitive data, not the database, and lives at http://expo.survex.com/repositories/troggle/.git/ The versions have diverged markedly, not just in the software but also in the implicit convnetions of how the directory structures of the survex files, the drawings and the scans are arranged. For the server setup, see /_deploy/debian/wookey-exposerver-recipe.txt and see http://expo.survex.com/handbook/troggle/serverconfig.html Much material which was in this file has been moved to http://expo.survex.com/handbook/troggle/serverconfig.html See copyright notices in http://expo.survex.com/handbook/computing/contribute.html and for context see http://expo.survex.com/handbook/computing/onlinesystems.html Troggle setup ============= 0. read the very extensive online documentation and stop reading this README... well, come back to this README after you have read the HTML pages. Not everything has been transferred. http://expo.survex.com/handbook/troggle/troglaptop.html http://expo.survex.com/handbook/troggle/serverconfig.html http://expo.survex.com/handbook/troggle/trogdangoup.html and at troggle/debian/serversetup 1. set up the ssh key-exchange with the git server so you can clone troggle http://expo.survex.com/handbook/computing/keyexchange.html Setting up directories ---------------------- see http://expo.survex.com/handbook/troggle/troglaptop.html and http://expo.survex.com/handbook/troggle/serverconfig.html Next, you need to fill in your local settings. Copy _deploy/WSL/localsettingsWSL.py to a new file called localsettings.py and edit it and settings.py to match your machine's file locations. Follow the instructions contained in the file to fill out your settings. { in _deploy/old/ we have these which are all very out of date: localsettings-expo-live.py is the python2.7 settings for the server. localsettingsubuntu.py localsettingsdocker.py localsettingswindows.py localsettingspotatohut.py } Python3, Django, and Database setup ----------------------------------- We are now using Django 3.2 and will move to 4.2 in 2024 We are installing with python 3.11 (the server is running 3.9) Install Django using pip, not with apt, on your test system in a venv. Conventionally on our main master expo server we install everything that we can as debian packages, not using pip. [installation instructions removed - now in http://expo.survex.com/handbook/troggle/troglaptop.html ] [venv description removed - read it in http://expo.survex.com/handbook/troggle/troglaptop.html ] READ the os-trog.sh script ! READ the venv-trog.sh script ! Automatic Provisioning and Configuration ---------------------------------------- We don't do this - yet. The most appropriate configuration tools today (2021) appear to be Bolt or Ansible https://puppet.com/docs/bolt/latest/bolt.html (declarative, local) https://docs.ansible.com/ansible/latest/user_guide/intro_getting_started.html (procedural, remote) https://puppet.com/blog/automating-from-zero-to-something/ We don't need anything for the deploy server itself, but we could do with something for setting up test servers quickly to help get newbie developers up to speed faster. But learning a new tool creates a barrier in itself. This is one reason most of us don't use Docker. CSS and media files ------------------- We are not using the STATICFILES capability. We are serving css files from troggle/media/.. (see urls.py) Plain CSS pages --------------- When running the test server manage.py runserver 0.0.0.0:8000 and without Apache running, we are serving CSS using using this Django 'view': view_surveys.cssfilessingle i.e. cssfilessingle() in core/view_surveys.py Setting up survex ----------------- You need to have survex installed as the command line tools 'cavern' is used as part of the survex import process. $ sudo apt install survex Setting up tables and importing survey data ------------------------------------------- Run $ sudo python databaseReset.py from the troggle directory will give you instructions. [ NB Adding a new year/expedition requires adding a column to the folk/folk.csv table - a year doesn't exist until that is done.] MariaDB database ---------------- Start it up with $ sudo mysql -u -p when it will prompt you to type in the password. Get this by reading the settings.py file in use on the server. then > CREATE DATABASE troggle; > use troggle; > exit; Note the semicolons. You can check the status of the db service: $ sudo systemctl status mysql You can start and stop the db service with $ sudo systemctl restart mysql.service $ sudo systemctl stop mysql.service $ sudo systemctl start mysql.service While logged in at a terminal session as expo on expo.survex.,com $ mysql -h localhost -u expo -p<password> will get you the MariasDb command prompt: https://www.hostwinds.com/guide/how-to-use-mysql-mariadb-from-command-line/ then (Note the SEMICOLONS !): >drop database troggle; >create database troggle; >quit Somewhere I have notes for the GRANT PRIVS type runes... Ah yes: CREATE DATABASE troggle; GRANT ALL PRIVILEGES ON troggle.* TO 'expo'@'localhost' IDENTIFIED BY 'somepassword'; FLUSH PRIVILEGES; (at mysql root prompt) (explained on https://chartio.com/resources/tutorials/how-to-grant-all-privileges-on-a-database-in-mysql/) (but you need to create the database too) The GRANT ALL PRIVILEGES bit requires you to logon in to MariaDB as root. sudo doesn't cut it. these permissions are set in a different 'info' database which usually is untouched even if database troggle gets creamed. The 'somepassword' is specified int he localsettings.py file. PERMISSIONS https://linuxize.com/post/usermod-command-in-linux/ THIS MAY BE OUT OF DATE - from 2022 we are running Apache as user 'expo' not 'www-data' so that the online editing system for SVX files works. The same goes for /expoweb/ files, so that "edit this page" works and the New Cave and New Entrance forms work. sudo usermod -a expo expocvs the expocvs group is used for git all the users should be in this group Running a Troggle server with Apache ------------------------------------ Troggle also needs these aliases to be configured. These are set in /home/expo/config/apache/expo.conf on the expo server. At least these need setting: DocumentRoot /home/expo/expoweb WSGIScriptAlias / /home/expo/troggle/wsgi.py <Directory /home/expo/troggle> <Files wsgi.py> Require all granted </Files> </Directory> the instructions for apache Alias commands are in comments at the end of the urls.py file. Unlike the django "manage.py runserver" method, apache requires a restart before it will use any changed files: sudo service apache2 restart Olly's comments 20 July 2020: olly: looking at /lib/systemd/system/apache2.service suggests so olly: ExecStart=/usr/sbin/apachectl start olly: ExecStop=/usr/sbin/apachectl stop olly: ExecReload=/usr/sbin/apachectl graceful Additions --------- The python code has been manually cleaned using the 'black' and 'ruff' lint tools, and the 'deptry' dependency checker. This needs doing every year or so. See dependencies-check-deptry.txt See troggle/pyproject.toml for configurations Experimental additions ---------------------- These are untried tools which may help us document how troggle works in future. pip install pygraphviz pip install pyparsing pydot # installs fine django extension graph_models # https://django-extensions.readthedocs.io/en/latest/graph_models.html