troggle/README.txt

Updated 2 May 2023

Troggle is an application for caving expedition data management, 
originally created for use on Cambridge University Caving Club (CUCC)expeditions 
and licensed under the GNU Lesser General Public License.

Troggle has been forked into two projects. The original one is maintained by Aaron Curtis 
and was used for Erebus caves in Antarctica. 
The CUCC variant uses files as the definitive data, not the database, and lives at http://expo.survex.com/repositories/troggle/.git/

For the server setup, see /_deploy/debian/wookey-exposerver-recipe.txt
and see http://expo.survex.com/handbook/troggle/serverconfig.html

Much material which was in this file has been moved to 
http://expo.survex.com/handbook/troggle/serverconfig.html

See copyright notices in
http://expo.survex.com/handbook/computing/contribute.html
and for context see
http://expo.survex.com/handbook/computing/onlinesystems.html 

Troggle setup
=============
0. read the very extensive online documentation and stop reading this README...
well, come back to this README after you have read the HTML pages. Not everything has been transferred.

   http://expo.survex.com/handbook/troggle/troglaptop.html
   http://expo.survex.com/handbook/troggle/serverconfig.html
   http://expo.survex.com/handbook/troggle/trogdangoup.html
   and at troggle/debian/serversetup 
1. set up the ssh key-exchange with the git server so you can clone troggle
   http://expo.survex.com/handbook/computing/keyexchange.html

Setting up directories
----------------------
see  http://expo.survex.com/handbook/troggle/troglaptop.html and 
http://expo.survex.com/handbook/troggle/serverconfig.html

Next, you need to fill in your local settings. Copy _deploy/WSL/localsettingsWSL.py 
to a new file called localsettings.py and edit it and settings.py to match
your machine's file locations. 
Follow the instructions contained in the file to fill out your settings.

{ in _deploy/old/ we have these which are all very out of date:
  localsettings-expo-live.py is the python2.7 settings for the server.
  localsettingsubuntu.py
  localsettingsdocker.py
  localsettingswindows.py
  localsettingspotatohut.py
}

Python3, Django, and Database setup
-----------------------------------
We are now using Django 3.2 and will move to 4.2 in 2024
We are installing with python 3.11 (the server is running 3.9)

Install Django using pip, not with apt, on your test system in a venv.
Conventionally on our main master expo server we install everything that we can as debian packages, not using pip.

[installation instructions removed - now in http://expo.survex.com/handbook/troggle/troglaptop.html ]

[venv description removed - read it in http://expo.survex.com/handbook/troggle/troglaptop.html ]

READ the os-trog.sh script !
READ the venv-trog.sh script !


Automatic Provisioning and Configuration
----------------------------------------
We don't do this - yet.

The most appropriate configuration tools today (2021) appear to be Bolt or Ansible
https://puppet.com/docs/bolt/latest/bolt.html (declarative, local)
https://docs.ansible.com/ansible/latest/user_guide/intro_getting_started.html (procedural, remote)
https://puppet.com/blog/automating-from-zero-to-something/ 

We don't need anything for the deploy server itself, but we could do with something for setting
up test servers quickly to help get newbie developers up to speed faster. But learning a new tool
creates a barrier in itself. This is one reason most of us don't use Docker.

CSS and media files
-------------------
We are not using the STATICFILES capability.
We are serving css files from troggle/media/.. (see urls.py) 

Plain CSS pages
---------------
When running the test server 
manage.py runserver 0.0.0.0:8000
and without Apache running, we are serving CSS using using this Django 'view':
view_surveys.cssfilessingle 
i.e.
cssfilessingle() in core/view_surveys.py

Setting up survex
-----------------
You need to have survex installed as the command line tools 'cavern' is
used as part of the survex import process.
$ sudo apt install survex

Setting up tables and importing survey data
-------------------------------------------
Run 
$ sudo python databaseReset.py
from the troggle directory will give you instructions.

[ NB Adding a new year/expedition requires adding a column to the
folk/folk.csv table - a year doesn't exist until that is done.]


MariaDB database
----------------
Start it up with
$ sudo mysql -u -p
when it will prompt you to type in the password. Get this by reading the settings.py file in use on the server.
then
> CREATE DATABASE troggle;
> use troggle;
> exit;

Note the semicolons.

You can check the status of the db service:
$ sudo systemctl status mysql

You can start and stop the db service with
$ sudo systemctl restart mysql.service
$ sudo systemctl stop mysql.service
$ sudo systemctl start mysql.service

While logged in at a terminal session as expo on expo.survex.,com

$ mysql -h localhost -u expo -p<password>
will get you the MariasDb command prompt: https://www.hostwinds.com/guide/how-to-use-mysql-mariadb-from-command-line/

then (Note the SEMICOLONS !):
>drop database troggle;
>create database troggle;
>quit
Somewhere I have notes for the GRANT PRIVS type runes...

Ah yes:
CREATE DATABASE troggle;
GRANT ALL PRIVILEGES ON troggle.* TO 'expo'@'localhost' IDENTIFIED BY 'somepassword'; FLUSH PRIVILEGES; (at mysql root prompt)

(explained on https://chartio.com/resources/tutorials/how-to-grant-all-privileges-on-a-database-in-mysql/)
(but you need to create the database too)

The GRANT ALL PRIVILEGES  bit requires you to logon in to MariaDB as root. sudo doesn't cut it.
these permissions are set in a different 'info' database which usually is untouched even if database troggle gets creamed.

The 'somepassword' is specified int he localsettings.py file.


PERMISSIONS
https://linuxize.com/post/usermod-command-in-linux/

THIS MAY BE OUT OF DATE - from 2022 we are running Apache as user 'expo' not 'www-data'

so that the online editing system for SVX files works.
The same goes for /expoweb/ files, so that "edit this page" works and the New Cave
and New Entrance forms work.

sudo usermod -a expo expocvs
the expocvs group is used for git 

all the users should be in this group


Running a Troggle server with Apache
------------------------------------
Troggle also needs these aliases to be configured. These are set in 
/home/expo/config/apache/expo.conf
on the expo server.

At least these need setting:
DocumentRoot              /home/expo/expoweb
WSGIScriptAlias        /  /home/expo/troggle/wsgi.py
<Directory /home/expo/troggle>
	<Files wsgi.py>
		Require all granted
	</Files>
</Directory>

the instructions for apache Alias commands are in comments at the end of
the urls.py file.

Unlike the django "manage.py runserver" method, apache requires a restart before it will use
any changed files:

sudo service apache2 restart

Olly's comments 20 July 2020:
olly: looking at /lib/systemd/system/apache2.service suggests so

olly: ExecStart=/usr/sbin/apachectl start
olly: ExecStop=/usr/sbin/apachectl stop
olly: ExecReload=/usr/sbin/apachectl graceful

Additions
---------
The python code has been manually cleaned using the 'black' and 'ruff' lint tools,
and the 'deptry' dependency checker. This needs doing every year or so.
See dependencies-check-deptry.txt

See troggle/pyproject.toml for configurations

Experimental additions
----------------------
These are untried tools which may help us document how troggle works in future.

pip install pygraphviz 
pip install pyparsing pydot # installs fine
django extension graph_models # https://django-extensions.readthedocs.io/en/latest/graph_models.html