expo/troggle
2
0
Fork 1
mirror of https://expo.survex.com/repositories/troggle/.git synced 2024-11-25 08:41:51 +00:00
Code Issues 19 Projects 3 Activity

clear cache on parsing & 2.2.19 notes

Browse Source
This commit is contained in:
Philip Sargent 2021-04-10 02:12:13 +01:00
parent 876868506f
commit a7e59b2bb0
3 changed files with 30 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
README.txt
View File

@ -1,8 +1,10 @@
Updated 10 April 2021
Troggle is an application for caving expedition data management, Troggle is an application for caving expedition data management,
originally created for use on Cambridge University Caving Club (CUCC)expeditions originally created for use on Cambridge University Caving Club (CUCC)expeditions
and licensed under the GNU Lesser General Public License. and licensed under the GNU Lesser General Public License.
Troggle has been forked into two projects. The original one is maintained by Aron Curtis Troggle has been forked into two projects. The original one is maintained by Aaron Curtis
and was used for Erebus caves. The CUCC variant uses files as the definitive data, and was used for Erebus caves. The CUCC variant uses files as the definitive data,
not the database and lives at http://expo.survex.com/repositories/troggle/.git/ not the database and lives at http://expo.survex.com/repositories/troggle/.git/
@ -14,10 +16,13 @@ and at troggle/debian/serversetup .
Troggle setup Troggle setup
============= =============
0. read the very extensive oinline documentation and stop reading this README.
1. git clone troggle into correct directory structure 1. git clone troggle into correct directory structure
2. install pip, django & patch django 2. install pip, django & patch django
3. configure django to recognise troggle 3. configure django to recognise troggle
See http://expo.survex.com/handbook/computing/onlinesystems.html
Setting up directories Setting up directories
---------------------- ----------------------
- create a directory in which you also have the loser/, expoweb/ and drawings/ repositories - create a directory in which you also have the loser/, expoweb/ and drawings/ repositories
@ -58,11 +63,11 @@ Follow the instructions contained in the file to fill out your settings.
Python3, Django, and Database setup Python3, Django, and Database setup
----------------------------------- -----------------------------------
First version of troggle using python3 required Django 1.8.19 (16 June 2019) First version of troggle using python3 required Django 1.8.19 (16 June 2019)
We are now using Django 1.11.29 We are now using Django 2.2.19
Read this: https://docs.djangoproject.com/en/3.0/topics/install/ Read this: https://docs.djangoproject.com/en/3.0/topics/install/
We are installing with python3.7.5 We are installing with python3.7.3
Also : https://linuxize.com/post/how-to-install-pip-on-ubuntu-20.04/ ] Also : https://linuxize.com/post/how-to-install-pip-on-ubuntu-20.04/ ]
Install Django using pip, not with apt, on your test system. Install Django using pip, not with apt, on your test system.
@ -79,13 +84,15 @@ $ sudo pip3 install -r requirements.txt
where requirements.txt is: where requirements.txt is:
confusable-homoglyphs==3.2.0 confusable-homoglyphs==3.2.0
Django==1.11.29 Django==2.2.19
django-registration==2.5.2 docutils==0.14
docutils==0.16 gunicorn==20.1.0
Pillow==7.1.2 Pillow==5.4.1
pytz==2020.1 pytz==2019.1
sqlparse==0.3.1 sqlparse==0.2.4
Unidecode==1.1.1 Unidecode==1.0.23
These minor things are as-standard on Debian Buster (10).
if you don't use sudo it will install them all in ~/.local/ if you don't use sudo it will install them all in ~/.local/
and so will only be available for you, not everyone; and and so will only be available for you, not everyone; and
@ -106,18 +113,13 @@ venv option
Or use a python3 virtual environment: (python3.5 not later) Or use a python3 virtual environment: (python3.5 not later)
$ cd troggle $ cd troggle
$ cd .. $ cd ..
$ python3.5 -m venv pyth35d2 $ python3.7 -m venv pyth37d2
(creates folder with virtual env) (creates folder with virtual env)
$ cd pyth35d2 $ cd pyth37d2
$ source bin/activate $ source bin/activate
(now install everything ) (now install everything )
$ pip install -r requirements.txt $ pip install -r requirements.txt
This works if you need a specific version of python3
but doesn't really work with python2.7 as you also need to obtain
an elderly version of pip that will work and use and older virtual
environment utility
Testing the django installation Testing the django installation
------------------------------- -------------------------------
Test things by running these commands: Test things by running these commands:
@ -157,11 +159,9 @@ which will give security warnings for deployment.
python manage.py check -Wall python manage.py check -Wall
Gives warnings of deprecated Django which should be fixed asap. Gives warnings of deprecated Django which should be fixed asap.
python manage.py test core python manage.py test -v 2
Runs our test suite Runs our test suite
python manage.py test
Runs tests for all the django plugins ("apps")
Registering troggle as a django application Registering troggle as a django application
------------------------------------------- -------------------------------------------
@ -219,7 +219,7 @@ folk/folk.csv table - a year doesn't exist until that is done.]
Database Database
-------- --------
If you want to use MySQL or Postgresql, download and install them. If you want to use MySQL or Postgresql, download and install them.
However, you can also use Django with sqlite3, which is included in Python and thus requires no extra installation. However, you can also use Django with sqlite3
MariaDB database MariaDB database
---------------- ----------------
@ -252,6 +252,9 @@ $ python manage.py runserver 8000 -v 3
from the troggle directory. This runs it on port 8000 so you see the website from the troggle directory. This runs it on port 8000 so you see the website
at http://localhost:8000/ at http://localhost:8000/
gunicorn also works. This runs with 9 workers (suitable for a 4-core processor):
$ gunicorn --reload -w 9 -b :8000 wsgi
EXTRAS EXTRAS
------ ------
cgit - https://git.zx2c4.com/cgit/about/ cgit - https://git.zx2c4.com/cgit/about/
@ -301,7 +304,7 @@ olly: ExecReload=/usr/sbin/apachectl graceful
Experimental additions Experimental additions
---------------------- ----------------------
These are modern tools which help us document how troggle works. These are untried tools which help us document how troggle works.
pip install pygraphviz pip install pygraphviz
pip install pyparsing pydot # installs fine pip install pyparsing pydot # installs fine

6
parsers/logbooks.py
View File

@ -10,7 +10,7 @@ from django.conf import settings
from django.template.defaultfilters import slugify from django.template.defaultfilters import slugify
from django.utils.timezone import get_current_timezone, make_aware from django.utils.timezone import get_current_timezone, make_aware
from troggle.core.models import DataIssue, Expedition from troggle.core.models import DataIssue, Expedition, TROG
from troggle.core.models_caves import Cave, LogbookEntry, PersonTrip, GetCaveLookup from troggle.core.models_caves import Cave, LogbookEntry, PersonTrip, GetCaveLookup
from parsers.people import GetPersonExpeditionNameLookup from parsers.people import GetPersonExpeditionNameLookup
from utils import save_carefully from utils import save_carefully
@ -86,7 +86,6 @@ def EnterLogIntoDbase(date, place, title, text, trippeople, expedition, logtime_
""" saves a logbook entry and related persontrips """ saves a logbook entry and related persontrips
Does NOT save the expeditionday_id - all NULLs. why? Does NOT save the expeditionday_id - all NULLs. why?
""" """
trippersons, author = GetTripPersons(trippeople, expedition, logtime_underground) trippersons, author = GetTripPersons(trippeople, expedition, logtime_underground)
if not author: if not author:
print(" ! - Skipping logentry: " + title + " - no author for entry") print(" ! - Skipping logentry: " + title + " - no author for entry")
@ -417,7 +416,7 @@ def SetDatesFromLogbookEntries(expedition):
persontrip.save() # also saved in EnterLogIntoDbase. MERGE these to speed up import. persontrip.save() # also saved in EnterLogIntoDbase. MERGE these to speed up import.
def LoadLogbookForExpedition(expedition,expect): def LoadLogbookForExpedition(expedition, expect):
""" Parses all logbook entries for one expedition """ Parses all logbook entries for one expedition
If a cache is found it uses it. If not found, or fails sanity checks, parses source file. If a cache is found it uses it. If not found, or fails sanity checks, parses source file.
""" """
@ -544,6 +543,7 @@ def LoadLogbooks():
expd ={} expd ={}
with open("loadlogbk.log", "a") as log: with open("loadlogbk.log", "a") as log:
for expo in expos: for expo in expos:
TROG['pagecache']['expedition'][expo.year] = None # clear cache
if expo.year not in nologbook: if expo.year not in nologbook:
print((" - Logbook for: " + expo.year)) print((" - Logbook for: " + expo.year))
numentries = LoadLogbookForExpedition(expo, entries[expo.year]) numentries = LoadLogbookForExpedition(expo, entries[expo.year])

2
templates/expedition.html
View File

@ -20,6 +20,8 @@
{% endfor %} {% endfor %}
</p> </p>
<p>Reparse and reload this year's logbook by clicking here: <a href="/expedition/{{expedition.year}}?reload">RELOAD</a>
<p><b>At a single glance:</b> The table shows all expo cavers and their recorded trips. <p><b>At a single glance:</b> The table shows all expo cavers and their recorded trips.
The columns are the date in the month (July or August), with a "T" for a logbook entry, and The columns are the date in the month (July or August), with a "T" for a logbook entry, and
an "S" for a survey trip. The colours are the same for people on the same trip.</p> an "S" for a survey trip. The colours are the same for people on the same trip.</p>