From 450375190779523036cbfde379a37b64162708e9 Mon Sep 17 00:00:00 2001 From: "Philip Sargent (laptop)" Date: Thu, 13 May 2021 00:00:39 +0300 Subject: [PATCH] improving README for a new dev install --- README.txt | 92 ++++++++++++++++--- .../{old => debian}/localsettingsserver.py | 7 +- .../{old => docker}/localsettingsdocker.py | 0 .../localsettingspotatohut.py | 1 + .../{old => ubuntu}/localsettingsubuntu.py | 4 +- security-warnings.txt | 3 +- 6 files changed, 90 insertions(+), 17 deletions(-) rename _deploy/{old => debian}/localsettingsserver.py (96%) rename _deploy/{old => docker}/localsettingsdocker.py (100%) rename _deploy/{old => potatohut}/localsettingspotatohut.py (97%) rename _deploy/{old => ubuntu}/localsettingsubuntu.py (94%) diff --git a/README.txt b/README.txt index 091cccf..a1c5f5c 100644 --- a/README.txt +++ b/README.txt @@ -69,6 +69,8 @@ We are installing with python3.7 Also : https://linuxize.com/post/how-to-install-pip-on-ubuntu-20.04/ ] Install Django using pip, not with apt, on your test system. +You will also definitely need sqlite3 even if you are planning to use another +database. Sqlite3 is used by the test harness system. Your Linux installation almost certainly already includes python3 and pip3 but in case it doesn't install those like this: @@ -76,13 +78,18 @@ $ sudo apt update $ sudo apt dist-upgrade $ sudo apt install python3 $ sudo apt install python3-pip +$ sudo apt install sqlite3 sqlite3-doc +$ cd /usr/bin +$ sudo ln -s python3 python +$ sudo ln -s pip3 pip Now install django etc. -$ sudo pip3 install -r requirements.txt +$ cd [troggle] +$ sudo pip install -r requirements.txt where requirements.txt is: confusable-homoglyphs==3.2.0 -Django==2.2.19 +Django==2.2 docutils==0.14 gunicorn==20.1.0 Pillow==5.4.1 @@ -96,19 +103,18 @@ if you don't use sudo it will install them all in ~/.local/ and so will only be available for you, not everyone; and the paths won't work to find troggle properly. -Unidecode handle some of the python2-3 conversions -and Pillow is an image handling package used to make +Pillow is an image handling package used to make the prospecting map. tinymce is the wysiwyg in-browser editor (disabled pending reinstatement) -$ pip3 list -o +$ pip list -o will list all the pip python packages installed. [NB we should test whether later verisons of tinymce work.] venv option ----------- -Or use a python3 virtual environment: (python3.5 not later) +Or use a python3 virtual environment: (python3.5 or later) $ cd troggle $ cd .. $ python3.7 -m venv pyth37d2 @@ -137,11 +143,29 @@ from within any folder on your machine. Now do $ cd troggle $ python manage.py + +You will get this error if everything is OK: (error: No module named 'localsettings') +So now rename one of the relevant platform files, e.g. if you are on WSL on Windows: +$ mv localsettingsWSl.py localsettings.py + +otherwise, if you are on debian: +$ cp _deploy/debian/localsettings.py . + +WARNING: only the WSL and debian variants are current in May 2021. All the others are so old that +they will need serious work to be useable. Copy what you need from WSL and debian variants of +localsettings.py + +Now try this again: +$ python manage.py + +and in addiiton to the [django] command list, you will now gets command lsists for [auth], +[contenttypes], [core], and [sessions]. These are the modules (plugins) loaaded into django. +[core] is the core of troggle (but not all of it: the input file parsers are not in [core]). + $ python manage.py check -v 3 --traceback -You should see the same list of commands that you saw with django-admin but wth -a lot of extra ones and no complaints. This means it is reading at least some of your -settings correctly. +Ideally this will list settings imports and then say +System check identified no issues (0 silenced). If you get an error your python sys.path is probably not set correctly yet. Do $ python -m site @@ -152,14 +176,56 @@ It should be at the top, which is where the current working directory is. Now: python manage.py check -v 3 --deploy -which will give security warnings for deployment. +which will give security warnings for deployment. Various middleware settings and +a warning not to use DEBUG=true in deployment. We always have DEBUG=True set for troggle +as otherwise the users get useless error messages. -python manage.py check -Wall +python -Wall manage.py check Gives warnings of deprecated Django which should be fixed asap. -python manage.py test -v 2 -Runs our test suite +Now you need to edit the following settings in your localsettings.py file to match your +development machine: +FILES = Path('/mnt/f/expofiles/') +EXPOFILES = Path('/mnt/f/expofiles/') + +All the other settings (drawings, expoweb etc.) will work fine if they are parallel directories +to the directory you installed troggle into. The troggle code can find out itself where it +is living. + +If you do not have a local copy of /expofiles/ (40 GB), you can use the expo server copy if +you set: +EXPOFILESREMOTE = TRUE +and then the FILES and EXPOFILES setings will be ignored. + +You will probably want to change the various EMAIL settings too. + +Now do +python manage.py test -v 2 +Runs our test suite. + +If you get an error, and you probably will, have a look in the test file, e.g. for this error: +FAIL: test_page_folk (troggle.core.TESTS.tests.PageTests) +look in the file troggle/code/TESTS/tests.py in the class PageTests. +It will also say: + File "/mnt/c/EXPO/troggle/core/TESTS/tests.py", line 266, in test_page_folk +which means that the asssert failure in on line 266 of troggle/code/TESTS/tests.py +and that the failure function is test_page_folk() +If you look at this you will see that line 264 is: + response = self.client.get('/folk/index.htm') + +so this file is missing. Duh. Of course it is. We downloaded troggle from git but we didn't run +the standalone script to generate the folk list. It is top of the list in +http://expo.survex.com/handbook/troggle/scriptscurrent.html#folk + +So do this: +cd ../expoweb/folk +python ../scripts/make-folklist.py index.htm +cd ../../troggle + +and run the tests again: + +python manage.py test -v 2 Registering troggle as a django application ------------------------------------------- diff --git a/_deploy/old/localsettingsserver.py b/_deploy/debian/localsettingsserver.py similarity index 96% rename from _deploy/old/localsettingsserver.py rename to _deploy/debian/localsettingsserver.py index 79cd6ef..7379a3c 100644 --- a/_deploy/old/localsettingsserver.py +++ b/_deploy/debian/localsettingsserver.py @@ -16,7 +16,12 @@ it overwrites defaults in that file. print(" * importing troggle/localsettings.py") -# DO NOT check this file into the git repo - it contains real passwords. +# DO NOT check this file into the git repo - it contains real passwords. [not this copy] +SECRET_KEY = "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" +EXPOUSERPASS = "nope" +EXPOADMINUSERPASS = "nope" +EMAIL_HOST_PASSWORD = "nope" + DATABASES = { 'default': { diff --git a/_deploy/old/localsettingsdocker.py b/_deploy/docker/localsettingsdocker.py similarity index 100% rename from _deploy/old/localsettingsdocker.py rename to _deploy/docker/localsettingsdocker.py diff --git a/_deploy/old/localsettingspotatohut.py b/_deploy/potatohut/localsettingspotatohut.py similarity index 97% rename from _deploy/old/localsettingspotatohut.py rename to _deploy/potatohut/localsettingspotatohut.py index 2e5c08b..184f0ab 100644 --- a/_deploy/old/localsettingspotatohut.py +++ b/_deploy/potatohut/localsettingspotatohut.py @@ -5,6 +5,7 @@ import sys # your/our password to the world! # This will ALL NEED TO BE CHANGED to match localsettingsWSL / python3 / Django v1.18.29 +# This WILL NOT WORK as it is for an earlier version of Django DATABASES = { diff --git a/_deploy/old/localsettingsubuntu.py b/_deploy/ubuntu/localsettingsubuntu.py similarity index 94% rename from _deploy/old/localsettingsubuntu.py rename to _deploy/ubuntu/localsettingsubuntu.py index 74ebd3c..c348cd8 100644 --- a/_deploy/old/localsettingsubuntu.py +++ b/_deploy/ubuntu/localsettingsubuntu.py @@ -1,7 +1,9 @@ import sys # link localsettings to this file for use on expo computer in austria -# This will ALL NEED TO BE CHANGED to match localsettingsWSL / python3 / Django v1.18.29 +# This will ALL NEED TO BE CHANGED to match localsettingsWSL / python3 / Django v2.2 +# This WILL NOT WORK as it is for an earlier version of Django +# consult localsettingsWSL for updates required. DATABASES = { 'default': { diff --git a/security-warnings.txt b/security-warnings.txt index edfad59..db85a63 100644 --- a/security-warnings.txt +++ b/security-warnings.txt @@ -5,6 +5,5 @@ WARNINGS: ?: (security.W012) SESSION_COOKIE_SECURE is not set to True. Using a secure-only session cookie makes it more difficult for network traffic sniffers to hijack user sessions. ?: (security.W016) You have 'django.middleware.csrf.CsrfViewMiddleware' in your MIDDLEWARE, but you have not set CSRF_COOKIE_SECURE to True. Using a secure-only CSRF cookie makes it more difficult for network traffic sniffers to steal the CSRF token. ?: (security.W018) You should not have DEBUG set to True in deployment. -?: (security.W019) You have 'django.middleware.clickjacking.XFrameOptionsMiddleware' in your MIDDLEWARE, but X_FRAME_OPTIONS is not set to 'DENY'. The default is 'SAMEORIGIN', but unless there is a good reason for your site to serve other parts of itself in a frame, you should change it to 'DENY'. -System check identified 5 issues (0 silenced). +System check identified 4 issues (0 silenced).