expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2024-11-25 16:52:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updated and hopefully partially simplified troggle installation - online edit of handbook/troggle/troglaptop.html

Browse Source
This commit is contained in:
Expo on server 2023-07-09 12:56:00 +01:00
parent 7e54c3634b
commit fb45dd1eed
1 changed files with 50 additions and 149 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

199
handbook/troggle/troglaptop.html
View File

@ -88,70 +88,41 @@ File permissions are a big issue.
provisioning so that systems are rebuilt cleanly. <a href="http://www.cuyc.org.uk">CUYC</a> (who also use Django) have a bash script which sets up a new django
development system. We should copy that in the first instance. Alas, we haven't got around to doing any of this yet.
<h2 id="git">Configuring git</h2>
<p>On a new machine you need to configure your git identity:
<pre><code>git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git config --global pull.rebase true
</pre></code>
<p>We use pull.rebase as this seems to work best with our repos and the way we work.
<h2 id="python">Installing python</h2>
<a href="https://xkcd.com/1987/"><img src="https://imgs.xkcd.com/comics/python_environment.png" align="right" hspace="20" width="230" alt='XKCD python install'></a>
<p>Python is not installed by default on Debian, and in any case we need specific versions to be installed. For Ubuntu 20.04 the
default installation is python3.9 whereas Ubuntu 22.04 gives you python3.10 (but it may not if using WSL, depending on your history. See the <a href="https://expo.survex.com/handbook/computing/wsllaptop.html">Expo WSL page</a> for more).
<p>If you are planning on eventually helping the migration of troggle from debian Bullseye (v11) to
<a href="https://www.debian.org/releases/">Bookworm (v12)</a> to then you will also want the appropriate later versions of python e.g. 3.10 on Ubuntu 20.04 like this:
<h2 id="git">Configuring ubnutu</h2>
<pre><code>sudo apt install python3 python3-pip -y
sudo apt install sqlite3 sqlite3-doc -y
sudo apt install survex -y
sudo apt install software-properties-common -y
sudo apt install git gitk -y
</code></pre>
Martin Green is not sure why, but historically we may have also installed:
<pre><code>
sudo apt install software-properties-common -y
sudo apt install default-jdk -y
sudo apt install binutils binfmt-support -y
</code></pre>
#sudo add-apt-repository ppa:deadsnakes/ppa
#sudo apt install python3.10 python3.10-venv python3.10-doc
#sudo apt --fix-broken install
<h2 id="git">Configuring git</h2>
<h4>key exchange</h4>
<p>You need this so that you can upload your edited code to the git repo on the server.
<p>Follow this link to <a href="../computing/keyexchange.html">register a key with the expo server</a> to get git access if you have
not already cloned the <var>:troggle:</var> repo.
cd /usr/bin
sudo ln -s python3 python
sudo ln -s pip3 pip </code></pre>
<h4>Set your indentity</h4>
<p>On a new machine you need to configure your git identity:
<pre><code>git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git config --global pull.rebase true
</code></pre>
<p>You will also definitely need sqlite3 even if you are planning to use another
database. Sqlite3 is used by the test harness system.
<p>Note that when you install survex it installs a shed load of packages that it needs.
<p>We do <em>not</em> install Django at this point. We will be installing Django in a separate virtual
environment (a 'venv'), not in the main linux system.
<p><b>Buggeration #1:</b> The <var>sudo add-apt-repository ppa:deadsnakes/ppa</var> line <a href="https://opensource.com/article/22/9/deprecated-linux-apt-key">won't work</a> after Debian 11 or Ubuntu 22.04. So this handbook page will need to be changed to use the <a href="https://askubuntu.com/questions/1286545/what-commands-exactly-should-replace-the-deprecated-apt-key">new method</a>. And this way of installing python3.10 is fine on Ubuntu but not on Debian, where you are stuck with 3.9 on Debian 11 (Bullseye).
<p><b>Buggeration #2:</b> In Ubuntu 23.04 you can’t run pip install outside a virtual environment. SO use venv, see <a href="https://www.omgubuntu.co.uk/2023/04/pip-install-error-externally-managed-environment-fix">Pip Install Error</a> (but not, don't use pipx unless you have used it before).
<p>In case you have python version problems with venv, look at the instructions for <a href="/handbook/computing/wsllaptop.html_edit">python3.11 on WSL2</a>.
<p>Note that default-jre is Java which is used by tunnel and <a href="https://activityworkshop.net/software/gpsprune/download.html">gpsprune</a>.
<h2 id="links">Creating folders and soft links</h2>
<p>We need to have the folder structure that troggle expects. On your own machine
you will have your own logon id, and you do not need to create an 'expo' user, but
you do need to create folders where an 'expo' user would have them. Make links there
to wherever you have actually installed the repos. So if you have installed all your repos
in <var>/mnt/c/EXPO/</var>, then you would need to do this:
<pre><code>cd ~
cd ..
sudo mkdir expo
cd expo
sudo ln -s /mnt/c/EXPO/expoweb expoweb
sudo ln -s /mnt/c/EXPO/troggle troggle
sudo ln -s /mnt/c/EXPO/loser loser
sudo ln -s /mnt/c/EXPO/drawings drawings
sudo ln -s /mnt/c/EXPO/expofiles expofiles
sudo ln -s /mnt/c/EXPO/expowebcache expowebcache
sudo mkdir expowebcache/3d
cd ..
ls -tlA expo</pre></code>
<h4>Pill the repositories</h4>
We now pull the required repositories:
<pre><code>cd ~/expo
git clone ssh://expo@expo.survex.com/home/expo/troggle
git clone ssh://expo@expo.survex.com/home/expo/loser
git clone ssh://expo@expo.survex.com/home/expo/expoweb
git clone ssh://expo@expo.survex.com/home/expo/drawings
</code></pre>
<h4 id="EXPOFILESREMOTE">Remote EXPOFILES</h4>
<p>If you do not have a local copy of the 40GB /expofiles/, don't worry. Later on we can set <var>'EXPOFILESREMOTE = True'</var> in the
@ -165,103 +136,39 @@ confusing and apparently inconsistent behaviour: e.g. you will upload a file but
<p>You can, if you like, have the expo photo archive collection somewhere else, not inside <var>expofiles</var>, on a troggle development machine. To do this set the PHOTOS_ROOT appropriately in <var>troggle/localsettings.py</var>. By default it is <var>PHOTOS_ROOT = EXPOFILES / 'photos'</var>. This is handy if you want everything else in expofiles in your Linux home drive for speed, but don't care about speed for the 29GB of photos which can sit on an SDdrive.
<h2>Installing Django and troggle</h2>
<img src="Django_Logo-420x180.png" align="right" hspace="20" width='210' alt='django logo'>
<p>The important point to note
here is that unless you are doing something fairly trivial, or you are a git genius,
it is sensible to set up a python virtual environments to hold duplicate copies of both troggle and Django code.
Then you will be able to check very quickly that your edited version of troggle runs with old, current
and pre-release versions of python and of Django;
and you will more easily be able to manage problems with incompatible versions of Django plugins as installing and upgrading the dependent packages is very fast.
<h2 id="python">Installing python libraries</h2>
<a href="https://xkcd.com/1987/"><img src="https://imgs.xkcd.com/comics/python_environment.png" align="right" hspace="20" width="230" alt='XKCD python install'></a>
<p>We do <em>not</em> install Django at this point. We will be installing Django in a separate virtual
environment (a 'venv'), not in the main linux system.
<h3 id="venv">Installing a venv</h3>
<p>We set up a <var>venv</var> specifically for python 3.9 (which is the standard version on our server which is running unmodified
Bullseye (debian v11) and <a href="https://docs.djangoproject.com/en/3.2/releases/3.2/">Django 3.2.12</a>. See the <a
href="https://docs.python.org/3.9/library/venv.html">standard python documentation on venv</a> for python 3.9.10.
Create a new venv for each version of python you are using. Get that venv installed right by explicitly stating the
python version to create it <var>python3.9 -m venv py39d32</var>. Note that we are creating it as a sibling folder to /expoweb/ .
Note also that up to now we have been using 'sudo ..' but for installing things inside the venv we do not use 'sudo ..':
<pre><code>cd ~
cd ../expo
python3.9 -m venv py39d32
cd py39d32
source bin/activate
pip list -o</pre></code>
<p>The last command lists the default packages installed in the venv. This is for comparison later, after we
have installed troggle, Django and dependencies. You will get a warning that you have an out of date version of pipbut this is as we want: we are using a version of pip appropriate for the older version of python within
the venv.
<img border="1" class="onright" width="150px" src='tricky-troggle.jpg' hspace="20" alt='troggle logo'/></a>
<p>In over to avoid compatability issues when deploying our code, we should develop using the same libraries that will be used in prduction of expo.survex.com. Expo.survex.com currently uses python3.9.2, python 3.x is backwardly compatable as long as you do not use any new fuctions it should be fine. Venv allows us to specify which python libraries to use.</p>
<h3 id="venv">Installing the troggle dependencies</h3>
<p>The first time you do this on a new machine you can't complete the pip installation of Django as you have not yet got the
dependencies appropriate for troggle - because you have not yet cloned the troggle repo. So the first time it is easiest to just create requirements.txt yourself with a text editor. Without using git yourself, you can get the file from the website at <a href="http://expo.survex.com/repositories/troggle/.git/tree/requirements.txt">requirements.txt</a>. If you have already cloned all the repos, then just copy it.
<pre><code>cp ../troggle/requirements.txt .</pre></code>
where <var>requirements.txt</var> (note the capitalisation of the listed packages) is:
<pre><code>asgiref==3.3.4
confusable-homoglyphs==3.2.0
coverage==5.5
Django==3.2
docutils==0.14
gunicorn==20.1.0
pytz==2019.1
sqlparse==0.2.4
typing-extensions==3.7.4.3
Unidecode==1.0.23</pre></code>
This will pick up the latest Django 3.2.x available, but all the other dependencies are pinned. These
dependencies are as-standard on debian Buster (10) and you will want to experiment with upgrading them
for Bullseye (v11).
<p>Once you have the file, install the listed dependencies like this:
<pre><code>pip install -r requirements.txt
pip list -o</pre></code>
Create and activate the virtual enviroment:
<pre><code>cd ~/expo
python3 -m venv venvexpo
source venvexpo/bin/activate
</code></pre>
Install the python libraries
<pre><code>pip install -r requirements.txt</code></pre>
<p>Pillow (not currently in that package list) is an image handling package used to make
the prospecting map (currently disabled,
see <a href="http://expo.survex.com/prospecting_guide/">/prospecting_guide/</a>).<br>
tinymce is the wysiwyg in-browser
editor (disabled pending reinstatement)
<p>This is also documented in the <a href="trogdjangup.html">updating Django for troggle page</a>
where it describes upgrading and testing with later versions of Django.
<p>If you have not used pip before, read <a href="https://linuxize.com/post/how-to-install-pip-on-ubuntu-20.04/">this</a>
<p>In case you have python version problems with venv, look at the instructions for <a href="/handbook/computing/wsllaptop.html_edit">python3.11 on WSL2</a>.
<h3 id="troginstall">Installing troggle</h3>
<p>The <var>:troggle:</var> repo is the python source code for troggle. This is what you will be editing. There are over 9,000 lines
of python code (excluding comments and blank lines) and nearly 3,000 lines of HTML/Django template code. This is over 600 files in
over 400 folders, but only 49MB in size (plus a .git repo of 32MB).
<h4>key exchange</h4>
<p>You need this so that you can upload your edited code to the git repo on the server.
<p>Follow this link to <a href="../computing/keyexchange.html">register a key with the expo server</a> to get git access if you have
not already cloned the <var>:troggle:</var> repo.
<h4>git clone troggle</h4>
<p>You will do a git clone to create a folder /troggle/ containing all the troggle code. You need to clone the 'python-3' branch, if
you an see multiple branches.</p>
<pre><code>cd ~
cd ../expo
git clone ssh://expo@expo.survex.com/home/expo/troggle</code></pre>
<p>Now we create soft links within the venv to all the repo folders and wherever you have a local copy of /expofiles/ and /expowebcache/:
<pre><code>cd py39d32
sudo ln -s /mnt/c/EXPO/expoweb expoweb
sudo ln -s /mnt/c/EXPO/troggle troggle
sudo ln -s /mnt/c/EXPO/loser loser
sudo ln -s /mnt/c/EXPO/drawings drawings
sudo ln -s /mnt/c/EXPO/expofiles expofiles
sudo ln -s /mnt/c/EXPO/expowebcache expowebcache</code></pre>
<p>If you have not been using git on this machine until now, you will need to identify yourself before you go any further.
<pre><code>git config --global user.email "you@example.com"
git config --global user.name "Your Name"</code></pre>
<h4>do the basic Django health checks</h4>
<p>This all checks that the installation has completed properly.
<pre><code>django-admin</code></pre>
@ -269,19 +176,13 @@ git config --global user.name "Your Name"</code></pre>
See <a href="djangostart.html">django-admin initial output</a> for what you should expect to see at this point.
If it crashes though, you have not managed to install the software completely.
If you get an error when running
<pre><code>django-admin</code></pre>
Then run django-admin like this (using the python version you have installed in the folder underneath /lib/):
<pre><code>python /usr/local/lib/python3.8/dist-packages/django/bin/django-admin.py</code></pre>
which should fix the paths, but this means that you installed Django directly on your machine and not in a venv. From now on you should be able to run <var>django-admin </var>
from within any folder on your machine.
<p>Now try
<pre><code>django-admin --version</code></pre>
The version number is the version of Django you have installed. Double check that it is the one you meant to install and check with <a href="trogdjangup.html">our Django versions page</a>.
<h4>do the basic troggle health checks</h4>
<p>Now try
<pre><code>python manage.py</code></code></pre>
<pre><code>python manage.py</code></pre>
You will get an error: No module named 'localsettings'. Fixing this is described below, but for now try: