expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2025-03-21 09:42:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

New laptop for troggle software dev documentation

Browse Source
This commit is contained in:
Philip Sargent 2021-10-22 21:04:23 +03:00
parent 7a614ccec8
commit cf4dc30251
5 changed files with 162 additions and 12 deletions

4
handbook/computing/basiclaptop.html

@ -43,12 +43,12 @@
</ul>
<h2 id="basic">Your own basic laptop</h2>
<p>If you are new to expo please use the <em>expo laptop</em> first. You don't <em>need</em> to use your own laptop - which can take several hours to configure completely.
<p>If you are new to expo and can't do what you want with just a browser and email, then please use the <em>expo laptop</em> in the potato hut first. You don't <em>need</em> to use your own laptop - which can take several hours to configure completely.
<p>To set up your own basic laptop for all cave data maintenance you need to do this:</p>
<ol>
<li>Register an SSH key</a> with an expo nerd i.e 'get a login'. (see "Key Configuration" below)</li>
<li>Install <a href="#software">git version control software</a> to download ("clone"), view and edit caving data.</li>
<li>Clone two <a href="../computing/repos.html">expo repositories</a> <var>loser</var> and <var>drawings</var> so you have the files on your machine. (Use the <a href="qstart-git.html">git reminder</a> for how to do this, e.g. <em>git clone ssh://expo@expo.survex.com:/home/expo/expoweb</em> [actually <var>los</var> is still (Oct.2021) using mercurial not git.</li>
<li>Clone two <a href="../computing/repos.html">expo repositories</a> <var>loser</var> and <var>drawings</var> so you have the files on your machine. (Use the <a href="qstart-git.html">git reminder</a> for how to do this, e.g. <em>git clone ssh://expo@expo.survex.com:/home/expo/expoweb</em> [actually <var>loser</var> is still (Oct.2021) using mercurial not git, so this won't work.]</li>
<li>Install survex, and therion or tunnel for editing cave data.
<li>Install image editing software such as Irfanview or gimp.
<li>If you are also planning on extensive work rewriting parts of the handbook, then you will also need the <a href="../computing/repos.html">expo repository</a> <var>expoweb</var>.

29
handbook/computing/keyexchange.html

@ -41,9 +41,34 @@
<li>You may already have a key on this machine. If you already have <tt>~/.ssh/id_rsa.pub</tt>, then send that.</li>
<li>If not, run <tt>ssh-keygen</tt>. It may ask about passwords: you can add a password for extra
security, but a passwordless key is fine, and more convenient.</li>
<li>That will create a file: <tt>.ssh/id_rsa.pub</tt> in your home directory. Email that file to one of the admins listed above. </li>
<li>That will create a file called (by default) <tt>.ssh/id_rsa.pub</tt> in your home directory. Email that file to one of the admins listed above. </li>
</ol>
<p>This is an example of the whole interaction where the key file has been given a different name:
<pre>
<tt>$ <b>ssh-keygen -C "philip@muscogee-wsl"</b>
Generating public/private rsa key pair.
Enter file in which to save the key (/home/philip/.ssh/id_rsa): <b>id_rsa_wsl</b>
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in <em>id_rsa_wsl</em>.
Your public key has been saved in <em>id_rsa_wsl.pub</em>.
The key fingerprint is:
SHA256:ySs0YD5IG2ZD50+riUDHWosNq+WJdqpkDlINXh709r0 philip@muscogee-wsl
The key's randomart image is:
+---[RSA 2048]----+
| . o |
| ..+ . |
| oB+* + |
|.=O%.* + o |
|.+*o= = S . |
|.* o = . . . |
|=++.o . . E |
|B o . |
|oo |
+----[SHA256]-----+
$
</tt>
</pre>
<h3>MacOS</h3>

13
handbook/computing/qstart-git.html

@ -16,8 +16,19 @@
<p>If you can get to the <em>expo laptop</em> try these commands on that first as the key exchange has already been done. If the key exchnage has not been done then none of this will work.</p>
<p>You can read or clone these repos without any control, but to write ("push") to them you will need to use <var>ssh://expo@expo.survex.com</var> and set up the <a href="keyexchange.html">key exchange</a>.</p>
<p>NOTE: always use user <var>'expo'</var> as the login user (<var>ssh://expo@...</var>) even though within git you will be identified by your own ssh key name.</p>
<p>Open a terminal in a new directory, e.g. /tmp/experiments/ in which you want to create the repo. It will automatically create a folder with the repo name e.g.'troggle' in that directory.</p>
<dl><dt>expoweb (The data management system)</dt><dd><tt>git clone ssh://expo@expo.survex.com/home/expo/expoweb</tt> (read/write)</dd><dt>troggle (The data management system backend)</dt><dd><tt>git clone ssh://expo@expo.survex.com/home/expo/troggle</tt> (read/write)</dd><dt>loser (The survey data) - this will fail until it is moved from hg to git</dt><dd><tt>git clone ssh://expo@expo.survex.com:loser</tt> (read/write)</dd><dt>drawings</dt><dd><tt>git clone ssh://expo@expo.survex.com/home/expo/drawings</tt> (read/write)</dd></dl>
<dl>
<dt>loser (The survey data) - <em>this will fail until it is moved from hg to git</em></dt><dd>This is all the survex files. You need this if you are rearranging and editing a lot of survey files.<tt>git clone ssh://expo@expo.survex.com:loser</tt> (read/write)</dd>
<dt>drawings</dt><dd>These are the therion and tunnel cave vector drawing files. You need this if you are 'tunneling' a lot of cave surveys.<tt>git clone ssh://expo@expo.survex.com/home/expo/drawings</tt> (read/write)</dd>
<dt>expoweb</dt><dd>These are all webpages, mostly the expo handbook. You need this only if you are editing a lot of handbook pages.<tt>git clone ssh://expo@expo.survex.com/home/expo/expoweb</tt> (read/write)</dd>
<dt>troggle</dt><dd>This is the python code that runs troggle and generates 2,000 webpages of reports and tables. This is what you need for software development.<tt>git clone ssh://expo@expo.survex.com/home/expo/troggle</tt> (read/write)</dd>
</dl>
<h3>Using git</h3>
<p><a href="https://stevebennett.me/2012/02/24/10-things-i-hate-about-git/"><img class="onright" src="git-arrows31.png" alt="git bread &amp; butter commands subset" width="250" /></a>You may find these useful:</p>
<ul>

8
handbook/troggle/trogdjangup.html

@ -58,9 +58,11 @@
<h4>Major, minor and releases</h4>
<p>Django release 2.2.20 is major-version 2, minor-version 2, and patch-release 20. 2.2 is the "feature release" and patch releases within each feature release are not meant to break anything. They are just to fix bugs.
<p>Things <em>will break</em> between <a href="https://docs.djangoproject.com/en/dev/internals/release-process/">feature releases</a> which come out every 8 months.
<p>You will come to rely extensively on <a href="https://docs.djangoproject.com/en/3.2/releases/">the release notes and versions documentation</a> which is maintained by django.
<p>You will also need to read the django <a href="https://docs.djangoproject.com/en/3.2/howto/upgrade-version/">guide to upgrading to newer versions</a>.
<h4>Django plugins ("apps")</h4>
<p>Documentation for the plugins is highly variable and plugin projects, being run by volunteers, can just die unexpectedly. For django-registration there are two sources of current information:<br />
<p>Documentation for the plugins is highly variable and plugin projects, being run by volunteers, can just die unexpectedly. For the django-registration plugin there are two sources of current information:<br />
<a href="https://django-registration.readthedocs.io/en/3.1/">Docs: django-registration 3.1</a><br />
<a href="https://pypi.org/project/django-registration/">PyPi: django-registration 3.1</a><br />
<p>but only one of these (PyPi) gives release history data - which is what you need if you get behind with the django upgrades.
@ -74,8 +76,8 @@
<p>There are six critical tricks that make everything much, much easier:
<ol>
<li>Use <a href="https://docs.python.org/3/library/venv.html">pip venv</a> or virtualenv to set up a virtual python environment within which you can easily and quickly change the specific releases of python, django, django's dependencies and django plugins.
With a previously created venv <var>t37</var> start it up like this:<br />
<var>cd t37<br />
With a previously created venv <var>py37d2</var> start it up like this:<br />
<var>cd py37d22<br />
source bin/activate<br />
python --version<br />
cd troggle<br />

120
handbook/troggle/troglaptop.html

@ -13,19 +13,131 @@
<h2>Software development machine</h2>
<p>For troggle itself, you need a linux machine. We all use Debian or Debian-derived machines (Debian itself, Ubuntu, Xubuntu etc.) but other forms of linux may work. Running Debian or Ubuntu under WSL on Windows 10 works fine.
<p>If you want to work on the troggle source code and be able to commit to the server git repo,
your account will need to be added to the troggle project members list. Contact wookey at wookware dot org to get this set up.
<p>This page is a work in progess. Text will be moved here from
<img border="1" class="onright" width="150px" src='tricky-troggle.jpg' alt='troggle logo'/></a>
<pre>
http://expo.survex.com/repositories/troggle/.git/tree/README.txt
handbook/troggle/serverconfig.html
handbook/computing/yourlaptop.html
handbook/computing/basiclaptop.html
handbook/computing/manual.html
handbook/troggle/trogdjangup.html
handbook/computing/qstart-git.html
</pre>
<h2 id="os">Installing linux</h2>
<p>If you don't already know how to do this, then you should probably not be attempting this. But in case you are an experienced linux user who has always had someone else set up the system for them, then Ubuntu is the easiest and more forgiving to install, either directly on the computer or inside WSL. Because we are using fairly old releases of Django, you will want Ubuntu-20.04
<ul>
<li>Install in <a href="https://docs.microsoft.com/en-us/windows/wsl/install">WSL on Windows 10</a>.
<li>Install <a href="https://ubuntu.com/download/desktop">Ubuntu 20.04 LTS</a>.
</ul>
<p>and familiarise yourself with the directory structure on the expo server, which we will be duplicating (partly) as docmented in <a href="serverconfig.html">The Expo Server</a>. We are not here doing a full install of all the software and scripts on the server, just the minimum to run, test and debug troggle on django.
<h3>WSL on Windows</h3>
<p>The standard documentation for Ubuntu or debian below all works, but you should first skim the
<a href="../computing/winlaptop.html">Windows expo laptop</a> description of WSL1 and WSL2. The default
is that WSL2 will be installed, but all our practical experience so far is with WSL1.
<h2 id="python">Installing python</h2>
<p>Python is not installed by default usually, and in any case we need specific versions to be installed. For Ubuntu 20.04 the default is python3.9 but this is incompatible with standard debian Buster, so we also need python3.7,
and also python3.8 if you are planning on migrating troggle from debian Buster (v10) to Bullseye (v11).
<pre><code>sudo apt install python3 python3-pip
sudo apt install python3.7
sudo apt install python3.8
sudo apt install sqlite3 sqlite3-doc
sudo apt install survex
cd /usr/bin
sudo ln -s python3 python
sudo ln -s pip3 pip </code></pre>
<p>You will also definitely need sqlite3 even if you are planning to use another
database. Sqlite3 is used by the test harness system.
<p>Note that when you install survex it installs a shed load of packages that it needs.
<p>We do <em>not</em> install django at this point. We will be installing django in a separate virtual
environment (a 'venv'), not in the main linux system.
<h2 id="links">Creating folders and soft links</h2>
<p>We need to have the folder structure that troggle expects. On your own machine
you will have your own logon id, and you do not need to create an 'expo' user, but
you do need to create folders where and an 'expo' user would have them. Make links there
to wherever you have actually installed the repos. So if you have installed all your repos
in <var>/mnt/c/EXPO/</var>, then you would need to do this:
<pre><code>cd ~
cd ..
sudo mkdir expo
cd expo
sudo ln -s /mnt/c/EXPO/expofiles expofiles
sudo ln -s /mnt/c/EXPO/expoweb expoweb
sudo ln -s /mnt/c/EXPO/troggle troggle
sudo ln -s /mnt/c/EXPO/loser loser
sudo ln -s /mnt/c/EXPO/drawings drawings
sudo ln -s /mnt/c/EXPO/expowebcache expowebcache
sudo mkdir expowebcache/3d
cd ..
ls -tlA expo</pre></code>
<h2>Installing Django and troggle</h2>
<p>The important point to note
here is that unless you are doing something fairly trivial, or you are a git genius,
it is sensible to set up a python virtual environments to hold duplicate copies of both troggle and django code.
Then you will be able to check that your edited version of troggle runs with old, current
and pre-release versions of python and of django;
and you will more easily be able to manage problems with incompatible versions of django plugins.
<h3 id="venv">Installing a venv</h3>
<p>We set up
a <var>venv</var> specifically for python 3.7 (which is the standard version on our server which is running Buster (debian v10) and <a href="https://docs.djangoproject.com/en/3.2/releases/2.2/">django 2.2.19</a>. See the <a href="https://docs.python.org/3.7/library/venv.html">standard python dcoumentation on venv</a> for python 3.7.12. Note that we are creating it as a sibling folder to /troggle/ .
<pre><code>cd ~
cd ../expo
python3.7 -m venv py37d22
cd py37d22
source bin/activate
pip list -o</pre></code>
<p>The last command lists the default packages installed in the venv. This is for comparison later, after we
have installed troggle, django and dependencies. You will get a warning that you have an out of date version of pipbut this is as we want: we are using a version of pip appropriate for the older version of python within
the venv.
<p>The first time you do this you can't complete the pip installation of django as you have not yet got the
dependencies - because you have not yet cloned the troggle repo, so the first time it is easiest to just create requirements.txt yourself with a text editor.
<pre><code>cp ../troggle/requirements.txt .
pip install -r requirements.txt
pip list -o</pre></code>
where <var>requirements.txt</var> (note the capitalisation of the listed packages) is:
<pre><code>confusable-homoglyphs==3.2.0
Django==2.2
docutils==0.14
gunicorn==20.1.0
Pillow==5.4.1
pytz==2019.1
sqlparse==0.2.4
Unidecode==1.0.23</pre></code>
This will pick up the latest django 2.2.x available, but all the other dependencies are pinned. These
dependencies are as-standard on debian Buster (10) and you will want to experiment with upgrading them
for Bullseye (v11).
<p>Pillow is an image handling package used to make
the prospecting map (currently disabled,
see <a href="http://expo.survex.com/prospecting_guide/">/prospecting_guide/</a>).<br>
tinymce is the wysiwyg in-browser
editor (disabled pending reinstatement)
<p>This is also documented in the <a href="trogdjangup.html">updating django for troggle page</a>
where it describes upgrading and testing with later versions of django.
<p>If you have not used pip before, read <a href="https://linuxize.com/post/how-to-install-pip-on-ubuntu-20.04/">this</a>
<h3 id="troginstall">Installing troggle</h3>
<dl><dt>key exchange</dt><dd>Follow this link to <a href="../computing/keyexchange.html">register a key with the expo server</a> to get git access. </dd>
<dt>troggle</dt><dd>The <var>:troggle:</var> repo is the python code that runs troggle and generates 2,000 webpages of reports and tables. This is what you need for software development.<tt>cd /home/expo/p37d22<br>git clone ssh://expo@expo.survex.com/home/expo/troggle</tt> (read/write)</dd>
</dl>
<hr />
Go on to: <a href="trogarch.html">Troggle architecture</a><br />