expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2025-01-18 17:02:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Trggle Django documentation

Browse Source
This commit is contained in:
Philip Sargent 2023-02-22 23:36:05 +00:00
parent 4232d47ea2
commit 5197d20bf7
3 changed files with 37 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
handbook/survey/onlinewallet.html
View File

@ -214,8 +214,7 @@ the wallet upload form, which similarly lists the files in each wallet but also
on the progress data for the wallet.
<h3>Setting up the online wallets</h3>
At the beginning of a new expo you do not need to do any setup for new wallets. The necessary folders etc. will be created automatically
as soon as you upload a new scan for a new year.
At the beginning of a new expo you do not need to do any setup for new wallets. The necessary folders etc. will be created automatically when any user creates a wallet with the 'Upload Scans' form. [In past years before 2022, a nerd would have needed to create all the wallets in advance.]
<h3>Maintaining the online wallets</h3>
@ -251,6 +250,7 @@ to do everything online to the server, and occasionally make a copy to your loca
<h4>Surface trips</h4>
<p>For prospecting and surface surveying it is not clear whether the default folder
for the url link to a survex file should be repo :loser: surface/1623/allplateau.svx
[More documentation required here.]
<h3>More Troggle magic</h3>
<p> Troggle also
@ -259,7 +259,8 @@ for the url link to a survex file should be repo :loser: surface/1623/allplateau
<li>checks for the presence of notesXXX.jpg, planXXX.jpg and elevXXX.jpg files
<li>checks for the presence of XXXnotes.jpg, XXXplan.jpg, XXXelevation.jpg and XXXelev.jpg files
<li>creates helpful URL links to the existing online survey documentation for the cave being surveyed
<li>creates helpful URL links to the working files you are managing on your own laptop
<li>creates links to other things happening on the same day as a wallet's date: logbook entries and other survex files
,!--<li>creates helpful URL links to the working files you are managing on your own laptop-->
</ul>
<p>Things it might do in future (if someone gets around to it) include: <br>
@ -268,7 +269,7 @@ for the url link to a survex file should be repo :loser: surface/1623/allplateau
- being more intelligent about .topo files and thus the lack of scan files,<br>
- checking the date is in the recent past etc.<br><br>
<hr>
<!--
The old wallets.py script runs without errors on earliers years before 1999 when there was not the same wallet system, but the results are less useful, e.g. see
http://expo.survex.com/expofiles/surveyscans/1999/
and compare with
@ -289,7 +290,7 @@ http://expo.survex.com/survey/2018%2330
This is probably not worth doing except maybe for specific critical connections.
</pre>
-->
<hr />

26
handbook/troggle/trogdjango.html
View File

@ -12,7 +12,8 @@
<h3>Django Versions and troggle</h3>
<p>Troggle is built on top of the django framework. In 2019 we had let ourselves get rather behind in keeping up to date with the current django version and this held us back from upgrading the operating system - which we needed to do. Troggle uses several community-contributed django plugins and each of these had their own version dependencies. We also got caught up with out of date and deprecated python language features.
<p>A ridiculous amount of work was required to get to a situation where troggle was running on relatively recent and nearly security-in-date versions of python, django, plugins and linux by Summer 2020. We don't want that to happen again.
<h3>Three sentence intro to Django for python programmers</h3>
<h3 id="3sent">Three sentence intro to Django for python programmers</h3>
<ul>
<li>There are many 'magic' function calls autogenerated. So when we create a <var>'Cave'</var> class (subclass of Django <var>Model</var>), with an instance <var>'caveenetrance'</var> we automagically define functions such as:
<ul>
@ -20,11 +21,11 @@
<li><var>Cave.objects.all()</var>
<li><var>Cave.caveandentrance_set.all</var>
</ul>
note that the suffix on a name '<var>'_set</var> actually denotes a function call to the database. Don't let this upset you.
note that the suffix on a name '<var>'_set</var> actually denotes a function call to the database on an object of class "CaveAndEntrance". Don't let this upset you. (The underline character and lower-casing denotes a dereferencing operation in Django queries.)
<li>You will get very familiar with the <a href="https://docs.djangoproject.com/en/2.2/">online documentation</a>. be sure to be looking that the flavour for the right release of Django (indicated on the bottom-right of the screen: the link here goes to v2.2). You can read it in Polish, Indonesian, Brazilian-Portuguese or whatever if you prefer.
<li>You will get very familiar with the <a href="https://docs.djangoproject.com/en/3.2/">online documentation</a>. be sure to be looking that the flavour for the right release of Django (indicated on the bottom-right of the screen: the link here goes to v3.2). You can read it in Polish, Indonesian, Brazilian-Portuguese or whatever if you prefer.
<li>Django has a good process for migrating both the python code and the SQL database to later versions of Django via <a href="https://docs.djangoproject.com/en/2.2/topics/migrations/">'migrations'</a>. Note that even though Expo imports all the data from files and its database is ephemeral (but it is intensely used for cross-referencing), the database schema still needs to be migrated.
<li>Django has a good process for migrating both the python code and the SQL database to later versions of Django via <a href="https://docs.djangoproject.com/en/dev/topics/migrations/">'migrations'</a>. Note that even though Expo imports all the data from files and its database is ephemeral (but it is intensely used for cross-referencing), the database schema still needs to be migrated.
</ul>
<h4>Django versions and deprecation</h4>
<a href="https://www.djangoproject.com/download#supported-versions">
@ -63,7 +64,24 @@ href="https://docs.djangoproject.com/en/dev/misc/design-philosophies/">loosely c
is important: it is a key reason why we need a framework and not just a mess of javascript. We have nearly 100 URL patterns.
We should try to reduce the number of distinct URLs perhaps, but they help document the structure too.
<h4 id="coding">More django coding tricks</h4>
<p>[This section, and the <a href="#3sent">3-sentence intro</a>, to go to another page ?]
<p>Much of our coding is still stuck in the early Django v1 era, before 2011. So many idioms in the code are not the best way of doing things these days. In particular there are powerful query optimisations now available that none of our code uses (yet):
<ul>
<li><a href="https://ctrlzblog.com/django-queryset-filter-15-examples/">.distinct()</a> - returns a list with no duplicated items, and many other clever filters
<li><a href="https://zerotobyte.com/how-to-use-django-select-related-and-prefetch-related/">select_related()</a> – creates INNER JOIN between tables
<li><a href="https://hakibenita.com/all-you-need-to-know-about-prefetching-in-django">prefetch_related()</a> – connects tables using Python (but queries both tables)
</ul>
and also much more about the manipulation of QuerySets:
<ul>
<li><a href="https://docs.djangoproject.com/en/dev/ref/models/querysets/#bulk-create">bulk_create()</a> - just started using it in data imports in 2023.
</ul>
<h4>More refactoring still to do</h4>
<p>Some of the more bizarre data model structures are probably fossils from Django v0.6, e.g. the separate classes that join Entrances to Caves. We haven't got to the bottom of all these yet.
<h4>Django Forms</h4>
<p>Just don't. Django has several generations of quite different clever mechanisms to make creating HTML forms "easier". Yes, making them might be easier, but maintaining this opinionated stuff is a nightmare. Really don't.
<hr />
Go on to: <a href="trogdjangup.html">Troggle: updating Django</a><br />
Return to: <a href="trogintro.html">Troggle intro</a><br />

9
handbook/troggle/trognotes.html
View File

@ -109,6 +109,15 @@ converts to a more asynchronous architecture using python <a href="https://docs.
Many of the things it was intended to replace are still operating as a motley collection written by many different people in
several languages (but mostly perl and python; we won't talk about the person who likes to use OCamL).
<h3>Writing python</h3>
<p>We aspire to a coding standard, and the <a href="https://github.com/google/styleguide/blob/gh-pages/pyguide.md">Google python coding standard</a> is a pretty good one. We don't use their TODO system though (para <a href="https://github.com/google/styleguide/blob/gh-pages/pyguide.md#312-todo-comments">3.12</a>) and the function (re)naming in troggle is still work in progress (para. 3.16.4).
<p>We don't have any type hints in troggle. It is not a big enough project to be worth it - probably.
<p>We have run much of the code through <a href="https://pypi.org/project/black/">black</a> in 2022 so that is our default formatting standard.
<p>We have also used ruff, isort and chill to trim and deduplicate the various pip installation packages and to fix the order of python packages in import statements.
<hr />
<h3><a id="arch">Archived updates</a></h3>
<p>Since 2008 we have been keeping detailed records of all data management system updates in the version control system.