mirror of
https://expo.survex.com/repositories/expoweb/.git/
synced 2024-11-21 23:01:55 +00:00
Sorting out troggle design docum.
This commit is contained in:
parent
91fc44a486
commit
6e3b781a27
BIN
handbook/computing/go-caving-th.jpg
Normal file
BIN
handbook/computing/go-caving-th.jpg
Normal file
Binary file not shown.
After Width: | Height: | Size: 6.2 KiB |
BIN
handbook/computing/go-caving.jpg
Normal file
BIN
handbook/computing/go-caving.jpg
Normal file
Binary file not shown.
After Width: | Height: | Size: 40 KiB |
BIN
handbook/computing/mtv_drawing2.jpg
Normal file
BIN
handbook/computing/mtv_drawing2.jpg
Normal file
Binary file not shown.
After Width: | Height: | Size: 22 KiB |
@ -48,10 +48,10 @@ This page is the overview. The online systems manuals are split into these secti
|
||||
<li><a href="../survey/newcave.html">Recording a new cave discovery</a></li>
|
||||
<li><a href="../survey/status.html">Monitoring the cave survey workflow status</a></li>
|
||||
</ul>
|
||||
<p>Surveying and recording survey data is the primary reason all this exists. The "survey handbook" part of this handbook gives instructions to beginners for how to record discoveries and surveys. The "data maintenance" manual is where to look for help when things go wrong: the data has been entered incorrectly, or software has crashed due to corrupt input data.
|
||||
<p><strong>Surveying and recording survey data is the reason all this exists.</strong> The "survey handbook" part of this handbook gives instructions to beginners for how to record discoveries and surveys. <p><a href="../survey/why.htm"><img class="onleft" src="go-caving-th.jpg"></a>The "data maintenance" manual is where to look for help when things go wrong: the data has been entered incorrectly, or software has crashed due to corrupt input data.
|
||||
<p>The "system maintenance" manual may of interest to a caver who wonders how it is all put together but should not be needed by most people.
|
||||
<p>
|
||||
A fuller list of "How do I..." instruction pages are on <a href="../index.htm">the handbook opening page</a>. If you are new to this you may need a beer.
|
||||
A fuller list of "How do I..." instruction pages are on <a href="../index.htm">the handbook opening page</a>. If you are new to this you may need a beer. A very fine place for a beer is the Panorama Restaurant at the top of the toll road - as illustrated.
|
||||
<hr />
|
||||
|
||||
<p>We have <a href="wookey-slides.html">an Overview Presentation</a> (2015: many parts out of date)
|
||||
|
@ -53,7 +53,7 @@ as the scanned notes, i.e. (for wallet #19) you would put them in:
|
||||
<tt>
|
||||
/home/expo/expofiles/surveyscans/2018/2018#19/
|
||||
</tt>
|
||||
|
||||
(but this is not where you will put your finished drawings.)
|
||||
<h3 id="therion">Using tunnel or therion for final survey production</h3>
|
||||
|
||||
<ul>
|
||||
@ -67,9 +67,11 @@ as the scanned notes, i.e. (for wallet #19) you would put them in:
|
||||
<p>The tunnel (or therion) files should NOT stored in the same folder as the scanned notes. They will eventually
|
||||
be uploaded to the version control repository
|
||||
<var><a href="../computing/repos.html">drawings</a></var>
|
||||
but for a first attempt store them on the <em>expo laptop</em> in /home/expo/drawings/{cavenumber}.
|
||||
but for a first attempt store them on the <em>expo laptop</em> in <var>/home/expo/drawings/{cavenumber}</var>.
|
||||
Look at what is in there already and ask someone whcih directory to put them in. It will probably be a folder like this:
|
||||
/home/expo/drawings/264-and-258/toimport/ .
|
||||
<tt>
|
||||
/home/expo/drawings/264-and-258/toimport/
|
||||
</tt>
|
||||
|
||||
<p>Take the printed centre lines and redraw the survey round it, working from
|
||||
the original sketches as if this was to be the final published survey. You
|
||||
@ -121,10 +123,11 @@ which will be encountered (eg. if it is a climb, are bolts going to be
|
||||
needed ? If a dig, is it a few loose boulders or a crawl over mud?)</p>
|
||||
|
||||
<h3>Archaic: hand-drawing the final survey</h3>
|
||||
|
||||
<p>The actual published cave-survey is produced by software these days.
|
||||
These notes come from a different age but reading them will make your tunneling better
|
||||
and more polished.
|
||||
and more polished:
|
||||
<div style="margin-left: 3em; margin-right: 4em"><em>
|
||||
|
||||
<p>Drawing a cave entirely by hand is not easy but anyone can learn to do it.
|
||||
Read the brief <a href="/expofiles/documents/surveying/XI-2-11.pdf">Cave Mapping - Sketching the Detail"</a>
|
||||
5-page llustrated guide by Ken Grimes which makes everything clear.
|
||||
@ -160,6 +163,8 @@ experience: it's now late April 2004, and the 204 survey is only just
|
||||
approaching completion. This shows how easy it is for these things to go wrong.
|
||||
The chief problems were a change of software and the fact that the Expo printer
|
||||
broke down last summer, so a number of surveys never got drawn up. -->
|
||||
</em></div>
|
||||
|
||||
<hr />
|
||||
<p>Back to the previous page in this sequence <a href="newsurvex.html">Starting a new survex file</a>.
|
||||
<br />Now go the the next page in this sequence <a href="newrig.html">Creating a new rigging guide"</a>.
|
||||
|
@ -19,6 +19,10 @@ didn't. Without detailed recording and surveying of the caves, it would
|
||||
rapidly become more difficult to find new passage, or to be sure that round
|
||||
the next corner wouldn't be a load of previous explorers' footprints.</p>
|
||||
|
||||
<a href="../computing/onlinesystems.html">
|
||||
<img style="margin:10px auto 20px; display:block"
|
||||
width=70% src="../computing/go-caving.jpg"></a>
|
||||
|
||||
<p>The main aim of the expedition is to explore new passages - to boldly
|
||||
explore what noone has seen before. Indeed, in many cases, what noone even
|
||||
suspected was there. This is the fun and excitement of expo, so why spoil it
|
||||
|
314
handbook/troggle/archnotes.html
Normal file
314
handbook/troggle/archnotes.html
Normal file
@ -0,0 +1,314 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||||
<title>Handbook Troggle NOTES</title>
|
||||
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
|
||||
</head>
|
||||
<body><style>body { background: #fff url(/images/style/bg-system.png) repeat-x 0 0 }</style>
|
||||
<h2 id="tophead">CUCC Expedition Handbook</h2>
|
||||
<h1>Cave data management Archive Notes</h1>
|
||||
|
||||
<h2>NOTES - collected here</h2>
|
||||
|
||||
|
||||
<h3>Historic "Future" Developments: Preamble</h3>
|
||||
<p><em>Assumptions</em> (points to necessarily agree upon)
|
||||
<ol>
|
||||
<li>Let's NOT try to design a generic catalogue for storing all kind of data about caves of the whole world, intended for every kind of user (sports, exploration, science). Let's just settle for a generic framework. Let geeks in individual countries or individual communities write their tools operating within this framework.
|
||||
<li>Let's try make it available for the layman, but still well-playable for the geeks.
|
||||
<li>Let's rely on already existing, popular technologies. Let's keep it open source and multiplatform. Let's try not to reinvent the wheel.
|
||||
<li>Let's not assume everyone has an Internet connection while working with their data.
|
||||
<li>Let's version-control as much as possible.
|
||||
<li>Let's support i18n - let's use UTF-8 everywhere and cater for data in many languages(entrance names, cave descriptions, location descriptions etc.)
|
||||
</ol>
|
||||
|
||||
<tt><em>Everything here is not current - this page just records a lot of unfinished ideas.
|
||||
Most people will not want to read this at all. This is for speleosoftwarearcheologists only.</em>
|
||||
</tt>
|
||||
|
||||
<p>Two page preliminary design document for <a href="../../documents/caca_arch2.pdf">'caca' (Cave Catalogue) rev.2 2013-07-26</a> by Wookey (copied from http://wookware.org/software/cavearchive/caca_arch2.pdf)
|
||||
|
||||
|
||||
<h2>The data management system conventions bit</h2>
|
||||
<p>This is likely to change with structural change to the site, with style changes which we expect to implement and with the method by which the info is actually stored and served up.</p>
|
||||
<p>... and it's not written yet, either :-)</p>
|
||||
<ul>
|
||||
|
||||
<li>Structure</li>
|
||||
<li>Info for each cave – automatically generated by [perl script] <tt>make-indxal4.pl</tt></li>
|
||||
<li>Contents lists & relative links for multi-article publications like journals. Complicated by expo articles being in a separate hierarchy from journals.</li>
|
||||
<li>Translations</li>
|
||||
<li>Other people's work - the noinfo hierarchy.</li>
|
||||
<li>Style guide for writing cave descriptions: correct use of boldface (<em>once</em> for each passage name, at the primary definition thereof; other uses of the name should be links to this, and certainly should not be bold.) </li>
|
||||
</ul>
|
||||
|
||||
<h3>CUCC wiki on troggle</h3>
|
||||
|
||||
<p>CUCC still has an archive list of things that at one time were live tasks:
|
||||
from <a href="https://camcaving.uk/Documents/Expo/Legacy/Misc/Troggle%20-%20Cambridge%20University%20Caving%20Club.htm">camcaving.uk/Documents/Expo/Legacy/Misc/...</a> and that page is reproduced in the table below (so don't worry if the URL link goes dark when CUCC reorganise their legacy pages).
|
||||
<p>Troggle is a system under development for keeping track of all expo data in a logical and accessible way, and displaying it on the web. At the moment, it is [no longer] under development at <u>http://troggle.cavingexpedition.com/</u>
|
||||
|
||||
<tt>But note that this is Aaron's version of troggle, forked from the version of troggle we use. Aaron uses this for the <a href="https://expeditionwriter.com/new-expedition-to-mount-erebus-antarctica/">Erebus expedition</a>.</tt>
|
||||
</p>
|
||||
<p>Note that the information there is incomplete and editing is not yet enabled.
|
||||
</p>
|
||||
<table border="1" cellspacing="0">
|
||||
<tr>
|
||||
<th><p>Feature</p></th>
|
||||
<th><p>Old expo website</p></th>
|
||||
<th><p>Troggle: planned</p></th>
|
||||
<th><p>Troggle: progress so far</p></th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Logbook</p></td>
|
||||
<td><p>Yes; manually formatted each year</p></td>
|
||||
<td><p>Yes; wiki-style</p></td>
|
||||
<td><p>Start at the front page, <a rel="nofollow" class="external autonumber" href="http://expo.survex.com/expedition/2007">troggle.cavingexpedition.com/ [1]</a> and click to logbook for year. The logbooks have been parsed back to 1997. </p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Cave index and stats generated from survex file</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Done; see <a rel="nofollow" class="external autonumber" href="http://expo.survex.com/survexfile/caves/264">troggle.cavingexpedition.com/survey/caves/264 [2]</a> </p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Survey workflow helper</p></td>
|
||||
<td><p>Yes; minimal. surveys.csv produced an html table of whose surveys were not marked “finished”</p></td>
|
||||
<td><p>Yes. Makes table of surveys per expo which shows exactly what needs doing. Displays scans. Integrated with survex, scanner software, and tunnel.</p></td>
|
||||
<td><p>See it at <a rel="nofollow" class="external free" href="http://expo.survex.com/survey_scans/">troggle.cavingexpedition.com/survey</a> . Be sure to try a recent year when we should have data. Survex, scanner, and tunnel integration still needs doing.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>QM lists generated automatically</p></td>
|
||||
<td><p>Depends on the cave. Each cave had a different system.</p></td>
|
||||
<td><p>Yes; unified system.</p></td>
|
||||
<td><p>Done, but only 204 and 234 Qms have been imported from old system so far. No view yet.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Automatic calendar for each year of who will be on expo when</p></td>
|
||||
<td><p>No, manually produced some years</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Done; see <a rel="nofollow" class="external free" href="http://expo.survex.com/expedition/2007">troggle.cavingexpedition.com/calendar/2007</a> (replace 2007 with year in question)</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Web browser used to enter data</p></td>
|
||||
<td><p>No</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Everything can be edited through admin, at <a rel="nofollow" class="external free" href="http://expo.survex.com/admin/">troggle.cavingexpedition.com/admin</a> . Ask aaron, martin, or julian for the password if you want to have a look / play around with the admin site. Any changes you make will be overwritten. Eventually, data entry will probably be done using custom forms.
|
||||
</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Cave and passage descriptions</p></td>
|
||||
<td><p>Yes, manually html coded.</p></td>
|
||||
<td><p>Yes, wiki-style.</p></td>
|
||||
<td><p>Not done yet.<br />
|
||||
</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Expo handbook</p></td>
|
||||
<td><p>Yes, manually html coded.<br />
|
||||
</p>Maybe. Needs to be discussed further.</td>
|
||||
<td><p><br />
|
||||
</p></td>
|
||||
<td><p>Not done yet.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Table of who was on which expo</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Data has been parsed, this view hasn't been written yet. </p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Signup form, System for keeping contact, medical and next of kin info</p></td>
|
||||
<td><p>No</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Signup form should be ready by 20 Jan.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Automated photo upload and gallery</p></td>
|
||||
<td><p>No; some manual photo galleries put together with lots of effort</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Photo upload done, gallery needs writing.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Search</p></td>
|
||||
<td><p>No</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p></p></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<h3>List of cave database software</h3>
|
||||
from <a href="http://wookware.org/software/cavearchive/databasesoftwarelist">wookware.org/software/cavearchive/databasesoftwarelist</a>
|
||||
|
||||
<pre>
|
||||
ckan is something like this - could we use it?
|
||||
esri online
|
||||
|
||||
CUCC (troggle) http://cucc.survex.com/ - this site.
|
||||
virgina caves database (access+arcgis) (futrell)
|
||||
each country database
|
||||
Austria (spelix) ( <a href="https://www.spelix.at/">www.spelix.at/</a>
|
||||
UK cave registry
|
||||
mendip cave registry: (access) <a href="http://www.mcra.org.uk/wiki/doku.php">www.mcra.org.uk/wiki/doku.php</a>
|
||||
White mountains database (gpx + google earth)
|
||||
Matienzo (?)
|
||||
Fisher ridge (stephen cladiux)
|
||||
hong meigui (erin) <a href="http://www.hongmeigui.net/"http://www.hongmeigui.net/</a> (ask erin later)
|
||||
Wikicaves <a href="http://www.grottocenter.org/">www.grottocenter.org/</a>
|
||||
multilingual, slippymap, wiki data entry. includes coordinate-free caves.
|
||||
focus on sport-caving type info (access, basic gear list, overall description, bibliography)
|
||||
e.g. australians only publish coordinates to nearest 10km
|
||||
turkey <a href="http://www.tayproject.org">www.tayproject.org</a>.
|
||||
|
||||
<a href="http://www.uisic.uis-speleo.org/contacts.html">www.uisic.uis-speleo.org/contacts.html</a> change link. no-one looks for list of databases under 'contacts'
|
||||
|
||||
graziano ferrari northern italy list (access + google earth)
|
||||
</pre>
|
||||
|
||||
<h3>Wookey's notes on things to do</h3>
|
||||
from <a href="http://wookware.org/software/cavearchive/goliczmail">wookware.org/software/cavearchive/goliczmail</a>
|
||||
<pre>
|
||||
Generally I'd like to find some people (geeks) that share these technical
|
||||
ideas: (1) store things in a file system, (2) use XML, (3) do not aim too high
|
||||
(do not try designing a general system for handling all caving-related data
|
||||
for the whole world).
|
||||
|
||||
If I could find some people that agree with this, then we could try to reach a
|
||||
compromise on:
|
||||
(1) how do we store our data in a file system,
|
||||
(2) how do we use this XML (let's do a common spec, but keep it simple)
|
||||
(3) how do we aim not too high and not end up dead like CaveXML :)
|
||||
|
||||
After we do that, everyone goes away to do their own projects and write their
|
||||
own code. Or maybe we have some degree of co-operation in actually writing the
|
||||
code. Normal life. But the idea is that all geeks working on "cave inventory"
|
||||
and systems making extensive use of cave inventories try to adhere to this
|
||||
framework as much as possible. So that we can then exchange our tools.
|
||||
|
||||
I think things like "which revision system do we use" or "do we use web or
|
||||
Python" are really secondary. Everyone has their own views, habits,
|
||||
backgrounds.
|
||||
|
||||
My idea is to work on this in a small group (no more than a few persons) - to
|
||||
get things going fast, even if they are not perfect from the beginning. If it
|
||||
works, we try to convince others to use it and maybe push it through UIS.
|
||||
</pre>
|
||||
|
||||
<h3>Wookey's other notes on things to do</h3>
|
||||
from <a href="http://wookware.org/software/cavearchive/troggle2design">wookware.org/software/cavearchive/troggle2design</a>
|
||||
<pre>
|
||||
forms
|
||||
-----
|
||||
1) members read/write folk.csv and year/members
|
||||
2) cave read/write cave_data, entrance_data, surveys/pics
|
||||
3) trips -> logbook , QMs, or surveys (more than one survey or location possible)
|
||||
4) logbook reads/write year/logbook
|
||||
5) survey
|
||||
6) prospecting app
|
||||
|
||||
forms show who is logged in.
|
||||
|
||||
databases
|
||||
---------
|
||||
trips, read from
|
||||
logbook entry
|
||||
folder year#index
|
||||
.svx files
|
||||
description
|
||||
QMs
|
||||
|
||||
members (cache from form)
|
||||
|
||||
caves
|
||||
caves_data
|
||||
entrance_data
|
||||
|
||||
storage:
|
||||
expoweb
|
||||
data/
|
||||
cave_entrances
|
||||
caves
|
||||
descriptions
|
||||
|
||||
loser
|
||||
foo.svx
|
||||
</pre>
|
||||
|
||||
|
||||
<h3>Yet more of Wookey's notes</h3>
|
||||
from <a href="http://wookware.org/software/cavearchive/expoweb-design">wookware.org/software/cavearchive/expoweb-design</a>
|
||||
<pre>
|
||||
frontpage
|
||||
---------
|
||||
quick to load:
|
||||
Links:
|
||||
Caves number, name, location
|
||||
Years
|
||||
Handbook
|
||||
Data Entry
|
||||
Main Index
|
||||
|
||||
Slippy map:
|
||||
Indexes to cave page
|
||||
|
||||
Cave page:
|
||||
Access, description, photos, QMs, Survey
|
||||
|
||||
Years:
|
||||
Logbooks/surveynotes/survexdata/people matrix
|
||||
Documents
|
||||
|
||||
Data Entry:
|
||||
Logbook entry
|
||||
Survey data
|
||||
Survey Notes
|
||||
Cave description
|
||||
QMs
|
||||
Photos
|
||||
New cave
|
||||
|
||||
Backend datafiles:
|
||||
caves/
|
||||
cave_entrance
|
||||
cave_data
|
||||
directory of info
|
||||
|
||||
years/
|
||||
year/
|
||||
logbook
|
||||
pubs/
|
||||
reports
|
||||
admin/
|
||||
lists
|
||||
who_and_when
|
||||
travel
|
||||
jobs
|
||||
|
||||
surveyscans/
|
||||
year/
|
||||
index
|
||||
#num
|
||||
handbook/
|
||||
(all static info)
|
||||
|
||||
Storage:
|
||||
non-html or > 280K go in 'files' (PDF, PNG, JPEG, DOC, ODF, SVG)
|
||||
convert small 1024x768 version into website by default. (matching structure?
|
||||
</pre>
|
||||
|
||||
<hr />
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<h3><a id="arch">Archived updates</a></h3>
|
||||
<p>Since 2008 we have been keeping detailed records of all data management system updates in the version control system.
|
||||
Before then we manually maintained <a href="../computing/update.html">a list of updates</a> which are now only of historical interest.
|
||||
<p>A history of the expo website and software was published in Cambridge Underground 1996. A copy of this article <a href="../c21bs.html">Taking Expo Bullshit into the 21st Century</a> is archived here.
|
||||
<hr />
|
||||
Go on to: <a href="trogarch.html">Troggle architecture</a><br />
|
||||
Return to: <a href="trognotes.html">Troggle notes</a><br />
|
||||
<hr />
|
||||
</body>
|
||||
</html>
|
BIN
handbook/troggle/mtv_drawing2.jpg
Normal file
BIN
handbook/troggle/mtv_drawing2.jpg
Normal file
Binary file not shown.
After Width: | Height: | Size: 22 KiB |
296
handbook/troggle/schema.json
Normal file
296
handbook/troggle/schema.json
Normal file
@ -0,0 +1,296 @@
|
||||
{"Entrance": [{"name": "ids",
|
||||
"data type": "ids",
|
||||
"help": "List of short strings by which this entrance can be identified. It is preferable for these ids to be unique."},
|
||||
{"name": "Names",
|
||||
"data type": "translations",
|
||||
"help": "Names of this entrance"},
|
||||
{"name": "Areas",
|
||||
"data type": "manytomany:Area",
|
||||
"help": "Areas that the entrance is in. Where an area has been specified there is no need to add the areas parent area."},
|
||||
{"name": "Description",
|
||||
"data type": "translation",
|
||||
"help": "A description of the entrance"},
|
||||
{"name": "Exploration",
|
||||
"data type": "translation",
|
||||
"help": "A description of the exploration/discovers of the entrance"},
|
||||
{"name": "Location description",
|
||||
"data type": "translation",
|
||||
"help": "A description of where the cave is located"},
|
||||
{"name": "Approach",
|
||||
"data type": "translation",
|
||||
"help": "A description of how to get to the entrance"},
|
||||
{"name": "Photos",
|
||||
"data type": "photos",
|
||||
"help": "Photos of the entrance"},
|
||||
{"name": "Marking",
|
||||
"data type": ["classification", [{"class": "Unmarked",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Unknown",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Paint",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Probably Paint",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Temporary Tag",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Probably Temporary Tag",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Spit",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Probably Spit",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Tag"},
|
||||
{"class": "Probably Tag",
|
||||
"extra information": {"desciption": "translation"}}
|
||||
]],
|
||||
"help": "Marking of the entrance"},
|
||||
{"name": "Findability",
|
||||
"data type": ["classification", [{"class": "Unknown",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Surveyed Coordinates",
|
||||
"extra information": {"desciption": "translation",
|
||||
"coordinates": "coordinates"}},
|
||||
{"class": "Surveyed Survex",
|
||||
"extra information": {"desciption": "translation",
|
||||
"station": "survex station"}},
|
||||
{"class": "Lost",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Refindable",
|
||||
"extra information": {"desciption": "translation"}},
|
||||
{"class": "Bearings",
|
||||
"extra information": {"desciption": "translation"}}
|
||||
]],
|
||||
"help": "How can the entrance be found?"},
|
||||
{"name": "References",
|
||||
"data type": "references",
|
||||
"help": "External articles about this entrance"},
|
||||
{"name": "Notes",
|
||||
"data type": "translation",
|
||||
"help": "Any miscellaneous information"},
|
||||
{"name": "URLs",
|
||||
"data type": "urls",
|
||||
"help": "URLs which should redirect to this entrance. (Legacy)"}
|
||||
],
|
||||
"Cave": [{"name": "ids",
|
||||
"data type": "ids",
|
||||
"help": "List of short strings by which this cave can be identified. It is preferable for these ids to be unique."},
|
||||
{"name": "Names",
|
||||
"data type": "translations",
|
||||
"help": "Names of this cave"},
|
||||
{"name": "Areas",
|
||||
"data type": "manytomany:Area",
|
||||
"help": "Areas that the cave is in. Where an area has been specified there is no need to add the area's parent area."},
|
||||
{"name": "Description",
|
||||
"data type": "translation",
|
||||
"help": "A description of the cavee"},
|
||||
{"name": "Exploration",
|
||||
"data type": "translation",
|
||||
"help": "A description of the exploration/discovers of the cave"},
|
||||
{"name": "Entrances",
|
||||
"data type": "manytomany:Entrance",
|
||||
"help": "Entrances connected to this piece of cave"},
|
||||
{"name": "Equipment",
|
||||
"data type": "translation",
|
||||
"help": "Equipment required in this cave"},
|
||||
{"name": "References",
|
||||
"data type": "references",
|
||||
"help": "External articles about this cave"},
|
||||
{"name": "Survey",
|
||||
"data type": "translation",
|
||||
"help": "Description (including relevant images?) of any drawn up surveys"},
|
||||
{"name": "Notes",
|
||||
"data type": "translation",
|
||||
"help": "Any miscilanious information"},
|
||||
{"name": "Centreline data",
|
||||
"data type": ["classification", [{"class": "Surveyed Survex",
|
||||
"extra information": {"desciption": "translation",
|
||||
"subsurvey": "survex name"}},
|
||||
{"class": "External",
|
||||
"extra information": {"desciption": "translation",
|
||||
"length": "float",
|
||||
"length unit": "length unit",
|
||||
"depth": "float",
|
||||
"depth unit": "length unit",
|
||||
"extent": "float",
|
||||
"extent unit": "length unit"
|
||||
}}]],
|
||||
"help": "Statistics of cave size, or reference to centre line data where this can be calculated"},
|
||||
{"name": "URLs",
|
||||
"data type": "urls",
|
||||
"help": "URLs which should redirect to this cave. (Legacy)"},
|
||||
{"name": "Parents",
|
||||
"data type": "parents",
|
||||
"help": "Larger Caves which this is part of"},
|
||||
{"name": "Is wet cave",
|
||||
"data type": "boolean",
|
||||
"help": "Specific to caves in the Austrian Kataster?"},
|
||||
{"name": "Is rock shelter",
|
||||
"data type": "boolean",
|
||||
"help": "Specific to caves in the Austrian Kataster?"},
|
||||
{"name": "Kataser number",
|
||||
"data type": "integer",
|
||||
"help": "Specific to caves in the Austrian Kataster?"},
|
||||
{"name": "Exploration status",
|
||||
"data type": ["classification", [{"class": "Unexplored"},
|
||||
{"class": "Partially Explored"},
|
||||
{"class": "Fully Explored"}]],
|
||||
"help": "Specific to caves in the Austrian Kataster?"}
|
||||
],
|
||||
"Area": [{"name": "ids",
|
||||
"data type": "ids",
|
||||
"help": "List of short strings by which this cave can be identified. It is preferable for these ids to be unique."},
|
||||
{"name": "Names",
|
||||
"data type": "translations",
|
||||
"help": "Names of this cave"},
|
||||
{"name": "Description",
|
||||
"data type": "translation",
|
||||
"help": "A description of the cavee"},
|
||||
{"name": "URLs",
|
||||
"data type": "urls",
|
||||
"help": "URLs which should redirect to this cave. (Legacy)"},
|
||||
{"name": "Parents",
|
||||
"data type": "parents",
|
||||
"help": "Larger areas of which this is a part"}
|
||||
],
|
||||
"Trip": [{"name": "ids",
|
||||
"data type": "ids",
|
||||
"help": "List of short strings by which this trip can be identified. It is preferable for these ids to be unique."},
|
||||
{"name": "When",
|
||||
"data type": ["classification", [{"class": "Date",
|
||||
"extra information": {"date": "date",
|
||||
"duration": "survex name",
|
||||
"duration units": "time unit"}},
|
||||
{"class": "Date and Time",
|
||||
"extra information": {"start": "datetime",
|
||||
"end": "datetime"
|
||||
}}]],
|
||||
"help": "When the trip happened"},
|
||||
{"name": "Authors",
|
||||
"data type": "manytomany:Person",
|
||||
"help": "Who wrote up the trip?"},
|
||||
{"name": "Participants",
|
||||
"data type": "manytomany:Person",
|
||||
"help": "Who was on the trip?"},
|
||||
{"name": "Description",
|
||||
"data type": "translation",
|
||||
"help": "Trip report"},
|
||||
{"name": "Expeditions",
|
||||
"data type": "manytomany:Expedition",
|
||||
"help": "The expedition(s) on which the trip occured."},
|
||||
{"name": "URLs",
|
||||
"data type": "urls",
|
||||
"help": "URLs which should redirect to this cave. (Legacy)"}
|
||||
],
|
||||
"Expedition": [{"name": "ids",
|
||||
"data type": "ids",
|
||||
"help": "List of short strings by which this expedition can be identified. It is preferable for these ids to be unique."},
|
||||
{"name": "Names",
|
||||
"data type": "translations",
|
||||
"help": "Names of this expedition"},
|
||||
{"name": "Description",
|
||||
"data type": "translation",
|
||||
"help": "A description of the expedition"},
|
||||
{"name": "URLs",
|
||||
"data type": "urls",
|
||||
"help": "URLs which should redirect to this expedition. (Legacy)"},
|
||||
{"name": "Parents",
|
||||
"data type": "parents",
|
||||
"help": "Larger expediton groupings of which this is a part"}
|
||||
],
|
||||
"Person": [{"name": "ids",
|
||||
"data type": "ids",
|
||||
"help": "List of short strings by which this person can be identified. It is preferable for these ids to be unique. To get links from team members in survex files etc, ISOdate search terms + '+' + nicknames can be added here, eg 2008-06-*+Nickname"},
|
||||
{"name": "Name",
|
||||
"data type": "name",
|
||||
"help": "Fullname"},
|
||||
{"name": "Description",
|
||||
"data type": "translation",
|
||||
"help": "A description of the caver"},
|
||||
{"name": "URLs",
|
||||
"data type": "urls",
|
||||
"help": "URLs which should redirect to this caver. (Legacy)"}
|
||||
],
|
||||
"Drawn up survey": [{"name": "ids",
|
||||
"data type": "ids",
|
||||
"help": "List of short strings by which this survey can be identified. It is preferable for these ids to be unique."},
|
||||
{"name": "Caves",
|
||||
"data type": "manytomany:Cave",
|
||||
"help": "What caves are in the survey?"},
|
||||
{"name": "Date",
|
||||
"data type": "date",
|
||||
"help": "When was the survey drawn?"},
|
||||
{"name": "Description",
|
||||
"data type": "translation",
|
||||
"help": "A description of the survey"},
|
||||
{"name": "File",
|
||||
"data type": "file",
|
||||
"help": "filename"},
|
||||
{"name": "Localisation",
|
||||
"data type": ["classification", [{"class": "Therion"},
|
||||
{"class": "Image - 3 point transform",
|
||||
"extra information": {"coordinate system": "coordinate system",
|
||||
"pixel1x": "float",
|
||||
"pixel1y": "float",
|
||||
"pixel2x": "float",
|
||||
"pixel2y": "float",
|
||||
"pixel3x": "float",
|
||||
"pixel3y": "float",
|
||||
"postion1E": "float",
|
||||
"postion1N": "float",
|
||||
"postion1H": "float",
|
||||
"postion2E": "float",
|
||||
"postion2N": "float",
|
||||
"postion2H": "float",
|
||||
"postion3E": "float",
|
||||
"postion3N": "float",
|
||||
"postion3H": "float"
|
||||
}}]],
|
||||
"help": "filename"}
|
||||
],
|
||||
"Photo": [{"name": "ids",
|
||||
"data type": "ids",
|
||||
"help": "List of short strings by which this photo can be identified. It is preferable for these ids to be unique."},
|
||||
{"name": "Caves",
|
||||
"data type": "manytomany:Cave",
|
||||
"help": "What caves is the photo of?"},
|
||||
{"name": "Areas",
|
||||
"data type": "manytomany:Area",
|
||||
"help": "Does the photo give information about a particular area?"},
|
||||
{"name": "People",
|
||||
"data type": "manytomany:Person",
|
||||
"help": "Who is in the photo?"},
|
||||
{"name": "Photo team",
|
||||
"data type": "manytomany:Person",
|
||||
"help": "Who helped take the photo?"},
|
||||
{"name": "Expeditions",
|
||||
"data type": "manytomany:Expedition",
|
||||
"help": "What expedition(s) was the photo taken on?"},
|
||||
{"name": "Date",
|
||||
"data type": "date",
|
||||
"help": "When was the photo taken"},
|
||||
{"name": "Description",
|
||||
"data type": "translation",
|
||||
"help": "A description of the survey"},
|
||||
{"name": "File",
|
||||
"data type": "file",
|
||||
"help": "filename"}],
|
||||
"QM": [{"name": "ids",
|
||||
"data type": "ids",
|
||||
"help": "List of short strings by which this QM can be identified. It is preferable for these ids to be unique."},
|
||||
{"name": "Caves",
|
||||
"data type": "manytomany:Cave",
|
||||
"help": "What caves is the QM in?"},
|
||||
{"name": "Expeditions",
|
||||
"data type": "manytomany:Expedition",
|
||||
"help": "What expedition(s) was the photo taken on?"},
|
||||
{"name": "Date",
|
||||
"data type": "date",
|
||||
"help": "When was the photo taken"},
|
||||
{"name": "Description",
|
||||
"data type": "translation",
|
||||
"help": "A description of the survey"},
|
||||
{"name": "Completion",
|
||||
"data type": "translation",
|
||||
"help": "A description of the survey"}]
|
||||
}
|
||||
|
80
handbook/troggle/trog2030.html
Normal file
80
handbook/troggle/trog2030.html
Normal file
@ -0,0 +1,80 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||||
<title>Handbook Troggle Design</title>
|
||||
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
|
||||
</head>
|
||||
<body><style>body { background: #fff url(/images/style/bg-system.png) repeat-x 0 0 }</style>
|
||||
<h2 id="tophead">CUCC Expedition Handbook</h2>
|
||||
<h1>Troggle in 2025-2030</h1>
|
||||
|
||||
<h2>5-Year Plan</h2>
|
||||
<p>[Philip Sargent, 1 June 2020]
|
||||
<ul>
|
||||
<li>I reckon django has at least another 4-5 years left as a very active project (~2025).
|
||||
<li>I reckon python has another 10-20 years at least.
|
||||
<li>I reckon SQL databases have another 30 years at least.
|
||||
</ul>
|
||||
<p>I don't think writing our own object/SQL code is sensible:
|
||||
there is such a lot going on we would create a large volume of software even if we stick close to the metal.
|
||||
[I could well be wrong. That is Option 1.]
|
||||
|
||||
<h3>Option 2</h2>
|
||||
<p>
|
||||
We keep the same architecture as now, and incrementally replace modules that use django/SQL with direct object storage of collections using pickle(), shelve() and json().
|
||||
Concurrency is not a problem as all data is read-only.
|
||||
We keep the url-rewriting and html-template things in django.[and migrate the unit-tests (a recent innovation) from django to be run stand-alone.]
|
||||
<p>
|
||||
This means that the "django-ness" part of troggle becomes quite small.
|
||||
The more modules we replace, the easier it becomes for new people to work on it - but also the easier it becomes to migrate it to newer django versions. Or the easier it becomes to move entirely from django to Jinja2 + a URL-router + a HTTP-request/response system.
|
||||
The data flow through the system becomes obvious to any python programmer with no django knowledge needed.
|
||||
<p>
|
||||
[This could be harder than it looks if cross-referencing and pointers between collections become unmaintainable - a risk we need to watch.]
|
||||
<p>
|
||||
Being memory-resident, we get a 20x speed increase. Which we don't need.
|
||||
|
||||
<p>
|
||||
So the proposed Option 2 looks a bit like this (django is the "flawed supplier" and pickle() is the "new supplier")<br><a href="
|
||||
https://martinfowler.com/bliki/BranchByAbstraction.html">
|
||||
<img src="https://martinfowler.com/bliki/images/branch-by-abstraction/step-2.png">Migrate to Replacement Abstraction Layer</a>
|
||||
|
||||
<h3>Option 2A</h2>
|
||||
<p>
|
||||
We also use a noSQL db with a direct and easy mapping to python collections.
|
||||
[This needs to be explored, but I suspect we don't gain much compared with the effort of forcing maintainers to learn a new query language. Shelve() is already adequate.]
|
||||
|
||||
<h3>Option Zero</h2>
|
||||
<div style="color:blue">
|
||||
<p>We need to de-cruft troggle *first*: remove 'good ideas' which were never used, trim redundancies and generally reduce it in size significantly.
|
||||
<p>
|
||||
We should have a good look at modifying everything so that we do not read in every survey station. This means making a list of functions that we will drop and some we replace by parsing cavern output and some we calculate during importing/reading svx files.
|
||||
|
||||
<p>
|
||||
Documentation is the key to keeping troggle in a state where someone can pick it up and do a useful week's work, e.g. extracting the parsed logbooks to use shelve() storage throughout instead of SQL. The next time someone like Radost comes along during the next 5 years we want to be able to use them effectively.
|
||||
</div>
|
||||
<h3>Things that could be a bit sticky</h3>
|
||||
<p>
|
||||
New functionality: e.g. making the whole thing GIS-centric is a possibility.
|
||||
A GIS db could make a lot of sense. Not in scope for this discussion.
|
||||
<p>There is not yet a front-end (javascript) framework on the client, i.e. a phone app or webpage, which is stable enough for us to commit effort to. Bits of troggle use very old jQuery ("edit this page", and the svx file editor) , and Flask looks interesting, but maybe in 2025 we could see a good way to move all the user interface to the client and just have an API on the server.
|
||||
|
||||
<h3>API</h3>
|
||||
<p>We will also need an API now-ish, whatever we do, so that keen kids can write their own special-purpose front-ends using new cool toys. Which will keep them out of our hair.
|
||||
<h3>Postscript</h3>
|
||||
|
||||
<p>Andy Waddington, who wrote the first expo website in 1996, mentioned that he could never get the hang of Django at all, and working with SQL databases would require some serious book-revision:
|
||||
<p>
|
||||
So a useful goal, I think, is to make 'troggle2' accessible to a generic python programmer with no specialist skills in any databases or frameworks. Put against that is the argument that that might double the volume of code to be maintained, which would be worse. Nevertheless, an aim to keep in mind.
|
||||
|
||||
But even 'just Python' is not that easy. Python is a much bigger language now than it used to be, with some esoteric corners. (Some of which could be very useful, such as the self-testing unit test
|
||||
capability: <a href="https://docs.python.org/3.8/library/doctest.html">docs.python.org/3.8/library/doctest</a> )
|
||||
|
||||
|
||||
|
||||
<hr />
|
||||
|
||||
Return to: <a href="trogdesign.html">Troggle design</a><br />
|
||||
<hr />
|
||||
</body>
|
||||
</html>
|
@ -28,12 +28,29 @@ The core of troggle is the data architecture: the set of tables into which all t
|
||||
<br><figcaption>Packages</figcaption>
|
||||
</figure>
|
||||
</div></div>
|
||||
<h3>Architecture description</h3>
|
||||
<p>Read the proposal: "<a href="/expofiles/documents/troggle/troggle_paper.pdf" download>Troggle: a novel system for cave exploration information management</a>", by Aaron Curtis</em>. But remember that this paper is an over-ambitious proposal. Only the core data management features have been built. We have none of the "person management" features, none of the "wallet progress" stuff and only two forms in use: for entering cave and cave entrance data.
|
||||
<p>
|
||||
ALSO there have been tables added to the core representation which are not anticipated in that document of this diagram, e.g. Scannedimage, Survexdirectory, Survexscansfolder, Survexscansingle, Tunnelfile, TunnelfileSurvexscansfolders, Survey. See <a href="datamodel.html">Troggle data model</a> python code (15 May 2020).
|
||||
<h3>Architecture and purpose</h3>
|
||||
<p>Read the proposal: "<a href="/expofiles/documents/troggle/troggle_paper.pdf" download>Troggle: a novel system for cave exploration information management</a>", by Aaron Curtis</em>. But remember that this paper is an over-ambitious proposal. Only the core data management features have been built. We have none of the "person management" features, none of the "wallet progress" stuff and only two forms in use: for entering <var>cave</var> and <var>cave entrance</var> data.
|
||||
|
||||
<h3>Implementation in software</h3>
|
||||
<p><a href="http://www.djangoproject.com/"><img class="onright" src="https://www.djangoproject.com/m/img/badges/djangopowered126x54.gif" border="0" alt="Powered by Django." title="Powered by Django." /></a>
|
||||
Troggle is written in Python (about 9,000 lines including comments) and is built on the Django framework. Before starting to work on Troggle it might be a good idea to run through an initial install and exploration of <a href="https://code.djangoproject.com/wiki/Tutorials">a tutorial Django project</a>.
|
||||
|
||||
<p>
|
||||
|
||||
Django is the thing that puts the survey data in a database in a way that helps us write far less code to get it in and out again, and provides templates which make it quicker to maintain all the webpages.
|
||||
|
||||
|
||||
|
||||
|
||||
<h3>Troggle parsers and input files</h3>
|
||||
<div class="onright">
|
||||
<figure>
|
||||
<a href="https://djangobook.com/mdj2-django-structure/"><img src="mtv_drawing2.jpg"></a>
|
||||
<br><figcaption>Django server and webpage (client)</figcaption>
|
||||
</figure>
|
||||
</div>
|
||||
<p>To understand how troggle imports the data from the survex files, tunnel files, logbooks etc., see the <a href="trogimport.html">troggle import (databaseReset.py)</a> documentation.
|
||||
<p>The following separate import operations are managed by the import utility <a href="trogimport.html">(databaseReset.py)</a>:
|
||||
<ul>
|
||||
|
@ -12,8 +12,9 @@
|
||||
<p>Open issues being worked on:
|
||||
<ul>
|
||||
<li>New systems for <a href="menudesign.html">website menus</a>
|
||||
<li>Speculations on <a href="trogdesignx.html">future architectures</a>
|
||||
<li>Critiqued proposal on <a href="trogsimpler.html">future architecture</a>
|
||||
<li>A possible <a href="trog2030.html">5 year plan</a>
|
||||
<li>Speculations on <a href="trogspeculate.html">future architectures</a>
|
||||
</ul>
|
||||
<hr />
|
||||
Go on to: <a href="trogarch.html">Troggle architecture</a><br />
|
||||
|
@ -15,13 +15,13 @@ faced with user-generated data and how to fix them. Common error messages. How t
|
||||
<p>It concentrates on showing how to find and fix import errors so that the troggle-generated webpages are complete and not full of gaps.
|
||||
<p>Intended audience: Ideally someone who is not a python programmer will be able to use this page to help them fix import errors.
|
||||
<p>Also here we will explain the useful diagnostic pages such as <br>
|
||||
<a href="expo.survex.com/experimental">experimental</a><br>
|
||||
<a href="expo.survex.com/pathsreport">pathsreport</a><br>
|
||||
<a href="expo.survex.com/survey_scans">survey_scans</a><br>
|
||||
<a href="expo.survex.com//admin/core/dataissue/l">/admin/core/dataissue/</a><br>
|
||||
<a href="expo.survex.com/tunneldata">tunneldata</a><br>
|
||||
<a href="expo.survex.com/statistics">statistics</a><br>
|
||||
<a href="expo.survex.com/people">people</a><br>
|
||||
<a href="http://expo.survex.com/experimental">experimental</a><br>
|
||||
<a href="http://expo.survex.com/pathsreport">pathsreport</a><br>
|
||||
<a href="http://expo.survex.com/survey_scans">survey_scans</a><br>
|
||||
<a href="http://expo.survex.com//admin/core/dataissue/l">/admin/core/dataissue/</a><br>
|
||||
<a href="http://expo.survex.com/tunneldata">tunneldata</a><br>
|
||||
<a href="http://expo.survex.com/statistics">statistics</a><br>
|
||||
<a href="http://expo.survex.com/people">people</a><br>
|
||||
|
||||
<hr />
|
||||
Go on to: <a href="trogarch.html">Troggle architecture</a><br />
|
||||
|
@ -7,13 +7,24 @@
|
||||
</head>
|
||||
<body><style>body { background: #fff url(/images/style/bg-system.png) repeat-x 0 0 }</style>
|
||||
<h2 id="tophead">CUCC Expedition Handbook</h2>
|
||||
<h1>Troggle - TO BE RESTRUCTURED</h1>
|
||||
<h1>Troggle - the beginnings of a manual</h1>
|
||||
<p>Troggle runs much of the the cave survey data management, presents the data on the website and manages the Expo Handbook.
|
||||
|
||||
<h2>NOTES - taken from elsewhere</h2>
|
||||
<h2>Who needs to know What and When</h2>
|
||||
|
||||
<p>This part of the handbook is intended for people maintaining the troggle software. Day to day cave recording and surveying tasks are documented in the expo "survey handbook"
|
||||
|
||||
<p>We have several quite different sorts of cavers who interact with troggle:
|
||||
<ul>
|
||||
<li>The youthful hard caver, who is trained in underground survey techniques but whose interest is limited to handing over the grubby survey notes when she emerges into daylight. Is keen to know how many km of cave she surveyed each year and to see pretty drawn-up surveys (done by someone else). Walks through walls.
|
||||
<li>The surface walker who is happy to do route-finding over the plateau, takes lots of photos of cave entrances and cavers enjoying sunshine and may sometimes be able to provide GPS tracks of where he has been. He needs a prospecting guide to find previously identified entrances and be able to find photos of caves in past years. Writes up his explorations in execrable handwriting in the logbook. Looks at walls.
|
||||
<li>The diligent student who types up the survey notes into survex file format, transcribes sketch notes onto survex centre-lines, and uses Therion to produce beautiful survey graphics of the caves he has digitised - but who is not a computer geek and whose brain oozes out of his ears when Wookey explains what git is. Applies artistic graffiti to walls.
|
||||
<li>The archivist who takes the survex files, the therion files, the GPS files, the scanned survex centrelines and files them in the right places on the <em> expo laptop</em>, uses the troggle reports to help ensure that these are consistent and are filed correctly. Uses troggle input forms to "create new cave" in the system and adds to the directory structures to match the recently discovered caves. Is learning git. When transcribing bad handwriting in logbook (or struggling with git), climbs walls.
|
||||
<li><em>Nerdus maximus</em>: talks python in his sleep and can rebase a hairy git branch without error after 7 bottles of Gosser. Painfully averse to writing documentation. Overstressed, over-caffeinated and with a tendency to mutter that it's all obvious. Oblivious to walls.
|
||||
</ul>
|
||||
<p>These are some of the "use cases" for which troggle needs to be (re)designed to cope with.
|
||||
|
||||
<h3>Rewrite from here on...</h3>
|
||||
<p>This troggle manual describes these:
|
||||
<ul>
|
||||
<li>Annual tasks: preparing for next year, finishing last year (troggle & scripts)
|
||||
@ -65,302 +76,14 @@ can do this by filling-in some online forms.
|
||||
All Survex | Scans | Tunneldata | 107 | 161 | 204 | 258 | 264 | Expo2016 | Expo2017 | Expo2018 | Django admin
|
||||
</pre>
|
||||
|
||||
<h3>Future Developments: Preamble</h3>
|
||||
<p><em>Assumptions</em> (points to necessarily agree upon)
|
||||
<ol>
|
||||
<li>Let's NOT try to design a generic catalogue for storing all kind of data about caves of the whole world, intended for every kind of user (sports, exploration, science). Let's just settle for a generic framework. Let geeks in individual countries or individual communities write their tools operating within this framework.
|
||||
<li>Let's try make it available for the layman, but still well-playable for the geeks.
|
||||
<li>Let's rely on already existing, popular technologies. Let's keep it open source and multiplatform. Let's try not to reinvent the wheel.
|
||||
<li>Let's not assume everyone has an Internet connection while working with their data.
|
||||
<li>Let's version-control as much as possible.
|
||||
<li>Let's support i18n - let's use UTF-8 everywhere and cater for data in many languages(entrance names, cave descriptions, location descriptions etc.)
|
||||
</ol>
|
||||
|
||||
<tt><em>Everything here should be updated or replaced - this page just records a lot of unfinished ideas.
|
||||
Most people will not want to read this at all. This is for speleosoftwarearcheologists only.</em>
|
||||
</tt>
|
||||
|
||||
<p>Two page preliminary design document for <a href="../../documents/caca_arch2.pdf">'caca' (Cave Catalogue) rev.2 2013-07-26</a> by Wookey (copied from http://wookware.org/software/cavearchive/caca_arch2.pdf)
|
||||
|
||||
|
||||
<h2>The data management system conventions bit</h2>
|
||||
<p>This is likely to change with structural change to the site, with style changes which we expect to implement and with the method by which the info is actually stored and served up.</p>
|
||||
<p>... and it's not written yet, either :-)</p>
|
||||
<ul>
|
||||
|
||||
<li>Structure</li>
|
||||
<li>Info for each cave – automatically generated by <tt>make-indxal4.pl</tt></li>
|
||||
<li>Contents lists & relative links for multi-article publications like journals. Complicated by expo articles being in a separate hierarchy from journals.</li>
|
||||
<li>Translations</li>
|
||||
<li>Other people's work - the noinfo hierarchy.</li>
|
||||
<li>Style guide for writing cave descriptions: correct use of boldface (<em>once</em> for each passage name, at the primary definition thereof; other uses of the name should be links to this, and certainly should not be bold.) </li>
|
||||
</ul>
|
||||
|
||||
<h3>CUCC wiki on troggle</h3>
|
||||
|
||||
<p>CUCC still has an archive list of things that at one time were live tasks:
|
||||
from <a href="https://camcaving.uk/Documents/Expo/Legacy/Misc/Troggle%20-%20Cambridge%20University%20Caving%20Club.htm">camcaving.uk/Documents/Expo/Legacy/Misc/...</a> and that page is reproduced in the table below (so don't worry if the URL link goes dark when CUCC reorganise their legacy pages).
|
||||
<p>Troggle is a system under development for keeping track of all expo data in a logical and accessible way, and displaying it on the web. At the moment, it is [no longer] under development at <u>http://troggle.cavingexpedition.com/</u>
|
||||
|
||||
<tt>But note that this is Aaron's version of troggle, forked from the version of troggle we use. Aaron uses this for the <a href="https://expeditionwriter.com/new-expedition-to-mount-erebus-antarctica/">Erebus expedition</a>.</tt>
|
||||
</p>
|
||||
<p>Note that the information there is incomplete and editing is not yet enabled.
|
||||
</p>
|
||||
<table border="1" cellspacing="0">
|
||||
<tr>
|
||||
<th><p>Feature</p></th>
|
||||
<th><p>Old expo website</p></th>
|
||||
<th><p>Troggle: planned</p></th>
|
||||
<th><p>Troggle: progress so far</p></th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Logbook</p></td>
|
||||
<td><p>Yes; manually formatted each year</p></td>
|
||||
<td><p>Yes; wiki-style</p></td>
|
||||
<td><p>Start at the front page, <a rel="nofollow" class="external autonumber" href="http://expo.survex.com/expedition/2007">troggle.cavingexpedition.com/ [1]</a> and click to logbook for year. The logbooks have been parsed back to 1997. </p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Cave index and stats generated from survex file</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Done; see <a rel="nofollow" class="external autonumber" href="http://expo.survex.com/survexfile/caves/264">troggle.cavingexpedition.com/survey/caves/264 [2]</a> </p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Survey workflow helper</p></td>
|
||||
<td><p>Yes; minimal. surveys.csv produced an html table of whose surveys were not marked “finished”</p></td>
|
||||
<td><p>Yes. Makes table of surveys per expo which shows exactly what needs doing. Displays scans. Integrated with survex, scanner software, and tunnel.</p></td>
|
||||
<td><p>See it at <a rel="nofollow" class="external free" href="http://expo.survex.com/survey_scans/">troggle.cavingexpedition.com/survey</a> . Be sure to try a recent year when we should have data. Survex, scanner, and tunnel integration still needs doing.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>QM lists generated automatically</p></td>
|
||||
<td><p>Depends on the cave. Each cave had a different system.</p></td>
|
||||
<td><p>Yes; unified system.</p></td>
|
||||
<td><p>Done, but only 204 and 234 Qms have been imported from old system so far. No view yet.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Automatic calendar for each year of who will be on expo when</p></td>
|
||||
<td><p>No, manually produced some years</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Done; see <a rel="nofollow" class="external free" href="http://expo.survex.com/expedition/2007">troggle.cavingexpedition.com/calendar/2007</a> (replace 2007 with year in question)</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Web browser used to enter data</p></td>
|
||||
<td><p>No</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Everything can be edited through admin, at <a rel="nofollow" class="external free" href="http://expo.survex.com/admin/">troggle.cavingexpedition.com/admin</a> . Ask aaron, martin, or julian for the password if you want to have a look / play around with the admin site. Any changes you make will be overwritten. Eventually, data entry will probably be done using custom forms.
|
||||
</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Cave and passage descriptions</p></td>
|
||||
<td><p>Yes, manually html coded.</p></td>
|
||||
<td><p>Yes, wiki-style.</p></td>
|
||||
<td><p>Not done yet.<br />
|
||||
</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Expo handbook</p></td>
|
||||
<td><p>Yes, manually html coded.<br />
|
||||
</p>Maybe. Needs to be discussed further.</td>
|
||||
<td><p><br />
|
||||
</p></td>
|
||||
<td><p>Not done yet.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Table of who was on which expo</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Data has been parsed, this view hasn't been written yet. </p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Signup form, System for keeping contact, medical and next of kin info</p></td>
|
||||
<td><p>No</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Signup form should be ready by 20 Jan.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Automated photo upload and gallery</p></td>
|
||||
<td><p>No; some manual photo galleries put together with lots of effort</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p>Photo upload done, gallery needs writing.</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>Search</p></td>
|
||||
<td><p>No</p></td>
|
||||
<td><p>Yes</p></td>
|
||||
<td><p></p></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<h3>List of cave database software</h3>
|
||||
from <a href="http://wookware.org/software/cavearchive/databasesoftwarelist">wookware.org/software/cavearchive/databasesoftwarelist</a>
|
||||
|
||||
<pre>
|
||||
ckan is something like this - could we use it?
|
||||
esri online
|
||||
|
||||
CUCC (troggle) http://cucc.survex.com/ - this site.
|
||||
virgina caves database (access+arcgis) (futrell)
|
||||
each country database
|
||||
Austria (spelix) ( <a href="https://www.spelix.at/">www.spelix.at/</a>
|
||||
UK cave registry
|
||||
mendip cave registry: (access) <a href="http://www.mcra.org.uk/wiki/doku.php">www.mcra.org.uk/wiki/doku.php</a>
|
||||
White mountains database (gpx + google earth)
|
||||
Matienzo (?)
|
||||
Fisher ridge (stephen cladiux)
|
||||
hong meigui (erin) <a href="http://www.hongmeigui.net/"http://www.hongmeigui.net/</a> (ask erin later)
|
||||
Wikicaves <a href="http://www.grottocenter.org/">www.grottocenter.org/</a>
|
||||
multilingual, slippymap, wiki data entry. includes coordinate-free caves.
|
||||
focus on sport-caving type info (access, basic gear list, overall description, bibliography)
|
||||
e.g. australians only publish coordinates to nearest 10km
|
||||
turkey <a href="http://www.tayproject.org">www.tayproject.org</a>.
|
||||
|
||||
<a href="http://www.uisic.uis-speleo.org/contacts.html">www.uisic.uis-speleo.org/contacts.html</a> change link. no-one looks for list of databases under 'contacts'
|
||||
|
||||
graziano ferrari northern italy list (access + google earth)
|
||||
</pre>
|
||||
|
||||
<h3>Wookey's notes on things to do</h3>
|
||||
from <a href="http://wookware.org/software/cavearchive/goliczmail">wookware.org/software/cavearchive/goliczmail</a>
|
||||
<pre>
|
||||
Generally I'd like to find some people (geeks) that share these technical
|
||||
ideas: (1) store things in a file system, (2) use XML, (3) do not aim too high
|
||||
(do not try designing a general system for handling all caving-related data
|
||||
for the whole world).
|
||||
|
||||
If I could find some people that agree with this, then we could try to reach a
|
||||
compromise on:
|
||||
(1) how do we store our data in a file system,
|
||||
(2) how do we use this XML (let's do a common spec, but keep it simple)
|
||||
(3) how do we aim not too high and not end up dead like CaveXML :)
|
||||
|
||||
After we do that, everyone goes away to do their own projects and write their
|
||||
own code. Or maybe we have some degree of co-operation in actually writing the
|
||||
code. Normal life. But the idea is that all geeks working on "cave inventory"
|
||||
and systems making extensive use of cave inventories try to adhere to this
|
||||
framework as much as possible. So that we can then exchange our tools.
|
||||
|
||||
I think things like "which revision system do we use" or "do we use web or
|
||||
Python" are really secondary. Everyone has their own views, habits,
|
||||
backgrounds.
|
||||
|
||||
My idea is to work on this in a small group (no more than a few persons) - to
|
||||
get things going fast, even if they are not perfect from the beginning. If it
|
||||
works, we try to convince others to use it and maybe push it through UIS.
|
||||
</pre>
|
||||
|
||||
<h3>Wookey's other notes on things to do</h3>
|
||||
from <a href="http://wookware.org/software/cavearchive/troggle2design">wookware.org/software/cavearchive/troggle2design</a>
|
||||
<pre>
|
||||
forms
|
||||
-----
|
||||
1) members read/write folk.csv and year/members
|
||||
2) cave read/write cave_data, entrance_data, surveys/pics
|
||||
3) trips -> logbook , QMs, or surveys (more than one survey or location possible)
|
||||
4) logbook reads/write year/logbook
|
||||
5) survey
|
||||
6) prospecting app
|
||||
|
||||
forms show who is logged in.
|
||||
|
||||
databases
|
||||
---------
|
||||
trips, read from
|
||||
logbook entry
|
||||
folder year#index
|
||||
.svx files
|
||||
description
|
||||
QMs
|
||||
|
||||
members (cache from form)
|
||||
|
||||
caves
|
||||
caves_data
|
||||
entrance_data
|
||||
|
||||
storage:
|
||||
expoweb
|
||||
data/
|
||||
cave_entrances
|
||||
caves
|
||||
descriptions
|
||||
|
||||
loser
|
||||
foo.svx
|
||||
</pre>
|
||||
|
||||
|
||||
<h3>Yet more of Wookey's notes</h3>
|
||||
from <a href="http://wookware.org/software/cavearchive/expoweb-design">wookware.org/software/cavearchive/expoweb-design</a>
|
||||
<pre>
|
||||
frontpage
|
||||
---------
|
||||
quick to load:
|
||||
Links:
|
||||
Caves number, name, location
|
||||
Years
|
||||
Handbook
|
||||
Data Entry
|
||||
Main Index
|
||||
|
||||
Slippy map:
|
||||
Indexes to cave page
|
||||
|
||||
Cave page:
|
||||
Access, description, photos, QMs, Survey
|
||||
|
||||
Years:
|
||||
Logbooks/surveynotes/survexdata/people matrix
|
||||
Documents
|
||||
|
||||
Data Entry:
|
||||
Logbook entry
|
||||
Survey data
|
||||
Survey Notes
|
||||
Cave description
|
||||
QMs
|
||||
Photos
|
||||
New cave
|
||||
|
||||
Backend datafiles:
|
||||
caves/
|
||||
cave_entrance
|
||||
cave_data
|
||||
directory of info
|
||||
|
||||
years/
|
||||
year/
|
||||
logbook
|
||||
pubs/
|
||||
reports
|
||||
admin/
|
||||
lists
|
||||
who_and_when
|
||||
travel
|
||||
jobs
|
||||
|
||||
surveyscans/
|
||||
year/
|
||||
index
|
||||
#num
|
||||
handbook/
|
||||
(all static info)
|
||||
|
||||
Storage:
|
||||
non-html or > 280K go in 'files' (PDF, PNG, JPEG, DOC, ODF, SVG)
|
||||
convert small 1024x768 version into website by default. (matching structure?
|
||||
</pre>
|
||||
|
||||
<hr />
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<h3><a id="arch">Archived updates</a></h3>
|
||||
<p>Since 2008 we have been keeping detailed records of all data management system updates in the version control system.
|
||||
Before then we manually maintained <a href="../computing/update.html">a list of updates</a> which are now only of historical interest.
|
||||
<p>A history of the expo website and software was published in Cambridge Underground 1996. A copy of this article <a href="c21bs.html">Taking Expo Bullshit into the 21st Century</a> is archived here.
|
||||
<p>A history of the expo website and software was published in Cambridge Underground 1996. A copy of this article <a href="../c21bs.html">Taking Expo Bullshit into the 21st Century</a> is archived here.
|
||||
<hr />
|
||||
Go on to: <a href="trogarch.html">Troggle architecture</a><br />
|
||||
Dubiously explore: <a href="archnotes.html">Historic ideas for cave data management</a><br />
|
||||
Return to: <a href="trogdesign.html">Troggle design notes</a><br />
|
||||
<hr />
|
||||
</body>
|
||||
|
Loading…
Reference in New Issue
Block a user