expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2025-12-08 14:54:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge lots of stuff from server

Browse Source
This commit is contained in:
Wookey
2020-03-28 04:25:02 +00:00
parent 0c80b5fa9e e54850ad65
commit 25af122e7e
155 changed files with 443 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
documents/troggle-tables.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 163 KiB

0
photos/jenny.jpg → folk/i/jenny.jpg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

84
handbook/computing/x-todo-styles.css Normal file
View File

@@ -0,0 +1,84 @@
button {
color: #ffffff;
font: 18px Georgia, "Times New Roman", Times, serif;
letter-spacing: 1px;
text-shadow: 0 1px 1px #000000;
#display: block;
float:right;
margin: auto;
font-size: 1.2em;
line-height: 1.25em;
padding: 7px 25px;
cursor: pointer;
position: static;
background: #bf3700;
border: 1px solid #60AABF;
-moz-border-radius: 20px;
-webkit-border-radius: 20px;
-khtml-border-radius: 20px;
border-radius: 20px;
/*
-webkit-box-shadow: rgba(0, 0, 0, .25) 0 1px 1px;
-moz-box-shadow: rgba(0, 0, 0, .25) 0 1px 1px;
-o-box-shadow: rgba(0, 0, 0, .25) 0 1px 1px;
box-shadow: rgba(0, 0, 0, .25) 0 1px 1px;
*/
background: #39b2e5;
background: -webkit-gradient(linear, left top, left bottom, from(#39b2e5), to(#6083BF));
background: -moz-linear-gradient(top, #39b2e5, #6083BF);
background: -o-linear-gradient(top, #39b2e5, #6083BF);
background: linear-gradient(top, #39b2e5, #6083BF);
}
button:hover,
button:focus {
cursor: pointer;
border-color: blue;
-webkit-box-shadow: aqua 0 0 8px;
-moz-box-shadow: aqua 0 0 8px;
-o-box-shadow: aqua 0 0 8px;
box-shadow: aqua 0 0 8px;
}
button:active {
background: #39b2e5;
background: -webkit-gradient(linear, left bottom, left top, from(#39b2e5), to(#6083BF));
background: -moz-linear-gradient(bottom, #39b2e5, #6083BF);
background: -o-linear-gradient(bottom, #39b2e5, #6083BF);
background: linear-gradient(bottom, #39b2e5, #6083BF);
}
h2, h3, h4 {
color: #39b2e5;
margin-bottom: 0;
}
ul:not(#links) li ul, ul:not(#links) ul {
font: 18px Calibri, sanserif;
font-weight: normal;
list-style-type: disc;
line-height: 1.3;
}
/* Default State */
ul:not(#links) {
font: 20px Calibri, sanserif;
font-weight: bold;
color: 6083BF;
margin-top: 0.4em;
margin-left: -1.2em;
margin-right: 5em;
list-style-type: none;
line-height: 1.4;
}
/* Toggled State */
input[type=checkbox] ~ ul:not(#links) li ul {
display: none;
}
input[type=checkbox]:checked ~ ul:not(#links) li ul {
display: block;
}

120
handbook/computing/x-todo.html Normal file
View File

@@ -0,0 +1,120 @@
<html>
<head>
<title>Expo Systems Exptl. To-Do List</title>
<meta charset="utf-8">
</head>
<body>
<link rel="stylesheet" type="text/css" href="x-todo-styles.css">
<h1>Expo Systems Exptl. To-Do List</h1>
<button onclick="window.location.href = 'x-todo.html_edit';">Update this to-do list</button>
<label for="toggle-1">Toggle sub-items</label>
<input type="checkbox" id="toggle-1">
<h2>Documentation</h2>
<ul>
<li><!--2020-03-26 psargent-->Wallets & new-cave
<ul>
<li><!--2020-03-26 psargent-->Explain (in the wallets process) how to view the surveys
online so that <br>(1) you can see an example of how to do your new one,
and <br>(2) so that you can check that you have done it properly when you
have finished.</li>
<li><!--2020-03-26 psargent-->Explain how to link a new cave into the other caves in
troggle filesystem</li>
</ul>
</li>
<li><!--2020-03-26 psargent-->Troggle architecture
<ul>
<li><!--2020-03-26 psargent-->Handbook: explain how troggle works - much better and simpler. Fix
mistakes I wrote. For future system maintainers</li>
<li><!--2020-03-26 psargent-->Copy the .wiki material into the documentation somewhere</li>
</ul>
</li>
<li><!--2020-03-26 psargent-->Surveyscans
<ul>
<li><!--2020-03-26 psargent-->explain where the image files are referenced from: svx, tunnel/therion, and why there are so many broken links</li>
</ul>
</li>
</ul>
<h2>Photos</h2>
<ul>
<li><!--2020-03-26 psargent-->Capitalisation in filenames problem
<ul>
<li><!--2020-03-26 psargent-->run the duplicate filename script on the server in
/photos/xml/ as there are files which differ only in
capitalisation of their names.</li>
<li><!--2020-03-26 psargent-->Fix such duplicates in the /photos/ folders where this
problem originates</li>
<li><!--2020-03-26 psargent-->Document this issue of duplicate photo filenames produced by
multiple cameras running similar software</li>
<li><!--2020-03-26 psargent-->somehow fix the photo filenames which use capital letters on
import or upload, or by a cron job</li>
</ul>
</li>
</ul>
<h2>Surveys data entry</h2>
<ul>
<li><!--2020-03-26 psargent-->2019 surveyscans
<ul>
<li><!--2020-03-26 psargent-->edit all the content.json</li>
<li><!--2020-03-26 psargent-->try to autocomplete the content.json by parsing .svx files</li>
<li><!--2020-03-26 psargent-->Put personal names into canonical form</li>
</ul>
</li>
</ul>
<h2>Troggle</h2>
<h2>Expo Server</h2>
<ul>
<li><!--2020-03-26 psargent--> www-data:www-data permissions
<ul>
<li><!--2020-03-26 psargent-->Fix permissions so that all files served by troggle have
www-data:www-data ownership and group ownership, otherwise the
"Edit this page" fails with a permissions error, or
</li>
<li><!--2020-03-26 psargent-->Or fix group
membership for the process running django so that it can write
files owned by 'expo'
</li>
</ul>
</li>
</ul>
<h2>
<font color="darkviolet">This to-do-list gadget itself</font>
</h2>
<ul>
<li><!--2020-03-26 psargent-->Eventual aim
<ul>
<li><!--2020-03-26 psargent-->to replace GitHub <a href="https://github.com/CaveSurveying/CUCCexposurveyissues/issues">
CUCCexposurveyissues</a> for non-tunnel, non-GIS issues.</li>
<li><!--2020-03-26 psargent-->We need two of these: <br>1.for expo systems work (troggle & scripts) - the current priority as we have nowhere currently that Wookey is willing to use
<br>2.for survey data progress/issues (though troggle itself can be used for some of this if cleaned up a bit)
</ul>
</li>
<li><!--2020-03-26 psargent-->Try to automate simple functions in javascript
<ul>
<li><!--2020-03-26 psargent-->to control this document
in-place (tick, delete) instead of using the <a
href="https://www.tiny.cloud/docs/general-configuration-guide/use-tinymce-inline/">
tinymce</a> HTML editor in Django</li>
<li><!--2020-03-26 psargent-->Maybe restructure whole page as JSON and use
javascript editor to create (edit), re-order and demote items. So long
as the content is inserted into the DOM, it should work ?</li>
<li><!--2020-03-26 psargent-->Implement comments on an issue, mimicking GitHub issues wiki</li>
<li><!--2020-03-26 psargent-->Use javascript to skip over the _edit page and submit changes
direct to server when updating</li>
<li><!--2020-03-26 psargent-->checkboxes so that items can be ticked
(which also does strike-through)</li>
<li><!--2020-03-26 psargent-->bin icon to delete an item? Better to have an archive process.</li><!--2020-03-26-->
</ul>
</li>
</ul>
<hr>
<!--2020-03-26--> Testing area..
<div id="demo">Demonstation as initially read from disc</div>
<script>
document.getElementById("demo").innerHTML = "Demonstation innerHTML change CONFIRMED";
</script>
<!--Bother. The DOM is not carried across to the _edit page. It looks like the _edit page just re-reads
the original file from disc. So storing changes done by js will mean working directly with the POST action and a form. -->
</body>
</html>

10
handbook/survey/README Normal file
View File

@@ -0,0 +1,10 @@
Add new folders of the form
2017#01 (where 01 is a serial number) for surveys with physical notes
or
2017#X01 (where 01 is a serial number) for surveys with only digital data
<html><body>
Dummy text to stop mistaken messages about broken links
from the linkchecker at
<a href="https://www.brokenlinkcheck.com/broken-links.php">brokenlinkcheck.com</a>
</body></html>

89
handbook/survey/how-to-do-it Normal file
View File

@@ -0,0 +1,89 @@
# Instructions
# 2019-03-02
# Philip Sargent
Wookey told me to sort out the contents.json files in expofiles/surveyscans/2018/
and these are my notes to remind myself what this entails.
This note was written during August 2018.
The job is to populate the contents.json file in each folder, e.g.
expofiles/surveyscans/2018/2018#03/contents.json
using the following input materials:
- the wallet 2018#03 and the papers inside it. This is in the 2018 lever-arch file.
- the folder in repo 'loser' holding the appropriate .svx files e.g.
"caves-1623/2017-cucc-24/gshclimb.svx"
- the script expofiles/surveyscans/2018/wallets.py (run by "python wallets.py")
the "wallets.py" script creates index.html files in each folder /2018/2018#nn/
and creates or updates a webpage for each person listed in any of the contents.json files
in the folder/2018/ e.g. "Becka Lawson.html".
The script wallets.py requires that the //loser// repo is populated on the machine that you
run the script on so that it can find the.svx files. Do not run this script on
the expo server itself as the loser repo is in the wrong place and all the .svx files
will be flagged as "not found" (Pauul Fox did this once).
wallets.py now (Feb.2019) takes a command-line argument for the location of the loser repo, e.g.
python wallets.py "/mnt/d/CUCC-Expo/loser/"
Before doing anything else, run wallets.py. This will create empty template contents.json
files in each folder.
You may need to create missing folders,e.g. I just had to create /2018/2018#30 to #32.
Every time you finish entering the data in contents.json in a folder,
run wallets.py to update the "person" html files and to
re-generate the index.html file for the 2018 folder as
a whole (surveyscans/2018/index.html).
But wallets.py does not delete old files which are not regenerated.
So it is a good idea to delete all the .html files
before running it.
The first difficulty is finding out which cave a wallet describes..
The wallet may say "radaghost to blitzkriek"
(or whatever) but without the name of the cave you can't find the .svx files
as you don't know that you need to look in e.g. loser/caves-1626/2018-dm-07/
Usually this is written by hand on the label of the wallet.
There are ambiguities about how the entries in the contents.json actually lead to
reminder instructions in the html files produced, and this is particularly
difficult for electronic caves where the topo files are missing
and for surface prospecting where it is not clear which of the actions
should be done and thus which products should be produced.
This needs to be documented.
For prospecting and surface surveying it is not clear whether the default folder
for the url link should be repo //loser// surface/1623/allplateau.svx
When there are more than one .svx file there seems to be no way of recording the list
in contents.json so it is impossible to tell what was done on that trip or whether
there is anything missing. This is especially true if it was electronic and the
.topo files are missing. Wookey confirms that this ios the case.
wallets.py will not overwrite any contents.json file, it only creates one where one
doesn't exist.
wallets.py also creates an index.html file inside each wallet folder dewcribing the status of this
folder. This is linked from the index.html file created in /2018/ by the script which
summarises everything for this year.
NAMING
The script detects if there are notesX.jpg planX.jpg and elevX.jpgfiles present, and
produces a reminder/warning if they are not,even if these have all been scanned
and given different names.
The job of the checker (perhaps on a second pass) if to rename files so that these
warnings disappear.
HINT
When there are a lot on wallets all with the same cave, make your own template
with the cave name and the right folder prefix for the svx folder
(in the loser repo) and copy it in to all those wallet folders - overwriting
the blank template produced by the wallets.py

82
handbook/survey/onlinewallet.html
View File

@@ -17,7 +17,7 @@
</ol>
<h3>The scanned pages</h3>
<p>These are simply the scanned imaages (or digital photographs) of each page of the original survey notes.
<p>These are simply the scanned images (or digital photographs) of each page of the original survey notes.
They should be named <em><span style="font-family:monospace">notesXXX.jpg</span></em> where "XXX" can be
anything you like. Typically we have the scanned pages called notes1.jpg, notes2.jpg, notes3.jpg.
<p>It is important that you use use the .jpg (JPEG) file format, and definitely not PNG (very voluminous)
@@ -25,7 +25,7 @@ or PDF (very hard to re-use elsewhere). Set the scanner at 300 dpi and adjust th
by using photo-editing software to enhance the writing. Also please crop each image to just the area containing
the survey data.
<p>As soon as the notes have been scanned you should (a) copy them to a USB stick or email them to someone, (b) upload the entire online wallet to the expo server in Cambridge
<span style="font-family:monospace">expo.survex.com</span>. This is so that these precious files are backed-up as soon as possible.
<span style="font-family:monospace">expo.survex.com</span>. This is so that these precious files are backed-up as soon as possible. (Assuming you have done the <a href="../computing/keyexchange.html">key-pair procedure</a>, do the upload by <a href="../uploading.html">sFTP</a> as you do for photos, but to the directory described in the <a href="newwallet.html">introduction to online wallets</a>.)
<h3>The other files and online index <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span></h3>
<p>All the other files are part of the multi-step process of producing the cave survey - see <a href="newcave.html">
@@ -51,7 +51,7 @@ have a different Austrian Kataster number issued for it in due course).
*ref 2018#06
; the #number is on the clear pocket containing the original notes
</pre>
But sometime in mid-Expo 2015 everyone stopped using the survex template file and so this information was not recorded since then. This will be fixed by hand-editing indue course.
But sometime in mid-Expo 2015 everyone stopped using the survex template file and so this information was not recorded since then. This will be fixed by hand-editing in due course.
(Note that many old .svx files were processed with an older version of survex which did not suppport this feature and so a comment was used instead.)
<p>Troggle produces very useful auto-generated reports of the status of the wallets and the survex files
@@ -127,17 +127,21 @@ in filenames on Windows computers.
</pre>
will, after the appropriate magic has happened, contain a file
<p>
<span style="font-family:monospace; size=x-small; background-color: lightgray">index.html</span>
<span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span>
<p>
which lists all the wallets which have uncompleted tasks, and lists all the people responsible for completing them.
You can see <a href="http://expo.survex.com/expofiles/surveyscans/2018/">the index.html for 2018</a> . Also there will be a linked file for each individual
You can see <a href="http://expo.survex.com/expofiles/surveyscans/2018/walletindex.html">the walletindex.html for 2018</a> . Also there will be a linked file for each individual
for their personal to-do list, and each online wallet contains its own
<span style="font-family:monospace">index.html</span> file which describes the survey production status for all the wallets.
<p>The magic creates index.html files in each folder /2018/2018#nn/
<p>The magic also creates walletindex.html files in each folder /2018/2018#nn/
and creates or updates a webpage for each person listed in any of the contents.json files
in the folder/2018/ e.g. <a href="http://expo.survex.com/expofiles/surveyscans/2018/Becka%20Lawson.html">Becka Lawson.html</a>.
<p>If you want to see what the files are in each online wallet then look at <br>
<a href="http://expo.survex.com/expofiles/surveyscans/2018/">expo.survex.com/expofiles/surveyscans/2018/</a> <br>
and you can navigate to the individual wallet folders too and look directly at the files there.
<p>All this magic is created by a script <span style="font-family:monospace; size=x-small; background-color: lightgray">wallets.py</span>.
<h3>Setting up the online wallets</h3>
@@ -153,8 +157,8 @@ new trip surveys.
<span style="font-family:monospace; size=x-small; background-color: lightgray">python wallets.py</span>
<br>
<p>This will create a default <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> and
<span style="font-family:monospace; size=x-small; background-color: lightgray">index.html</span> in each online wallet subfolder and also a
<span style="font-family:monospace; size=x-small; background-color: lightgray">index.html</span> in the <span style="font-family:monospace; size=x-small; background-color: lightgray">/2018/</span> folder.
<span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span> in each online wallet subfolder and also a
<span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span> in the <span style="font-family:monospace; size=x-small; background-color: lightgray">/2018/</span> folder.
<p>This script works fine on Linux (Debian, Xubuntu, etc.) and also now works fine in the <a href="https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/">Windows 10 bash system</a>.
@@ -177,7 +181,7 @@ You will run
<span style="font-family:monospace; size=x-small; background-color: lightgray">python wallets.py</span>
<p>
regularly, after every batch of survey data is entered or scanned.
<p>This will always overwrite all the <span style="font-family:monospace">index.html</span> files but it will never touch
<p>This will always overwrite all the <span style="font-family:monospace">walletindex.html</span> files but it will never touch
the <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> files.
<p>You will also regularly synchronise your laptop
and the expo laptop with <br />
@@ -206,7 +210,7 @@ As all this is not under version control the timestamps of the files are really
<p>
So script <span style="font-family:monospace; size=x-small; background-color: lightgray">wallets.py</span> has been fixed so that
<ul>
<li>the generated <span style="font-family:monospace; size=x-small; background-color: lightgray">index.html</span> file in each wallet folder is given the same timestamp as the <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> file there, <em>not the time of when the script is run</em>. This is unusual but intentional and in practice very helpful.
<li>the generated <span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span> file in each wallet folder is given the same timestamp as the <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> file there, <em>not the time of when the script is run</em>. This is unusual but intentional and in practice very helpful.
<li>the script no longer overwrites the <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> files every time it runs. It only changes that file's timestamp if it actually changes anything in the contents.json file.
</ul>
@@ -221,9 +225,11 @@ different formats. It also
<ul>
<li>checks whether the .svx files listed are actually present <br>in the <a href="http://expo.survex.com/repositories/home/expo">::loser::</a> repository
<li>checks for the presence of notesXXX.jpg, planXXX.jpg and elevXXX.jpg files
<li>checks for the presence of XXXnotes.jpg, XXXplan.jpg and XXXelev.jpg files
<li>creates a template <span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span> in any wallet which does not have one.
<li>creates helpful URL links to the existing online survey documentation for the cave being surveyed
<li>creates helpful URL links to the working files you are managing on your own laptop
<li>Inserts the date & time into each generated file so that anyone can see when it was last updated
</ul>
<p>Things it might do in future (if someone gets around to it) include: <br>
@@ -237,60 +243,14 @@ different formats. It also
- checking the date is in the recent past etc.<br><br>
<h3>How <em><span style="font-family:monospace; size=x-small; background-color: lightgray">contents.json</span></em> fields match
<em><span style="font-family:monospace; size=x-small; background-color: lightgray">index.html</span></em> reports</h3>
<em><span style="font-family:monospace; size=x-small; background-color: lightgray">walletindex.html</span></em> reports</h3>
<p>
<em>to be written...</em>
<hr>
Old notes, being turned into real documentation...
<pre>
# Instructions
# 2018-08-14
# Philip Sargent
Wookey told me to sort out the contents.json files in expofiles/surveyscans/2018/
and these are my notes to remind myself what this entails.
The job is to populate the contents.json file in each folder, e.g.
expofiles/surveyscans/2018/2018#03/contents.json
using the following input materials:
- the wallet 2018#03 and the papers inside it. This is in the 2018 lever-arch file.
- the folder in repo 'loser' holding the appropriate .svx files e.g.
"caves-1623/2017-cucc-24/gshclimb.svx"
- the script expofiles/surveyscans/2018/wallets.py (run by "python wallets.py")
the "wallets.py" script creates index.html files in each folder /2018/2018#nn/
and creates or updates a webpage for each person listed in any of the contents.json files
in the folder/2018/.
The script wallets.py requires that the //loser// repo is populated on the machine that you
run the script on so that it can find the.svx files.
If your machine has the ::loser:: repo in a different place from that expected by the script, you can just
put the path on the command line:
python wallets.py "/mnt/d/CUCC-Expo/loser/"
Before doing anything else, run wallets.py. This will create empty template contents.json
files in each folder.
You may need to create missing folders,e.g. I just had to create /2018/2018#30 to #32.
Every time you finish entering the data in contents.json in a folder,
run wallets.py to update the "person" html files and to
re-generate the index.html file for the 2018 folder as
a whole (surveyscans/2018/index.html).
There are ambiguities about how the entries in the contents.json actually lead to
reminder instructions in the html files produced, and this is particularly
difficult for electronic caves where the topo files are missing
and for surface prospecting where it is not clear which of the actions
should be done and thus which products should be produced.
This needs to be documented.
For prospecting and surface surveying it is not clear whether the default folder
for the url link should be repo ::loser:: surface/1623/allplateau.svx
@@ -309,14 +269,14 @@ the blank template produced by the wallets.py
# Update March 2019
a consolidated to-do list of the last 3 years on the server:
http://expo.survex.com/expofiles/surveyscans/2016-18/index.html
http://expo.survex.com/expofiles/surveyscans/2016-18/walletindex.html
This is a hand-done kludge and only the first level of links works - which is to the individual person's page.
the lists for the last 3 years individually and all the links are working for each wallet page:
both local links to your PC and to the right location of the .svx files on the troggle server.
http://expo.survex.com/expofiles/surveyscans/2016/index.html
http://expo.survex.com/expofiles/surveyscans/2017/index.html
http://expo.survex.com/expofiles/surveyscans/2018/index.html
http://expo.survex.com/expofiles/surveyscans/2016/walletindex.html
http://expo.survex.com/expofiles/surveyscans/2017/walletindex.html
http://expo.survex.com/expofiles/surveyscans/2018/walletindex.html
and all the names of people have been hand-edited in the .json files to be consistent and identical.
2015 has now been done stand-alone but there is no consolidated report for 2015-18 yet.

61
noinfo/wallets.py Normal file → Executable file
View File

@@ -1,43 +1,55 @@
#!/usr/bin/env python
import sys, os, operator, urllib, json, re, time
from datetime import datetime
# 2017 originally by Martin Green
# 2018-08-27 edited Philip Sargent
# 2019-03-02 extended to take command line argument of loser_dir and set mod time of index.html to be sane as json file
# 2019-03-02 extended to take command line argument of loser_dir and set mod time of index.html to be same as json file
# 2019-12-17 extra output of links to troggle-generated trip data
# 2019-12-31 bits to make website link-checker not barf so much. Added endswith() to .startswith() for notes, elev, plan filenames
# 2020-01-21 Now we are using Windows10-WSL1, +links to expedition logbook on every generated page
# 2020-03-14 Adding timestamp tovisible outputs
loser_dir = "/home/expo/loser/"
#loser_dir = "/mnt/d/CUCC-Expo/loser/" # when running on Win10/bash
loser_dir = "/home/expo/loser"
#loser_dir = "/mnt/d/CUCC-Expo/loser/" # when running on Win10-WSL1
#loser_dir = "/media/philip/SD-huge/CUCC-Expo/loser/" # when running on xubuntu laptop 'barbie'
if len(sys.argv) > 1 :
if sys.argv[1] != "":
loser_dir = sys.argv[1]
print "Loser repo is assumed to be in: " + loser_dir
dateTimeObj=datetime.now(tz=None)
timestamp = dateTimeObj.strftime("%d-%b-%Y (%H:%M)")
print "Loser repo (for svx files) is assumed to be in: " + loser_dir + "/"
drawings_dir = loser_dir[0:len(loser_dir)-5] + "drawings"
print "Drawings repo (for drawings files) is assumed to be in: " + drawings_dir + "/"
html_base = "<html><body>%(body)s</body></html>"
html_year_index = html_base % {"body": "<H1>%(year)s surveys: wallets status</H1>\n<p>List of trips: <a href=\"http://expo.survex.com/expedition/%(year)s\">expedition/%(year)s</a> - troggle-processed .svx files and logbook entries on server</p>\n<H2>Persons</H2>\n<UL>\n%(persons)s</UL>\n<H2>Wallets</H2>\n<table>%(wallets)s</table>\n<H2>Needing Scanning</H2>\n<UL>\n%(needing scanning)s</ul>\n<H2>Website (Guidebook description) needing updating\n</H2>\n<UL>\n%(website needing updating)s</ul>\n"}
html_year_index = html_base % {"body": "<H1>%(year)s surveys: wallets status</H1>\n<p>List of trips: <a href=\"http://expo.survex.com/expedition/%(year)s\">expedition/%(year)s</a> - troggle-processed .svx files and logbook entries on server</p>\nAs of %(timestamp)s\n<H2>Persons</H2>\n<UL>\n%(persons)s</UL>\n<H2>Wallets</H2>\n<table>%(wallets)s</table>\n<H2>Needing Scanning</H2>\n<UL>\n%(needing scanning)s</ul>\n<H2>Website (Guidebook description) needing updating\n</H2>\n<UL>\n%(website needing updating)s</ul>\n"}
html_year_person = "<li><a href='%(person)s.html'>%(person)s</a><UL>\n%(complaints)s</ul></li>\n"
html_year_wallet_entry = "<tr><td><a href='%(walletindex)s'>%(walletname)s %(cave)s %(name)s</a></td> <td>%(complaints)s</td></tr>\n"
html_person_wallet_entry = "<li><a href='%(walletindex)s'>%(walletname)s</a> <UL>\n%(complaints)s</ul></li>\n"
html_year_scanning_entry = "<li><a href='%(walletindex)s'>%(walletname)s %(cave)s %(name)s</a></li>\n"
html_wallet_file_entry = "<li><a href='%(fileurl)s'>%(filename)s</a></li>\n"
html_wallet_index = html_base % {"body": "<H1>%(title)s : %(cave)s : %(name)s</H1>\n<p>Date: %(date)s</p><p>People: %(people)s</p>\n<p>Cave <a href='%(description)s'>Guidebook description</a> - %(description_needed)s \n<p>Survex file:<br>&nbsp;&nbsp;Local (Windows): <a href='%(loser_dirw)s%(survex)s' download>%(loser_dirw)s%(survex)s</a><br>&nbsp;&nbsp;Local (Linux): <a href='%(loser_dir)s%(survex)s' download>%(loser_dir)s%(survex)s</a><br>&nbsp;&nbsp;Server: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://expo.survex.com/survexfile/%(survex)s' download>%(survex)s</a></p><a href='../index.html'>Wallet index for this year</a> </p>\n<H2>Issues</H2>\n%(complaints)s\n<H2>Files</H2>\n<UL>\n%(files)s</UL>\n"}
html_wallet_index = html_base % {"body": "<H1>%(title)s : %(cave)s : %(name)s</H1>\n<p>List of trips: <a href=\"http://expo.survex.com/expedition/%(year)s\">expedition/%(year)s</a> - troggle-processed .svx files and logbook entries on server</p>\n<p>Date: %(date)s</p><p>People: %(people)s</p>\n<p>Cave <a href='%(description)s'>Guidebook description</a> - %(description_needed)s \n<p>Survex file:<br>&nbsp;&nbsp;<br>&nbsp;&nbsp;&nbsp;Local: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href='file:///%(loser_dir)s/%(survex)s' download>file:///%(loser_dir)s/%(survex)s</a><br>&nbsp;Server: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://expo.survex.com/survexfile/%(survex)s' download>http://expo.survex.com/survexfile/%(survex)s</a></p><a href='../index.html'>Wallet index for this year</a><br/>Local location for ::loser:: repo specified on command line is <a href='file:///%(loser_dir)s'>%(loser_dir)s</a>. </p>\n<H2>Issues</H2>\n%(complaints)s\n<H2>Files</H2>\n<UL>\n%(files)s</UL>\n"}
html_survex_required = {True: "Survex ", False: ""}
html_plan_scanned = {True: "", False: "Plan "}
html_elev_scanned = {True: "", False: "Elev "}
html_description_written = {True: "", False: "Desc "}
html_qms_written = {True: "", False: "QMs "}
html_status = {True: "Issues: ", False: ""}
html_person = html_base % {"body": "<H1>%(person)s</H1><H2>Outstanding Wallets</H2><UL>\n%(wallets)s</UL>"}
html_person = html_base % {"body": "<H1>%(person)s</H1>\n<p>List of trips: <a href=\"http://expo.survex.com/expedition/%(year)s\">expedition/%(year)s</a> - troggle-processed .svx files and logbook entries on server</p>\n<H2>Outstanding Wallets</H2>\nAs of %(timestamp)s\n<UL>\n%(wallets)s</UL>"}
html_complaint_items = "<li>%(count)i %(complaint)s</li>"
html_items = "<li>%s</li>"
blank_json = {
"cave": "",
"date": "",
"description url": "",
"description url": "/caves",
"description written": False,
"electronic survey": False,
"elev drawn": False,
@@ -53,6 +65,7 @@ blank_json = {
"survex not required": False,
"website updated": False}
#need to use wallets as a dict/tuple (id,cave,name) - not sure how.
wallets = []
wallets_needing_scanning = set()
@@ -106,7 +119,8 @@ for item in sorted(os.listdir(".")):
print "Writing file %s" % (contents_path)
json_file = open(contents_path, "w")
json.dump(data, json_file, indent = 1)
json_file.close()
json_file.close()
# Get modification time of contents.json
# print("json last modified: %s" % time.ctime(os.path.getmtime(contents_path)))
json_mtime = os.path.getmtime(contents_path)
@@ -131,36 +145,39 @@ for item in sorted(os.listdir(".")):
#Notes
notes_scanned = reduce(operator.or_, [f.startswith("note") for f in files], False)
notes_scanned = reduce(operator.or_, [f.endswith("note") for f in files], notes_scanned)
if not notes_scanned:
complaints.append("The notes needs scanning (no noteN.jpg file found)")
complaints.append("The notes needs scanning (no noteNN.jpg or XXnote.jpg file found)")
wallets_needing_scanning.add(item)
#Plan drawing required
plan_scanned = reduce(operator.or_, [f.startswith("plan") for f in files], False)
plan_scanned = reduce(operator.or_, [f.endswith("plan") for f in files], plan_scanned)
plan_drawing_required = not (plan_scanned or data["plan drawn"])
if plan_drawing_required:
complaints.append("The plan needs drawing (no planN.jpg file found)")
person_complaints.append(" plan(s) needs drawing (no planN.jpg file found)")
complaints.append("The plan needs drawing (no planNN.jpg or XXplan.jpg file found)")
person_complaints.append(" plan(s) needs drawing (no planNN.jpg or XXplan.jpg file found)")
if not plan_drawing_required and not plan_scanned:
complaints.append("The plan needs <em>scanning</em> (no planN.jpg file found)")
complaints.append("The plan needs <em>scanning</em> (no planNN.jpg or XXplan.jpg file found)")
wallets_needing_scanning.add(item)
#Elev drawing required
elev_scanned = reduce(operator.or_, [f.startswith("elev") for f in files], False)
elev_scanned = reduce(operator.or_, [f.endswith("elev") for f in files], elev_scanned)
elev_drawing_required = not (elev_scanned or data["elev drawn"])
if elev_drawing_required:
complaints.append("The elev needs drawing (no elevN.jpg file found)")
person_complaints.append(" elev(s) needs drawing (no elevN.jpg file found)")
complaints.append("The elev needs drawing (no elevNN.jpg or XXelev.jpg file found)")
person_complaints.append(" elev(s) needs drawing (no elevNN.jpg or XXelev.jpg file found)")
if not elev_drawing_required and not elev_scanned:
complaints.append("The elev needs <em>scanning</em> (no elevN.jpg file found)")
complaints.append("The elev needs <em>scanning</em> (no elevNN.jpg or XXelev.jpg file found)")
wallets_needing_scanning.add(item)
#Description
if not data["description written"]:
complaints.append("The description needs writing")
person_complaints.append(" description(s) needs writing")
description_needed = "Not present, needs doing."
description_needed = "A description is indicated as being needed, so may need adding into this cave page."
#QMS
@@ -170,12 +187,12 @@ for item in sorted(os.listdir(".")):
#Website
if not data["website updated"]:
complaints.append("The guidebook description on the website needs updating")
complaints.append("The website is marked as needing updating (using the guidebook description)")
website_needing_updating.add(item)
#Electronic Surveys
if not data["electronic survey"]:
complaints.append("Tunnel / Therion files need drawing")
complaints.append("Tunnel / Therion drawing files need drawing")
if data["survex file"]:
survex_description = data["survex file"]
@@ -183,7 +200,7 @@ for item in sorted(os.listdir(".")):
survex_description = "Not specified"
wallet_index_file = open(os.path.join(item, "index.html"), "w")
wallet_index_file.write(html_wallet_index % {"title": item,
wallet_index_file.write(html_wallet_index % {"title": item, "year": year,
"cave": data["cave"],
"name": data["name"],
"date": data["date"],
@@ -237,7 +254,7 @@ for person, person_wallets in people.items():
person_summary = dict(person_summary)
year_index_file = open("index.html", "w")
year_index_file.write(html_year_index % {"year": year, "persons": reduce(operator.add, [html_year_person % {"person": person,
year_index_file.write(html_year_index % {"year": year, "timestamp": timestamp, "persons": reduce(operator.add, [html_year_person % {"person": person,
"complaints": reduce(operator.add,
[html_complaint_items % {"complaint": complaint,
"count": count}
@@ -270,7 +287,7 @@ year_index_file.close()
for person, item_complaint_list in people.items():
person_file = open(person + ".html", "w")
person_file.write(html_person % {"person": person,
person_file.write(html_person % {"person": person, "year": year, "timestamp": timestamp,
"wallets": reduce(operator.add, [html_person_wallet_entry % {"walletname": wallet,
"walletindex": urllib.quote(wallet) + "/index.html",
"complaints": reduce(operator.add,

BIN
photos/adamc.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 3.9 KiB

BIN
photos/aggy_finn.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 9.1 KiB

BIN
photos/ajday.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 27 KiB

BIN
photos/alexc.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 23 KiB

BIN
photos/alim.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 3.7 KiB

BIN
photos/andy_farrow.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/andya.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 10 KiB

BIN
photos/andyc.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.6 KiB

BIN
photos/andyw.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.3 KiB

BIN
photos/bderby.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 23 KiB

BIN
photos/beckal.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 40 KiB

BIN
photos/benvm.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.1 KiB

BIN
photos/bill_miners.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 25 KiB

BIN
photos/billh.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 9.7 KiB

BIN
photos/bishop.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 12 KiB

BIN
photos/bobb.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.6 KiB

BIN
photos/brian.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 13 KiB

BIN
photos/budge.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 14 KiB

BIN
photos/callum.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 19 KiB

BIN
photos/chrisd.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/chriss.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 19 KiB

BIN
photos/claire_purnell.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/clive_george.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.2 KiB

BIN
photos/dan_mace.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 7.0 KiB

BIN
photos/dave_collins.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 13 KiB

BIN
photos/dave_galvin.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 6.9 KiB

BIN
photos/dave_horsley.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.8 KiB

BIN
photos/daveb.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 7.7 KiB

BIN
photos/davef.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 5.6 KiB

BIN
photos/daveh.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.0 KiB

BIN
photos/davel.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 52 KiB

BIN
photos/del.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 10 KiB

BIN
photos/dolby.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 6.0 KiB

BIN
photos/dougf.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 12 KiB

BIN
photos/earl.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 21 KiB

BIN
photos/emma_wilson.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 41 KiB

BIN
photos/erin.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 13 KiB

BIN
photos/fay.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.4 KiB

BIN
photos/fran.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 9.4 KiB

BIN
photos/francis_turner.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/frankt.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 17 KiB

BIN
photos/gails.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 25 KiB

BIN
photos/gill_lindsey.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 10 KiB

BIN
photos/glen.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.4 KiB

BIN
photos/gnash.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 13 KiB

BIN
photos/haines.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 10 KiB

BIN
photos/henri_welbourne.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.7 KiB

BIN
photos/hilary_greaves.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 16 KiB

BIN
photos/hippo.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.8 KiB

BIN
photos/hughs.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.8 KiB

BIN
photos/ian_harris.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 6.8 KiB

BIN
photos/ianm.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.7 KiB

BIN
photos/jab.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 5.8 KiB

BIN
photos/janetm.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.4 KiB

BIN
photos/jared.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 7.0 KiB

BIN
photos/jennyb.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 33 KiB

BIN
photos/jerry.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 2.9 KiB

BIN
photos/jjr.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 10 KiB

BIN
photos/joel.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 7.8 KiB

BIN
photos/john_cordingley.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 22 KiB

BIN
photos/johnp.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/jont_leach.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 13 KiB

BIN
photos/jonw.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 9.4 KiB

BIN
photos/jtodd.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 9.1 KiB

BIN
photos/judith.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 10 KiB

BIN
photos/juliab.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.8 KiB

BIN
photos/julian_shilton.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 9.4 KiB

BIN
photos/juliette_kelly.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/justin_powell.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 14 KiB

BIN
photos/katej.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 6.5 KiB

BIN
photos/keith.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.7 KiB

BIN
photos/kenb.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/kristian.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 33 KiB

BIN
photos/mark_byers.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 9.2 KiB

BIN
photos/mark_scott.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 12 KiB

BIN
photos/markf.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.4 KiB

BIN
photos/markm.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/markr.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 5.6 KiB

BIN
photos/marks.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 19 KiB

BIN
photos/martin_sluka.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/marting.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 34 KiB

BIN
photos/mattk.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.2 KiB

BIN
photos/mike_baslington.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

BIN
photos/mike_perryman.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 7.8 KiB

BIN
photos/mikem.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 27 KiB

BIN
photos/miketa.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.8 KiB

BIN
photos/msabry.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 6.5 KiB

BIN
photos/msd.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 6.8 KiB

BIN
photos/msd204.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 44 KiB

BIN
photos/mts.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 7.3 KiB

BIN
photos/muff.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 5.5 KiB

BIN
photos/nadia.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 5.1 KiB

Some files were not shown because too many files have changed in this diff Show More