expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2025-05-08 02:45:46 +01:00
Code Issues Packages Projects Releases Wiki Activity

updating wallets scripts (expofiles copy not git)

Browse Source
This commit is contained in:
Philip Sargent 2021-04-24 23:16:44 +01:00
parent 2e6a9a7f50
commit c34d6b2280
5 changed files with 296 additions and 237 deletions

2
handbook/troggle/scriptscurrent.html

@ -33,7 +33,7 @@ p {
<p>It coordinates producing the 3d surveys used in the cave description pages, updates the area pages, runs the folk script, runs the QM list generation within each of the cave pages that needs it, runs svxtrace, and reports on everything using "bigbro" which we don't have any other reference to. (Generation of the .3d files as required is now done by troggle.) <p>It coordinates producing the 3d surveys used in the cave description pages, updates the area pages, runs the folk script, runs the QM list generation within each of the cave pages that needs it, runs svxtrace, and reports on everything using "bigbro" which we don't have any other reference to. (Generation of the .3d files as required is now done by troggle.)
<h4 id="wallets">Wallets</h4> <h4 id="wallets">Wallets</h4>
<p><a href="../survey/onlinewallet.html">Online wallets</a> are initially maintained using the <a href="/expofiles/surveyscans/wallets.py">wallets.py</a> script, but troggle also directly imports all the expofiles/surveyscans/ directories of scanned survey notes and produces <a href="/survey_scans/">reports</a> on then. <p><a href="../survey/onlinewallet.html">Online wallets</a> are initially maintained using the <a href="/expofiles/surveyscans/wallets.py">wallets.py</a> script, but troggle also directly imports all the expofiles/surveyscans/ directories of scanned survey notes and produces <a href="/survey_scans/">reports</a> on them. There are several bash and python scripts in the <a href="/expofiles/surveyscans/">surveyscans</a> directory to create wallets for the coming year, and to re-run the wallet processing on all past years (for when we improve the script). For 2021 we have converted wallets.py to python3, so be careful of older versions which are python2.
<h4 id="folk">Folk</a></h4> <h4 id="folk">Folk</a></h4>

2
handbook/troggle/scriptsother.html

@ -22,7 +22,7 @@
<li><a href="scriptscurrent.html#latex">bierbook.tex</a> LaTeX script for generating the bierbook - a new list of names and dates each year <li><a href="scriptscurrent.html#latex">bierbook.tex</a> LaTeX script for generating the bierbook - a new list of names and dates each year
<li><a href="scriptscurrent.html#latex">seshbook.tex</a> LaTeX script for generating the seshbook - works from the same list of names <li><a href="scriptscurrent.html#latex">seshbook.tex</a> LaTeX script for generating the seshbook - works from the same list of names
<li><a href="scriptscurrent.html#latex">therionpage.tex</a> LaTeX script and makefile for generating therion-style protractors</li><br /> <li><a href="scriptscurrent.html#latex">therionpage.tex</a> LaTeX script and makefile for generating therion-style protractors</li><br />
<li><a href="scriptscurrent.html#latex">wallets.py</a> generates statuspages and to-do list pages for survey data production. <li><a href="scriptscurrent.html#wallets">wallets.py</a> generates statuspages and to-do list pages for survey data production.
<li><a href="scriptsqms.html">svx2qm.py</a> extracts QMs from the survex files (and <a href="scriptsqms.html">find-dead-qms.py</a>) <li><a href="scriptsqms.html">svx2qm.py</a> extracts QMs from the survex files (and <a href="scriptsqms.html">find-dead-qms.py</a>)
<li><a href="scriptsqms.html">tablize-qms.pl</a> turns the list of QMs extracted into an HTML file <li><a href="scriptsqms.html">tablize-qms.pl</a> turns the list of QMs extracted into an HTML file
<li><a href="scriptscurrent.html#surface">make_svx.sh</a> generates surface Survex tracks <li><a href="scriptscurrent.html#surface">make_svx.sh</a> generates surface Survex tracks

2
noinfo/walletscripts/allwallets.sh

@ -1,6 +1,6 @@
#/bin/sh #/bin/sh
for i in 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019; do for i in 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2021; do
#for i in 2016 2017 2018 2019; do #for i in 2016 2017 2018 2019; do
echo $i echo $i
cp -p wallets.py $i cp -p wallets.py $i

11
noinfo/walletscripts/mkdirs.sh Normal file

@ -0,0 +1,11 @@
#!/bin/bash
# Make the first set of directories for the coming year
# run from /surveyscans/ and tell it the year, e.g.
# $ ./mkdirs.sh 2021
if [$1 -eq ""]; then echo -e "mkdirs [year]\nProvide a year as the argument."; exit; fi
for i in {100..135}; do
ds=${i:1:3}
echo mkdir $1/$1"#"$ds
mkdir $1/$1"#"$ds
done

510
noinfo/walletscripts/wallets.py

@ -1,32 +1,52 @@
#!/usr/bin/env python #!/usr/bin/env python
import sys, os, operator, urllib, json, re, time import sys, os, operator, urllib.request, urllib.parse, urllib.error, json, re, time
from datetime import datetime from datetime import datetime
from functools import reduce
from pathlib import Path
# 2017 originally by Martin Green # 2017 originally by Martin Green
# 2018-08-27 edited Philip Sargent # 2018-08-27 edited Philip Sargent
# 2019-03-02 extended to take command line argument of loser_dir and set mod time of index.html to be same as json file # 2019-03-02 extended to take command line argument of loser_dir and set mod time of index.html
# to be same as json file
# 2019-12-17 extra output of links to troggle-generated trip data # 2019-12-17 extra output of links to troggle-generated trip data
# 2019-12-31 bits to make website link-checker not barf so much. Added endswith() to .startswith() for notes, elev, plan filenames # 2019-12-31 bits to make website link-checker not barf so much. Added endswith() to .startswith()
# for notes, elev, plan filenames
# 2020-01-21 Now we are using Windows10-WSL1, +links to expedition logbook on every generated page # 2020-01-21 Now we are using Windows10-WSL1, +links to expedition logbook on every generated page
# 2020-03-15 Adding timestamp to visible outputs, changing name of produced files to walletindex.html so that contents can be browsed # 2020-03-15 Adding timestamp to visible outputs, changing name of produced files to walletindex.html
# so that contents can be browsed
# 2020-03-15 Added "ignore" to the <year>#00 folder containing scraps - then removed as we do # 2020-03-15 Added "ignore" to the <year>#00 folder containing scraps - then removed as we do
# want it to appear in the reports under "UNKNOWN" # want it to appear in the reports under "UNKNOWN"
# 2021-04-24 Converted from python2 to python3 - god almighty did I really once think this was an
# acceptable python layout?
'''This stand-alone programe processes all the wallet folders for one year and produces the
list of actions that need to be done.
It produces
- an overall summary page for all the wallets in this year
- a summary page for each wallet
- a page specific to each person.listing what they need to do across all wallets
It scans the subdirectories only one level deep
e.g. we are in /2020/ so it scans /2020/2020#01, /2020/2020#02 et seq.
All the files in one folder must be for only one cave, but in principle coule be for several trips.
However all the files in one folder should relate to a single survex file (troggle assumes this) and
a survex file should relate to a single trip (we do this, the Austrians and Germans don't)
'''
loser_dir = "/home/expo/loser" loser_dir = "/home/expo/loser"
#loser_dir = "/mnt/d/CUCC-Expo/Loser/" # when running on Win10-WSL1 #loser_dir = "/mnt/d/CUCC-Expo/Loser/" # when running on Win10-WSL1
#loser_dir = "/media/philip/SD-huge/CUCC-Expo/loser/" # when running on xubuntu laptop 'barbie' #loser_dir = "/media/philip/SD-huge/CUCC-Expo/loser/" # when running on xubuntu laptop 'barbie'
if len(sys.argv) > 1 : # GLOBALS
if sys.argv[1] != "": wallets_needing_scanning = set()
loser_dir = sys.argv[1] website_needing_updating = set()
wallets = [] #need to use wallets as a dict/tuple (id,cave,name)
people = {}
cave = ""
name = ""
dateTimeObj=datetime.now(tz=None)
timestamp = dateTimeObj.strftime("%d-%b-%Y (%H:%M)")
print "Loser repo (for svx files) is assumed to be in: " + loser_dir + "/"
drawings_dir = loser_dir[0:len(loser_dir)-5] + "drawings"
print "Drawings repo (for drawings files) is assumed to be in: " + drawings_dir + "/"
html_base = "<html><body>%(body)s</body></html>" html_base = "<html><body>%(body)s</body></html>"
html_year_index = html_base % {"body": "<H1>%(year)s surveys: wallets status</H1>\n<p>List of trips: <a href=\"http://expo.survex.com/expedition/%(year)s\">expedition/%(year)s</a> - troggle-processed .svx files and logbook entries on server</p>\nAs of %(timestamp)s\n<H2>Persons</H2>\n<UL>\n%(persons)s</UL>\n<H2>Wallets</H2>\n<table>%(wallets)s</table>\n<H2>Needing Scanning</H2>\n<UL>\n%(needing scanning)s</ul>\n<H2>Website (Guidebook description) needing updating\n</H2>\n<UL style=\"column-count: 3; \">\n%(website needing updating)s</ul>\n"} html_year_index = html_base % {"body": "<H1>%(year)s surveys: wallets status</H1>\n<p>List of trips: <a href=\"http://expo.survex.com/expedition/%(year)s\">expedition/%(year)s</a> - troggle-processed .svx files and logbook entries on server</p>\nAs of %(timestamp)s\n<H2>Persons</H2>\n<UL>\n%(persons)s</UL>\n<H2>Wallets</H2>\n<table>%(wallets)s</table>\n<H2>Needing Scanning</H2>\n<UL>\n%(needing scanning)s</ul>\n<H2>Website (Guidebook description) needing updating\n</H2>\n<UL style=\"column-count: 3; \">\n%(website needing updating)s</ul>\n"}
@ -46,8 +66,6 @@ html_person = html_base % {"body": "<H1>%(person)s</H1>\n<p>List of trips: <a hr
html_complaint_items = "<li>%(count)i %(complaint)s</li>" html_complaint_items = "<li>%(count)i %(complaint)s</li>"
html_items = "<li>%s</li>" html_items = "<li>%s</li>"
blank_json = { blank_json = {
"cave": "", "cave": "",
"date": "", "date": "",
@ -67,239 +85,269 @@ blank_json = {
"survex not required": False, "survex not required": False,
"website updated": False} "website updated": False}
def do_item(year, item):
global loser_dir
global wallets
global people
global cave, name
global wallets_needing_scanning
global website_needing_updating
#need to use wallets as a dict/tuple (id,cave,name) - not sure how.
wallets = []
wallets_needing_scanning = set()
website_needing_updating = set()
people = {}
#use dir this file is in to get current year files = []
path,year = os.path.split(os.path.dirname(os.path.realpath(__file__))) for f in os.listdir(os.path.join(".", item)):
if f not in ["contents.json", "contents.json~","walletindex.html"] and os.path.isfile(os.path.join(".", item, f)):
print "Year: " + year files.append(f)
contents_path = os.path.join(".", item, "contents.json")
for item in sorted(os.listdir(".")): #print "Trying to read file %s" % (contents_path)
if os.path.isdir(item) and item != year+"indexpages": if not os.path.isfile(contents_path):
files = [] print("Creating file %s from template" % (contents_path))
for f in os.listdir(os.path.join(".", item)): json_file = open(contents_path, "w")
if f not in ["contents.json", "contents.json~","walletindex.html"] and os.path.isfile(os.path.join(".", item, f)): json.dump(blank_json, json_file, sort_keys=True, indent = 1)
files.append(f) json_file.close()
contents_path = os.path.join(".", item, "contents.json") #print "Reading file %s" % (contents_path)
#print "Trying to read file %s" % (contents_path) json_file = open(contents_path)
if not os.path.isfile(contents_path): try:
print "Creating file %s from template" % (contents_path) data = json.load(json_file)
json_file = open(contents_path, "w") except:
json.dump(blank_json, json_file, sort_keys=True, indent = 1) print("FAILURE parsing JSON file %s" % (contents_path))
json_file.close() # Python bug: https://github.com/ShinNoNoir/twitterwebsearch/issues/12
#print "Reading file %s" % (contents_path) raise
json_file = open(contents_path) if not data["people"]:
try: data["people"]=["NOBODY"]
data = json.load(json_file) json_file.close()
except: write_required = False
print "FAILURE parsing JSON file %s" % (contents_path) try:
# Python bug: https://github.com/ShinNoNoir/twitterwebsearch/issues/12 wallet, cave, name = re.match("(\d\d\d\d#\d\d)-(.*) (.*)", item).groups()
raise except:
if not data["people"]: wallet, cave, name = "", "", ""
data["people"]=["NOBODY"] #print data
for k, v in list(blank_json.items()):
if k not in data:
if k == "cave":
data[k] = cave
elif k == "name":
data[k] = name
else:
data[k] = v
write_required = True
#print write_required
if write_required:
print("Writing file %s" % (contents_path))
json_file = open(contents_path, "w")
json.dump(data, json_file, indent = 1)
json_file.close() json_file.close()
write_required = False
try:
wallet, cave, name = re.match("(\d\d\d\d#\d\d)-(.*) (.*)", item).groups()
except:
wallet, cave, name = "", "", ""
#print data
for k, v in blank_json.items():
if not data.has_key(k):
if k == "cave":
data[k] = cave
elif k == "name":
data[k] = name
else:
data[k] = v
write_required = True
#print write_required
if write_required:
print "Writing file %s" % (contents_path)
json_file = open(contents_path, "w")
json.dump(data, json_file, indent = 1)
json_file.close()
# Get modification time of contents.json # Get modification time of contents.json
# print("json last modified: %s" % time.ctime(os.path.getmtime(contents_path))) # print("json last modified: %s" % time.ctime(os.path.getmtime(contents_path)))
json_mtime = os.path.getmtime(contents_path) json_mtime = os.path.getmtime(contents_path)
#make wallet descriptions #make wallet descriptions
#Survex #Survex
survex_required = (data["survex not required"] and data["survex file"] == "") or \ not_req = (data["survex not required"] and data["survex file"] == "")
not (not data["survex not required"] and os.path.isfile(os.path.join(loser_dir, data["survex file"]))) req = (not data["survex not required"] and os.path.isfile(os.path.join(loser_dir, data["survex file"])))
survex_complaint = "" survex_required = not_req or not req
if data["survex not required"] and data["survex file"] != "":
survex_complaint = "Survex is not required and yet there is a survex file!"
if not data["survex not required"] and data["survex file"] == "":
survex_complaint = "A survex file is required, but has not been specified!"
if not data["survex not required"] and not os.path.isfile(os.path.join(loser_dir, data["survex file"])):
survex_complaint = "The specified survex file (%s) does not exist here!" % os.path.join(loser_dir, data["survex file"])
complaints = []
person_complaints = []
if survex_required:
complaints.append(survex_complaint)
person_complaints.append(survex_complaint)
#Notes survex_complaint = ""
notes_scanned = reduce(operator.or_, [f.startswith("note") for f in files], False) if data["survex not required"] and data["survex file"] != "":
notes_scanned = reduce(operator.or_, [f.endswith("note") for f in files], notes_scanned) survex_complaint = "Survex is not required and yet there is a survex file!"
if not notes_scanned: if not data["survex not required"] and data["survex file"] == "":
complaints.append("The notes needs scanning (no noteNN.jpg or XXnote.jpg file found)") survex_complaint = "A survex file is required, but has not been specified!"
wallets_needing_scanning.add(item) if not data["survex not required"] and not os.path.isfile(os.path.join(loser_dir, data["survex file"])):
survex_complaint = "The specified survex file (%s) does not exist here!" % os.path.join(loser_dir, data["survex file"])
complaints = []
person_complaints = []
if survex_required:
complaints.append(survex_complaint)
person_complaints.append(survex_complaint)
#Plan drawing required #Notes
plan_scanned = reduce(operator.or_, [f.startswith("plan") for f in files], False) notes_scanned = reduce(operator.or_, [f.startswith("note") for f in files], False)
plan_scanned = reduce(operator.or_, [f.endswith("plan") for f in files], plan_scanned) notes_scanned = reduce(operator.or_, [f.endswith("note") for f in files], notes_scanned)
plan_drawing_required = not (plan_scanned or data["plan drawn"]) if not notes_scanned:
if plan_drawing_required: complaints.append("The notes needs scanning (no noteNN.jpg or XXnote.jpg file found)")
complaints.append("The plan needs drawing (no planNN.jpg or XXplan.jpg file found)") wallets_needing_scanning.add(item)
person_complaints.append(" plan(s) needs drawing (no planNN.jpg or XXplan.jpg file found)")
if not plan_drawing_required and not plan_scanned: #Plan drawing required
complaints.append("The plan needs <em>scanning</em> (no planNN.jpg or XXplan.jpg file found)") plan_scanned = reduce(operator.or_, [f.startswith("plan") for f in files], False)
wallets_needing_scanning.add(item) plan_scanned = reduce(operator.or_, [f.endswith("plan") for f in files], plan_scanned)
plan_drawing_required = not (plan_scanned or data["plan drawn"])
if plan_drawing_required:
complaints.append("The plan needs drawing (no planNN.jpg or XXplan.jpg file found)")
person_complaints.append(" plan(s) needs drawing (no planNN.jpg or XXplan.jpg file found)")
if not plan_drawing_required and not plan_scanned:
complaints.append("The plan needs <em>scanning</em> (no planNN.jpg or XXplan.jpg file found)")
wallets_needing_scanning.add(item)
#Elev drawing required #Elev drawing required
elev_scanned = reduce(operator.or_, [f.startswith("elev") for f in files], False) elev_scanned = reduce(operator.or_, [f.startswith("elev") for f in files], False)
elev_scanned = reduce(operator.or_, [f.endswith("elev") for f in files], elev_scanned) elev_scanned = reduce(operator.or_, [f.endswith("elev") for f in files], elev_scanned)
elev_drawing_required = not (elev_scanned or data["elev drawn"]) elev_drawing_required = not (elev_scanned or data["elev drawn"])
if elev_drawing_required: if elev_drawing_required:
complaints.append("The elev needs drawing (no elevNN.jpg or XXelev.jpg file found)") complaints.append("The elev needs drawing (no elevNN.jpg or XXelev.jpg file found)")
person_complaints.append(" elev(s) needs drawing (no elevNN.jpg or XXelev.jpg file found)") person_complaints.append(" elev(s) needs drawing (no elevNN.jpg or XXelev.jpg file found)")
if not elev_drawing_required and not elev_scanned: if not elev_drawing_required and not elev_scanned:
complaints.append("The elev needs <em>scanning</em> (no elevNN.jpg or XXelev.jpg file found)") complaints.append("The elev needs <em>scanning</em> (no elevNN.jpg or XXelev.jpg file found)")
wallets_needing_scanning.add(item) wallets_needing_scanning.add(item)
#Description #Description
if not data["description written"]: if not data["description written"]:
complaints.append("The description needs writing") complaints.append("The description needs writing")
person_complaints.append(" description(s) needs writing") person_complaints.append(" description(s) needs writing")
description_needed = "A description is indicated as being needed, so may need adding into this cave page." description_needed = "A description is indicated as being needed, so may need adding into this cave page."
#QMS
if not data["qms written"]:
complaints.append("The QMs needs writing")
person_complaints.append(" set(s) of QMs needs writing")
#Website
if not data["website updated"]:
complaints.append("The website is marked as needing updating (using the guidebook description)")
website_needing_updating.add(item)
#Electronic Surveys
if not data["electronic survey"]:
complaints.append("Tunnel / Therion drawing files need drawing")
if data["survex file"]:
survex_description = data["survex file"]
else:
survex_description = "Not specified"
wallet_index_file = open(os.path.join(item, "walletindex.html"), "w")
wallet_index_file.write(html_wallet_index % {"title": item, "year": year,
"cave": data["cave"],
"name": data["name"],
"date": data["date"],
"people": reduce(operator.add, [" %s," % person for person in data["people"]], ""),
"description": "http://expo.survex.com"+data["description url"],
"description_needed": description_needed,
"loser_dir": loser_dir,
"loser_dirw": loser_dir[5].upper() + ':/' + loser_dir[7:],
"survex": survex_description,
"complaints": reduce(operator.add, ["<p>" + complaint + "</p>" for complaint in complaints], ""),
"files": reduce(operator.add,
[html_wallet_file_entry % {"fileurl": urllib.parse.quote(f),
"filename": f}
for f
in files],
"")})
wallet_index_file.close()
wallets.append((item, data["cave"], data["name"], survex_required, plan_scanned, elev_scanned, data["description written"], data["qms written"]))
# Set modification time to be the same as that of contents.json
index_file = item+"/walletindex.html"
os.utime(index_file, ( json_mtime,json_mtime))
#QMS #People
if not data["qms written"]:
complaints.append("The QMs needs writing")
person_complaints.append(" set(s) of QMs needs writing")
#Website
if not data["website updated"]:
complaints.append("The website is marked as needing updating (using the guidebook description)")
website_needing_updating.add(item)
#Electronic Surveys
if not data["electronic survey"]:
complaints.append("Tunnel / Therion drawing files need drawing")
if data["survex file"]:
survex_description = data["survex file"]
else:
survex_description = "Not specified"
wallet_index_file = open(os.path.join(item, "walletindex.html"), "w")
wallet_index_file.write(html_wallet_index % {"title": item, "year": year,
"cave": data["cave"],
"name": data["name"],
"date": data["date"],
"people": reduce(operator.add, [" %s," % person for person in data["people"]], ""),
"description": "http://expo.survex.com"+data["description url"],
"description_needed": description_needed,
"loser_dir": loser_dir,
"loser_dirw": loser_dir[5].upper() + ':/' + loser_dir[7:],
"survex": survex_description,
"complaints": reduce(operator.add, ["<p>" + complaint + "</p>" for complaint in complaints], ""),
"files": reduce(operator.add,
[html_wallet_file_entry % {"fileurl": urllib.quote(f),
"filename": f}
for f
in files],
"")})
wallet_index_file.close()
wallets.append((item, data["cave"], data["name"], survex_required, plan_scanned, elev_scanned, data["description written"], data["qms written"]))
# Set modification time to be the same as that of contents.json
index_file = item+"/walletindex.html"
os.utime(index_file, ( json_mtime,json_mtime))
#People
for person in data["people"]:
# delete all person.html as we are recreating all the ones that matter and old ones have old data
if os.path.isfile(person + ".html"):
os.remove(person + ".html")
if person_complaints:
for person in data["people"]: for person in data["people"]:
# delete all person.html as we are recreating all the ones that matter and old ones have old data if person not in people:
if os.path.isfile(person + ".html"): people[person] = []
os.remove(person + ".html") people[person].append((item, person_complaints))
if person_complaints:
for person in data["people"]: def main():
if not people.has_key(person): global loser_dir
people[person] = [] global wallets
people[person].append((item, person_complaints)) global people
global cave, name
global wallets_needing_scanning
global website_needing_updating
if len(sys.argv) > 1 :
if sys.argv[1] != "":
loser_dir = sys.argv[1]
dateTimeObj=datetime.now(tz=None)
timestamp = dateTimeObj.strftime("%d-%b-%Y (%H:%M)")
print("Loser repo (for svx files) is assumed to be in: " + loser_dir + "/")
drawings_dir = loser_dir[0:len(loser_dir)-5] + "drawings"
print("Drawings repo (for drawings files) is assumed to be in: " + drawings_dir + "/")
wallets.sort() #use dir this file is in to get current year
website_needing_updating = list(website_needing_updating) path,year = os.path.split(os.path.dirname(os.path.realpath(__file__)))
website_needing_updating.sort()
wallets_needing_scanning = list(wallets_needing_scanning)
wallets_needing_scanning.sort()
person_summary = [] print("Year: " + year)
for person, person_wallets in people.items():
complaints = reduce(operator.add, [complaints for wallet, complaints in person_wallets], [])
complaints_summary = []
for complaint in set(complaints):
complaints_summary.append((complaint, complaints.count(complaint)))
person_summary.append((person, complaints_summary))
person_summary = dict(person_summary) for item in sorted(os.listdir(".")):
if os.path.isdir(item) and item != year+"indexpages":
do_item(year, item)
year_index_file = open("walletindex.html", "w")
year_index_file.write(html_year_index % {"year": year, "timestamp": timestamp, "persons": reduce(operator.add, [html_year_person % {"person": person,
"complaints": reduce(operator.add,
[html_complaint_items % {"complaint": complaint,
"count": count}
for complaint, count
in complaints],
"")}
for person, complaints
in person_summary.items()], ""),
"needing scanning": reduce(operator.add, [html_year_scanning_entry % {"walletname": wallet,
"cave": cave,
"name": name,
"walletindex": urllib.quote(wallet) + "/walletindex.html"}
for (wallet)
in wallets_needing_scanning], ""),
"website needing updating": reduce(operator.add, [html_year_scanning_entry % {"walletname": wallet,
"cave": cave,
"name": name,
"walletindex": urllib.quote(wallet) + "/walletindex.html"}
for (wallet)
in website_needing_updating], ""),
"wallets": reduce(operator.add,
[html_year_wallet_entry % {"walletname": wallet,
"cave": cave,
"name": name,
"walletindex": urllib.quote(wallet) + "/walletindex.html",
"complaints": html_status[survex_required or not plan_scanned or not elev_scanned or description_written] + html_survex_required[survex_required] + html_plan_scanned[plan_scanned] + html_elev_scanned[elev_scanned] + html_description_written[description_written] + html_qms_written[qms_written] }
for (wallet, cave, name, survex_required, plan_scanned, elev_scanned, description_written, qms_written)
in wallets])})
year_index_file.close()
for person, item_complaint_list in people.items(): wallets.sort()
person_file = open(person + ".html", "w") website_needing_updating = list(website_needing_updating)
person_file.write(html_person % {"person": person, "year": year, "timestamp": timestamp, website_needing_updating.sort()
"wallets": reduce(operator.add, [html_person_wallet_entry % {"walletname": wallet, wallets_needing_scanning = list(wallets_needing_scanning)
"walletindex": urllib.quote(wallet) + "/walletindex.html", wallets_needing_scanning.sort()
"complaints": reduce(operator.add,
[html_items % complaint person_summary = []
for complaint for person, person_wallets in list(people.items()):
in complaints], complaints = reduce(operator.add, [complaints for wallet, complaints in person_wallets], [])
"") } complaints_summary = []
for wallet, complaints for complaint in set(complaints):
in item_complaint_list], "") complaints_summary.append((complaint, complaints.count(complaint)))
}) person_summary.append((person, complaints_summary))
person_file.close()
person_summary = dict(person_summary)
year_index_file = open("walletindex.html", "w")
year_index_file.write(html_year_index % {"year": year, "timestamp": timestamp, "persons": reduce(operator.add, [html_year_person % {"person": person,
"complaints": reduce(operator.add,
[html_complaint_items % {"complaint": complaint,
"count": count}
for complaint, count
in complaints],
"")}
for person, complaints
in list(person_summary.items())], ""),
"needing scanning": reduce(operator.add, [html_year_scanning_entry % {"walletname": wallet,
"cave": cave,
"name": name,
"walletindex": urllib.parse.quote(wallet) + "/walletindex.html"}
for (wallet)
in wallets_needing_scanning], ""),
"website needing updating": reduce(operator.add, [html_year_scanning_entry % {"walletname": wallet,
"cave": cave,
"name": name,
"walletindex": urllib.parse.quote(wallet) + "/walletindex.html"}
for (wallet)
in website_needing_updating], ""),
"wallets": reduce(operator.add,
[html_year_wallet_entry % {"walletname": wallet,
"cave": cave,
"name": name,
"walletindex": urllib.parse.quote(wallet) + "/walletindex.html",
"complaints": html_status[survex_required or not plan_scanned or not elev_scanned or description_written] + html_survex_required[survex_required] + html_plan_scanned[plan_scanned] + html_elev_scanned[elev_scanned] + html_description_written[description_written] + html_qms_written[qms_written] }
for (wallet, cave, name, survex_required, plan_scanned, elev_scanned, description_written, qms_written)
in wallets])})
year_index_file.close()
for person, item_complaint_list in list(people.items()):
person_file = open(person + ".html", "w")
person_file.write(html_person % {"person": person, "year": year, "timestamp": timestamp,
"wallets": reduce(operator.add, [html_person_wallet_entry % {"walletname": wallet,
"walletindex": urllib.parse.quote(wallet) + "/walletindex.html",
"complaints": reduce(operator.add,
[html_items % complaint
for complaint
in complaints],
"") }
for wallet, complaints
in item_complaint_list], "")
})
person_file.close()
#if __name__ == "__main__":
main()