expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2024-11-22 07:11:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

updating wallets scripts (expofiles copy not git)

Browse Source
This commit is contained in:
Philip Sargent 2021-04-24 23:16:44 +01:00
parent 2e6a9a7f50
commit c34d6b2280
5 changed files with 296 additions and 237 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
handbook/troggle/scriptscurrent.html
View File

@ -33,7 +33,7 @@ p {
<p>It coordinates producing the 3d surveys used in the cave description pages, updates the area pages, runs the folk script, runs the QM list generation within each of the cave pages that needs it, runs svxtrace, and reports on everything using "bigbro" which we don't have any other reference to. (Generation of the .3d files as required is now done by troggle.) <p>It coordinates producing the 3d surveys used in the cave description pages, updates the area pages, runs the folk script, runs the QM list generation within each of the cave pages that needs it, runs svxtrace, and reports on everything using "bigbro" which we don't have any other reference to. (Generation of the .3d files as required is now done by troggle.)
<h4 id="wallets">Wallets</h4> <h4 id="wallets">Wallets</h4>
<p><a href="../survey/onlinewallet.html">Online wallets</a> are initially maintained using the <a href="/expofiles/surveyscans/wallets.py">wallets.py</a> script, but troggle also directly imports all the expofiles/surveyscans/ directories of scanned survey notes and produces <a href="/survey_scans/">reports</a> on then. <p><a href="../survey/onlinewallet.html">Online wallets</a> are initially maintained using the <a href="/expofiles/surveyscans/wallets.py">wallets.py</a> script, but troggle also directly imports all the expofiles/surveyscans/ directories of scanned survey notes and produces <a href="/survey_scans/">reports</a> on them. There are several bash and python scripts in the <a href="/expofiles/surveyscans/">surveyscans</a> directory to create wallets for the coming year, and to re-run the wallet processing on all past years (for when we improve the script). For 2021 we have converted wallets.py to python3, so be careful of older versions which are python2.
<h4 id="folk">Folk</a></h4> <h4 id="folk">Folk</a></h4>

2
handbook/troggle/scriptsother.html
View File

@ -22,7 +22,7 @@
<li><a href="scriptscurrent.html#latex">bierbook.tex</a> LaTeX script for generating the bierbook - a new list of names and dates each year <li><a href="scriptscurrent.html#latex">bierbook.tex</a> LaTeX script for generating the bierbook - a new list of names and dates each year
<li><a href="scriptscurrent.html#latex">seshbook.tex</a> LaTeX script for generating the seshbook - works from the same list of names <li><a href="scriptscurrent.html#latex">seshbook.tex</a> LaTeX script for generating the seshbook - works from the same list of names
<li><a href="scriptscurrent.html#latex">therionpage.tex</a> LaTeX script and makefile for generating therion-style protractors</li><br /> <li><a href="scriptscurrent.html#latex">therionpage.tex</a> LaTeX script and makefile for generating therion-style protractors</li><br />
<li><a href="scriptscurrent.html#latex">wallets.py</a> generates statuspages and to-do list pages for survey data production. <li><a href="scriptscurrent.html#wallets">wallets.py</a> generates statuspages and to-do list pages for survey data production.
<li><a href="scriptsqms.html">svx2qm.py</a> extracts QMs from the survex files (and <a href="scriptsqms.html">find-dead-qms.py</a>) <li><a href="scriptsqms.html">svx2qm.py</a> extracts QMs from the survex files (and <a href="scriptsqms.html">find-dead-qms.py</a>)
<li><a href="scriptsqms.html">tablize-qms.pl</a> turns the list of QMs extracted into an HTML file <li><a href="scriptsqms.html">tablize-qms.pl</a> turns the list of QMs extracted into an HTML file
<li><a href="scriptscurrent.html#surface">make_svx.sh</a> generates surface Survex tracks <li><a href="scriptscurrent.html#surface">make_svx.sh</a> generates surface Survex tracks

2
noinfo/walletscripts/allwallets.sh
View File

@ -1,6 +1,6 @@
#/bin/sh #/bin/sh
for i in 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019; do for i in 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2021; do
#for i in 2016 2017 2018 2019; do #for i in 2016 2017 2018 2019; do
echo $i echo $i
cp -p wallets.py $i cp -p wallets.py $i

11
noinfo/walletscripts/mkdirs.sh Normal file
View File

@ -0,0 +1,11 @@
#!/bin/bash
# Make the first set of directories for the coming year
# run from /surveyscans/ and tell it the year, e.g.
# $ ./mkdirs.sh 2021
if [$1 -eq ""]; then echo -e "mkdirs [year]\nProvide a year as the argument."; exit; fi
for i in {100..135}; do
ds=${i:1:3}
echo mkdir $1/$1"#"$ds
mkdir $1/$1"#"$ds
done

516
noinfo/walletscripts/wallets.py
View File

@ -1,32 +1,52 @@
#!/usr/bin/env python #!/usr/bin/env python
import sys, os, operator, urllib, json, re, time import sys, os, operator, urllib.request, urllib.parse, urllib.error, json, re, time
from datetime import datetime from datetime import datetime
from functools import reduce
from pathlib import Path
# 2017 originally by Martin Green # 2017 originally by Martin Green
# 2018-08-27 edited Philip Sargent # 2018-08-27 edited Philip Sargent
# 2019-03-02 extended to take command line argument of loser_dir and set mod time of index.html to be same as json file # 2019-03-02 extended to take command line argument of loser_dir and set mod time of index.html
# to be same as json file
# 2019-12-17 extra output of links to troggle-generated trip data # 2019-12-17 extra output of links to troggle-generated trip data
# 2019-12-31 bits to make website link-checker not barf so much. Added endswith() to .startswith() for notes, elev, plan filenames # 2019-12-31 bits to make website link-checker not barf so much. Added endswith() to .startswith()
# for notes, elev, plan filenames
# 2020-01-21 Now we are using Windows10-WSL1, +links to expedition logbook on every generated page # 2020-01-21 Now we are using Windows10-WSL1, +links to expedition logbook on every generated page
# 2020-03-15 Adding timestamp to visible outputs, changing name of produced files to walletindex.html so that contents can be browsed # 2020-03-15 Adding timestamp to visible outputs, changing name of produced files to walletindex.html
# so that contents can be browsed
# 2020-03-15 Added "ignore" to the <year>#00 folder containing scraps - then removed as we do # 2020-03-15 Added "ignore" to the <year>#00 folder containing scraps - then removed as we do
# want it to appear in the reports under "UNKNOWN" # want it to appear in the reports under "UNKNOWN"
# 2021-04-24 Converted from python2 to python3 - god almighty did I really once think this was an
# acceptable python layout?
'''This stand-alone programe processes all the wallet folders for one year and produces the
list of actions that need to be done.
It produces
- an overall summary page for all the wallets in this year
- a summary page for each wallet
- a page specific to each person.listing what they need to do across all wallets
It scans the subdirectories only one level deep
e.g. we are in /2020/ so it scans /2020/2020#01, /2020/2020#02 et seq.
All the files in one folder must be for only one cave, but in principle coule be for several trips.
However all the files in one folder should relate to a single survex file (troggle assumes this) and
a survex file should relate to a single trip (we do this, the Austrians and Germans don't)
'''
loser_dir = "/home/expo/loser" loser_dir = "/home/expo/loser"
#loser_dir = "/mnt/d/CUCC-Expo/Loser/" # when running on Win10-WSL1 #loser_dir = "/mnt/d/CUCC-Expo/Loser/" # when running on Win10-WSL1
#loser_dir = "/media/philip/SD-huge/CUCC-Expo/loser/" # when running on xubuntu laptop 'barbie' #loser_dir = "/media/philip/SD-huge/CUCC-Expo/loser/" # when running on xubuntu laptop 'barbie'
if len(sys.argv) > 1 : # GLOBALS
if sys.argv[1] != "": wallets_needing_scanning = set()
loser_dir = sys.argv[1] website_needing_updating = set()
wallets = [] #need to use wallets as a dict/tuple (id,cave,name)
people = {}
cave = ""
name = ""
dateTimeObj=datetime.now(tz=None)
timestamp = dateTimeObj.strftime("%d-%b-%Y (%H:%M)")
print "Loser repo (for svx files) is assumed to be in: " + loser_dir + "/"
drawings_dir = loser_dir[0:len(loser_dir)-5] + "drawings"
print "Drawings repo (for drawings files) is assumed to be in: " + drawings_dir + "/"
html_base = "<html><body>%(body)s</body></html>" html_base = "<html><body>%(body)s</body></html>"
html_year_index = html_base % {"body": "<H1>%(year)s surveys: wallets status</H1>\n<p>List of trips: <a href=\"http://expo.survex.com/expedition/%(year)s\">expedition/%(year)s</a> - troggle-processed .svx files and logbook entries on server</p>\nAs of %(timestamp)s\n<H2>Persons</H2>\n<UL>\n%(persons)s</UL>\n<H2>Wallets</H2>\n<table>%(wallets)s</table>\n<H2>Needing Scanning</H2>\n<UL>\n%(needing scanning)s</ul>\n<H2>Website (Guidebook description) needing updating\n</H2>\n<UL style=\"column-count: 3; \">\n%(website needing updating)s</ul>\n"} html_year_index = html_base % {"body": "<H1>%(year)s surveys: wallets status</H1>\n<p>List of trips: <a href=\"http://expo.survex.com/expedition/%(year)s\">expedition/%(year)s</a> - troggle-processed .svx files and logbook entries on server</p>\nAs of %(timestamp)s\n<H2>Persons</H2>\n<UL>\n%(persons)s</UL>\n<H2>Wallets</H2>\n<table>%(wallets)s</table>\n<H2>Needing Scanning</H2>\n<UL>\n%(needing scanning)s</ul>\n<H2>Website (Guidebook description) needing updating\n</H2>\n<UL style=\"column-count: 3; \">\n%(website needing updating)s</ul>\n"}
@ -46,8 +66,6 @@ html_person = html_base % {"body": "<H1>%(person)s</H1>\n<p>List of trips: <a hr
html_complaint_items = "<li>%(count)i %(complaint)s</li>" html_complaint_items = "<li>%(count)i %(complaint)s</li>"
html_items = "<li>%s</li>" html_items = "<li>%s</li>"
blank_json = { blank_json = {
"cave": "", "cave": "",
"date": "", "date": "",
@ -67,239 +85,269 @@ blank_json = {
"survex not required": False, "survex not required": False,
"website updated": False} "website updated": False}
def do_item(year, item):
global loser_dir
global wallets
global people
global cave, name
global wallets_needing_scanning
global website_needing_updating
#need to use wallets as a dict/tuple (id,cave,name) - not sure how. files = []
wallets = [] for f in os.listdir(os.path.join(".", item)):
wallets_needing_scanning = set() if f not in ["contents.json", "contents.json~","walletindex.html"] and os.path.isfile(os.path.join(".", item, f)):
website_needing_updating = set() files.append(f)
people = {} contents_path = os.path.join(".", item, "contents.json")
#print "Trying to read file %s" % (contents_path)
#use dir this file is in to get current year if not os.path.isfile(contents_path):
path,year = os.path.split(os.path.dirname(os.path.realpath(__file__))) print("Creating file %s from template" % (contents_path))
json_file = open(contents_path, "w")
print "Year: " + year json.dump(blank_json, json_file, sort_keys=True, indent = 1)
for item in sorted(os.listdir(".")):
if os.path.isdir(item) and item != year+"indexpages":
files = []
for f in os.listdir(os.path.join(".", item)):
if f not in ["contents.json", "contents.json~","walletindex.html"] and os.path.isfile(os.path.join(".", item, f)):
files.append(f)
contents_path = os.path.join(".", item, "contents.json")
#print "Trying to read file %s" % (contents_path)
if not os.path.isfile(contents_path):
print "Creating file %s from template" % (contents_path)
json_file = open(contents_path, "w")
json.dump(blank_json, json_file, sort_keys=True, indent = 1)
json_file.close()
#print "Reading file %s" % (contents_path)
json_file = open(contents_path)
try:
data = json.load(json_file)
except:
print "FAILURE parsing JSON file %s" % (contents_path)
# Python bug: https://github.com/ShinNoNoir/twitterwebsearch/issues/12
raise
if not data["people"]:
data["people"]=["NOBODY"]
json_file.close() json_file.close()
write_required = False #print "Reading file %s" % (contents_path)
try: json_file = open(contents_path)
wallet, cave, name = re.match("(\d\d\d\d#\d\d)-(.*) (.*)", item).groups() try:
except: data = json.load(json_file)
wallet, cave, name = "", "", "" except:
#print data print("FAILURE parsing JSON file %s" % (contents_path))
for k, v in blank_json.items(): # Python bug: https://github.com/ShinNoNoir/twitterwebsearch/issues/12
if not data.has_key(k): raise
if k == "cave": if not data["people"]:
data[k] = cave data["people"]=["NOBODY"]
elif k == "name": json_file.close()
data[k] = name write_required = False
else: try:
data[k] = v wallet, cave, name = re.match("(\d\d\d\d#\d\d)-(.*) (.*)", item).groups()
write_required = True except:
#print write_required wallet, cave, name = "", "", ""
if write_required: #print data
print "Writing file %s" % (contents_path) for k, v in list(blank_json.items()):
json_file = open(contents_path, "w") if k not in data:
json.dump(data, json_file, indent = 1) if k == "cave":
json_file.close() data[k] = cave
elif k == "name":
# Get modification time of contents.json data[k] = name
# print("json last modified: %s" % time.ctime(os.path.getmtime(contents_path))) else:
json_mtime = os.path.getmtime(contents_path) data[k] = v
write_required = True
#print write_required
if write_required:
print("Writing file %s" % (contents_path))
json_file = open(contents_path, "w")
json.dump(data, json_file, indent = 1)
json_file.close()
# Get modification time of contents.json
# print("json last modified: %s" % time.ctime(os.path.getmtime(contents_path)))
json_mtime = os.path.getmtime(contents_path)
#make wallet descriptions #make wallet descriptions
#Survex #Survex
survex_required = (data["survex not required"] and data["survex file"] == "") or \ not_req = (data["survex not required"] and data["survex file"] == "")
not (not data["survex not required"] and os.path.isfile(os.path.join(loser_dir, data["survex file"]))) req = (not data["survex not required"] and os.path.isfile(os.path.join(loser_dir, data["survex file"])))
survex_complaint = "" survex_required = not_req or not req
if data["survex not required"] and data["survex file"] != "":
survex_complaint = "Survex is not required and yet there is a survex file!"
if not data["survex not required"] and data["survex file"] == "":
survex_complaint = "A survex file is required, but has not been specified!"
if not data["survex not required"] and not os.path.isfile(os.path.join(loser_dir, data["survex file"])):
survex_complaint = "The specified survex file (%s) does not exist here!" % os.path.join(loser_dir, data["survex file"])
complaints = []
person_complaints = []
if survex_required:
complaints.append(survex_complaint)
person_complaints.append(survex_complaint)
#Notes
notes_scanned = reduce(operator.or_, [f.startswith("note") for f in files], False)
notes_scanned = reduce(operator.or_, [f.endswith("note") for f in files], notes_scanned)
if not notes_scanned:
complaints.append("The notes needs scanning (no noteNN.jpg or XXnote.jpg file found)")
wallets_needing_scanning.add(item)
#Plan drawing required
plan_scanned = reduce(operator.or_, [f.startswith("plan") for f in files], False)
plan_scanned = reduce(operator.or_, [f.endswith("plan") for f in files], plan_scanned)
plan_drawing_required = not (plan_scanned or data["plan drawn"])
if plan_drawing_required:
complaints.append("The plan needs drawing (no planNN.jpg or XXplan.jpg file found)")
person_complaints.append(" plan(s) needs drawing (no planNN.jpg or XXplan.jpg file found)")
if not plan_drawing_required and not plan_scanned:
complaints.append("The plan needs <em>scanning</em> (no planNN.jpg or XXplan.jpg file found)")
wallets_needing_scanning.add(item)
survex_complaint = ""
#Elev drawing required if data["survex not required"] and data["survex file"] != "":
elev_scanned = reduce(operator.or_, [f.startswith("elev") for f in files], False) survex_complaint = "Survex is not required and yet there is a survex file!"
elev_scanned = reduce(operator.or_, [f.endswith("elev") for f in files], elev_scanned) if not data["survex not required"] and data["survex file"] == "":
elev_drawing_required = not (elev_scanned or data["elev drawn"]) survex_complaint = "A survex file is required, but has not been specified!"
if elev_drawing_required: if not data["survex not required"] and not os.path.isfile(os.path.join(loser_dir, data["survex file"])):
complaints.append("The elev needs drawing (no elevNN.jpg or XXelev.jpg file found)") survex_complaint = "The specified survex file (%s) does not exist here!" % os.path.join(loser_dir, data["survex file"])
person_complaints.append(" elev(s) needs drawing (no elevNN.jpg or XXelev.jpg file found)") complaints = []
if not elev_drawing_required and not elev_scanned: person_complaints = []
complaints.append("The elev needs <em>scanning</em> (no elevNN.jpg or XXelev.jpg file found)") if survex_required:
wallets_needing_scanning.add(item) complaints.append(survex_complaint)
person_complaints.append(survex_complaint)
#Notes
notes_scanned = reduce(operator.or_, [f.startswith("note") for f in files], False)
notes_scanned = reduce(operator.or_, [f.endswith("note") for f in files], notes_scanned)
if not notes_scanned:
complaints.append("The notes needs scanning (no noteNN.jpg or XXnote.jpg file found)")
wallets_needing_scanning.add(item)
#Plan drawing required
plan_scanned = reduce(operator.or_, [f.startswith("plan") for f in files], False)
plan_scanned = reduce(operator.or_, [f.endswith("plan") for f in files], plan_scanned)
plan_drawing_required = not (plan_scanned or data["plan drawn"])
if plan_drawing_required:
complaints.append("The plan needs drawing (no planNN.jpg or XXplan.jpg file found)")
person_complaints.append(" plan(s) needs drawing (no planNN.jpg or XXplan.jpg file found)")
if not plan_drawing_required and not plan_scanned:
complaints.append("The plan needs <em>scanning</em> (no planNN.jpg or XXplan.jpg file found)")
wallets_needing_scanning.add(item)
#Description
if not data["description written"]: #Elev drawing required
complaints.append("The description needs writing") elev_scanned = reduce(operator.or_, [f.startswith("elev") for f in files], False)
person_complaints.append(" description(s) needs writing") elev_scanned = reduce(operator.or_, [f.endswith("elev") for f in files], elev_scanned)
description_needed = "A description is indicated as being needed, so may need adding into this cave page." elev_drawing_required = not (elev_scanned or data["elev drawn"])
if elev_drawing_required:
complaints.append("The elev needs drawing (no elevNN.jpg or XXelev.jpg file found)")
person_complaints.append(" elev(s) needs drawing (no elevNN.jpg or XXelev.jpg file found)")
if not elev_drawing_required and not elev_scanned:
complaints.append("The elev needs <em>scanning</em> (no elevNN.jpg or XXelev.jpg file found)")
wallets_needing_scanning.add(item)
#QMS #Description
if not data["qms written"]: if not data["description written"]:
complaints.append("The QMs needs writing") complaints.append("The description needs writing")
person_complaints.append(" set(s) of QMs needs writing") person_complaints.append(" description(s) needs writing")
description_needed = "A description is indicated as being needed, so may need adding into this cave page."
#Website #QMS
if not data["website updated"]: if not data["qms written"]:
complaints.append("The website is marked as needing updating (using the guidebook description)") complaints.append("The QMs needs writing")
website_needing_updating.add(item) person_complaints.append(" set(s) of QMs needs writing")
#Electronic Surveys #Website
if not data["electronic survey"]: if not data["website updated"]:
complaints.append("Tunnel / Therion drawing files need drawing") complaints.append("The website is marked as needing updating (using the guidebook description)")
website_needing_updating.add(item)
if data["survex file"]: #Electronic Surveys
survex_description = data["survex file"] if not data["electronic survey"]:
else: complaints.append("Tunnel / Therion drawing files need drawing")
survex_description = "Not specified"
wallet_index_file = open(os.path.join(item, "walletindex.html"), "w") if data["survex file"]:
wallet_index_file.write(html_wallet_index % {"title": item, "year": year, survex_description = data["survex file"]
"cave": data["cave"], else:
"name": data["name"], survex_description = "Not specified"
"date": data["date"],
"people": reduce(operator.add, [" %s," % person for person in data["people"]], ""),
"description": "http://expo.survex.com"+data["description url"],
"description_needed": description_needed,
"loser_dir": loser_dir,
"loser_dirw": loser_dir[5].upper() + ':/' + loser_dir[7:],
"survex": survex_description,
"complaints": reduce(operator.add, ["<p>" + complaint + "</p>" for complaint in complaints], ""),
"files": reduce(operator.add,
[html_wallet_file_entry % {"fileurl": urllib.quote(f),
"filename": f}
for f
in files],
"")})
wallet_index_file.close()
wallets.append((item, data["cave"], data["name"], survex_required, plan_scanned, elev_scanned, data["description written"], data["qms written"]))
# Set modification time to be the same as that of contents.json
index_file = item+"/walletindex.html"
os.utime(index_file, ( json_mtime,json_mtime))
wallet_index_file = open(os.path.join(item, "walletindex.html"), "w")
#People wallet_index_file.write(html_wallet_index % {"title": item, "year": year,
"cave": data["cave"],
"name": data["name"],
"date": data["date"],
"people": reduce(operator.add, [" %s," % person for person in data["people"]], ""),
"description": "http://expo.survex.com"+data["description url"],
"description_needed": description_needed,
"loser_dir": loser_dir,
"loser_dirw": loser_dir[5].upper() + ':/' + loser_dir[7:],
"survex": survex_description,
"complaints": reduce(operator.add, ["<p>" + complaint + "</p>" for complaint in complaints], ""),
"files": reduce(operator.add,
[html_wallet_file_entry % {"fileurl": urllib.parse.quote(f),
"filename": f}
for f
in files],
"")})
wallet_index_file.close()
wallets.append((item, data["cave"], data["name"], survex_required, plan_scanned, elev_scanned, data["description written"], data["qms written"]))
# Set modification time to be the same as that of contents.json
index_file = item+"/walletindex.html"
os.utime(index_file, ( json_mtime,json_mtime))
#People
for person in data["people"]:
# delete all person.html as we are recreating all the ones that matter and old ones have old data
if os.path.isfile(person + ".html"):
os.remove(person + ".html")
if person_complaints:
for person in data["people"]: for person in data["people"]:
# delete all person.html as we are recreating all the ones that matter and old ones have old data if person not in people:
if os.path.isfile(person + ".html"): people[person] = []
os.remove(person + ".html") people[person].append((item, person_complaints))
if person_complaints:
for person in data["people"]: def main():
if not people.has_key(person): global loser_dir
people[person] = [] global wallets
people[person].append((item, person_complaints)) global people
global cave, name
global wallets_needing_scanning
global website_needing_updating
if len(sys.argv) > 1 :
if sys.argv[1] != "":
loser_dir = sys.argv[1]
dateTimeObj=datetime.now(tz=None)
timestamp = dateTimeObj.strftime("%d-%b-%Y (%H:%M)")
print("Loser repo (for svx files) is assumed to be in: " + loser_dir + "/")
drawings_dir = loser_dir[0:len(loser_dir)-5] + "drawings"
print("Drawings repo (for drawings files) is assumed to be in: " + drawings_dir + "/")
#use dir this file is in to get current year
path,year = os.path.split(os.path.dirname(os.path.realpath(__file__)))
print("Year: " + year)
for item in sorted(os.listdir(".")):
if os.path.isdir(item) and item != year+"indexpages":
do_item(year, item)
wallets.sort() wallets.sort()
website_needing_updating = list(website_needing_updating) website_needing_updating = list(website_needing_updating)
website_needing_updating.sort() website_needing_updating.sort()
wallets_needing_scanning = list(wallets_needing_scanning) wallets_needing_scanning = list(wallets_needing_scanning)
wallets_needing_scanning.sort() wallets_needing_scanning.sort()
person_summary = [] person_summary = []
for person, person_wallets in people.items(): for person, person_wallets in list(people.items()):
complaints = reduce(operator.add, [complaints for wallet, complaints in person_wallets], []) complaints = reduce(operator.add, [complaints for wallet, complaints in person_wallets], [])
complaints_summary = [] complaints_summary = []
for complaint in set(complaints): for complaint in set(complaints):
complaints_summary.append((complaint, complaints.count(complaint))) complaints_summary.append((complaint, complaints.count(complaint)))
person_summary.append((person, complaints_summary)) person_summary.append((person, complaints_summary))
person_summary = dict(person_summary) person_summary = dict(person_summary)
year_index_file = open("walletindex.html", "w") year_index_file = open("walletindex.html", "w")
year_index_file.write(html_year_index % {"year": year, "timestamp": timestamp, "persons": reduce(operator.add, [html_year_person % {"person": person, year_index_file.write(html_year_index % {"year": year, "timestamp": timestamp, "persons": reduce(operator.add, [html_year_person % {"person": person,
"complaints": reduce(operator.add, "complaints": reduce(operator.add,
[html_complaint_items % {"complaint": complaint, [html_complaint_items % {"complaint": complaint,
"count": count} "count": count}
for complaint, count for complaint, count
in complaints], in complaints],
"")} "")}
for person, complaints for person, complaints
in person_summary.items()], ""), in list(person_summary.items())], ""),
"needing scanning": reduce(operator.add, [html_year_scanning_entry % {"walletname": wallet, "needing scanning": reduce(operator.add, [html_year_scanning_entry % {"walletname": wallet,
"cave": cave, "cave": cave,
"name": name, "name": name,
"walletindex": urllib.quote(wallet) + "/walletindex.html"} "walletindex": urllib.parse.quote(wallet) + "/walletindex.html"}
for (wallet) for (wallet)
in wallets_needing_scanning], ""), in wallets_needing_scanning], ""),
"website needing updating": reduce(operator.add, [html_year_scanning_entry % {"walletname": wallet, "website needing updating": reduce(operator.add, [html_year_scanning_entry % {"walletname": wallet,
"cave": cave, "cave": cave,
"name": name, "name": name,
"walletindex": urllib.quote(wallet) + "/walletindex.html"} "walletindex": urllib.parse.quote(wallet) + "/walletindex.html"}
for (wallet) for (wallet)
in website_needing_updating], ""), in website_needing_updating], ""),
"wallets": reduce(operator.add, "wallets": reduce(operator.add,
[html_year_wallet_entry % {"walletname": wallet, [html_year_wallet_entry % {"walletname": wallet,
"cave": cave, "cave": cave,
"name": name, "name": name,
"walletindex": urllib.quote(wallet) + "/walletindex.html", "walletindex": urllib.parse.quote(wallet) + "/walletindex.html",
"complaints": html_status[survex_required or not plan_scanned or not elev_scanned or description_written] + html_survex_required[survex_required] + html_plan_scanned[plan_scanned] + html_elev_scanned[elev_scanned] + html_description_written[description_written] + html_qms_written[qms_written] } "complaints": html_status[survex_required or not plan_scanned or not elev_scanned or description_written] + html_survex_required[survex_required] + html_plan_scanned[plan_scanned] + html_elev_scanned[elev_scanned] + html_description_written[description_written] + html_qms_written[qms_written] }
for (wallet, cave, name, survex_required, plan_scanned, elev_scanned, description_written, qms_written) for (wallet, cave, name, survex_required, plan_scanned, elev_scanned, description_written, qms_written)
in wallets])}) in wallets])})
year_index_file.close() year_index_file.close()
for person, item_complaint_list in people.items(): for person, item_complaint_list in list(people.items()):
person_file = open(person + ".html", "w") person_file = open(person + ".html", "w")
person_file.write(html_person % {"person": person, "year": year, "timestamp": timestamp, person_file.write(html_person % {"person": person, "year": year, "timestamp": timestamp,
"wallets": reduce(operator.add, [html_person_wallet_entry % {"walletname": wallet, "wallets": reduce(operator.add, [html_person_wallet_entry % {"walletname": wallet,
"walletindex": urllib.quote(wallet) + "/walletindex.html", "walletindex": urllib.parse.quote(wallet) + "/walletindex.html",
"complaints": reduce(operator.add, "complaints": reduce(operator.add,
[html_items % complaint [html_items % complaint
for complaint for complaint
in complaints], in complaints],
"") } "") }
for wallet, complaints for wallet, complaints
in item_complaint_list], "") in item_complaint_list], "")
}) })
person_file.close() person_file.close()
#if __name__ == "__main__":
main()