expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2024-11-25 16:52:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

Documenting how to add a new year and new people in folk.csv and test that it works by running the databaseRest.py people command

Browse Source
This commit is contained in:
Philip Sargent 2020-02-21 00:23:59 +00:00
parent 9de86c739c
commit 98e4f9763e
5 changed files with 162 additions and 49 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
css/main2.css
View File

@ -83,8 +83,8 @@ div.clear {
clear: both; clear: both;
} }
tt, code { /*code bits*/ tt, code { /*code bits*/
background: #ccc; background: #eee;
font-size: 0.8em; font-size: 0.9em;
padding: 10px 20px; padding: 10px 20px;
display: block; display: block;
} }

59
handbook/computing/folkupdate.html Normal file
View File

@ -0,0 +1,59 @@
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>CUCC Expedition Handbook: People Update</title>
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
</head>
<body>
<h2 id="tophead">CUCC Expedition Handbook</h2>
<h1>The list of people on expo</h1>
<p>These are nerd-instructions. Find a nerd if you think this needs to be done.
<h3 id="import">The folk list</a></h3>
<p>This is part of the <a href="newyear.html">"new expo year"</a> preparations.
The folk.csv file is stored on the server under version control in the ::expoweb:: repository in
<code>expoweb/folk/folk.csv</code>
<p>Before expo starts the folk.csv file is updated.
<p>Edit folk/folk.csv, adding the new year to the end of the header line, a new column, with just a comma (blank cell) for people who weren't there, a 1 for people who are there, and a -1 for people who are there but didn't go caving.
This is adding an extra column to every line in the file.
<p>
Add new lines for new people, with the right number of columns.
<p>This process is tedious and error-prone and ripe for improvement especially as expo proceeds and youhave many more names to add. Adding a list of people from the bier book and their aliases (the name in brackets) would be a lot better, but some way to make sure that names match with previous years would be good.
<p>
Contrary to what you might expect we find that it is much more reliable to edit this using a text editor rather than a spreadsheet program.
<p>The nerd can log in using the 'expo' userid.
<p>The nerd needs to do this:
<ol>
<li>Upload the edited copy of folk.csv to the server using the version control system.
<li>Log in to the expo server and run the update script (see below for details)
<li>Watch any error messages scroll by.
<li>Look at the new folk list HTML file at <a href="http://expo.survex.com/folk/">http://expo.survex.com/folk/</a>
<li>Re-run the import script until you have got rid of all the import errors.
</ol>
<p>This is how you login and run the script:
<pre><code>ssh expo@expo.survex.com
cd folk
python ../scripts/make-folklist.py &lt;folk.csv &gt;index.htm
</code></pre>
<p>If there are no errors, you will just see this on your ssh session:
<pre><codeLoading expeditions
Loading personexpeditions
</code></pre>
<p>And you will see the result online at <a href="http://expo.survex.com/folk/">http://expo.survex.com/folk/</a>.
<p>It is recommended that the nerd does this locally on their own laptop first. It does not need to be an expo laptop, even a Windows 10 laptop is fine if the script is run in a WSL terminal. The only software you need to install is the default python package.
<hr />
</body>
</html>

13
handbook/computing/newyear.html
View File

@ -38,17 +38,8 @@
<li>create initial versions of the bierbook and the seshbook in expoweb/documents/bierbook (the final copies after expo will be scanned and put in e.g. expofiles/writeups/2020/ along with the scanned logbook). <li>create initial versions of the bierbook and the seshbook in expoweb/documents/bierbook (the final copies after expo will be scanned and put in e.g. expofiles/writeups/2020/ along with the scanned logbook).
<li>copy the contents of the previous year's expoweb/years/ folder, e.g. /2019/ to 2020/ and edit all the files therein. You should start with at least an index.html and a mission.html <li>copy the contents of the previous year's expoweb/years/ folder, e.g. /2019/ to 2020/ and edit all the files therein. You should start with at least an index.html and a mission.html
</ul> </ul>
<p>Edit folk/folk.csv, adding the new year to the end of the header <p>Edit folk/folk.csv according to the instructions in the <a href="folkupdate.html">Folk Update process</a>
line, a new column, with just a comma (blank </p>
cell) for people who weren't there, a 1 for people who were there, and
a -1 for people who were there but didn't go caving. Add new lines for
new people, with the right number of columns.</p>
<p>This proces is tedious and error-prone and ripe for improvement.
Adding a list of people, from the bier book, and their aliases would be
a lot better, but some way to make sure that names match with previous
years would be
good.</p>
Return to the main <a href="../manual.html">online systems manual</a>. Return to the main <a href="../manual.html">online systems manual</a>.

130
handbook/logbooks.html
View File

@ -1,5 +1,7 @@
<!DOCTYPE html>
<html> <html>
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>CUCC Expedition Handbook: The Website</title> <title>CUCC Expedition Handbook: The Website</title>
<link rel="stylesheet" type="text/css" href="../css/main2.css" /> <link rel="stylesheet" type="text/css" href="../css/main2.css" />
</head> </head>
@ -7,32 +9,54 @@
<h2 id="tophead">CUCC Expedition Handbook</h2> <h2 id="tophead">CUCC Expedition Handbook</h2>
<h1>Logbooks</h1> <h1>Logbooks</h1>
<p>As soon as possible after a trip finishes, a <b>hand-written write-up</b> of the trip is made in the nearest logbook: <p>As soon as possible after a trip finishes, a <b>hand-written write-up</b> of the trip is made in the nearest logbook:
the base camp logbook or the top camp logbook. All these logbook entries are then typed into a laptop (often the expo laptop) the base camp logbook or the top camp logbook.
which is then synchronised the version control system.
<p>Recent logbooks:
<ul> <ul>
<li><a href="../years/2019/logbook.html">2019</a> <li><a href="#why">Why</a> the logbook is so important
<li><a href="../years/2018/logbook.html">2018</a> <li><a href="#type">Typing on the <em>Expo Laptop</em></a>
<li><a href="../years/2017/logbook.html">2017</a> <li><a href="#another">Typing on another laptop</a>
<li><a href="#format">Formatting</a> a typed logbook entry
</br></br>
<li><a href="#import">Importing</a> the logbook into troggle (nerds only) - error checking
<li><a href="#history">Historical</a> logbookformat (nerds only)
</ul> </ul>
<h3 id="why">Why the logbook is important</h3>
<p>The logbook writeup is the oldest and most basic way of recording your trip but it must not be neglected. This is also where you put <p>The logbook writeup is the oldest and most basic way of recording your trip but it must not be neglected. This is also where you put
your speculations and ideas for what looks promising and what is obvious but doesn't go: things that are vital to future expoers. And please, please your speculations and ideas for what looks promising and what is obvious but doesn't go: things that are vital to future expoers. And please, please
do lots of sketches in the logbook. do lots of sketches in the logbook.
<p>If you are at basecamp, then it is an excellent idea to <p>Always, always write the date, <em>your</em> name and the names of other people involved at the top of each entry.
<b>type your logbook trip report</b> instead of writing it by hand - see <a href=#type">below</a>. If you mention a cave location, please also write down the cave name somewhere. In 10 years no-one will know
where "Lemon Snout" is.
<p>The contents of both the topcamp logbook and the basecamp logbook are typed into the same
"logbook.html" file for archiving. The drawings are scanned and stored in the same place, and hand-edited
into the logbook.html file after expo finished.
<p>If this is all new to you, please now read <a href="datamgt.html">Cave data management</a>, <p>If this is all new to you, please now read <a href="datamgt.html">Cave data management</a>,
and <a href="survey/why.htm">why we make surveys</a> and then the and <a href="survey/why.htm">why we make surveys</a> and then the
<a href="survey/index.htm">Survey Handbook</a> <a href="survey/index.htm">Survey Handbook</a>
<p>If you are at basecamp, then it is an excellent idea to
<b>type the text of your logbook trip report</b> instead of writing it by hand - see <a href="#type">below</a>. But still do lots of drawings in the paper logbook.
<p>The contents of both the topcamp logbook and the basecamp logbook are typed into the same
"logbook.html" file for future use in tracking down leads and surveys. The drawings are scanned and stored in the same place, and hand-edited
into the logbook.html file after expo finished. The typed notes are uploaded into the expo server database and
correlated with survey data done on the same day or by the same people.
<p>Recent logbooks:
<ul>
<li><a href="../years/2018/logbook.html">2018</a>
<li><a href="../years/2017/logbook.html">2017</a>
</ul>
<p>All these logbook entries are then typed into a laptop (often the expo laptop)
which is then synchronised the version control system.
<p>The result is a webpage reporting who did what and what was done by whom on expo,
e.g. see <a href="http://expo.survex.com/expedition/2018">the 2018 expo report</a>.
<h3 =id="type">Typing just your trip report (at the <i>Expo Laptop</i>)</h3> <h3 =id="type">Typing just your trip report (at the <i>Expo Laptop</i>)</h3>
<p>If you are at basecamp, then it is an excellent idea to type up your logbook trip report. <p>If you are at basecamp, then it is an excellent idea to type up your logbook trip report.
You can then print this and stick it in the logbook, adding any sketches by hand. You can then print this and stick it in the logbook, adding any sketches by hand.
@ -60,11 +84,13 @@ Then start it up again from the vertical icon menu as before.
<p>Now type in your trip report using whatever format you like, but please leave a blank line between paragraphs. <p>Now type in your trip report using whatever format you like, but please leave a blank line between paragraphs.
<h3>Adding your trip to the logbook online file</h3> <h4>Adding your trip to the logbook online file</h4>
<p>If you are using the <em>expo laptop</em> just edit this file: <p>If you are using the <em>expo laptop</em> just edit this file (if you are on expo in 2025):
<pre>
/home/expo/expoweb/years/2018/logbook.html <code>
</pre> /home/expo/expoweb/years/2025/logbook.html
</code>
copy the format you can see other people have used; copy the format you can see other people have used;
and other people will take care of synchronising it with the version control system. and other people will take care of synchronising it with the version control system.
@ -83,46 +109,84 @@ ensure that the updates from all the people entering trip data are OK and don't
<p>Logbooks are typed up and kept in the [expoweb]/years/[nnnn]/ directory as 'logbook.html'.</p> <p>Logbooks are typed up and kept in the [expoweb]/years/[nnnn]/ directory as 'logbook.html'.</p>
<h3>Recommended procedure (on another machine)</h3> <h3 id="another">Typing on another machine</h3>
<p> <p>
Rather than editing logbook.html when you type up your trip, it is a much better Rather than editing logbook.html when you type up your trip, it is a much better
idea to type up <i>just your trip(s)</i> in a separate file, e.g. "logbook-mynewtrip.txt", and email it to a nerd if you are sitting at a different laptop. idea to type up <i>just your trip(s)</i> in a separate file, e.g. "logbook-mynewtrip.txt", and email it to a nerd if you are sitting at a different laptop.
<h3>Format of the online logbooks</a></h3> <h3 id="format">Format of the online logbooks</a></h3>
<p>Do whatever you like to try and represent the logbook in html. The only rigid structure is the markup to allow troggle to parse the files into 'trips':</p> <p>Do whatever you like to try and represent the logbook in html. The only rigid structure is the markup to allow troggle to parse the files into 'trips':</p>
<pre> <code><pre>
&lt;div class="tripdate" id="t2007-07-12B"&gt;2007-07-12&lt;/div&gt; &lt;div class="tripdate" id="t2007-07-12B"&gt;2007-07-12&lt;/div&gt;
&lt;div class="trippeople"&gt;&lt;u&gt;Jenny Black&lt;/u&gt;, Olly Betts&lt;/div&gt; &lt;div class="trippeople"&gt;&lt;u&gt;Jenny Black&lt;/u&gt;, Olly Betts&lt;/div&gt;
&lt;div class="triptitle"&gt;Top Camp - Setting up 76 bivi&lt;/div&gt; &lt;div class="triptitle"&gt;Top Camp - Setting up 76 bivi&lt;/div&gt;
&lt;div class="timeug"&gt;T/U 10 mins&lt;/div&gt; &lt;div class="timeug"&gt;T/U 10 mins&lt;/div&gt;</pre></code>
</pre>
<p>Note that the ID's must be unique, so are generated from 't' plus the trip date plus a,b,c etc. <p>Note that the ID's must be unique, so are generated from 't' plus the trip date plus a,b,c etc.
when there is more than one trip on a day.</p> when there is more than one trip on a day.</p>
<p>T/U stands for "Time Underground" in hours (6 minutes would be "0.1 hours"). <p>T/U stands for "Time Underground" in hours (6 minutes would be "0.1 hours").
<hr /> <hr />
<h3>Historical logbooks</h3> <h3 id="history">Historical logbooks</h3>
<p>Older logbooks (prior to 2007) were stored as logbook.txt with just a bit of consistent markup to allow troggle parsing.</p> <p>Older logbooks (prior to 2007) were stored as logbook.txt with just a bit of consistent markup to allow troggle parsing.</p>
<p>The formatting was largely freeform, with a bit of markup ('===' around header, bars separating date, <place> - <description>, and who) which allows the troggle import script to read it correctly. The underlines show who wrote the entry. There is also a format for time-underground info so it can be automagically tabulated.</p> <p>The formatting was largely freeform, with a bit of markup ('===' around header, bars separating date, <place> - <description>, and who) which allows the troggle import script to read it correctly. The underlines show who wrote the entry. There is also a format for time-underground info so it can be automagically tabulated.</p>
<p>So the format should be:</p> <p>So the format should be:</p>
<pre> <code>
===2009-07-21|204 - Rigging entrance series| Becka Lawson, Emma Wilson, Jess Stirrups, Tony Rooke=== ===2009-07-21|204 - Rigging entrance series| Becka Lawson, Emma Wilson ===
</br>
&lt;Text of logbook entry&gt; &#123;Text of logbook entry&#125;
</br>
T/U: Jess 1 hr, Emma 0.5 hr T/U: Jess 1 hr, Emma 0.5 hr
</pre> </code>
<h3 id="import">Importing the logbook into troggle</a></h3>
<p>This is usually done after expo but it is in excellent idea to have a nerd do this a couple of times during expo to discover problems while the people are still around to ask.
<p>The nerd needs to login to the expo server using <em>their own userid</em>, not the 'expo' userid. The nerd also needs to be in the group that is allowed to do 'sudo'.
<p>The nerd needs to do this:
<ol>
<li>Look at the list of pre-existing old import errors at </br> <a href="http://expo.survex.com/admin/core/dataissue/">http://expo.survex.com/admin/core/dataissue/</a> </br>
The nerd will have to login to the troggle management console to do this, not just the usual troggle login.
<li>You need to get the list of people on expo sorted out first. </br>
This is documented in the <a href="computing/folkupdate.html">Folk Update</a> process.
<li>Log in to the expo server and run the update script (see below for details)
<li>Watch the error messages scroll by, they are more detailed than the messages archived in the old import errors list
<li>Edit the logbook.html file to fix the errors. These are usually typos, non-unique tripdate ids or unrecognised people. Some unrecognised people will mean that you have to fix them using the <a href="computing/folkupdate.html">Folk Update</a> process first.
<li>Re-run the import script until you have got rid of all the import errors.
<li>Pat self on back. Future data managers and people trying to find missing surveys will worship you.
</ol>
<p>The procedure is like this. It will be familiar to you because
you will have already done most of this for the <a href="computing/folkupdate.html">Folk Update</a> process.
<pre><code>ssh {youruserid}@expo.survex.com
cd ~expo
cd troggle
sudo python databaseReset.py logbooks
</code></pre>
<p>It will produce a list of errors like this, starting with the most recent logbook which will be the one for the expo you are working on.
You can abort the script (Ctrl-C) when you have got the errors for the current expo that you are going to fix
<pre><code>Loading Logbook for: 2017
- Parsing logbook: 2017/logbook.html
- Using parser: Parseloghtmltxt
Calculating GetPersonExpeditionNameLookup for 2017
- No name match for: 'Phil'
- No name match for: 'everyone'
- No name match for: 'et al.'
("can't parse: ", u'\n\n&lt;img src="logbkimg5.jpg" alt="New Topo" /&gt;\n\n')
- No name match for: 'Goulash Regurgitation'
- Skipping logentry: Via Ferata: Intersport - Klettersteig - no author for entry
- No name match for: 'mike'
- No name match for: 'Mike'</code></pre>
<hr /> <hr />
<ul id="links">
<li><a href="index.htm">Expedition Handbook</a>
</ul>
</body> </body>
</html> </html>

5
years/2018/logbook.html
View File

@ -1617,8 +1617,8 @@ Needs surveying and tagging.
<p>Removed water-collection poly-sheet from 115 ent. Now only has 9 litres of water and some flapjack & museli in it. <p>Removed water-collection poly-sheet from 115 ent. Now only has 9 litres of water and some flapjack & museli in it.
<div class="timeug">T/U: 0.1 hours</div> <div class="timeug">T/U: 0.1 hours</div>
<hr />
</hr>
<div class="tripdate" id="t2018-08-02b">2018-08-02</div> <div class="tripdate" id="t2018-08-02b">2018-08-02</div>
<div class="trippeople"><u>Jon Arne Toft</u>, Haydon</div> <div class="trippeople"><u>Jon Arne Toft</u>, Haydon</div>
<div class="triptitle">Homecoming Lizard King</div> <div class="triptitle">Homecoming Lizard King</div>
@ -1638,12 +1638,11 @@ From a ledge, a 4-bolt traverse pushed the continuation against the draft,into a
(larger) rift that would require bolting. (larger) rift that would require bolting.
<p> Further leads in the opposite direction or further down the rift (BIG RIFT!). <p> Further leads in the opposite direction or further down the rift (BIG RIFT!).
Alternatively the opposite direction of the phreatic tube from the Lizard King. Alternatively the opposite direction of the phreatic tube from the Lizard King.
Lotsof bolting, were were very cold. Lots of bolting, were were very cold.
Haydon says he was a 2 on a 1-10 scale where 1 is frozen solid.I was also very cold. Haydon says he was a 2 on a 1-10 scale where 1 is frozen solid.I was also very cold.
In other words, an altogether excellent pushing front with a brilliant windchill effect. In other words, an altogether excellent pushing front with a brilliant windchill effect.
<div class="timeug">T/U: 5 hours</div> <div class="timeug">T/U: 5 hours</div>
</hr>
<hr /> <hr />
<div class="tripdate" id="t2018-08-02f">2018-08-02</div> <div class="tripdate" id="t2018-08-02f">2018-08-02</div>