expoweb/handbook/computing/regular.html

<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>CUCC Expedition Handbook: Programmers manual</title>
<link rel="stylesheet" type="text/css" href="../../css/main2.css" />
</head>
<body>
<style>body { background: #fff url(/images/style/bg-system.png) repeat-x 0 0 }</style>
<h2 id="tophead">CUCC Expedition Handbook - Online systems</h2>
<h1>Expo Files Maintenance - Regular tasks</h1>

<h2>Regular operations</h2>


<p>This page <i>is a checklist</i> who are helping the survey data reduction process during and after expo. 
<p>All of these things should be checked <em>every week</em> for a couple of months after expo, and then every couple of months. Then again when preparing for the new expo in May.

<h3>Re-importing all the data</h3>
<p>We do this regularly during software maintenance to check that everything is still working. See the <a href="../troggle/trogimport.html">database reset</a> documentation. But it must be done: 
<ul>
<li>Whenever anybody has uploaded anything directly to the repos using git, without using the online forms: drawings, survey data (in :loser: repo) or handbook updates (in :expoweb:),
<li>Every couple of months, to keep things clean and honest (and to check that the semiautomated OS updates on the server haven't broken anything).
</ul>


<h3>Data entry TO-DO list</h3>
<p>A user-editable online to-do list for data management is now <a href="todo-data.html">part of the expo online systems</a>. Review this regularly to see what needs doing, and please *delete* jobs that have been done.

<h3>Wallets and Surveyscans</h3>
<h4>2020#00</h4>
<p> The  <b>#00</b> wallet directory (e.g. /2020/2020<b>#00</b>/ ) contains orphan files that have been found on the expo laptop in odd places, or have been scanned from bits of notebook found inside other documents. Keep an eye on it and re-file the contents as you discover what they are. 
<h4>EOL</h4>
<p>Run dos2unix to convert any uploaded Windows text files to the format expected by our software. e.g. <pre><code>cd expofiles/surveyscans
find . -not -type d -exec file "{}" ";" | grep CRLF >crlf.txt
`awk -F: '// {print "dos2unix \"" $1 "\""}' crlf.txt`</code></pre>
<p>Also a good idea to run on all of expofiles once every few years as many GPX exports 
from phones are a bit variable in how they do EOL characters.

<h3>Tunnel files (Drawings)</h3>

<h3>QMs</h3>
<p>As the caves get written up, run <a href="scriptsqms.html">svx2qm.py</a> and  <a href="scriptsqms.html">find-dead-qms.py</a> to check that the QMs have all be entered correctly into the svx files and that thecave descriptions have been updated with (a) the new open QMs and (b) the old closed QMs.

<h3>Survex files</h3>
Look at the <a href="svxvalid.html">valid SVX refs</a> page to check that new svx files properly reference the wallet folders, and create the wallet folder link back to the svx if the contents.json file in the wallet folder needs updating.

<h3>Folk</h3>
<p>During prep. for the new expo the folklist will be updated with all the new people expected, but after expo the mugshots and blurb text for the new people will need to be added. See <a href="folkupdate.html">folkupdate</a> for the procedure.  

<hr />
Annual tasks <a href="newyear.html">New expo year jobs</a>.<br />
Return to the main <a href="manual.html">online systems manual</a>.

<hr /></body>
</html>
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00			`<!DOCTYPE html>`
			`<html>`
			`<head>`
			`<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />`
			`<title>CUCC Expedition Handbook: Programmers manual</title>`
			`<link rel="stylesheet" type="text/css" href="../../css/main2.css" />`
			`</head>`
			`<body>`
Making handbook more consistent 2021-04-29 13:00:37 +01:00			`<style>body { background: #fff url(/images/style/bg-system.png) repeat-x 0 0 }</style>`
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00			`<h2 id="tophead">CUCC Expedition Handbook - Online systems</h2>`
Making handbook more consistent 2021-04-29 13:00:37 +01:00			`<h1>Expo Files Maintenance - Regular tasks</h1>`
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00
grey banner and lots of to-do fixes, to do items all updated from scribbled notes on printout 2020-04-22 19:37:10 +01:00			`<h2>Regular operations</h2>`
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00

grey banner and lots of to-do fixes, to do items all updated from scribbled notes on printout 2020-04-22 19:37:10 +01:00			`<p>This page <i>is a checklist</i> who are helping the survey data reduction process during and after expo.`
			`<p>All of these things should be checked <em>every week</em> for a couple of months after expo, and then every couple of months. Then again when preparing for the new expo in May.`
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00
Data import - documn pages links 2023-02-24 16:55:35 +00:00			`<h3>Re-importing all the data</h3>`
			`<p>We do this regularly during software maintenance to check that everything is still working. See the <a href="../troggle/trogimport.html">database reset</a> documentation. But it must be done:`
			`<ul>`
			`<li>Whenever anybody has uploaded anything directly to the repos using git, without using the online forms: drawings, survey data (in :loser: repo) or handbook updates (in :expoweb:),`
			`<li>Every couple of months, to keep things clean and honest (and to check that the semiautomated OS updates on the server haven't broken anything).`
			`</ul>`


grey banner and lots of to-do fixes, to do items all updated from scribbled notes on printout 2020-04-22 19:37:10 +01:00			`<h3>Data entry TO-DO list</h3>`
			`<p>A user-editable online to-do list for data management is now <a href="todo-data.html">part of the expo online systems</a>. Review this regularly to see what needs doing, and please delete jobs that have been done.`
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00
			`<h3>Wallets and Surveyscans</h3>`
grey banner and lots of to-do fixes, to do items all updated from scribbled notes on printout 2020-04-22 19:37:10 +01:00			`<h4>2020#00</h4>`
			`<p> The <b>#00</b> wallet directory (e.g. /2020/2020<b>#00</b>/ ) contains orphan files that have been found on the expo laptop in odd places, or have been scanned from bits of notebook found inside other documents. Keep an eye on it and re-file the contents as you discover what they are.`
			`<h4>EOL</h4>`
			`<p>Run dos2unix to convert any uploaded Windows text files to the format expected by our software. e.g. <pre><code>cd expofiles/surveyscans`
svx *ref validity checking stuff now working fine 2020-04-06 16:58:30 +01:00			`find . -not -type d -exec file "{}" ";" \| grep CRLF >crlf.txt`
grey banner and lots of to-do fixes, to do items all updated from scribbled notes on printout 2020-04-22 19:37:10 +01:00			`awk -F: '// {print "dos2unix \"" $1 "\""}' crlf.txt`</code></pre>
			`<p>Also a good idea to run on all of expofiles once every few years as many GPX exports`
			`from phones are a bit variable in how they do EOL characters.`
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00
			`<h3>Tunnel files (Drawings)</h3>`

documenting old scripts 2020-04-23 01:13:10 +01:00			`<h3>QMs</h3>`
			`<p>As the caves get written up, run <a href="scriptsqms.html">svx2qm.py</a> and <a href="scriptsqms.html">find-dead-qms.py</a> to check that the QMs have all be entered correctly into the svx files and that thecave descriptions have been updated with (a) the new open QMs and (b) the old closed QMs.`

Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00			`<h3>Survex files</h3>`
documenting old scripts 2020-04-23 01:13:10 +01:00			`Look at the <a href="svxvalid.html">valid SVX refs</a> page to check that new svx files properly reference the wallet folders, and create the wallet folder link back to the svx if the contents.json file in the wallet folder needs updating.`
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00
grey banner and lots of to-do fixes, to do items all updated from scribbled notes on printout 2020-04-22 19:37:10 +01:00			`<h3>Folk</h3>`
			`<p>During prep. for the new expo the folklist will be updated with all the new people expected, but after expo the mugshots and blurb text for the new people will need to be added. See <a href="folkupdate.html">folkupdate</a> for the procedure.`

Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00			`<hr />`
documenting old scripts 2020-04-23 01:13:10 +01:00			`Annual tasks <a href="newyear.html">New expo year jobs</a>.<br />`
Moving online systems overview and computing manual pages 2020-04-09 19:13:18 +01:00			`Return to the main <a href="manual.html">online systems manual</a>.`
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00
Making handbook more consistent 2021-04-29 13:00:37 +01:00			`<hr /></body>`
Generating more outline troggle documentation 2020-04-03 01:06:35 +01:00			`</html>`