expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2025-01-30 07:02:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

moving archaic stuff

Browse Source
This commit is contained in:
Philip Sargent 2019-07-17 14:44:38 +02:00
parent a0975c9418
commit 23f01ce82e
2 changed files with 63 additions and 39 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
handbook/manual.html
View File

@ -276,45 +276,8 @@ good.</p>
<p>This is generated by the script tablizebyname-csv.pl from the input file Surveys.csv</p>
<h3 id="automation">Automation on expo.survex.com</h3>
<p>Ths section is entirely out of date (June 2014), and awaiting deletion or removal</p>.
<p>The way things normally work, python or perl scripts turn CSV input into HTML for the data management system. Note that:</p>
<p>The CSV files are actually tab-separated, not comma-separated despite the extension.</p>
<p>The scripts can be very picky and editing the CSVs with microsoft excel has broken them in the past- not sure if this is still the case.</p>
<p>Overview of the automagical scripts on the expo data management system</p>
[Clearly very out of date is it is assuming the version control is svn whereas we changed to hg years ago.]
<pre>
Script location Input file Output file Purpose
/svn/trunk/expoweb/noinfo/make-indxal4.pl /svn/trunk/expoweb/noinfo/CAVETAB2.CSV many produces all cave description pages
/svn/trunk/expoweb/scripts/make-folklist.py /svn/trunk/expoweb/folk/folk.csv http://expo.survex.com/folk/index.htm Table of all expo members
/svn/trunk/surveys/tablize-csv.pl /svn/trunk/surveys/tablizebyname-csv.pl
/svn/trunk/surveys/Surveys.csv
http://expo.survex.com/expo/surveys/surveytable.html http://expo.survex.com/surveys/surtabnam.html
Survey status page: "wall of shame" to keep track of who still needs to draw which surveys
</pre>
<h3><a id="arch">Archived updates</a></h3>
<p>Since 2008 we have been keeping detailed records of all data management system updates in the version control system.
Before then we manually maintained <a href="../update.htm">a list of updates</a> which are now only of historical interest.
<p>A history of the expo website and software was published in Cambridge Underground 1996. A copy of this article <a href="c21bs.html">Taking Expo Bullshit into the 21st Century</a> is archived here.
<h2>The data management system conventions bit</h2>
<p>This is likely to change with structural change to the site, with style changes which we expect to implement and with the method by which the info is actually stored and served up.</p>
<p>... and it's not written yet, either :-)</p>
<ul>
<li>Structure</li>
<li>Info for each cave &ndash; automatically generated by <tt>make-indxal4.pl</tt></li>
<li>Contents lists &amp; relative links for multi-article publications like journals. Complicated by expo articles being in a separate hierarchy from journals.</li>
<li>Translations</li>
<li>Other people's work - the noinfo hierarchy.</li>
<li>Style guide for writing cave descriptions: correct use of boldface (<em>once</em> for each passage name, at the primary definition thereof; other uses of the name should be links to this, and certainly should not be bold.) </li>
</ul>
<hr />

65
handbook/website-history.html
View File

@ -127,11 +127,24 @@ data was split into separate repositories: the website,
troggle, the survey data, the tunnel data. Seagrass was turned off at
the end of 2013, and the site has been hosted by Sam Wenham at the
university since Feb 2014.
In 2018 we have 4 repositories, see <a href="manual.html#repositories">the website manual</a></p>.
<h3>2018</h3>
<p>In 2018 we have 4 repositories,
<ul>
<li><a href="http://expo.survex.com/repositories/home/expo/expoweb/graph">expoweb</a> - the handbook and data management system webpages, including generation scripts</li>
<li><a href="http://expo.survex.com/repositories/home/expo/troggle/graph/">troggle</a> - the database-driven part of the data management system - see <a href="troggle-ish.html">notes on troggle</a> for further explanation</li>
<li><a href="http://expo.survex.com/repositories/home/expo/loser/graph/">loser</a> - the survex survey data</li>
<li><a href="http://expo.survex.com/repositories/home/expo/tunneldata/graph/">tunneldata</a> - the tunnel (and therion) data and drawings</li>
</ul>
<p>In spring 2018 Sam, Wookey and Paul Fox updated the Linux version and the Django version (i.e. troggle) to
something vaguely acceptable to the university computing service and fixed all the problems that were then observed.
<h3>2019</h3>
<p>In early 2019 the university computing serviuce upgraded its firewall rules which took the
server offline completely. Wookey eventually managed to find us free space (a virtual machine)
on a debian mirror server somewhere in Leicestershire (we think).
@ -140,11 +153,59 @@ This move to a different secure server means that all ssh to the server now need
<p>At the beginning of the 2019 expo two repos had been moved from mercurial to git: troggle and drawings (formerly called tunneldata). The other two repos expoweb and loser remained on mercurial.
</div>
<h4>Wookey: 12 July 2019</h4>
<p>troggle has been migrated to git, and the old erebus and cvs branches (pre 2010) removed. Some decrufting was done to get rid of log files, old copies of embedded javascript (codemirror, jquery etc) and some fat images no longer used.
<p>
tunneldata has also been migrated to git, and renamed 'drawings' as it includes therion data too these days.
<p>
The loser repo and expoweb repo need more care in migration. Loser should have the old 1999-2004 CVS history restored, and maybe toms annual snapshots from before that, so ancient history can usefully be researched (sometimes useful). It's also a good idea to add the 2015, 2016 and 2017 ARGE data we got (in 2017) added in the correct years so that it's possible to go back to an 'end of this year' checkout and get an accurate view of what was found (for making plots and length stats). All of that requires some history rewriting, which is best done at the time of conversion.
<p>
Similarly expoweb is full of bloat from fat images and surveys and one 82MB thesis that got checked in and then removed. Clearing that out is a good idea. I have a set of 'unused fat blob' lists which can be stripped out with git-gilter. It's not hard to make a 'do the conversion' script, ready for sometime after expo 2019 has calmed down.
<hr>
<h3 id="automation">Automation on expo.survex.com</h3>
<p>Ths section is entirely out of date (June 2014), and moved here for historic interest</p>.
<p>The way things normally work, python or perl scripts turn CSV input into HTML for the data management system. Note that:</p>
<p>The CSV files are actually tab-separated, not comma-separated despite the extension.</p>
<p>The scripts can be very picky and editing the CSVs with microsoft excel has broken them in the past- not sure if this is still the case.</p>
<p>Overview of the automagical scripts on the expo data management system</p>
[Clearly very out of date is it is assuming the version control is svn whereas we changed to hg years ago.]
<pre>
Script location Input file Output file Purpose
/svn/trunk/expoweb/noinfo/make-indxal4.pl /svn/trunk/expoweb/noinfo/CAVETAB2.CSV many produces all cave description pages
/svn/trunk/expoweb/scripts/make-folklist.py /svn/trunk/expoweb/folk/folk.csv http://expo.survex.com/folk/index.htm Table of all expo members
/svn/trunk/surveys/tablize-csv.pl /svn/trunk/surveys/tablizebyname-csv.pl
/svn/trunk/surveys/Surveys.csv
http://expo.survex.com/expo/surveys/surveytable.html http://expo.survex.com/surveys/surtabnam.html
Survey status page: "wall of shame" to keep track of who still needs to draw which surveys
</pre>
<h3><a id="arch">Archived updates</a></h3>
<p>Since 2008 we have been keeping detailed records of all data management system updates in the version control system.
Before then we manually maintained <a href="../update.htm">a list of updates</a> which are now only of historical interest.
<p>A history of the expo website and software was published in Cambridge Underground 1996. A copy of this article <a href="c21bs.html">Taking Expo Bullshit into the 21st Century</a> is archived here.
<h2>The data management system conventions bit</h2>
<p>This is likely to change with structural change to the site, with style changes which we expect to implement and with the method by which the info is actually stored and served up.</p>
<p>... and it's not written yet, either :-)</p>
<ul>
<li>Structure</li>
<li>Info for each cave &ndash; automatically generated by <tt>make-indxal4.pl</tt></li>
<li>Contents lists &amp; relative links for multi-article publications like journals. Complicated by expo articles being in a separate hierarchy from journals.</li>
<li>Translations</li>
<li>Other people's work - the noinfo hierarchy.</li>
<li>Style guide for writing cave descriptions: correct use of boldface (<em>once</em> for each passage name, at the primary definition thereof; other uses of the name should be links to this, and certainly should not be bold.) </li>
</ul>
Return to<br>
<a href="onlinesystems.html">Website update</a><br>
<a href="expodata.html">Website developer information</a><br>
<hr>
</body>
</html>
</html>