make essentials updates, diagram and links

handbook/troggle/make-essentialsREADME.txt

@@ -1,3 +1,33 @@
+New in 2025:
+============
+We now have the boundaries for all the kataster areas in a shapefile (again), we have re-generated GPX files for both 1623 and 1626 as GPX files. Copies are in
+expoweb/documents/boundaries/
+1623.gpx
+1626.gpx
+
+The original shapefile data is in the files 
+expofiles/qgis_resources/boundaries
+4_Teilgruppen_M31_2021_01 .shp. ..shx, .dbf etc etc
+
+Download ogr2ogr and run this to convert the whole lot to a single 26 MB GPX files.
+ 
+$ ogr2ogr -f GPX -nlt MULTILINESTRING -t_srs EPSG:4326 -s_srs EPSG:32633 boundaries.gpx 4_Teilgruppen_UTM_33_N_2021_01.shp -dsco GPX_USE_EXTENSIONS=YES
+
+The problem is that the shapefile uses polygons, and is in UTM33N, whereas GPX files are just polylines and are in WGS84: hence all the command line parameters.
+
+The 1623.gpx file was extracted simply by editing it out of the 26 MB gpx file.. It was then run through GPSprune to add the  SRTM altitude data.
+
+Now we can re-engineer the process for making essentials2025.gpx from scratch, without having to bounce back and forth between survex and other formats.
+
+
+
+Status in 2024
+==============
+In 2024 we manually created essentials2024.gpx
+and then updated it to include the deviation around the seilbahn works
+at the end of the car park to make essentials2024divert.gpx
+
+
 The folders for each year are for general gpx files from recorded tracks, 
 which will be included as surface legs in all.svx.
 gpx_publish is for creating exportable files to personal 
@@ -31,12 +61,19 @@ the year directory following the 2018 example.  This is the place to rename
 stations to follow normal conventions and to mark waypoints as entrances,
 etc.  The file does not need to be present.
 
-4. Open a terminal in this directory loser/ and run 
+4. If you have no gpx files for the current year, copy the 'dummy.gpx' file
+from the 2024 year folder.
+
+5. Open a terminal in this directory loser/ and run 
 the make_svx.sh script like this: bash gpx/make_svx.sh
 
-5. Run "cavern all.svx" in loser/gpx/ then "aven all.3d".  Check that the
+6. Run "cavern all_gpx.svx" in loser/gpx/ then "aven all_gpx.3d".  Check that the
 tracks appear as expected when fixed points / surface surveys are enabled.
 
-6. Commit everything to git and push.
+7. cd /loser then run gpx/make_essentials.sh
+
+8. Commit everything to git and push.
+
+--HOWEVER the essentials.gpx created by this inserts TRKSEG around every pair 
+of points, so this is UNUSABLE on OSMand.
 
-7. run gpx/make_essentials.sh

handbook/troggle/scriptscurrent.html

@@ -95,14 +95,28 @@ and run
 </ul>
 
 <h4 id="gps">make_essentials.sh</h4>
-<p>In :loser:/gpx/
+
+<div class="onright">
+<figure>
+  <a href="/expofiles/qgis_resources/gpx_publish_process.jpg">
+    <img src="gpx_publish_process-small.jpg"  /></a>
+  <figcaption><br>Click for enlarged view
+  </figcaption>
+  </figure>
+</div>
+<p>This needs re-writing because the process is highly confusing: see the "documentation" in the diagram on the right. ALSO the 2024 process created a GPX track which was unusable in OSMand (our preferred GPS app) because it put every point into a separate <var>trkseg</var>.
+  <p>In :loser:/gpx/ [2024 text]
 <p>Makes essentials.gpx - see <a href="../essentials.html">GPS on expo</a>. 
 This used to require the gpx2survex program (written in OCAML) but now doesn't (since 2023).
 Get the OCAML file from GitHub <a href="https://github.com/mshinwell/gps2survex">https://github.com/mshinwell/gps2survex;</a>
 <p>Read the <a href="make-essentialsREADME.txt">README</a> file in :loser:/gpx/y.
 <p>Someone needs to document this and and make_svx.sh properly.
+<ul>
   
 <h4 id="surface">gpx2survex and make_svx.sh</h4>
+  [to be re-written in 2025]
+
+  
 <p>In :loser:/gpx/
 <p>Regenerates the surface tracks as survex files from GPS .gpx files. Also requires the gpx2survex program.
   <p>We used to use the OCAML program gpx2survex but we now also have a python equivalent gpx2survex.py which is used by make_svx2.sh This is part of the make_essentials generation process.
@@ -113,13 +127,17 @@ Get the OCAML file from GitHub <a href="https://github.com/mshinwell/gps2survex"
       <p>Documented, by <var>man survexport, survexport --help</var>,
         and in the Survex manual on <a href="https://survex.com/docs/manual/survexport.htm">survexport</a>.
         
-        
+<h4 id="ogr2ogr">ogr2ogr</h4> 
+  If you need to refresh everything from scratch, e.g. if the kataster boundary data shapefile is updated (unlikley), then download <a href="https://gdal.org/en/stable/download.html">GDAL</a> which contains the ogr2ogr2 executable and can convert to gps:
+  <a href="https://gdal.org/en/stable/drivers/vector/gpx.html">format gpx</a>. The command line to do this is:<br>
+<code>ogr2ogr -f GPX -nlt MULTILINESTRING -t_srs EPSG:4326 -s_srs EPSG:32633 boundaries.gpx 4_Teilgruppen_UTM_33_N_2021_01.shp -dsco GPX_USE_EXTENSIONS=YES
+  </code>
   
 <h4 id="ocaml">convert_shx.ml</h4>
-<p>Not quite obsolete, but nearly.
+<p>Obsolete. We do not need this as we use ogr2ogr on the command line to generate the boundaries as a GPX file from the shapefile (.shp) format. This only needs to be done once.
 <div style="margin-left: 5%">
 <p>Unusually, this is in the <var>:loser:</var> repository, in :loser:/fixedpoints/scripts/convert_shx/
-  <p>We think this turns a shapefile which holds the coordinates of the 1623, 1624 boundaries into GPX. But we have mislaid the shapefile containing this vital data. 
+  <p>We think this turns a shapefile which holds the coordinates of the 1623, 1624 boundaries into GPX. <strike>But we have mislaid the shapefile containing this vital data.</strike> We now have the shapefile again.
 <p>It runs  <var>ogr2ogr -f csv -lco GEOMETRY=AS_WKT outputfile inputfile</var> and then extensively post-processes the shapefile output.
 It is written in OCAML. Therefore it must be Mark Shinwell's responsibility.
 <p><a href="https://gdal.org/programs/ogr2ogr.html">ogr2ogr</a> is a file conversion utility. 
@@ -127,7 +145,7 @@ It seems to be being run to convert CSV files into something else. The "shx" par
 <a href="https://docs.fileformat.com/gis/shx/">shapefile index format</a>.
 <p>We suspect this was part of the production process for originally making essentials.gpx, but we don't need it as we now have the boundary data in other formats.
   </div>
-
+</ul>
 <h4>svx2qm.py, tablize-qms.pl, find-dead-qms.py,qmreader.pl</h4>
 See the entire page devoted to the various 
 <a href="scriptsqms.html">QM scripts</a>.