Expo Systems To-Do List

How this works

Click twice on a sub-heading to reveal the individual to-do items. Click again to hide.
If a heading is in italics, then there are hidden items.

Click on the main "How this works" heading to toggle reveal/hide everything.

Printing this page uses a much smaller font. Complain to the nerd list if you don't like this behaviour.

Edit this page by clicking on the big blue button. It uses the same "Edit this page" function that you may have used before. Just be careful not to touch the code at the top and bottom of the page.

We are working on a more capable and easier-to-use system which will mimic how GitHub Issues do this sort of thing.

To see the "data maintenance" and "survey data-entry" to-do lists, go to the other TO-DO list

Module to-do texts

These are extracted directly from the todo = '''Thing to do ''' string at the top of each file of troggle code:

Process Documentation

Wallets and new-cave
  • Explain how to link a new cave into the other caves in troggle filesystem
  • Explain how to add photos with the correct URL format to sub-HTML files attached to New Caves
    • svx check
  • explain command line to run on newly typed survex file to check for format errors '$cavern xxxx.svx'
  • explain when to use svxtrace.py current scripts
  • Making an svx - explain how to make link to *ref to wallet folder
  • explain how to put stuff in the wallet folder and how to look at expofiles/photos/xml/
  • Add to user data manual how to upload to expofiles - in the "survey manual"
  • Add in description of *where* to put the drawings files

    • Wallets re-programming

    Things wallets.py might do in future
  • - integrate into troggle, remove as a distinct script
  • - relates to conversion of the XRF shell script into python/troggle too
  • - checking the cave number specified matches the folder for the .svx file,
  • - checking that the *ref: filed in the survex file is the same as the wallet nameS
  • - detecting whether there is a description or a list of QMs in the survex file,
  • - accepting a list of .svx files and not just one (a very common thing),
  • - checking the name of the cave against the cave number,
  • - checking whether the website page even exists for this cave,
  • - being more intelligent about .topo files and thus the lack of scan files,
  • - checking the date is in the recent past etc.

    • Surveys data entry documentation

    Write code to automatically extract ref info about wallets from tunnel xml files
  • This will replace :drawings:chk-xml.txt as used by :drawings:check-xml.sh
  • Do similar for Therion files when people start using #REF comments
    • 2019 surveyscans
  • edit all the content.json files
  • try to autocomplete the content.json by parsing .svx files
  • Put personal names into canonical form
    • 2014 contents.json
  • parse all the svx files to extract people, dates & svx references

    • Handbook

    Navigation - Context, Situational Awareness
  • No overall view of what there is: need a user-friendly diagrammatic overview sitemap. There are 4,869 files in :expoweb:
  • Not fixed by menus - breadcrumbs needed?
  • Diagrams of major sections: survey handbook, rigging handbook, computing handbook, Areas, cave data..
  • Chase Airoa to help fix the menus for phones
  • Look at gov.uk guidelines for inspiration

    • Areas and prospecting guides

    Areas (these are the sub-areas)
  • Downgrade the "areas" aspect of the documentation as we don't use these now
    Prospecting Guides & slippy map & GPS tracks
  • Redesign to be used as overlay on phone/tablet using essentials.gpx data

    • Photos

    Fix the BINS package (Wookey is looking for replacement)
    set up git on the BINS software? Fix base url sautret.org
    set up a cron / Makefile job to run BINS
    why is 'updatephotos' script in expofiles?
    Document the ~expo/webphotos/ directory and how it works
    Why does expofiles/photos/xml/ get created containing all the generated xml files? Fix this.
    galley thumbs etc.
    :expoweb:/galley/0.htm thumbnails not loading

    Laptop config.

    imagemagick convert
    Not working for PDFs on current version of xubuntu ! Fix & document. Also not working on WSL ubuntu. Needed to work with logbook images easily & to extract PDFs, JPEGs. Need to edit /etc/ImageMagick-6/policy.xml cromwell-intl.com/open-source/pdf-not-authorized.html

    System Documentation

    Document how we generate the SMK overall lengths and stats
    scripts and manual actions in cavern
    smklengths.sh script
    smklengsths.odt
    :loser:/docs/ files
    Questions.txt
    SMKlog.txt
    Troggle architecture
    Regenerate the diagram of the tables in the database & document
    Describe the parsing/import procedures so non-nerds can do it
    Surveyscans
    explain where the image files are referenced from: svx, tunnel/therion, and
    Be exhaustive in checking all the broken links between svx & wallets
    non-troggle scripts
    DOCUMENT them all - have we got ? //expofiles/unidentified_scripts/ !
    Makefile
    Get it working ! How many of other scripts should be in it ? Should we do data imports to schedule too ?

    Troggle

    QM links in cave descriptions ALL BROKEN
    There are HREF links to individual QMs in the cave description in e.g. 204 Swings and Roundabouts such as "At the bottom there is a climb down over boulders [C2001-204-45 C]." where the troggle URL fails completely.
    Better import error messages
    REPORT 'folder not recognised' errors into error log inside database -> "DataIssue" register
    Troggle parser/viewer
    Explain how expopages (formerly flatpages) work. refer to computing/hbmanual1.html
    Completely new design of QM management needed
    see QM status and redesign needed document
    jQuery version and remove all unused CSS and JS files
    jQuery used for Edit This Page very very VERY out of date. Reinstate TinyMCE too ?
    19 names don't work in folk/troggle
    there are 3 separate name parsers: in the folk script, and two in different bits of troggle.
    Django migrations documentation needed
    How to upgrade python version, django version, django-plugins versions, pip install and useful/essential tools such as python -Wall manage.py check -v 3.
    django-estensions usefulness. Explain/document why troggle still needs database migraitons even though we rebuild it afresh each time

    Troggle Refactoring

    Make parsers use generators to yield one line at a time
    Both recursive load and linear load parsers/survex could do this.
    In the logbooks parsing and cache loading.
    Replace assert() with dataIssue [NOTIFICATION pattern] or Exceptions
    https://martinfowler.com/articles/replaceThrowWithNotification.html
    Python documentation: Django's use of modern unobvious idioms
    [DECORATOR] pattern used for login-required pages
    Get all trogglistas to read https://ebookscart.com/effective-python-by-brett-slatkin-pdf-download/ - downloded to /expofiles/
    Get all trogglistas to read https://martinfowler.com/tags/refactoring.html
    Flatfiles is used to deliver CSS and JS for django admin pages,
    staticfiles django plugin broken, not needed so use flatfiles but fix awkradnesses such as /expofiles/ directory listing not working

    Expo Server

    document cgit installation and config
    as per Wookey on new server Debian 10
    document directory structure
    as per Julian's GitHub list - then close that issue. Use 'tee' to create ascii art for this.
    permissions
    do we need a cron job or has the group membership fix fixed it ? either way, document it.
    Makefile - and permissions.
    Apache & MySQL configuration
    DOCUMENT How to set up a server
    Fix and document permissions for cron jobs
    Document "anacron" / cron system on server
    Explain why Docker containers can be a maintenance nightmare unless built from a script
    /expofiles/ needs config , refer to apache/django docm.
    admin/doc/ BROKEN PAGE needs config - refer to django docm.

    This to-do-list gadget itself

    Status
    Make it only click once to change display state of the DT tag
    Eventual aim
    to replace GitHub CUCCexposurveyissues for non-tunnel, non-GIS issues.
    Try to automate simple functions in javascript
    to control this document in-place (tick, delete) instead of using the tinymce HTML editor in Django
    Maybe restructure whole page as JSON and use javascript editor to create (edit), re-order and demote items.
    if using tinymcs, use the INLINE mode.
    Implement comments on an issue, mimicking GitHub issues wiki
    Use javascript to skip over the _edit page and submit changes direct to server when updating. Needs direct POST and include the javascript in the posted content.
    checkboxes so that items can be ticked (which also does strike-through)
    promotion/demotion options?
    bin icon to delete an item? Better to have an archive process.
    Local WSL server
    Why is it using recuced-capability tinymce, a paths problem ?
    Italics management
    Put DL into italics when *any* DD is hidden
    Testing area..
    Demonstation text as initially read from disc
    See also Data maintenance to-do list
    Back to Online Systems manual