expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2026-02-25 21:55:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

Troggle UML class diagram and text

Browse Source
This commit is contained in:
Philip Sargent
2020-06-29 16:33:29 +01:00
parent f266691baf
commit 6403b7f211
7 changed files with 115 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
handbook/troggle/trogimport.html
View File

@@ -34,20 +34,15 @@ Usage is 'python databaseReset.py <command> [runlabel]'
and [runlabel] is an optional string identifying this run of the script
in the stored profiling data 'import-profile.json'
if [runlabel] is absent or begins with "F-" then it will skip the :memory: pass
caves and logbooks must be run on an empty db before the others as they
set up db tables used by the others.
</pre></code>
<p>On a clean computer with 16GB of memory and using sqlite a complete import takes about 20 minutes if nothing else is running.
On the shared expo server it can take a couple of hours if the server is in use
(we have only a share of it). On your
own computer, the first in-memory sqlite pass takes only about 6 minutes.
We do this so that typos and data-entry errors
are found quickly.
<p>On a clean computer with 16GB of memory and using sqlite a complete import takes about 10 minutes now if nothing else is running.
On the shared expo server it could take a couple of hours if the server was in use
(we have only a share of it).
<p>Here is an example of the output after it runs, showing which options were used recently and how long
each option took (in seconds)
<code><pre>
each option took (in seconds). <code><pre>
-- troggle.sqlite django.db.backends.sqlite3
** Running job Profile
** Ended job Profile - 0.0 seconds total.
@@ -63,8 +58,9 @@ survexblks (s) 1153.1 - 3917.0 1464.1 1252.9
tunnel (s) - - 25.5 - 23.1
scans (s) - - 52.5 - 45.9
</pre></code>
<p>The 'survexblks' option loads all the survex files recursively following the <var>*include</var>
statements. It can take a long time if memory is low and the operating system has to page a lot.
[This data is from May 2020 immediately after troggle had been ported from python2 to python3 but before the survex import was re-engineered. It now takes ~600s in total.]
<p>The 'survexblks' option loaded all the survex files recursively following the <var>*include</var>
statements. It takes a long time if memory is low and the operating system has to page a lot. This has now been rewritten.
<p>(That value of 0 seconds for QMs looks suspicious..)
<p>The file <var>import_profile.json</var> holds these historic times. Delete it to get
a clean slate.