expo/expoweb
1
0
Fork 0
mirror of https://expo.survex.com/repositories/expoweb/.git/ synced 2024-11-25 08:41:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

update to describe WSL2 minimally

Browse Source
This commit is contained in:
Philip Sargent 2021-10-21 23:58:01 +03:00
parent e6efe0d1db
commit db0e1e634f
2 changed files with 39 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
handbook/computing/winlaptop.html
View File

@ -20,7 +20,7 @@
<p>It used to be quite simple: software equivalents existed for Windows for everything we needed. Since Spring 2018, we unfortunately have to use ssh keys. This has simply added an extra step for most things, but has made one thing (rsync) really quite hard to get working.
<p>What makes this all much more tiresome is that Microsoft are introducing some cool new ways of interacting with Linux systems but these are new and need some effort to get configured correctly. See the end of this page for more details.
<ul>
<li><a href="#well">Things that already work well with a Windows laptop</a><br />
<li><a href="#well">Things that already work well with a Windows laptop</a><br />
- Editing the handbook webpages, typing up SVX files and transcribing the logbook, <br />
- Anything where the file-transfer to the expo server is via the git version control software
<li><a href="#problems">Things that cause problems</a><br />
@ -87,7 +87,7 @@ it downloads a <em>copy</em> of the contents of essentials.gpx and not a link.
<h3 id="hard">Things that are really quite involved</h3>
<p>The core problem is integrating the PuTTy key management software (pagent.exe) with a terminal window. We need a terminal window to run rsync as none of the packaged software (Filezilla, PuTTy) includes an rsync client.
<p>The solution we have now is to use WSL1 and to create another key, distinct from the PuTTy one, and to upload that key to the expo server. Because this is treating WSL as if it were a different machine requiring its own key quite separate from the Windows key, we expect this to continue to work when WSL2 becomes the default behaviour on Windows10.
<p>The solution we have now for rsync is to use WSL1 and to create another key, distinct from the PuTTy one, and to upload that key to the expo server. Because this is treating WSL as if it were a different machine requiring its own key quite separate from the Windows key, we expect this to continue to work when WSL2 becomes the default behaviour on Windows10.
<p>So on a machine with WSL enabled, create an ordinary cmd window and get into the WSL environment using the wsl command:<br />
<span style="font-family:monospace; size=x-small; background-color: lightgray">
D:\CUCC-Expo\expoweb\ <font color=red>wsl</font>
@ -130,21 +130,22 @@ The generated key is in the current directory and you need to move them to ~/.ss
Now finally you can use all the usual command line tools at yor wsl command line to communicate with the server with ssh, scp, rsync, such as:
<pre>
<tt>rsync -nazv --delete-after --prune-empty-dirs expo@expo.survex.com:expofiles/ expofiles</tt></pre>
<p>All of which is really a bit ridiculous if all you want to do is to use rsync on a Windows machine to sort out some photos.
<h3 id="bold">"Are you feeling lucky, punk"</h3>
<p>So here is the current wild frontier. Currently these are the ways to get a terminal window which might work:
<p>So here is the current wild frontier. Currently these are the ways to get a terminal window which you might expect to work:
<ul>
<li>cmd window - the old faithful going all the way back to MS-DOS. But no path to an rsync.exe executable as standard.
<li>PowerShell terminal window - nope, no rsync.
<li>bash window - installed by default when you install <A href="https://gitforwindows.org/">gitforwindows</a>. Unfortunately while this <a href="https://en.wikipedia.org/wiki/MinGW">MINGW32</a> setup includes a comamnd-line git executable it doesn't include rsync.
<li>bash window - installed by default when you install <A href="https://gitforwindows.org/">gitforwindows</a>. Unfortunately while this <a href="https://en.wikipedia.org/wiki/MinGW">MINGW32</a> setup includes a command-line git executable it doesn't include rsync.
<li>cygwin - a cmd terminal where you have downloaded and installed <a href="https://www.cygwin.com/">Cygwin</a>. Yes, if you have selected the rsync package you will be able to run the rysnc executable, but it won't have access to the cyptographic key so it can't connect to the expo server. Please feel free to work out how to make this work. A more recent, graphical variant is <a href="https://hackaday.com/2017/03/29/swan-better-linux-on-windows/">Swan</a>.
<li><img src="wsl.jpg" align="right" hspace="10"><a href="https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux">Windows Subsystem for Linux</a> aka WSL1 available on all Windows10 machines since the November 2019 update. (The first versions of WSL1 didn't do the ssh key-pair setup easily: <a href="https://blog.anaisbetts.org/using-github-credentials-in-wsl2/">"fairly annoying because of how out-to-lunch SSH Agent is"</a> but it works now.)
<li><img src="wsl.jpg" align="right" hspace="10"><a href="https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux">Windows Subsystem for Linux</a> aka WSL1 available on all Windows10 machines since November 2019.
<ul>
<li>WSL1 <a href="https://code.visualstudio.com/remote-tutorials/wsl/enable-wsl">Install it like this</a>. This does all we want. This works using a key generated by its own version of ssh-keygen if you follow the instructions above about putting it in the right place.
<li>WSL <a href="https://docs.microsoft.com/en-us/windows/wsl/install">Install it like this</a>. This does what we need. This works using a key generated by its own version of ssh-keygen if you follow the instructions above about putting it in the right place.
<li>WSL1: <a href="https://hackaday.com/2019/12/23/linux-fu-wsl-tricks-blur-the-windows-linux-line/"> Converting Windows paths to Linux paths and vice-versa</a>.
<li>WSL1 will shortly be made obsolete by <a href="https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux#WSL_2">WSL2</a> - the new wonderful system which will solve all our problems, make the tea and carry our gear up to topcamp. Allegedly.
<li>The <a href="https://docs.microsoft.com/en-us/windows/wsl/install-win10">2019 WSL2 release</a> includes a selection of complete Linux kernels. If you want to use this, then please do - and then write the handbook documentation too. But beware that it has <a href="https://docs.microsoft.com/en-us/windows/wsl/wsl2-index">two different modes</a> which behave differently.
<li>WSL1 is being made obsolete by <a href="https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux#WSL_2">WSL2</a> - the new wonderful system which will solve all our problems, make the tea and carry our gear up to topcamp. Allegedly. WSL2 solves the filenaming problems because it is a full virtual Linux running on a Linux filesystem (ext4) inside Windows. Note that WSL2 has hardware requirements that WSL1 does not have, so an old machine may have to use WSL1.
</ul>
<li><a href="https://www.theregister.co.uk/2019/12/18/multipass/">Canonical Multipass</a> - a completely different alternative to WSL: more isolation, more understandable behaviour (?)
@ -153,9 +154,9 @@ Now finally you can use all the usual command line tools at yor wsl command line
<h4>WSL1 tricks and tips</h4>
<p>WSL1 unfortunately introduces a wonderful new problem of file permissions. Every file on the Windows filesystem NTFS has a set of permissions managed by the filesystem. Every NTFS file that WSL knows about (if mounted with -o metadata) acquires a completely parallel set of file permissions that are fundamentally different things and are never in sync in any sense. <a href="https://devblogs.microsoft.com/commandline/chmod-chown-wsl-improvements/">All sorts of fun</a> results. This will be fixed by WSL2 (expected May 2020) which will have <a href="https://docs.microsoft.com/en-us/windows/wsl/wsl2-ux-changes">an entirely separate filesystem</a>, a Virtual Hardware Disk (VHD). Which will introduce a quite different set of interesting problems.
<p>WSL1 unfortunately introduces a wonderful new problem of file permissions. Every file on the Windows filesystem NTFS has a set of permissions managed by the filesystem. Every NTFS file that WSL knows about (if mounted with -o metadata) acquires a completely parallel set of file permissions that are fundamentally different things and are never in sync in any sense. <a href="https://devblogs.microsoft.com/commandline/chmod-chown-wsl-improvements/">All sorts of fun</a> results. This is fixed by WSL2 which has <a href="https://docs.microsoft.com/en-us/windows/wsl/wsl2-ux-changes">an entirely separate filesystem</a>, a Virtual Hardware Disk (VHD).
<p>If you are disturbed by the instructions to produce an entirely different key for WSL1 to use when your PC already has a perfectly good PuTTy key installed on the server, then you are right. It is inelegant. But it works, the instructions are shorter and there are fewer things that go wrong. If you are terribly offended by that then you can set your PC up to use one key shared between WSL and normal-Windows as described in <a href="https://devblogs.microsoft.com/commandline/sharing-ssh-keys-between-windows-and-wsl-2/">this October 2019 article</a>. (Don't set up a password on the key because then you don't need to install keychain.) But beware, this sort of thing goes out of date quite rapidly and WSL2 is looming.
<p>If you are disturbed by the instructions to produce an entirely different key for WSL to use when your PC already has a perfectly good PuTTy key installed on the server, then you are right. It is inelegant. But it works, the instructions are shorter and there are fewer things that go wrong. If you are terribly offended by that then you can set your PC up to use one key shared between WSL and normal-Windows as described in <a href="https://devblogs.microsoft.com/commandline/sharing-ssh-keys-between-windows-and-wsl-2/">this October 2019 article</a>. (Don't set up a password on the key because then you don't need to install keychain.) But beware, this sort of thing goes out of date quite rapidly.
<p>Links to useful articles to help you work this out for yourself:
<ul>
@ -168,7 +169,7 @@ Now finally you can use all the usual command line tools at yor wsl command line
</ul>
<p>When things go wrong when trying to sort this out, you may find these pages useful. I did.
<tt><a href="https://serverfault.com/questions/423075/rsync-failed-to-set-permission-operation-not-permitted">rsync - failed to set permission - operation not permitted</a>
<tt><a href="https://serverfault.com/questions/423075/rsync-failed-to-set-permission-operation-not-permitted">rsync - failed to set permission - operation not permitted</a></tt>
<tt><a href="https://blogs.msdn.microsoft.com/wsl/2017/04/18/file-system-improvements-to-the-windows-subsystem-for-linux/">file-system-improvements-to-the-windows-subsystem-for-linux/</a></tt>
<tt><a href="https://stackoverflow.com/questions/48328446/id-rsa-pub-file-ssh-error-invalid-format">id-rsa-pub-file-ssh-error-invalid-format</a></tt>
<tt><a href="https://stackoverflow.com/questions/42863913/key-load-public-invalid-format">key-load-public-invalid-format</a></tt>

45
handbook/computing/yourlaptop.html
View File

@ -86,11 +86,7 @@ but all the recommended software here is open source (and please don't install p
<li><a href="https://git-scm.com/download/gui/linux">GUI for git</a> - There is a choice on Linux but many people just use the command line.
<li>Your favourite text editor. gedit is installed on the <em>expo laptop</em> and has syntax highlighting for .svx files too. This works with <a href="https://survex.com/changes.html">all GtkSourceView-based editors</a> (thanks to Phil Withnall). Download a local copy of the syntax colourizer <a download href="/site_media/survex.lang">survex.lang</a>.
</ul>
<h4>For Linux and WSL (text mode only) Windows users only:</h4>
<ul>
<li>Survex is a package on debian and ubuntu but is not up to date. You need to attach to a ppa to get the latest survex:
<a href="https://survex.com/download.html?platform=debian">ubuntu and debian</a> download. This is also the version to use on WSL for doing 'cavern' command-line things in WSL (such as doing troggle imports of all the svx data)
</ul>
<h4>For Windows users only:</h4>
@ -156,32 +152,45 @@ to <a href="https://blog.shvetsov.com/2010/03/making-pageant-automatically-load-
Only machines which have done the key-pair setup process can do scp, sftp or rsync.
</p>
<h4>For Linux and WSL Windows users doing software development only:</h4>
<p>We run <var>troggle</var> on older, stable versions of software. So you will need to install similarly old software on your development machine. A default Debian install in October 2021 will install v11 (Bullseye) and python3.9 whereas we are still using v10 (Buster) and python3.7 for development. Buster is in <a href=" https://wiki.debian.org/LTS">LTS support</a> until June 2024. Everything should work of course, but be aware.
<ul>
<li>Basic stuff, e.g. after a fresh install of Debian, either natively or under <a href="winlaptop.html#bold">WSL on Windows</a>:
<code>sudo apt install survex<br>
sudo apt install python3<br>
sudo apt install python-is-python3<br>
sudo apt install python3-pip<br>
sudo apt install software-properties-common<br>
sudo add-apt-repository ppa:deadsnakes/ppa<br>
sudo apt install python3.7 python3.7-venv python3.7-doc binutils binfmt-support<br>
</code>
<li>Survex is a package on debian and ubuntu, and will have been installed as shown above; but it may not be up to date within the distribution. If you need a later version of survex than the pone packaged with debian then you need to attach to a ppa to get the latest survex:
<a href="https://survex.com/download.html?platform=debian">ubuntu and debian</a> download. This is also the version to use on WSL for doing 'cavern' command-line things in WSL (such as doing troggle imports of all the svx data)
</ul>
<h4>Cheap hack for Linux</h4>
<p>On a Linux machine, it will not be necessary once you have everything set-up correctly, but
a useful initial short-cut is to create 'mimic' symbolic links to directories to match those on the
server and the <em>expo laptop</em>.
(You should not rely on these as other people will have set up their machines differently.)
So, assuming that you are storing /expofiles/ on a different drive /mnt/f/ from everything else which is on /mnt/d/, and your machine has
no user <var>expo</var>:<pre><code>cd /home
So, assuming that you are storing /expofiles/ on a different drive /mnt/f/ from everything else which is on /mnt/c/, and your machine has
no user <var>expo</var>:<pre><code>d /home
sudo mkdir expo
cd expo
sudo ln -s /mnt/f/expofiles expofiles
sudo ln -s /mnt/d/CUCC-Expo/expoweb expoweb
sudo ln -s /mnt/d/CUCC-Expo/troggle troggle
sudo ln -s /mnt/d/CUCC-Expo/loser loser
sudo ln -s /mnt/c/EXPO/expoweb expoweb
sudo ln -s /mnt/c/EXPO/troggle troggle
sudo ln -s /mnt/c/EXPO/loser loser
sudo ln -s /mnt/c/EXPO/drawings drawings
sudo ln -s /mnt/c/EXPO/expowebcache expowebcache
sudo mkdir expowebcache/3d
cd ..
sudo ln -s /mnt/d/CUCC-Expo/expowebcache expowebcache
cd expowebcache
mkdir 3d
ls -tlA
ls -tlA expo
</code></pre>
where obviously you will be using your own locations for expofiles, expoweb, loser etc. instead of <var>/mnt/d/CUCC...</var> etc. This creates valid directory paths for, e.g. <var>/home/expo/loser</var> etc.
where obviously you will be using your own locations for expofiles, expoweb, loser etc. instead of <var>/mnt/d/EXPO...</var> etc. This creates valid directory paths for, e.g. <var>/home/expo/loser</var> etc.
</p>
<h3>Learning how to use this software</h3>
<ul>