spacemacs/layers/+distributions/spacemacs-docker/README.org

spacemacs-docker distribution

• Description
• Features:
• Drawbacks:
• Getting started(AFTER THIS PR IS MERGED)
  • For all platform
    • How to build:
    • How to use(with the default settings)
      • Xpra over SSH(GUI):
      • In webbrowser:
      • Mosh/SSH(terminal):
  • Optimizations
    • Data transfer via mmap(GNU/Linux)
    • Use host's network
    • Use host's file system
    • Use host's X11 session instead of SSH/Xpra(GNU/Linux)
• FAQ
  • How to set localtime/zone?
  • What layers support(currently) automatic dependency installation?
  • How to contribute?
  • What's up with the Dockerfile formatting?
  • Why not use Docker compose?
  • Why Xpra?
  • How to update?
  • How to get an older build(backup) of my Spacemacs?
  • Add answers to the questions from the PR
• Troubleshooting
  • Font size/screen is messed-up. Help!
  • Mosh fonts/space-line doesn't look right
  • I have a lag/picture is blurry/clipboard sharing doesn't work…
  • Ranger sometimes lags with the message "… process … something …"
    • localize the bug. Current status: heisenbug :(
  • Sometimes(rarely) I have strange Emacs slowdowns
    • Find out what's going one.

Description

Cross-platform, cross-device persistent emacs distribution with automatic dependency management and free cloud builds/backups.

Features:

1. automatic dependency(SDK, tools) installation for Spacemacs layers. Instead of everyone figuring out how to install/build a layer dependencies, we can write an installation script once and everyone will automatically get them.
2. Run the same Spacemacs installation across all platforms and devices separately or shared (useful for mentoring, collaborative editing etc.) DEMO - same Spacemacs instance shared between Win 10 Notebook and Ubuntu PC. On the both systems Spacemacs window(frame) behaves like if it was a native app it even has proper taskbar icon :) Copy-pasta and full-screen also work. You can even use Spacemacs in a web-browser! (DEMO) It us useful, for example, if you want to showcase Spacemacs or your personal setup with the zero install requirements.
3. Client can disconnect and reconnect(from the same or another machine) to a Spacemacs instance without disturbing its execution. DEMO. It means that you can start Spacemacs at work and simply connect to it from a handheld devices on the way back home to check rss feed/email and then from a home machine and Spacemacs will be in the exact same state as you left it.
4. You can run multiply Spacemacs instances on the same host. DEMO. They can be fully isolated or share only what you want them to share. Bandwidth, CPU, disk space quota also supported. It may be really useful for education. Imagine a classroom with a bunch of cheap Raspberry PI clients and one Spacemacs host(local or cloud based).
5. Pausing(freezing) - may be handy if you want to temporary free CPU resources, but don't want to kill your development environment.
6. Snapshots for host migration and potentially the fastest way to start Spacemacs based development environment, especially if you use stuff like JVM that needs time to "heat up" DEMO. For GNU/Linux it uses native "snapshoter" CRIU
7. Disposable sandboxed development environment. Its much harder to damager your actual OS and if your experiment gone wrong you can drop all changes.
8. Free cloud builds and storage - especially handy if your internet upload is much slower or expensive than download.
9. Since spacemacs-docker distribution uses common base (Emacs version, OS, packages) it is much easer to reproduce and fix bugs.
10. Docker uses layered filesystem so for updates you'll need to download only the changed layer. Also you can make multiply images that share the most of data so they require less disk space.

Drawbacks:

1. While performance with the GNU/Linux host seems to be equal(perceptually) to a locally installed Spacemacs, even with a Windows 10 client on a fairly ancient laptop(first gen. mobile i3). It's actual better, if you take into account code linter, compiler etc.). But if you host both client and server on such laptop you'll experience noticeable keyboard lag (probably due to low specs and a poor Hyper-V support).
2. You will need Docker on the server machine and(for the most cases) Xpra at the client + ssh. But if you don't mind some hacks and want to use Linux and run spacemacs-docker locally, then you only need Docker. As a bonus you'll get max speed.

Getting started(AFTER THIS PR IS MERGED)

For all platform

How to build:

• Install docker like normal ppl do or like if there is no tomorrow(don't forget sudo). Also you'll need Xpra (optionally).
• Fork the seed repo, read its README.org
• Register at DockeHub and create an automated builds from the fork. Also you may want to link it to spacemacs/onbuild repository - this way your Spacemacs will automatically rebuild each time spacemacs/onbuild is updated.

How to use(with the default settings)

Xpra over SSH(GUI):

• docker run -ti --rm -p 22:22 <YOU_NAME>/spacemacs it will attach Xpra output to the console, it you detach it Docker will kill and delete the container (and all changes). So to make it useful you need to mount your workspace. To do so add -v <...>/workspace:/mnt/workspace where <...> is the absolute path to your local workspace. Also you probably would like to make your layouts and recent files persistent: First create a folder on the host's file system where you want to store those files. For example /tmp/spacemacs-cache then docker run with -v /tmp/spacemacs-cache:/home/spacemacs/.emacs.d/.cache (Linux example)

• Then you can attach to it via Xpra:

    xpra attach --encoding=rgb --pulseaudio=no --compress=0\
    ssh:<NAME_FROM_/hooks/build>@localhost:14

  on Windows executable is called Xpra_cmd also you can use Xpra's GUI app but it's a good idea to fist run it with log/console output, it will tell you if your client misses python libraries.

• You can share your Spacemacs by adding --sharing=yes for each Xpra client xpra attach ... --sharing=yes ...
• Read Docker run reference and Xpra FAQ.

In webbrowser:

• Same as with Xpra over SSH docker run -ti --rm <YOU_NAME>/spacemacs but instead of -p 22:22 map -p 10000:10000 and add set container's environment variable SDMODE to webbrowser_insecure i.e: docker run ... -e SDMODE=webbrowser_insecure ... Also you'll need to set a password ... -e XPRA_PASSWORD=<YOUR_PASSWORD> ... Now you can use Spacemacs via webbrowser. URL: http://localhost:10000/index.html?encoding=png&password=<YOUR_PASSWORD> or use http://localhost:10000/connect.html if you want to configure keyboard layout, encoding, full-screen mode etc. To get proper full-screen you need to fist enter browser full-screen at this page and then connect to the Spacemacs. The mode is called "webbrowser_insecure" because it uses http protocol. So you should use it only for local network behind system firewall or Docker's one i.e don't map container's port 10000 to the host's port. Find out local IP of the container and use it instead of localhost. For the remote use you can add TLS proxy and may be wrap your setup into Docker compose for the convenience.

Mosh/SSH(terminal):

• Use Xpra over SSH(GUI) method to launch a new Spacemacs instance (or use existing one). Set container's environment variable SDMODE to ssh i.e: docker run ... -e SDMODE=ssh ... if you don't need Xpra.
• Then use ssh to connect ssh <NAME_FROM_/hooks/build>@localhost
• To use Mosh (which is much better way than ssh) you'll also need to map an UDP port docker run ... -p 60001:60001/udp ... then to connect: mosh -p 60001 <NAME_FROM_/hooks/build>@localhost emacs

Optimizations

Data transfer via mmap(GNU/Linux)

You can use Xpra mmap protocol to greatly reduce CPU usage, but only if the host and the client are on the same machine(file system). For this first create a folder on the host's file system where the mmap will be created. For example spacemacs-mmap (it should be owned by user) then same as with Xpra over SSH(GUI) but also mount the folder -v /tmp/spacemacs-mmap/:/tmp/spacemacs-mmap/ to attach run:

  TMPDIR=/tmp/spacemacs-mmap/ xpra attach --mmap=yes ...

the mmap option only works for GNU/Linux See change-log.

Use host's network

Instead of using -p to map individual ports, you can add --network=host to the docker run... it's not only more convenient way to run stuff, but also more efficient (and less secure). If you use docker for Mac/Windows you'll may need to figure out your docker machine IP and use it instead of localhost.

Use host's file system

Docker's file system is somewhat slower. If you have "hot" folders you can mount them to the host's filesystem. See Docker volumes.

Use host's X11 session instead of SSH/Xpra(GNU/Linux)

This is a hacky way to get absolute best performance. For it to work host's X11 session owner should have same UID as the spacemacs-docker user (both are 1000 be the default). Simply mount /tmp/.X11-unix and run Emacs directly without ssh and Xpra:

  docker run -ti --rm -v /tmp/.X11-unix:/tmp/.X11-unix\
   -e DISPLAY=$DISPLAY\
   --entrypoint=/usr/bin/emacs\
   -e NO_AT_BRIDGE=1\
   -e SHELL=/bin/bash\
   <YOUR_NAME>/spacemacs

FAQ

How to set localtime/zone?

Try docker run ... -v /etc/localtime:/etc/localtime:ro ... or docker run ... -e TZ=UTC ... - Replace UTC with your time zone.

What layers support(currently) automatic dependency installation?

See deps-installers/README.org

How to contribute?

You can contribute a dependency installation script for a currently unsupported layer or fix/extend existing one. See lib/README.org. Also you can improve libraries, docs.

What's up with the Dockerfile formatting?

Docker build logs are ugly anyway. At least we can keep Dockerfiles as readable as possible. Also Emacs-Lisp build helpers prints out executed command and its output (if any) so we don't need to dig into usual docker builder's wall of text.

Why not use Docker compose?

It will make spacemacs-docker setup much less "beginner friendly". Currently user's .spacemacs file serves similar role. During the build process(at DockerHub) builder reads .spacemacs file and reads layer related variables from Spacemacs to configures the installation or even modify the .spacemacs file. If will be impossible to do with the modular design. Also modularity may increase the maintenance cost of the distribution.

Why Xpra?

• Besides all the Xpra features outlined in the Features section. It also has a client for all major platforms (even Android) and due to its implementation it is likely to work in the same way with all the client variations, so it should be easier to maintain.

How to update?

• First get a new version docker pull <YOUR_NAME>/spacemacs

• Then run it. And if it is fine, remove the old one:

  1. Get image list docker images.
  2. Remove an image docker rmi <IMAGE>.

How to get an older build(backup) of my Spacemacs?

You can pull by the digest. Also you can use digest in the FROM Dockerfile instruction.

TODO Add answers to the questions from the PR

Troubleshooting

Font size/screen is messed-up. Help!

If you have more than one display/projector or other displaying device connected, Xpra may use DPI from the wrong device. You can specify DPI at the client with xpra attach --dpi=<DPI> ... and/or at the server side. See Seed project's Dockerfile.

Mosh fonts/space-line doesn't look right

You probably need to install NanumGothic and source-code-pro fonts

I have a lag/picture is blurry/clipboard sharing doesn't work…

Xpra has many setting that can be changed on the fly(some of them have hot-keys). While attached, click Xpra logo near the system clock on the taskbar(Windows) and make sure that the configs are right.

Ranger sometimes lags with the message "… process … something …"

TODO localize the bug. Current status: heisenbug :(

Sometimes(rarely) I have strange Emacs slowdowns

TODO Find out what's going one.