Merge branch 'master' into staging

1 files changed, 315 insertions, 65 deletions

diff --git a/doc/guix.texi b/doc/guix.texi
index 7381c2c6ba..3b4ba487ad 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -453,7 +453,7 @@ If your host distro uses the systemd init system, this can be achieved
 with these commands:
 
 @example
-# cp ~root/.guix-profile/lib/systemd/system/guix-daemon.service \
+# ln -s ~root/.guix-profile/lib/systemd/system/guix-daemon.service \
         /etc/systemd/system/
 # systemctl start guix-daemon && systemctl enable guix-daemon
 @end example
@@ -461,7 +461,7 @@ with these commands:
 If your host distro uses the Upstart init system:
 
 @example
-# cp ~root/.guix-profile/lib/upstart/system/guix-daemon.conf /etc/init/
+# ln -s ~root/.guix-profile/lib/upstart/system/guix-daemon.conf /etc/init/
 # start guix-daemon
 @end example
 
@@ -567,6 +567,12 @@ guix import}).  It is of
 interest primarily for developers and not for casual users.
 
 @item
+@c Note: We need at least 0.10.2 for 'channel-send-eof'.
+Support for build offloading (@pxref{Daemon Offload Setup}) depends on
+@uref{https://github.com/artyom-poptsov/guile-ssh, Guile-SSH},
+version 0.10.2 or later.
+
+@item
 When @url{http://zlib.net, zlib} is available, @command{guix publish}
 can compress build byproducts (@pxref{Invoking guix publish}).
 @end itemize
@@ -814,9 +820,11 @@ available on the system---making it much harder to view them as
 
 @cindex offloading
 @cindex build hook
-When desired, the build daemon can @dfn{offload}
-derivation builds to other machines
-running Guix, using the @code{offload} @dfn{build hook}.  When that
+When desired, the build daemon can @dfn{offload} derivation builds to
+other machines running Guix, using the @code{offload} @dfn{build
+hook}@footnote{This feature is available only when
+@uref{https://github.com/artyom-poptsov/guile-ssh, Guile-SSH} is
+present.}.  When that
 feature is enabled, a list of user-specified build machines is read from
 @file{/etc/guix/machines.scm}; every time a build is requested, for
 instance via @code{guix build}, the daemon attempts to offload it to one
@@ -832,16 +840,18 @@ The @file{/etc/guix/machines.scm} file typically looks like this:
 (list (build-machine
         (name "eightysix.example.org")
         (system "x86_64-linux")
+        (host-key "ssh-ed25519 AAAAC3Nza@dots{}")
         (user "bob")
-        (speed 2.))    ; incredibly fast!
+        (speed 2.))     ;incredibly fast!
 
       (build-machine
         (name "meeps.example.org")
         (system "mips64el-linux")
+        (host-key "ssh-rsa AAAAB3Nza@dots{}")
         (user "alice")
         (private-key
          (string-append (getenv "HOME")
-                        "/.lsh/identity-for-guix"))))
+                        "/.ssh/identity-for-guix"))))
 @end example
 
 @noindent
@@ -875,31 +885,54 @@ The user account to use when connecting to the remote machine over SSH.
 Note that the SSH key pair must @emph{not} be passphrase-protected, to
 allow non-interactive logins.
 
+@item host-key
+This must be the machine's SSH @dfn{public host key} in OpenSSH format.
+This is used to authenticate the machine when we connect to it.  It is a
+long string that looks like this:
+
+@example
+ssh-ed25519 AAAAC3NzaC@dots{}mde+UhL hint@@example.org
+@end example
+
+If the machine is running the OpenSSH daemon, @command{sshd}, the host
+key can be found in a file such as
+@file{/etc/ssh/ssh_host_ed25519_key.pub}.
+
+If the machine is running the SSH daemon of GNU@tie{}lsh,
+@command{lshd}, the host key is in @file{/etc/lsh/host-key.pub} or a
+similar file.  It can be converted to the OpenSSH format using
+@command{lsh-export-key} (@pxref{Converting keys,,, lsh, LSH Manual}):
+
+@example
+$ lsh-export-key --openssh < /etc/lsh/host-key.pub 
+ssh-rsa AAAAB3NzaC1yc2EAAAAEOp8FoQAAAQEAs1eB46LV@dots{}
+@end example
+
 @end table
 
 A number of optional fields may be specified:
 
-@table @code
+@table @asis
 
-@item port
-Port number of SSH server on the machine (default: 22).
+@item @code{port} (default: @code{22})
+Port number of SSH server on the machine.
 
-@item private-key
-The SSH private key file to use when connecting to the machine.
+@item @code{private-key} (default: @file{~/.ssh/id_rsa})
+The SSH private key file to use when connecting to the machine, in
+OpenSSH format.
 
-Currently offloading uses GNU@tie{}lsh as its SSH client
-(@pxref{Invoking lsh,,, GNU lsh Manual}).  Thus, the key file here must
-be an lsh key file.  This may change in the future, though.
+@item @code{daemon-socket} (default: @code{"/var/guix/daemon-socket/socket"})
+File name of the Unix-domain socket @command{guix-daemon} is listening
+to on that machine.
 
-@item parallel-builds
-The number of builds that may run in parallel on the machine (1 by
-default.)
+@item @code{parallel-builds} (default: @code{1})
+The number of builds that may run in parallel on the machine.
 
-@item speed
+@item @code{speed} (default: @code{1.0})
 A ``relative speed factor''.  The offload scheduler will tend to prefer
 machines with a higher speed factor.
 
-@item features
+@item @code{features} (default: @code{'()})
 A list of strings denoting specific features supported by the machine.
 An example is @code{"kvm"} for machines that have the KVM Linux modules
 and corresponding hardware support.  Derivations can request features by
@@ -915,7 +948,7 @@ machines, since offloading works by invoking the @code{guix archive} and
 this is the case by running:
 
 @example
-lsh build-machine guile -c "'(use-modules (guix config))'"
+ssh build-machine guile -c "'(use-modules (guix config))'"
 @end example
 
 There is one last thing to do once @file{machines.scm} is in place.  As
@@ -1209,6 +1242,56 @@ data in the right format.
 This is important because the locale data format used by different libc
 versions may be incompatible.
 
+@subsection Name Service Switch
+
+@cindex name service switch, glibc
+@cindex NSS (name service switch), glibc
+@cindex nscd (name service caching daemon)
+@cindex name service caching daemon (nscd)
+When using Guix on a foreign distro, we @emph{strongly recommend} that
+the system run the GNU C library's @dfn{name service cache daemon},
+@command{nscd}, which should be listening on the
+@file{/var/run/nscd/socket} socket.  Failing to do that, applications
+installed with Guix may fail to look up host names or user accounts, or
+may even crash.  The next paragraphs explain why.
+
+@cindex @file{nsswitch.conf}
+The GNU C library implements a @dfn{name service switch} (NSS), which is
+an extensible mechanism for ``name lookups'' in general: host name
+resolution, user accounts, and more (@pxref{Name Service Switch,,, libc,
+The GNU C Library Reference Manual}).
+
+@cindex Network information service (NIS)
+@cindex NIS (Network information service)
+Being extensible, the NSS supports @dfn{plugins}, which provide new name
+lookup implementations: for example, the @code{nss-mdns} plugin allow
+resolution of @code{.local} host names, the @code{nis} plugin allows
+user account lookup using the Network information service (NIS), and so
+on.  These extra ``lookup services'' are configured system-wide in
+@file{/etc/nsswitch.conf}, and all the programs running on the system
+honor those settings (@pxref{NSS Configuration File,,, libc, The GNU C
+Reference Manual}).
+
+When they perform a name lookup---for instance by calling the
+@code{getaddrinfo} function in C---applications first try to connect to
+the nscd; on success, nscd performs name lookups on their behalf.  If
+the nscd is not running, then they perform the name lookup by
+themselves, by loading the name lookup services into their own address
+space and running it.  These name lookup services---the
+@file{libnss_*.so} files---are @code{dlopen}'d, but they may come from
+the host system's C library, rather than from the C library the
+application is linked against (the C library coming from Guix).
+
+And this is where the problem is: if your application is linked against
+Guix's C library (say, glibc 2.24) and tries to load NSS plugins from
+another C library (say, @code{libnss_mdns.so} for glibc 2.22), it will
+likely crash or have its name lookups fail unexpectedly.
+
+Running @command{nscd} on the system, among other advantages, eliminates
+this binary incompatibility problem because those @code{libnss_*.so}
+files are loaded in the @command{nscd} process, not in applications
+themselves.
+
 @subsection X11 Fonts
 
 @cindex fonts
@@ -3137,6 +3220,11 @@ the @code{#:python} parameter.  This is a useful way to force a package
 to be built for a specific version of the Python interpreter, which
 might be necessary if the package is only compatible with a single
 interpreter version.
+
+By default guix calls @code{setup.py} under control of
+@code{setuptools}, much like @command{pip} does.  Some packages are not
+compatible with setuptools (and pip), thus you can disable this by
+setting the @code{#:use-setuptools} parameter to @code{#f}.
 @end defvr
 
 @defvr {Scheme Variable} perl-build-system
@@ -5167,10 +5255,19 @@ gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.
 gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0
 @end example
 
-It does so by browsing the FTP directory of each package and determining
-the highest version number of the source tarballs therein.  The command
+Alternately, one can specify packages to consider, in which case a
+warning is emitted for packages that lack an updater:
+
+@example
+$ guix refresh coreutils guile guile-ssh
+gnu/packages/ssh.scm:205:2: warning: no updater for guile-ssh
+gnu/packages/guile.scm:136:12: guile would be upgraded from 2.0.12 to 2.0.13
+@end example
+
+@command{guix refresh} browses the upstream repository of each package and determines
+the highest version number of the releases therein.  The command
 knows how to update specific types of packages: GNU packages, ELPA
-packages, etc.---see the documentation for @option{--type} below.  The
+packages, etc.---see the documentation for @option{--type} below.  There
 are many packages, though, for which it lacks a method to determine
 whether a new upstream release is available.  However, the mechanism is
 extensible, so feel free to get in touch with us to add a new method!
@@ -5210,7 +5307,7 @@ usually run from a checkout of the Guix source tree (@pxref{Running
 Guix Before It Is Installed}):
 
 @example
-$ ./pre-inst-env guix refresh -s non-core
+$ ./pre-inst-env guix refresh -s non-core -u
 @end example
 
 @xref{Defining Packages}, for more information on package definitions.
@@ -5245,6 +5342,8 @@ the updater for GNOME packages;
 the updater for KDE packages;
 @item xorg
 the updater for X.org packages;
+@item kernel.org
+the updater for packages hosted on kernel.org;
 @item elpa
 the updater for @uref{http://elpa.gnu.org/, ELPA} packages;
 @item cran
@@ -5276,7 +5375,7 @@ In addition, @command{guix refresh} can be passed one or more package
 names, as in this example:
 
 @example
-$ ./pre-inst-env guix refresh -u emacs idutils gcc-4.8.4
+$ ./pre-inst-env guix refresh -u emacs idutils gcc@@4.8
 @end example
 
 @noindent
@@ -5295,6 +5394,9 @@ be used when passing @command{guix refresh} one or more package names:
 @itemx -L
 List available updaters and exit (see @option{--type} above.)
 
+For each updater, display the fraction of packages it covers; at the
+end, display the fraction of packages covered by all these updaters.
+
 @item --list-dependent
 @itemx -l
 List top-level dependent packages that would need to be rebuilt as a
@@ -6055,6 +6157,30 @@ add a call to @code{guix-publish-service} in the @code{services} field
 of the @code{operating-system} declaration (@pxref{guix-publish-service,
 @code{guix-publish-service}}).
 
+If you are instead running Guix on a ``foreign distro'', follow these
+instructions:”
+
+@itemize
+@item
+If your host distro uses the systemd init system:
+
+@example
+# ln -s ~root/.guix-profile/lib/systemd/system/guix-publish.service \
+        /etc/systemd/system/
+# systemctl start guix-publish && systemctl enable guix-publish
+@end example
+
+@item
+If your host distro uses the Upstart init system:
+
+@example
+# ln -s ~root/.guix-profile/lib/upstart/system/guix-publish.conf /etc/init/
+# start guix-publish
+@end example
+
+@item
+Otherwise, proceed similarly with your distro's init system.
+@end itemize
 
 @node Invoking guix challenge
 @section Invoking @command{guix challenge}
@@ -6641,27 +6767,26 @@ partition lives at @file{/dev/sda1}, a file system with the label
 mkfs.ext4 -L my-root /dev/sda1
 @end example
 
-@c FIXME: Uncomment this once GRUB fully supports encrypted roots.
-@c A typical command sequence may be:
-@c
-@c @example
-@c # fdisk /dev/sdX
-@c @dots{} Create partitions etc.@dots{}
-@c # cryptsetup luksFormat /dev/sdX1
-@c # cryptsetup open --type luks /dev/sdX1 my-partition
-@c # mkfs.ext4 -L my-root /dev/mapper/my-partition
-@c @end example
+@cindex encrypted disk
+If you are instead planning to encrypt the root partition, you can use
+the Cryptsetup/LUKS utilities to do that (see @inlinefmtifelse{html,
+@uref{https://linux.die.net/man/8/cryptsetup, @code{man cryptsetup}},
+@code{man cryptsetup}} for more information.)  Assuming you want to
+store the root partition on @file{/dev/sda1}, the command sequence would
+be along these lines:
 
-In addition to e2fsprogs, the suite of tools to manipulate
-ext2/ext3/ext4 file systems, the installation image includes
-Cryptsetup/LUKS for disk encryption.
+@example
+cryptsetup luksFormat /dev/sda1
+cryptsetup open --type luks /dev/sda1 my-partition
+mkfs.ext4 -L my-root /dev/mapper/my-partition
+@end example
 
 Once that is done, mount the target root partition under @file{/mnt}
-with a command like (again, assuming @file{/dev/sda1} is the root
-partition):
+with a command like (again, assuming @code{my-root} is the label of the
+root partition):
 
 @example
-mount /dev/sda1 /mnt
+mount LABEL=my-root /mnt
 @end example
 
 Finally, if you plan to use one or more swap partitions (@pxref{Memory
@@ -6724,6 +6849,10 @@ Be sure that your partition labels match the value of their respective
 @code{device} fields in your @code{file-system} configuration, assuming
 your @code{file-system} configuration sets the value of @code{title} to
 @code{'label}.
+
+@item
+If there are encrypted or RAID partitions, make sure to add a
+@code{mapped-devices} field to describe them (@pxref{Mapped Devices}).
 @end itemize
 
 Once you are done preparing the configuration file, the new system must
@@ -6968,7 +7097,9 @@ desired configuration.  In particular, notice how we use @code{inherit}
 to create a new configuration which has the same values as the old
 configuration, but with a few modifications.
 
-The configuration for a typical ``desktop'' usage, with the X11 display
+@cindex encrypted disk
+The configuration for a typical ``desktop'' usage, with an encrypted
+root partition, the X11 display
 server, GNOME and Xfce (users can choose which of these desktop
 environments to use at the log-in screen by pressing @kbd{F1}), network
 management, power management, and more, would look like this:
@@ -7293,13 +7424,16 @@ errors before being mounted.
 When true, the mount point is created if it does not exist yet.
 
 @item @code{dependencies} (default: @code{'()})
-This is a list of @code{<file-system>} objects representing file systems
-that must be mounted before (and unmounted after) this one.
+This is a list of @code{<file-system>} or @code{<mapped-device>} objects
+representing file systems that must be mounted or mapped devices that
+must be opened before (and unmounted or closed after) this one.
 
 As an example, consider a hierarchy of mounts: @file{/sys/fs/cgroup} is
 a dependency of @file{/sys/fs/cgroup/cpu} and
 @file{/sys/fs/cgroup/memory}.
 
+Another example is a file system that depends on a mapped device, for
+example for an encrypted partition (@pxref{Mapped Devices}).
 @end table
 @end deftp
 
@@ -7797,6 +7931,7 @@ declaration.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
 * Network File System::         NFS related services.
+* Continuous Integration::      The Cuirass service.
 * Miscellaneous Services::      Other services.
 @end menu
 
@@ -8407,13 +8542,22 @@ configure networking."
 @end deffn
 
 @cindex WPA Supplicant
-@deffn {Scheme Procedure} wpa-supplicant-service @
-       [#:wpa-supplicant @var{wpa-supplicant}]
-Return a service that runs @url{https://w1.fi/wpa_supplicant/,WPA
+@defvr {Scheme Variable} wpa-supplicant-service-type
+This is the service type to run @url{https://w1.fi/wpa_supplicant/,WPA
 supplicant}, an authentication daemon required to authenticate against
-encrypted WiFi or ethernet networks. Service is started to listen for
+encrypted WiFi or ethernet networks.  It is configured to listen for
 requests on D-Bus.
-@end deffn
+
+The value of this service is the @code{wpa-supplicant} package to use.
+Thus, it can be instantiated like this:
+
+@lisp
+(use-modules (gnu services networking)
+             (gnu packages admin))
+
+(service wpa-supplicant-type wpa-supplicant)
+@end lisp
+@end defvr
 
 @cindex NTP
 @cindex real time clock
@@ -9979,7 +10123,7 @@ Return a service that runs @command{mysqld}, the MySQL or MariaDB
 database server.
 
 The optional @var{config} argument specifies the configuration for
-@command{mysqld}, which should be a @code{<mysql-configuraiton>} object.
+@command{mysqld}, which should be a @code{<mysql-configuration>} object.
 @end deffn
 
 @deftp {Data Type} mysql-configuration
@@ -10001,16 +10145,11 @@ For MariaDB, the root password is empty.
 @cindex mail
 @cindex email
 The @code{(gnu services mail)} module provides Guix service definitions
-for mail services.  Currently the only implemented service is Dovecot,
-an IMAP, POP3, and LMTP server.
-
-Guix does not yet have a mail transfer agent (MTA), although for some
-lightweight purposes the @code{esmtp} relay-only MTA may suffice.  Help
-is needed to properly integrate a full MTA, such as Postfix.  Patches
-welcome!
+for email services: IMAP, POP3, and LMTP servers, as well as mail
+transport agents (MTAs).  Lots of acronyms!  These services are detailed
+in the subsections below.
 
-To add an IMAP/POP3 server to a GuixSD system, add a
-@code{dovecot-service} to the operating system definition:
+@subsubheading Dovecot Service
 
 @deffn {Scheme Procedure} dovecot-service [#:config (dovecot-configuration)]
 Return a service that runs the Dovecot IMAP/POP3/LMTP mail server.
@@ -11366,18 +11505,47 @@ could instantiate a dovecot service like this:
                   (string "")))
 @end example
 
+@subsubheading OpenSMTPD Service
+
+@deffn {Scheme Variable} opensmtpd-service-type
+This is the type of the @uref{https://www.opensmtpd.org, OpenSMTPD}
+service, whose value should be an @code{opensmtpd-configuration} object
+as in this example:
+
+@example
+(service opensmtpd-service-type
+         (opensmtpd-configuration
+           (config-file (local-file "./my-smtpd.conf"))))
+@end example
+@end deffn
+
+@deftp {Data Type} opensmtpd-configuration
+Data type regresenting the configuration of opensmtpd.
+
+@table @asis
+@item @code{package} (default: @var{opensmtpd})
+Package object of the OpenSMTPD SMTP server.
+
+@item @code{config-file} (default: @var{%default-opensmtpd-file})
+File-like object of the OpenSMTPD configuration file to use.  By default
+it listens on the loopback network interface, and allows for mail from
+users and daemons on the local machine, as well as permitting email to
+remote servers.  Run @command{man smtpd.conf} for more information.
+
+@end table
+@end deftp
 
 @node Kerberos Services
 @subsubsection Kerberos Services
 @cindex Kerberos
 
-The @code{(gnu services Kerberos)} module provides services relating to
+The @code{(gnu services kerberos)} module provides services relating to
 the authentication protocol @dfn{Kerberos}.
 
 @subsubheading PAM krb5 Service
 @cindex pam-krb5
 
-The pam-krb5 service allows for login authentication and password
+The @code{pam-krb5} service allows for login authentication and password
 management via Kerberos.
 You will need this service if you want PAM enabled applications to authenticate
 users using Kerberos.
@@ -11585,6 +11753,84 @@ If it is @code{#f} then the daemon will use the host's fully qualified domain na
 @end table
 @end deftp
 
+@node Continuous Integration
+@subsubsection Continuous Integration
+
+@cindex continuous integration
+@uref{https://notabug.org/mthl/cuirass, Cuirass} is a continuous
+integration tool for Guix.  It can be used both for development and for
+providing substitutes to others (@pxref{Substitutes}).
+
+The @code{(gnu services cuirass)} module provides the following service.
+
+@deffn {Scheme Procedure} cuirass-service @
+       [#:config @code{(cuirass-configuration)}]
+Return a service that runs @command{cuirass}.
+
+The @var{#:config} keyword argument specifies the configuration for
+@command{cuirass}, which must be a @code{<cuirass-configuration>}
+object, by default it doesn't provide any build job.  If you want to
+provide your own configuration you will most likely use the
+@code{cuirass-configuration} special form which returns such objects.
+@end deffn
+
+In order to add build jobs you will have to set the
+@code{specifications} field.  Here is an example of a cuirass service
+defining a build job based on a specification that can be found in
+Cuirass source tree.
+
+@example
+(let ((spec `((#:name . "guix")
+              (#:url . "git://git.savannah.gnu.org/guix.git")
+              (#:load-path . ".")
+              ;; Adapt to a valid absolute file name.
+              (#:file . "/.../cuirass/tests/gnu-system.scm")
+              (#:proc . hydra-jobs)
+              (#:arguments (subset . "hello"))
+              (#:branch . "master"))))
+  (cuirass-service #:config (cuirass-configuration
+                             (specifications (list spec)))))
+@end example
+
+While information related to build jobs are located directly in the
+specifications, global settings for the @command{cuirass} process are
+accessible in other @code{cuirass-configuration} fields.
+
+@deftp {Data Type} cuirass-configuration
+Data type representing the configuration of Cuirass.
+
+@table @asis
+@item @code{cache-directory} (default: @code{""})
+Location of the repository cache.
+
+@item @code{user} (default: @code{"cuirass"})
+Owner of the @code{cuirass} process.
+
+@item @code{group} (default: @code{"cuirass"})
+Owner's group of the @code{cuirass} process.
+
+@item @code{interval} (default: @code{60})
+Number of seconds between the poll of the repositories followed by the
+Cuirass jobs.
+
+@item @code{database} (default: @code{"/var/run/cuirass/cuirass.db"})
+Location of sqlite database which contains the build results and previously
+added specifications.
+
+@item @code{specifications} (default: @code{'()})
+A list of specifications, where a specification is an association list
+(@pxref{Associations Lists,,, guile, GNU Guile Reference Manual}) whose
+keys are keywords (@code{#:keyword-example}) as shown in the example
+above.
+
+@item @code{use-substitutes?} (default: @code{#f})
+This allows using substitutes to avoid building every dependencies of a job
+from source.
+
+@item @code{one-shot?} (default: @code{#f})
+Only evaluate specifications and build derivations once.
+@end table
+@end deftp
 
 @node Miscellaneous Services
 @subsubsection Miscellaneous Services
@@ -13649,7 +13895,6 @@ for instance, the module python-dateutil is packaged under the names
 starts with @code{py} (e.g. @code{pytz}), we keep it and prefix it as
 described above.
 
-
 @subsubsection Specifying Dependencies
 @cindex inputs, for Python packages
 
@@ -13666,6 +13911,12 @@ following check list to determine which dependency goes where.
 @itemize
 
 @item
+We currently package Python 2 with @code{setuptools} and @code{pip}
+installed like Python 3.4 has per default.  Thus you don't need to
+specify either of these as an input.  @command{guix lint} will warn you
+if you do.
+
+@item
 Python dependencies required at run time go into
 @code{propagated-inputs}.  They are typically defined with the
 @code{install_requires} keyword in @file{setup.py}, or in the
@@ -13679,8 +13930,7 @@ testing---e.g., those in @code{tests_require}---go into
 propagated because they are not needed at run time, and (2) in a
 cross-compilation context, it's the ``native'' input that we'd want.
 
-Examples are @code{setuptools}, which is usually needed only at build
-time, or the @code{pytest}, @code{mock}, and @code{nose} test
+Examples are the @code{pytest}, @code{mock}, and @code{nose} test
 frameworks.  Of course if any of these packages is also required at
 run-time, it needs to go to @code{propagated-inputs}.