When the daemon starts up it creates a fresh (empty) LVM configuration
and starts up lvmetad (which depends on the LVM configuration).
However this appears to cause problems: Some types of PV seem to
require lvmetad and don't work without it
(https://bugzilla.redhat.com/show_bug.cgi?id=1581810). If we don't
start lvmetad earlier, the device nodes are not created.
Therefore move the whole initialization step into appliance/init.
Two further changes had to be made:
Now we are using lvmetad all the time, using vgchange is incorrect.
With lvmetad activated early we must use ‘pvscan --cache --activate ay’
to scan all disks for PVs and activate any VGs on them (although the
documentation is complex, confusing and contradictory so I'm not
completely sure about this).
The ‘lvm_system_dir’ local variable in ‘daemon/lvm-filter.c’
previously contained the path of the directory above $LVM_SYSTEM_DIR
(eg. $LVM_SYSTEM_DIR = "/etc/lvm", lvm_system_dir = "/etc"). As this
was highly confusing, I have changed it so the local variable and the
environment variable have identical contents. This involved removing
the ‘lvm/’ component from a couple of paths since it is now included
in the local variable.
Apparently the jansson library on 64bit installation does not provide
the non-64bit name, so jansson is not pulled in the appliance, and thus
the daemon does not start.
Updates commit eb4fbe96c9.
While YAJL mostly works fine, it did not see any active development in
the last 3 years. OTOH, Jansson is another JSON C implementation, with
a very liberal license, and a much nicer API.
Hence, switch all of libguestfs from YAJL to Jansson:
- configure checks, and buildsystem in general
- packages pulled in the appliance
- actual implementations
- contrib scripts
- documentation
This also makes use of the better APIs available (e.g. json_object_get,
json_array_foreach, and json_object_foreach). This does not change the
API of our OCaml Yajl module.
The issue:
- raid1 will be in degraded state if one of its components is logical volume (LV)
- raid0 will be inoperable at all (inacessible from within appliance) if one of its component is LV
- raidN: you can expect the same issue for any raid level depends on how many components are inaccessible at the time mdadm is running and raid redundency.
It happens because mdadm is launched prior to lvm AND it is instructed to run found arrays immediately (--run flag) regardless of completeness of their components.
Later (when lvm activates found LVs) md signature on LV might be recognized BUT newly found raid components could't be inserted into already running (in degraded state)
or marked as inoperable raid arrays.
The patch fixes the issue in the following way:
1. Found arrays won't be run immediately unless ALL expected drives (components) are present. Here '--no-degraded' flag comes into a play. See mdadm(8).
2. Second mdadm call (after LVM is scanned) will scan UNUSED yet devices and make an attempt to run all found arrays (even they will be in degraded state).
There is no performance penalty because second pass scans UNUSED yet devices. Here is 'boot-benchmark' before and after patch:
: libvirt backend : direct backend
------------------------------------------------
master : 835.2ms ±1.1ms : 670.4ms ±0.3ms
master+patch : 837.7ms ±2.4ms : 671.8ms ±0.2ms
LVM is fine with a completely empty configuration file (meaning "all
defaults"), so start with one instead of copying the system
configuration file.
Also this means we can very easily implement lvm_set_filter
functionality without using Augeas, since we no longer have to worry
about existing filters being present.
Thanks: Alasdair Kergon, Zdenek Kabelac.
Commit a6330e9d3a enabled /run for
systemd-tmpfiles: while this works fine in most of the cases, there are
few tmpfiles configurations that still references /var/run instead of
/run. As result, include also /var/run in the systemd-tmpfiles
execution.
`virt-rescue -a disk -i' does the right thing.
`-m' was previously an alternate form of `--memsize'. By sniffing the
parameter we can make `-m MB' continue to work, while also allowing
`-m' to be used as a short form for the `--mount' option.
This also removes most of the description of `--suggest' from the man
page, since it is no longer needed.
Instead of using "direct mode" (which was basically a quick hack),
virt-rescue now launches the appliance with a running daemon.
The daemon doesn't do much -- there is still a bash shell which the
user interacts with. The daemon is there simply to provide the
initial GUESTFS_LAUNCH_FLAG message and to handle shutdown a bit more
gracefully.
To interact with the shell, and replacing direct mode, virt-rescue now
prints out log messages (the output of the shell), and sends input
typed by the user directly to the console socket. This uses the
guestfs_internal_get_console_socket API added previously. Most of the
complexity behind this is hidden in virt-rescue.
This fully fixes the handling of ^C (RHBZ#1152819). Also there were
earlier reports that full screen commands like 'vim' didn't work well,
(RHBZ#1171654), but in this version vim appears to work fine, albeit
only using 80x24 of the screen because of the serial console.
Introduce a new API to create a new squashfs filesystem out of a path
in the guest. It can be configured to exclude paths based on patterns,
and to select which compression use for the filesystem.
The advantage of running mksquashfs directly in the appliance is that
ownerships are properly saved, as opposed to tar_out + local untar.
Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
Tools could require the use of pseudo-terminals, so make sure we have
/dev/pts available in the appliance. The "command" API already
bind-mounts it when running commands, so this is the only bit needed.
initviocons package provides tools to resize the terminal. Having it
in the appliance will allow SUSE users to have proper line wrapping
in their terminal when using virt-rescue.
Added:
- cdrtools: added as alternative to cdrkit
- multipath-tools: contains kpartx (in AUR)
Removed:
- ntfsprogs: the package is no longer available, it has been completely
replaced by ntfs-3g (already in packagelist.in)
- zfs-fuse: no longer in AUR
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
By adding common CLEANFILES and DISTCLEANFILES variables to
common-rules.mk, we can remove these from most other Makefiles, and
also clean files more consistently.
Note that bin_PROGRAMS are already cleaned by 'make clean', so I
removed cases where these were unnecessarily added to CLEANFILES.
Setup the volatile /run in the appliance also with the tmpfiles
configurations available. In particular, setting up correctly the lvm
bits allow lvmetad to run.
Currently lvmetad is started in init, and thus using the system
(= appliance) configuration of lvm. Later on, in the daemon, a local
copy of the lvm configuration is setup, and set it for use using the
LVM_SYSTEM_DIR environment variable: this means only the programmes
executed by the daemon will use the local lvm configuration, and not
lvmetad.
Thus manually start lvmetad from the daemon, right after having setup
the local lvm configuration, and still without failing if it cannot be
executed.
Additionally, since lvmetad now respects the right configuration, make
sure to update its cache when rescanning the VGs by passing --cache to
vgscan.
Sadly, the dhclient-script shipped as part of isc-dhcp-client in Ubuntu
unconditionally reads from /etc/fstab without checking for its
existence. Since no package holds /etc/fstab, this file will not exist
in the appliance, cause dhclient to fail (actually keep looping calling
the failing dhclient-script) when the network is requested.
As a workaround, touch /etc/fstab just before enabling the network: if
that file exists nothing changes, while an empty file will be available
in the other case, making at least dhclient-script in Ubuntu working.
At least on openSUSE and SLES, the /etc/mtab file is already existing.
Skipping the symlink creation in init removes one error message during
the appliance boot.
SUSE Linux Entreprise Server doesn't have dhcpcd and the hivex package
is not in the default repositories. Better use dhcp-client and
libhivex0.
openSUSE needs ntfs-3g and ntfsprogs packages.
Previously we were running ldconfig to create /etc/ld.so.cache.
This is required, at least on Fedora, if we need to run any binary
that uses a library with a weird path. libiscsi (a dependency of
qemu-img, used by virt-dib) is an example of such a weird library,
since it puts its single library into /usr/lib64/iscsi for no readily
understandable reason, and drops a configuration file into
/etc/ld.so.conf.d/ so that this new directory gets picked up.
By copying the /etc/ld.so.cache from the host we get an already
configured cache which should contain every library on the host, so
there is no need to run ldconfig.
Running ldconfig adds about 100ms to the boot time. I would prefer
that we understood which libraries need ldconfig to be run, and fix
that. We could also consider running ldconfig in parallel, but since
it might be required by just about any binary that the init script
runs it's not clear what benefit that gives.
This reverts commit 66aa98265d.
Recently Debian switched to btrfs-progs, so almost all the distributions
(except openSUSE) have btrfs-progs which is then moved as common
package.
The old btrfs-tools name is left there, to support Debian Jessie and
older, and Ubuntu Xenial and older.
Some of the systemd-tmpfiles snippets need the machine ID of the running
system; the current lack of this file produces warning messages during
the appliance boot like:
[/usr/lib/tmpfiles.d/systemd.conf:26] Failed to replace specifiers: /run/log/journal/%m
[/usr/lib/tmpfiles.d/systemd.conf:28] Failed to replace specifiers: /run/log/journal/%m
[/usr/lib/tmpfiles.d/systemd.conf:29] Failed to replace specifiers: /run/log/journal/%m
Thus create a new randomly-generated /etc/machine-id on boot.
Since commit bb5d30ab2a, we don't
require any external programs like grep to parse the command line. We
only use bash intrinsics.
Therefore we can do it early (but after /proc is mounted).
This allows verbose mode to enable set -x early on, so we can trace
most things that the init script does.
Read the content of /proc/cmdline using bash features, and use its
[[ ... ]] expression to find texts in a variable.
This shaves off 5 grep invocations.
