This adds a contrib script which can be used to build the virt-p2v ISO
on top of RHEL 5 or RHEL 6, i686 (32 bit) or x86-64 (64 bit) base.
There is also a script for testing the ISOs produced this way.
I don't think this will really make any difference. However there is
some puzzling disk corruption in the recently built RHEL 5 & 6 i686
images, and I want to make absolutely sure they are not caused by an
unsynchronized disk image.
There were still a few places where we were testing bash rather than
the intended program. Add `libtool --mode=execute' to the beginning
of the @VG@ macro to try to get around this.
If supported, use socket activation to pass a pre-opened listening
socket to the NBD server. This means we can guarantee to choose an
unused port even if there are multiple instances of virt-p2v running
(ie. when testing) or if there are unexpected services running on the
same machine.
This change applies to both NBD servers, since both now (or will
shortly) support this feature.
In preparation for using socket activation, move the nbd_local_port
global inside nbd.c and make it private.
This is largely code motion. However I rearranged the order in which
the ssh data connection and the NBD server are started up. Previously
the ssh connection was made first, and the NBD server was then
started. Now the NBD server starts first. This happens so that nbd.c
can choose the local port, and when we implement socket activation it
will allow nbd.c to open the socket and choose a free port too.
This is almost pure code motion, but I changed the name and prototype
of the function 'wait_nbd' to make its purpose clearer, and to remove
the unnecessary timeout setting (which is hard-coded).
Parse the output disk as URI, and use all its attributes just like
it is done for the input disk. The only change is that the fsync of the
output disk is limited now for local URIs only, since it will not work
with remote protocols.
Implement the "tgz" output format, i.e. "tar" compressed as gzip.
This was implemented in diskimage-builder upstream as
commit da41ee6012b064aa6901c871a1104a3a3933117e.
Implement a system similar to the operations in virt-sysprep, so each
output format has its own file and attributes (such as command line
arguments).
The result is that there is no more need to spread everywhere the job
required by each output format, such as checking for tools, or hooking
at the right point.
Make sure they are handled as options for the actual mkfs driver,
instead of mkfs itself. Also, improve the documentation of
--mkfs-options accordingly.
This was fixed in diskimage-builder upstream as
commit 5e2330d89c6c0e55b270464abea7e99a4d3246ad.
Add a new option to create files with the MD5 and SHA512 checksums of
all the disk image types produced.
This was implemented in diskimage-builder upstream as
commit 2ea5feca5c4b64867ac327736edfb20408f8840e and
commit 22952b7ea0543bb4f446752976d1d8ba232b021a.
Improve the error message that is shown when trying to selecting an
element, and another element provides it.
This was implemented in diskimage-builder upstream as
commit 452f7b8d5aaa6e85e3b01b970cd9cbc842b34b86.
We don't have to always extract all files from the OVA archive. The OVA,
as defined in the standard, is plain tar. We can work directly over the
tar archive if we use correct 'offset' and 'size' options when defining
the backing file for QEMU. This puts much lower requirement on available
disk space.
Since the virt-v2v behaviour for OVA input now depends on QEMU version
available this affects some of the tests. Expected result of the
affected also has to depend on the QEMU used thus such tests will have
two *.expected files.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The function looks up file in tar archive and returns a tuple containing
which at byte it starts and how long the file is.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The function returns version of qemu-img as a tuple (major, minor), or
the value (0,9) in case there was an error detecting the version.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
In commit 08f82f2e3d we increased the
memory size to 800MB (on x86) so that the semodule command would work.
However it has been discovered that another SELinux command takes
large amounts of RAM (setfiles during the SELinux relabel step).
Therefore increase the memory size again, this time to 2000MB.
The information whether the disk is gzip compressed or not is stored
in the OVF. There is no reason to do the detection.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
- Make module name explicit, so it's more obvious which module a
function is defined in.
- Group lines of code by feature.
- Capitalize some module names properly.
Just code motion, no functional change.
Commit 004de6cf45 was not sufficient to
solve this problem. We are also using gperf in guestfish. Rewrite
the code in the same way to avoid having to prototype the hash
function.
Add code in virt-p2v so that it can use nbdkit (with the file plugin)
as an alternative to qemu-nbd.
This is controlled through the virt-p2v --nbd command line option,
allowing you to select which server (out of qemu-nbd or nbdkit) you
prefer (with a fallback). The default is: --nbd=qemu-nbd,nbdkit so
qemu-nbd will be used preferentially.
When testing virt-p2v (eg. on the host machine) this prevents us from
testing two instances of virt-p2v at the same time because both will
try to use the same port.
Generate the random filename using our utility function
guestfs_int_random_string. This also means that we will not need to
call srandom() in guestfish or virt-edit.
The prototype of the hash function changed in gperf 3.1:
* The 'len' parameter of the hash function and of the lookup function is now
of type 'size_t' instead of 'unsigned int'. This makes it safe to call these
functions with strings of length > 4 GB, on 64-bit machines.
This commit moves the guestfs_int_string_to_errno function into the
third (functions) section of the gperf file so it is able to use the
lookup function without needing a prototype.
Thanks: Marius Cirsta
The prototype of the hash function changed in gperf 3.1:
* The 'len' parameter of the hash function and of the lookup function is now
of type 'size_t' instead of 'unsigned int'. This makes it safe to call these
functions with strings of length > 4 GB, on 64-bit machines.
This change tries to detect the required type in ./configure
Thanks: Marius Cirsta
Three more pieces of common code are moved under the common/
subdirectory. This is just code motion.
Note that windows.[ch] wasn't even being used by guestfish. That code
was only used in other virt tools.
This is mostly code motion but:
(1) I had to remove the compile-time COMPILING_GUESTFISH and
COMPILING_VIRT_RESCUE macros and replace them with runtime constants
and checks.
(2) I moved the fish/config.c file into this library.
Just code motion.
This commit makes it clearer what is a utility and what is part of the
library. It also makes it clear that we should rename:
guestfs-internal-frontend.h -> utils.h
guestfs-internal-frontend-cleanups.h -> cleanups.h (?)
but this commit does not make that change.
This commit, which is just code motion, moves the common XDR protocol
code (libprotocol) and the common errno handling (liberrnostring) into
libraries which are each built once and shared between the library and
daemon.
The error is:
gobject/docs/gtk-doc.make:52: warning: CLEANFILES multiply defined in condition TRUE ...
gobject/docs/Makefile.am:95: 'gobject/docs/gtk-doc.make' included from here
common-rules.mk:28: ... 'CLEANFILES' previously defined here
gobject/docs/Makefile.am:18: 'subdir-rules.mk' included from here
subdir-rules.mk:20: 'common-rules.mk' included from here
In the ancient version of PCRE in RHEL 5, partial match support does
not support /.*/, returning PCRE_ERROR_BADPARTIAL. However the
equivalent regular expression /(?:.)*/ works fine, so use that
instead.
RHEL 5 has ancient Gtk 2.10 which lacks several functions and whole
features we need (eg the spinner). Add various macros and function
definitions to work around this.
The guestfs_canonical_device_name API was rewriting /dev/mdX as
/dev/sdX. This is wrong since (eg) /dev/sd0 is not a device name, so
if you pass the canonicalized name back to the API it will fail.
virt-v2v was one tool doing this.
Bug found by valgrind on aarch64:
==21350== Syscall param faccessat(pathname) points to unaddressable byte(s)
==21350== at 0x4F05274: access (access.c:29)
==21350== by 0x4954DC3: guestfs_int_get_uefi (appliance-uefi.c:90)
==21350== by 0x49752D7: launch_libvirt (launch-libvirt.c:413)
==21350== by 0x496FD83: guestfs_impl_launch (launch.c:98)
==21350== by 0x4902003: guestfs_launch (actions-3.c:145)
[...]
==21350== Address 0x0 is not stack'd, malloc'd or (recently) free'd
Because we used 'return' directly, the effect of the CAMLparam0()
macro was not undone, resulting in local roots corruption when the
visit callback throws an exception.
The visit_tests unit test picked this up, but the corruption was only
seen on aarch64.
Fixes commit 4a62e52111.
appliance-uefi.c: In function 'guestfs_int_get_uefi':
appliance-uefi.c:67:9: error: implicit declaration of function 'access' [-Werror=implicit-function-declaration]
if (access (codefile, R_OK) == 0 && access (varsfile, R_OK) == 0) {
^~~~~~
appliance-uefi.c:67:5: error: nested extern declaration of 'access' [-Werror=nested-externs]
if (access (codefile, R_OK) == 0 && access (varsfile, R_OK) == 0) {
^~
appliance-uefi.c:67:27: error: 'R_OK' undeclared (first use in this function)
if (access (codefile, R_OK) == 0 && access (varsfile, R_OK) == 0) {
^~~~
appliance-uefi.c:67:27: note: each undeclared identifier is reported only once for each function it appears in
cc1: all warnings being treated as errors
Fixes commit dd519e8a8e.
Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
EDD (https://en.wikipedia.org/wiki/Enhanced_Disk_Drive) is a BIOS INT 13h
subcall for communicating the extended size of the boot disk over 8GB
to the OS. Since libguestfs doesn't use a boot disk, and the
appliance disk is limited to 4GB, and we use virtio-scsi, this is all
useless.
EDD is also broken currently on RHEL 7.3, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1404287
Also the EDD probing takes significant extra time (about 80ms on my
laptop), and using edd=off reduces this time although doesn't entirely
eliminate it.
One of the tests for btrfs_subvolume_show tries to invoke it on the
toplevel subvolume, expecting its failure (since btrfs-progs used to not
be able to list that). However, since v4.8.3 btrfs-progs can do that
(even if the data returned for that subvolume is limited/incomplete),
and thus this test fails because the API does not fail anymore.
Since we can consider that functionality working (for some kind of it),
then drop this test.
Very recent versions of tar (most probably as a consequence of
CVE-2016-6321) may refuse archive members with '..', like the relative
paths to upper level directories.
Since these are just tests, simply copy the files in the temporary
directories where tar (or zip as well) is run, so all the files are in
the same directory.
The return value struct was freed using a simple free() instead of the
own cleanup function of each struct: this meant dynamically allocated
values (such as strings) were leaked.
Use the proper cleanup functions instead.
The return value struct was freed using a simple free() instead of the
own cleanup function of each struct: this meant dynamically allocated
values (such as strings) were leaked.
Use the proper cleanup functions instead.
The definition of BUILDFILES was accidentally removed, resulting
in the error:
cp /home/rjones/d/websites/libguestfs/download/builder/
cp: missing destination file operand after '/home/rjones/d/websites/libguestfs/download/builder/'
Fixes commit 65a0570385.
It fails with:
File "../builder/templates/make-template.ml", line 1, characters 0-1:
Parse error: [a_LIDENT] expected after "#" (in [implem])
End_of_file
An error occurs while processing.
In any case this doesn't matter because this file should not be
translated.
Fixes commit a1ea9a2599.
See:
http://caml.inria.fr/pub/docs/manual-ocaml/comp.html#ss%3Awarn57
I believe the code as written previously was incorrect. However we
are lucky because if neither clause matches then it will fall through
to displaying an error message, allowing the user to correct the
problem.
For a very long time we have maintained two sets of utility functions,
in mllib/common_utils.ml and generator/utils.ml. This changes things
so that the same set of utility functions can be shared with both
directories.
It's not possible to use common_utils.ml directly in the generator
because it provides several functions that use modules outside the
OCaml stdlib. Therefore we add some lightweight post-processing which
extracts the functions using only the stdlib:
(*<stdlib>*)
...
(*</stdlib>*)
and creates generator/common_utils.ml and generator/common_utils.mli
from that. The effect is we only need to write utility functions
once.
As with other tools, we still have generator-specific utility
functions in generator/utils.ml.
Also in this change:
- Use String.uppercase_ascii and String.lowercase_ascii in place
of deprecated String.uppercase/String.lowercase.
- Implement String.capitalize_ascii to replace deprecated
String.capitalize.
- Move isspace, isdigit, isxdigit functions to Char module.
Support for RHEV with RHEL 6 nodes required us to output the old style
qcow2 compat=0.10 images. Since RHEV 3.6 GA, RHEL 6 has not been
supported as a RHEV node type. Since RHV 4.1, compat=1.1 is
supported. (Support for compat=1.1 is uncertain in RHV 4.0 even on
RHEL 7 nodes.)
There are significant downsides to using qcow2 compat=0.10 instead of
the modern default (compat=1.1).
Therefore this patch does two things:
For -o rhev, it drops support for compat=0.10 completely. You must
use RHV 4.1.
For -o vdsm, it adds an interim flag (--vdsm-compat=0.10 or
--vdsm-compat=1.1) which controls the compat level of the qcow2 output
file. VDSM should use --vdsm-compat=1.1 when it is known that modern
qemu is available. We can make this the default later when all RHV
instances have moved to 4.1.
It also adds:
vdsm-compat-option
to the `virt-v2v --machine-readable' output to indicate that this flag
can be used.
Thanks: Yaniv Kaul, Michal Skrivanek.
Flag the filesystems for Linux /usr properly as USR role, and detect
some data out of it, like the distro information from an os-release
(if present), and the architecture (since the binaries used for our
architecture check will be available there only).
Later on, collect the results in a way similar to what is done for
CoreOS: for each non-CoreOS root, try to find its /usr filesystem, and
if found then merge what is missing from root; in the last case, also
override the distro inspection data (version, product name) if available
in /usr.
When checking for the existance of /dev/mapper devices found in the
fstab of a filesystem, using guestfs_exists means they are checked as
files in the guest, while they really appear as devices on the
appliance. Instead, try the lvm name resolution anyway, and ignore them
when they are reported as missing.
Thanks to: Richard W.M. Jones.
Fixes commit 96b6504b09.
Move the checks for empty xmlXPathObjectPtr, and for extracting the
result string out of it, to a new helper functions.
This is just code motion, there should be no behaviour changes.
Add a new inspect role for "/usr" partitions, and use that to mark the
/usr partition in CoreOS: this additional role allows to ease its lookup
later on, when merging its results into those of the root.
Introduce a new enum to classify the role of a filesystem, if available.
This will help later on when doing operations on non-root filesystems,
like detecting particular mountpoints such as /usr.
The new enum has only "root" as known role, which replaces the is_root
flag.
Tools could require the use of pseudo-terminals, so make sure we have
/dev/pts available in the appliance. The "command" API already
bind-mounts it when running commands, so this is the only bit needed.
This refactors command line parsing into a parse_cmdline function.
The function uses the Arg module to parse the command line instead of
ad hoc parsing.
This also enforces use of the ./run script to run the program, since
otherwise you can end up with mixed versions of the OCaml bindings and
the C library.
This updates commit 65a0570385.
initviocons package provides tools to resize the terminal. Having it
in the appliance will allow SUSE users to have proper line wrapping
in their terminal when using virt-rescue.
Rebuild these images with the new make-template.ml script. This fixes
the lack of virtio-scsi driver and broken initramfs.
These images all use LVM because without that both Fedora and RHEL put
the root filesystem on an extended partition (/dev/sda5) which
virt-resize cannot handle.
(Note the RHEL images are not public.)
Create a new directory (builder/template). Integrate all of the
scripts into a single program, so that templates are generated more
consistently.
This also changes how the index file is generated. The script now
generates the index file fragment and saves it under version control,
and then generates the final index file by concatenating these.
(Previously the index was written by hand which was tedious and
error-prone.)
The new script also saves the generated kickstart under version
control so it can be referenced later.
It turns out that adding the virtio-scsi driver does not help
to fix the dracut problem.
This partially reverts commit f766c84c39,
keeping the change to index.
Unfortunately I was unable to build s390 binaries since multilib was
dropped in Fedora 24 on s390x. Going from the source of the 'file'
command it seems as if it prints "32-bit" (the architecture is really
31 bit).
As noted by Pino in another patch, the logic passes the first member of
the struct which happens to be the right address to the callback
function.
This will break the callback if order of the members of the struct will
change.
As the callback is using the entire struct, better to pass the pointer
to the struct itself.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
If an environment variable such as XDG_RUNTIME_DIR or one of the
tmpdirs or cachedir is set to a non-existent directory, improve the
error message that the user will see so that (where possible) it
includes the environment variable or API call.
This is still not bullet-proof because it's hard to display the
environment variable if it is LIBGUESTFS_TMPDIR or
LIBGUESTFS_CACHEDIR, but the main problem is with XDG_RUNTIME_DIR
(because of systemd bugs).
Thanks: Hilko Bengen for identifying the bug.
This makes a number of small changes to how the internal documentation
is generated and what can be documented.
Header files are now permitted to contain internal documentation.
This works in the same wasy as .c files.
Also documentation can be added for structs as well as functions.
This commit also adds documentation to some header files and structs,
and fixes a few places which contained broken header documentation.
We do this by sending an Inhibit() message to logind and receiving a
file descriptor back, which we hold open until the conversion
completes (or fails). This is described here:
https://www.freedesktop.org/wiki/Software/systemd/inhibit/
This adds an additional optional dependency on DBus since we use DBus
to call the Inhibit() method.
Reported-by: Chris Cowley.
Create a single temporary directory for all the files created during the
virt-builder run (except big disk images, which already use the
libguestfs cache directory). A single directory means there is no need
to manually schedule all the temporary files and directories for removal
when the application quits.
Add a new optional parameter for the Curl ADT, so temporary files can be
created in a specified directory (which is supposed to be temporary, and
disposed only when the application quits).
$0 is set to the name of the test script, eg. test-settings.sh, and in
bash there is no way to change that.
Create a new environment variable, $script, which is set to the name
of the wrapper script (eg. test-settings-fedora-24.sh).
This should give more accurate error messages.
When TESTS_ENVIRONMENT already uses 'run', the VG variable
doesn't also need to use 'run'.
The specific problem is that if the command contains newlines
then double invocations of the 'run' script fails (in libtool).
ie the following command failed causing errors in check-valgrind:
$VG virt-builder phony-fedora \
-v --no-cache --no-check-signature $no_network \
...
--write '/etc/append4:line1
' \
...
Currently 'make install' installs the virt-p2v binary in
/usr/libexec/virt-p2v on the host. It is never supposed to be run
from there, even by another program, so use of /usr/libexec is
incorrect. It is only supposed to be copied into USB keys / ISOs /
etc created by virt-p2v-make-* scripts.
The other problem with shipping a "naked" binary on the host is that
packages built from that get all the dependencies of virt-p2v, for
example Gtk. This is unnecessary just for running the command line
scripts mentioned above.
This changes the Makefile and scripts so that the binary is stored
compressed in $libdir/virt-p2v/virt-p2v.xz. It is compressed to avoid
exposing the dependencies. It is stored under $libdir since the
binary is still architecture-dependent.
A further change is that when we copy the binary into the virt-p2v
ISO, it is now installed in /usr/bin instead of /usr/libexec. (And
note that we always use /usr/bin, not $bindir, since this path should
not need to be affected by the configuration of libguestfs).
Added:
- cdrtools: added as alternative to cdrkit
- multipath-tools: contains kpartx (in AUR)
Removed:
- ntfsprogs: the package is no longer available, it has been completely
replaced by ntfs-3g (already in packagelist.in)
- zfs-fuse: no longer in AUR
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The OVF standard allows the use of SHA256 hashes in the manifest file.
Adding support for this.
One of the tests was updated to put SHA256 into manifest file.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The regular expression for parsing the manifest line was wrong. There is
a mandatory space between '=' and the hash.
Another problem was that only the first line of the manifest file was
actually processed.
Also added some debugging info and warning to catch problems with
parsing.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Try to improve the way packages of VMware tools are removed from
YUM-based guests:
- when filtering the package itself from its providers, do a stricter
check so either the provide is the unversioned package, or it is
exactly its own name
- if the package has no other providers, then going further will cause
the invocation of 'yum install' with no packages, and thus the package
itself will not be added to the list of packages to be removed; to
overcome this issue, just mark the package as "to be removed" in that
case
Related to: RHBZ#1155150
When checking whether a kernel supports virtio or it is Xen-based, it is
assumed that the feature has the kernel module for it; this will fail if
the feature is built-in in the kernel, misrepresenting it.
The solution is to check the kernel configuration whether the feature
is built-in, in case there is no module available.
This fixes the virtio detection on Ubuntu kernels, where virtio is
built in and not as module.
Detect only once which method must be used to get and set the default
Grub2 kernel in the guest: this avoids the same checks in list_kernels
and set_default_kernel.
Also, add a "no method" option as well: Debian/Ubuntu guests do not have
neither grubby nor Perl's Bootloader::Tools, so there is no way to know
what is the default kernel, nor to change it. In this case, add a
warning about this situation.
Move the Checksums module from virt-builder mostly as it is; the only
change is that on checksum mismatch an exception is raised rather than
invoking "error" directly: this way users of verify_checksum &
verify_checksums can do their own handling of the situation.
Debian is also doing a UsrMove/UsrMerge:
https://wiki.debian.org/UsrMerge
However it is not finalized that Debian will actually make this
change.
Since some Debian systems have /sbin as a symlink and other have /sbin
as a real directory, and we should avoid choosing a symlinked
directory for the daemon, the easiest fix is simply to probe /usr/sbin
before /sbin since under all scenarios (and Fedora too) /usr/sbin is a
real directory.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=838995
Create a new libfishcommon convenience static library to build just once
(instead of 12 times!) the majority of the guestfish sources used in the
rest of the C tools (mostly for command line stuff, inspection, and
mount).
The notable exceptions not using libfishcommon are guestfish itself, and
virt-rescue: both need to build at least one of the common sources using
additional CPPFLAGS.
All the define was doing in options.h was masking the declaration of
inspect_mount_root, which was always built-in in inspect.c (because of
the unconditional #define there) anyway.
Since this is common code used by all the C tools, try to avoid extra
knobs which add different code paths for no benefit.
Some of the C tools were building also config.c as part of the shared
sources from guestfish, and thus bringing a dependency on libconfig.
Since none of them actually read the libguestfs configuration at all,
then exclude fish/config.c from their build, and stop linking to
libconfig.
Using update-initramfs is the native way of updating initrd on Debian
based systems.
To add some modules to the image we can list them in file
/etc/initramfs-tools/modules.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
A disk of type 'volume' is stored as
<source pool='pool_name' volume='volume_name'/>
and its real location is inside the 'volume_name', as 'pool_name': in
this case, query libvirt for the actual path of the specified volume in
the specified pool.
Adjust the code so that:
- for_each_disk gets the virConnectPtr, needed to do operations with
libvirt
- when extracting the disk filename depending on the type, the code
snippet doing it can directly set 'filename', without setting an XPath
result variable
Only file-based volumes are supported for now; more types can be added
(with proper testing) later on.
Since libvirt 2.0.0, these two new <listen/> types have been
supported: https://libvirt.org/formatdomain.html#elementsGraphics
This change just copies that configuration over from the source to the
destination if the destination is also libvirt.
Since we previously used 'LNone' to mean "no parseable <listen/>
element" I also had to change previous uses of 'LNone' to 'LNoListen',
so we can use 'LNone' to mean "<listen type='none'>".
Thanks: Ming Xie.
NTFS file system always has the MFT file at inode 0. This reliable
information helps testing the API.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Library's counterpart of the daemon's internal_find_inode command.
It writes the daemon's command output on a temporary file and parses it,
deserialising the XDR formatted tsk_dirent structs.
It returns to the caller the list of tsk_dirent structs generated by the
internal_find_inode command.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The internal_find_inode command searches all entries referring to the
given inode and returns a tsk_dirent structure for each of them.
The command is able to retrieve information regarding deleted
or unaccessible files where other commands such as stat or find
would fail.
The gathered list of tsk_dirent structs is serialised into XDR format
and written to a file by the appliance.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Make use of the additional command line arguments, and API needed to
decrypt LUKS partitions.
This affects only virt-customize, virt-get-kernel, virt-sparsify, and
virt-sysprep, as they are the main OCaml tools interacting with
user-provided images.
Expose via Common_utils the C functions & variables (part of guestfish)
that handle decryption of LUKS partitions, and the additional command
line arguments to tune the way they work. This way it will be easy to
provide (basic) crypto support also in OCaml-based tools.
Related to: RHBZ#1362649
With the current implementation, the root inode of the given partition
is ignored.
The root inode is now reported. Its name will be a single dot '.'
reproducing the TSK API.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
On Debian family of OSes Grub2 tools are prefixed with 'grub-', not with
'grub2-'. We have to detect the correct name of the tool to use it.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
When creating the XML for the new guest, always put the name of the pool
containing the disks, even when -os specified a pool UUID: libvirt does
not handle pool UUIDs for storage, but only names.
When running in verbose mode for debugging, run `glance --version` to
print the version of the glance client. This helps when dealing with
failures, or making its output actionable.
The glance client v1.0.0 defaults to the Image v2 API: this means that
an image can be referenced only by its UUID, and not anymore by the name
as well (which does not need to be unique). This caused our glance
invocation to set the properties for the newly created image to fail,
since we are using the name as identifier.
Instead of first creating the image and then setting all the properties
for it, set all the properties when creating the image: other than being
simpler, it also avoid parsing the output of the 'glance image-create'
command for the UUID ('id') of the image.
Create a new module [Linux_kernels] which does all kernel detection,
and also provides a place to define the kernel_info data structure.
This is essentially just code motion.
If the guest has no <Name> element in the OVF, previously we chose
"default" as the name. This changes that so it uses a name derived
from the basename of the OVA file instead.
For example:
virt-v2v -i ova /path/to/myguest.ova [...]
would use "myguest" as the name (assuming no <Name> was present).
Modifies the behaviour of
commit 1ae4252c93.
Upstream Perl is going to remove '.' from @INC (the include path for
modules) by default for the next major release (= 5.26) [1], as measure
to fix security issues. Debian already started backporting the fixes
for this [2], thus behaving this way in current Sid installations.
Since the affected Perl sources are only the local daemon testing
scripts, a simple fix is to force the 'requires' for the local
captive-daemon.pm module to start from the current directory: this way
there is no need to manually augment @INC, and only our local module is
loaded automatically.
[1] https://rt.perl.org/Public/Bug/Display.html?id=127810
[2] https://lists.debian.org/debian-devel-announce/2016/08/msg00013.html
Make use of the recently added 'getprogname' module in gnulib: replace
our guestfs_int_program_name with the getprogname() provided by the
module, since it does the same thing, and in a portable way.
As consequence of the above, use gnulib in a couple of tests that use
getprogname().
Since guestfs_int_program_name is gone, drop the configure checks
associated with it.
When running 'make maintainer-check-extra-dist', check also that all
generated files are included in the tarball. This is done so that end
users will not need OCaml to compile from tarball releases.
Previously we used an awkward hack to split up the large src/actions.c
into smaller files (which can benefit from being compiled in
parallel). This commit generalizes that code, so that we pass a
subsetted actions list to certain generator functions.
The output of the generator is identical after this commit and the
previous commit, except for the UUID encoded into tests/c-api/tests.c
since that is derived from the MD5 hash of generator/actions.ml.
This mostly mechanical change moves all of the libguestfs API
lists of functions into a struct in the Actions module.
It also adds filter functions to be used elsewhere to get
subsets of these functions.
Original code Replacement
all_functions actions
daemon_functions actions |> daemon_functions
non_daemon_functions actions |> non_daemon_functions
external_functions actions |> external_functions
internal_functions actions |> internal_functions
documented_functions actions |> documented_functions
fish_functions actions |> fish_functions
*_functions_sorted ... replacement as above ... |> sort
Commit ab2df2e659 attempted to rewrite
/etc/hosts if it contained the old hostname.
However this wasn't done reliably for a couple of reasons:
(1) Certain Debian/Ubuntu /etc/hostname contains
"localhost.localdomain" but still has "unassigned-hostname" or
"unassigned-hostname.unassigned-domain" in /etc/hosts.
(2) Even if (1) doesn't apply, you still need to split out the
hostname and domainname parts and deal with them separately.
Use /etc/os-release as first option, translating the distro name to the
current identifier used. The other options (the release files) are left
as following checks, avoiding them if any matches.
Don't remove the Processor and Intelppm nodes since that just breaks
the device driver.
The only remaining node being removed by the original code was
"rhelscsi" (the xenpv-win driver). I changed this so that instead of
deleting the whole node, it simply disables that driver. If you look
at RHBZ#737600, it's not even clear that deleting nodes was the right
fix for anything. I also renamed the function.
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
When put on new virtual hardware Windows will start driver installation
for newly discovered devices.
The problem is that it happens asynchronously and concurrently with
other activities, in particular, the firstboot scripts. This may result
in conflicts (because a firstboot script may want to install or
uninstall a driver, etc.) and eventually in the system left in
inconsistent state.
In order to prevent it, add another firstboot script which calls a
special utility, pnp_wait, whose sole purpose is to wait until all
PnP-related activities are finished. (It does so via a WinAPI call
which isn't available from a script so it has to be a compiled .exe.
The source code for it can be found in the corresponding directory in
https://github.com/rwmjones/rhsrvany). This firstboot script is put
first, so that other firstboot scripts do not have to worry about
interactions with PnP manager any more.
One caveat is that on Windows XP and Windows 2003 the PnP manager always
fires the "Found New Hardware" wizard, which doesn't allow PnP to make
any progress until the user closes it. As a result, this firstboot
script would never finish.
To work it around, follow the Microsoft KB
(https://support.microsoft.com/en-us/kb/938596) and set up a registry
key to make the the PnP manager not fire the wizard; the key is restored
to its initial state upon PnP completion. Unfortunately this only works
on systems having the mentioned update installed; otherwise the user
will have to interact with the UI.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
It will be used in new code in a followup patch.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
If multiple tests (eg. from different branches) are running in
parallel, they will attempt to use and delete the same temporary pool.
Use a random pool name to avoid this.
Other functions, such as ``-i ova'', also call Xml.parse_memory.
Therefore the error is not always from XML generated by libvirt, the
XML might be generated from other sources too.
Thanks: Ming Xie.
Following the same style used for the firstboot tests in the
customize/ subdirectory, use a separate test script for each real
guest that we test. This has two advantages: (a) the tests run in
parallel (and are thus faster) and (b) automake "knows" about each
test and reports the results separately.
Methods in OCaml which don't take any parameters don't require the
dummy unit arg, ie writing:
method foo = ...
is fine. The reason you might need the unit arg is if you need to
create a closure from the method without calling it, for example if
you need to use the method in a callback.
In lablgtk2 the convention is to use unitless methods if either: the
method shouldn't be used as a callback; or: (conceptually) the method
doesn't change the object's internal state. Let's do that here.
The name of the initrd image on Debian-based systems is different from
what used on Fedora/RHEL/SUSE and derived; set a different regexp to
avoid making the current regexp even more complex.
Implement the 'remove', 'file_list_of_package', and 'file_owner' methods
of the Linux module for the "deb" package manager (dpkg basically, on
Debian and derived distributions).
Also allow it for the main conversion code.
Move the actual job in an helper function, so the common bits (like the
check of the size of 'packages' and the reload of Augeas) can be done
for all the package manager implementations.
This should be code motion with no behaviour change.
Old perl/POD from Debian Wheezy has a bug where <name>NAME</name>
gives the error:
Unknown E content in E</name>
even though this is in a verbatim section. Fudge the documentation to
avoid this.
In OCaml 3.12, type inference was not as smart and could not use local
information to disambiguate same-named fields in different structures.
The easiest fix for this is just to rename the field so it doesn't
have the same name as a field in the Customize_cmdline.ops struct.
In Debian Wheezy with libvirt 0.9.12.3, HAVE_LIBVIRT was defined, but
HAVE_LIBVIRT_BACKEND was undefined, so src/libvirt-auth.c failed to
compile.
It's easy, and better, to check the minimum version in the configure
script.
Create an object hierarchy to represent different bootloaders for Linux
guests, moving the separate handling of grub1 and grub2 in different
classes: this isolates the code for each type of bootloader together,
instead of scattering it all around.
This is mostly code refactoring, with no actual behaviour change.
String.map was added in OCaml 4.00.0. However we use this function
to implement String.lowercase_ascii etc.
Therefore include a definition of the function for older versions of
OCaml. (Debian Wheezy has OCaml 3.12.1.)
By adding common CLEANFILES and DISTCLEANFILES variables to
common-rules.mk, we can remove these from most other Makefiles, and
also clean files more consistently.
Note that bin_PROGRAMS are already cleaned by 'make clean', so I
removed cases where these were unnecessarily added to CLEANFILES.
There was some code in configure.ac dating back to 2009
(commit b9014d6a0d) which attempted to
add the javac option `-source 1.5'. I don't think this code ever
worked.
However, if -source 1.5 is added, then you get this warning:
warning: [options] bootstrap class path not set in conjunction with -source 1.5
warning: [options] source value 1.5 is obsolete and will be removed in a future release
warning: [options] To suppress warnings about obsolete options, use -Xlint:-options.
and that's very hard to reliably fix because you have to know somehow
a magic path. For more details, see:
https://blogs.oracle.com/darcy/entry/bootclasspath_older_sourcehttps://blogs.oracle.com/darcy/entry/how_to_cross_compile_for
Anyway, remove the voodoo from configure.ac.
As a side effect, this allows you to set the EXTRA_JAVAC_FLAGS
environment variable before ./configure with any extra javac flags
that you want.
Add also /boot/efi/EFI/redhat/grub.conf as configuration of Grub 1;
since the "grub" lens of Augeas does not handle this path, add a
transformation so Augeas can parse it.
Expose the aug_transform API through the library, so it's possible to
add/remove Augeas transformations to handle files in custom places using
existing lenses.
Add new scripts for:
- guestunmount
- virt-copy-in
- virt-copy-out
- virt-customize
- virt-dib
- virt-diff
- virt-get-kernel
- virt-p2v-make-disk
- virt-p2v-make-kickstart
- virt-p2v-make-kiwi
- virt-tar-in
- virt-tar-out
Also combine the separate virt-resize script into the general script
virt-alignment-scan. There wasn't really any reason to have separate
scripts.
By moving these two functions out of the common options parsing code,
it means we don't need to depend on all the other machinery of options
parsing, such as the global variables ("verbose"), libconfig, etc.
The code to parse btrfs subvol entries in /etc/fstab failed if the
entry had more than one comma-separated option, for example:
/dev/sda4 /home btrfs rw,user,subvol=foo 0 0
This commit fixes that code to use Augeas correctly.
Fixes commit 7ba0e10501.
Reported by: Zhongfu Li
https://bugs.launchpad.net/ubuntu/+source/libguestfs/+bug/1615337
Libvirt >= 2.1.0 now allows you to open files which have a "json:"
QEMU pseudo-URL as backingfile, whereas previously it would fail hard
in this case (RHBZ#1134878).
When virt-v2v performs conversions from Xen (over SSH) or vCenter
(over HTTPS) it uses these pseudo-URLs as backingfiles. We had to
tell people to use LIBGUESTFS_BACKEND=direct to avoid libvirt in this
situation.
This commit narrows the check so it will now only print the error if
libvirt < 2.1.0 and LIBGUESTFS_BACKEND=direct is not set. Also the
error is modified to tell users they can either upgrade libvirt or set
LIBGUESTFS_BACKEND=direct to work around the problem.
Note there is not an easy way apart from checking the version number
of libvirt to tell if the json pseudo-URL is supported.
As a side-effect, this commit also prints the libvirt version number
in debugging output when virt-v2v starts up, which is sometimes useful
information for narrowing down bugs (it is in fact already printed by
libguestfs, so this is duplicate information, but it's a bit easier to
find when it's at the top of the debug).
Thanks: Peter Krempa.
Make 'which' gracefully handle the case where $PATH is not set
(it will raise Executable_not_found, but that is the expected thing to
do).
Related to RHBZ#1367839.
From RHEL 7.3, Red Hat have decided to only compile the secure boot
version of OVMF on x86-64, with flags -D SECURE_BOOT_ENABLE -D SMM_REQUIRE.
The filename has also changed to reflect this - it is now
/usr/share/OVMF/OVMF_CODE.secboot.fd. The old file
/usr/share/OVMF/OVMF_CODE.fd is no longer shipped.
However switching to using this variant of OVMF for UEFI guests is not
just a matter of changing the filename. The new OVMF variant won't
run unless we also change:
- The qemu machine model, from the default ("pc" ==
"pc-i440fx-rhel7.3.0" or later) to Q35 ("q35" == "pc-q35-rhel7.3.0"
or later).
- Add <smm> under <features>.
- Set <loader ... secure="yes">.
NB: On RHEL the changes requires qemu-kvm-rhev. It is no longer
possible to convert UEFI guests using the basic qemu-kvm.
Thanks: Laszlo Ersek, Ming Xie.
Previously we had a list of UEFI paths in src/uefi.c, which was
accessed in virt-v2v using a private (guestfs_int_*) API and some C
binding code. This was clumsy and required the paths to be replicated
in the virt-v2v unit tests.
Instead just generate the list of paths from the generator, creating
src/uefi.c and v2v/uefi.ml with the same content.
Remove the C bindings and the virt-v2v unit tests associated with UEFI
paths.
This *may* be required by some filesystems in order for fstrim to
work. I'm not actually sure if this is true, but it's what
virt-sparsify --in-place does, and that utility has been tested a lot
more in regards to trimming.
This reverts the change made for RHBZ#1168144. The warning is now
always displayed.
It would be nice to make the warning actionable, but there is not a
lot that end users can do since fstrim is such a complex topic
interacting with all filesystem and storage layers.
Do not print warning for 'sr' devices when converting fstab. Not all
systems create the /dev/cdrom symlink for SCSI CD-ROM devices. Moreover,
on systems with multiple CD-ROMs, having entries for /dev/sr* devices
may be inevitable.
RWMJ: Use String.is_prefix instead of String.find, to more accurately
match on the device name.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Conversion from Citrix Xen has now been tested and should work. Remove
the comment about it from the documentation.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The replacement string was wrong. There are only two match groups in the
regular expression, not three.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The Invalid_argument exception is there to catch unexpected situation
when rpm returns no output. Such situation should be reported rather
then hidden.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Because we run the external fstrim command we don't have access to the
kernel errno when it fails. However in the case where it prints this
specific error message, turn that into errno ENOTSUP.
Add makefile variables to enable silent rules for simple command
invocations, such as ocamlc, ocamlopt, javac, and erlc.
This reduces the log output when building with silent rules, still
showing the full command lines otherwise.
Allow to specify the format of the helper drive, to avoid relying on the
format autodetection. This is the same approach used also in all the
other tools.
Turn the rc.d symlinks for the guestfs-firstboot service as relative,
instead of absolute paths: the result is the same (the service works the
same), and this way is more coherent with symlinks created by
update-rc.d.
Currently we install a systemd service named firstboot.service and a
SysV service named virt-sysprep-firstboot. On systems where systemd is
the init system and runs with the SysV compatibility, the different
names make systemd handle them as different services, and thus trying to
run the firstboot script runner twice.
Rename both the systemd service and the SysV one to guestfs-firstboot:
the new name is less generic, and allows the systemd service to be
shadowed by the SysV service (and thus running just once).
Also cleanup the old services: the old SysV service can be removed
directly, since its former name had "virt-sysprep" in it, and so there
could not be much room for confusion and conflict. Regarding the old
systemd service: to avoid leaving it behind, a simple cleanup strategy
is in place, checking the content of the old firstboot.service to really
ensure we are removing one of our versions of this file.
Export FS_TYPE for the elements, so they can know what is the filesystem
created.
This was implemented in diskimage-builder upstream as
commit be521bdec6233530cc952be246a66f1957b0dd58.
Implement the "docker" output format, to import an image into docker.
This was implemented in diskimage-builder upstream as
commit 1187f9d5df6d6eddba478ac75d3834391480755f and
commit edc06a20e57bdf0a9a03949b54395008d73520dc.
Instead of running them before lanching the appliance with the disks and
then uploading the result after root.d hooks run, mount the root in the
local temporary directory using FUSE and then run the hooks on the
guest. Other than being closer to what diskimage-builder does, it also
avoids issues with the extra-data.d scripts assuming to find things
already, and we don't error out while trying to unpack files on the
guest.
Since virt-dib requires FUSE now, build it only if FUSE is supported.
run_command uses Unix.create_process which forks a child process, and
executes execve: the latter fails when the executable does not exist,
triggering the exit which, in older OCaml versions [1], also runs the
at_exit handlers.
Since there is not much that can be done to avoid this on the OCaml
side, to keep run_command working also in older OCaml version then
manually search for the existance of the given executable, exiting with
code 127 (as a shell does) in this case.
[1] http://caml.inria.fr/mantis/view.php?id=7209
If the user doesn't specify a format (ie. they want autodetection),
and the drive is set to read-only, previously we created a qcow2
overlay to protect the drive but didn't set the backing_fmt field.
However libvirt disabled this so now libvirt doesn't work at all on
qcow2 files that have no backing_fmt field set.
We must therefore do autodetection in libguestfs and use that to set
the backing_fmt field.
We were already doing autodetection in the non-readonly case, so this
commit refactors that code into a new function
("get_source_format_or_autodetect") and uses it on the readonly path
too.
Because we didn't give the type of 'g' there wasn't sufficient
information for the type checker to pick up this problem. By
specifying the real type of 'g' in a type annotation, that reveals the
problem.
Thanks: Pino Toscano, Nyoxi on IRC.
What was happening in file_owner function did not match the description
in the comment. When a path is owned by multiple packages the returned
string was in fact a concatenation of the names of all packages that own
it. E.g. for `Linux.is_file_owned g inspect "/etc"` the returned value
was "filesystemyum" (i.e. "filesystem" + "yum").
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
$OSINFO_SYSTEM_DIR (or its /usr/share/osinfo fallback) represents the
top-level directory of the osinfo data, while we need to read the "os"
subdirectory of it.
Fixes commit c2ae46a9eb.
Directly query the attributes in the media node, instead of using an
XPath expression to read it.
Saves ~10ms when parsing the libosinfo DB on a Fedora 24 installation
(other than making the code easier).
Check the presence of Gtk properly depending on the value of --with-gtk:
if a specific version is chosen, then let PKG_CHECK_MODULES fail if that
version if not found, otherwise fallback from gtk3 to gtk2 to no gtk.
Also move few common AC_SUBST in a single place.
It looks like flex 2.6.1 changed [1] the return code for EOF in
yyinput. Therefore, use the right value depending on the version of
flex which generates the lexer.
[1] f863c9490e
Setup the volatile /run in the appliance also with the tmpfiles
configurations available. In particular, setting up correctly the lvm
bits allow lvmetad to run.
Commit 2e16e3e993 added lv_active=active
as additional condition when listing LVs, to ignore those with the
activationskip flag set. OTOH, this check is too broad, and matches
also other kind of LVs.
Change the condition to lv_skip_activation!=1, so matching precisely
what was meant, and only that.
Related to: RHBZ#1306666
Currently lvmetad is started in init, and thus using the system
(= appliance) configuration of lvm. Later on, in the daemon, a local
copy of the lvm configuration is setup, and set it for use using the
LVM_SYSTEM_DIR environment variable: this means only the programmes
executed by the daemon will use the local lvm configuration, and not
lvmetad.
Thus manually start lvmetad from the daemon, right after having setup
the local lvm configuration, and still without failing if it cannot be
executed.
Additionally, since lvmetad now respects the right configuration, make
sure to update its cache when rescanning the VGs by passing --cache to
vgscan.
When editing the lvm configuration to set the LVM filter, edit the
'global_filter' key in addition to 'filter': the latter is not used when
lvmetad is running, when only the former works.
Commit b91b39e06a changed the separator to
':', although this creates parsing issues when there are fields with
colons (for example lv_tags=imgbased:pool).
Change once more the separator, but this time using a non-printable
character such as '\r': while it will produce uglier debug logs, this
should greatly reduce the possibilities of conflicts with texts of
metadata.
Switch to guestfs_int_version_from_x_y_or_x to parse version numbers --
although, restrict the parsing to what could look like a valid version
number, to discard quickly version strings like "unknown".
This makes sure that Debian, Fedora, Mageia, and SLED ISOs have the
right version number.
More recent versions of libosinfo switched the internal directory with
the XML files of OSes to a different layout (still with the same XML
format), causing libguestfs to not read them anymore. Furthermore, the
internal directory is going to disappear soon, replaced by a public
osinfo database [1].
Revamp the way libguestfs reads the data: first try the upcoming osinfo
layout, falling back to the current libosinfo layout (which is the same
as osinfo), and then to the old flat layout.
[1] https://gitlab.com/libosinfo/libosinfo/blob/master/docs/database-layout.txt
-Wnull-dereference and -Wshift-overflow are new warnings in GCC 6, so do
not try to disable them with pragmas on older GCC versions.
Fixes commit a8e15ea924.
One -Wnull-dereference warning is real: we deliberately cause a
segfault in one of the tests.
There is a -Wshift-overflow bug in a Gtk 2 header.
The others are the result of shortcomings in GCC.
In all cases we have to add GCC diagnostic overrides to ignore
the warnings when compiling with ./configure --enable-werror.
The commit claimed that SELinux was disabled in the
virt-p2v-make-kickstart script, and so this was just making things
consistent. However this is not true. SELinux *is* enabled in the
kickstart version. There is some code to disable it, but it is
commented out (admittedly it's hard to tell because the code is buried
in a multi-line sed expression).
The commit message also claimed (this time correctly) that the
'--selinux-relabel' step caused an extra reboot each time virt-p2v is
run from the ISO. However we have recently fixed this so it can
usually do the relabelling during the build, not at boot time, so this
is no longer a problem.
This reverts commit 25ffcc7d43.
For OCaml tools this does essentially nothing useful because the
--help output is automatically generated from the options, and so
cannot be wrong. However for C tools this is a useful check.
It would be nice to generate C tools --help output, but there isn't
enough information in the getopt data to do that.
This commit also includes fixes to the --help output for a few tools.
This function allows to download file system data units (blocks) from
the given partition.
The API can be used to detect data hidden within filesystem bad blocks
or slack space.
Moreover for filesystems such as Ext3 and Ext4, this function is the
only way to retrieve deleted files. An example is given in the function
tests.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Pass -q to virsh if virt-v2v was run with -q, to reduce its output in
quiet mode.
Unfortunately this does not currently work in virsh as it should, see
RHBZ#1358179, so only after that bug is fixed this change can actually
take effect.
ocamlmklib -g does not work in OCaml 4.01.0. It does work (and is
desirable) in OCaml 4.02.2. However since we would like to support
back to OCaml 3.11, we cannot use it.
In libguestfs 1.33.43 we managed to release a completely
non-functional virt-customize binary (it ran without error but did
nothing at all, whatever command line arguments you gave it).
Improve the test so this cannot happen again.
podcheck.pl is run as part of the tests to perform various checks on
the documentation and the tool.
Currently we check only that the documented options matches the
options that the tool implements and vice versa. This commit would
also allow us (in future) to check --help, --long-options,
--short-options, --version output.
This commit includes scripts to run the tests and various fixes to the
manual pages to ensure that the tests pass.
For guestfish, guestmount, remove '?' from short options. Currently
those tools don't process -?, so I believe these are erroneous:
$ guestfish -\?
Try `guestfish --help' for more information.
For virt-format, the -c, -d and -q options are removed. These options
just give errors because they appear in the short options list but not
in the case statement.
Store them directly in List_entries, an adding helper function to
convert from string.
Also use Getopt.Symbol for them, so there is no need to manually reject
invalid formats.
Instead of linking with individual objects, which is very tedious,
build a proper library and link virt-builder, virt-customize and
virt-sysprep to it.
This makes the binaries a tiny bit smaller because .cmxa/.a files
allow unused code to be removed by the linker, whereas explicitly
linking .cmx/.o files does not.
Instead of linking with individual objects, which is very tedious,
build a proper library and link the other tools with it.
This doesn't make the resulting binaries any larger.
This implements the --selinux-relabel option for virt-customize,
virt-builder and virt-sysprep. There is no need to autorelabel
functionality now.
Thanks: Stephen Smalley
Sadly, the dhclient-script shipped as part of isc-dhcp-client in Ubuntu
unconditionally reads from /etc/fstab without checking for its
existence. Since no package holds /etc/fstab, this file will not exist
in the appliance, cause dhclient to fail (actually keep looping calling
the failing dhclient-script) when the network is requested.
As a workaround, touch /etc/fstab just before enabling the network: if
that file exists nothing changes, while an empty file will be available
in the other case, making at least dhclient-script in Ubuntu working.
Since the function uses the StringMap module, I also had to
move this to mllib.
The function is renamed and the 'if verbose ()' part is moved into the
function, but apart from that, it's all just code motion.
Add a dummy description value to mark an option as "hidden", so it will
not be shown in the help text.
Mark few options as hidden:
- common: --short-options, --long-options, --debug-gc
- virt-sysprep: --dump-pod, --dump-pod-options
Since --debug-gc is now really considered internal, it is no more
documented.
Add a new Getopt module to mllib, to parse command line arguments with
handlers close to the ones used with Arg, but using getopt(3) (actually
getopt_long_only) to do the real parsing. This allow us to provide
options for OCaml tools with a syntax similar to the C tools, and use
the additional features getopt offers and Arg does not.
Getopt now handles every part of the command line handling, including
the output of short & long options.
Do a single-step conversion of Common_utils and all the OCaml tools to
the syntax of Getopt.
Move a couple of utility functions from Common_utils to Getopt, since
they fit better there (and Common_utils cannot be used in Getopt, as
the former already uses the latter).
As side-change due to the conversion, extra arguments for sysprep
operation can have more keys for the same argument.
In previous commit c5f12e47e4 we changed
the PATH environment variable to run the locally built virt-v2v, not
the system-installed virt-v2v.
However this meant that we were running the locally built virt-v2v
against the system-installed libguestfs.so.0. We also need to set
LD_LIBRARY_PATH.
Fixes commit c5f12e47e4.
In the case where we are building a Python dist (which can be
distributed separately), we generated calls to non-existent optargs
structures if an API was once no-optargs but had optargs added. A
typical error was this (library: 1.32.5, Python sdist: 1.33.42):
guestfs-py.c: In function ‘guestfs_int_py_glob_expand’:
guestfs-py.c:9648:40: error: storage size of ‘optargs_s’ isn’t known
struct guestfs_glob_expand_opts_argv optargs_s;
This is a simple solution to this, although it has the possible
disadvantage that such functions will not be bound at all.
Since commit 83e92b4a97, utils.c
includes "ignore-value.h". We copy utils.c into the python sdist
tarball, but it didn't not compile because of the missing header file.
Therefore we need to copy the header in too.
Fixes commit 83e92b4a97.
guestfs_set_uuid wants device as argument.
Moreover, btrfstune -U is unable to set uuid for subvolumes,
only unmounted partitions are supported.
Here we skip btrfs subvolumes.
Calling guestfs_is_lv on btrfs subvolume throws an error.
Here we workaround it by taking Mountable instead of Device
and returning 'false' for non-device mountables.
Access, modification, last status change and creation time in
Unix format as for statns.
Number of links pointing to a given entry.
If the entry is a symbolic link, report the its target path.
A new flag (DIRENT_COMPRESSED 0x04) indicating whether the file is
compressed using native filesystem compression support.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
This adds imperative list manipulation functions inspired by Perl.
The functions are passed list refs which get updated in place.
This allows us to replace some awkward pure functional code like:
let xs = ys in
let xs = if foo then xs @ zs else xs in
with:
let xs = ref ys in
if foo then append xs zs;
These are now passed using a curl configuration file, which is a
little bit safer than using command lines. virt-builder doesn't need
to pass usernames and passwords to curl, but if it ever does in future
this will be a lot safer.
Change the Curl module to use an ADT to store the name of the curl
binary and the arguments.
The callers in virt-v2v are changed accordingly.
This also adds a (currently unused) ?proxy argument to allow callers
to override the proxy. It also adds some safety arguments implicitly.
Declare most of the stringsbuf as CLEANUP_FREE_STRINGSBUF, so they are
freed completely on stack unwind: use take_stringsbuf() in return
places to take away from the stringsbuf its content, and remove all the
manual calls to free_stringslen (no more needed now).
This requires to not use free_stringslen anymore on failure in the
helper functions of stringsbuf, which now leave the content as-is (might
be still useful even on error).
This allows us to simplify the memory management of stringsbuf's, which
are not properly fully freed, fixing memory leaks in some error paths
(which were not calling free_stringslen).
If add_string_nodup fails free the passed string instead of leaking it,
as that string would have been owned by the stringbuf.
Adapt few places to this behaviour.
We have to download the old 32 bit explorer.exe in order to find the
icons as the 64 bit version doesn't contain any icons (where are they?).
Thus prefer the 32 bit (WoW64 subsystem) directory if found.
Fixes commit 7f16c346bb.
Thanks: Xiaoyun Hu
The current location doesn't exist unless you've installed GNOME,
which is not so common on Ubuntu. Unfortunately I couldn't find any
other location containing a clean, high quality logo.
This adds another low quality icon source, and also prevents any icon
being returned if the highquality flag was set (note this prevents
virt-manager from displaying an icon, but there's nothing we can do
about that, and it's no worse than the current situation).
Updates commit 1d0683964f.
Thanks: Xiaoyun Hu
This is now enabled by default and cannot be changed/disabled by the
user. The output is saved into /tmp/.../virt-v2v-conversion-log.txt
on the conversion server.
This makes sure that in the internal configure_kernel_modules function,
for virtio or SCSI block types:
a) the warnings about leftover Xen modules are printed
b) the changes in Augeas are saved
Fixes commit ee02191483.
First check for the existence of the directory /etc/modprobe.d, in case
using a file under it; this also skips all the other checks, since they
are not needed at all. Also /etc/modprobe.d exists on recent Linux
versions, so let's give priority to the more common methods.
When /etc/modprobe.d does not exist, check for the file to edit using a
single list of possible files, now in order of priority, where the first
find is used without checking further for the rest.
Also, make sure all the returned paths are absolute: they are used in
Augeas paths later on, so relative paths will not do anything useful.
We are now able to cancel the conversion instantly by sending ^C to
the remote virt-v2v process.
Also, this reverts:
"p2v: Poll to make Cancel Conversion button more responsive."
(commit 6da4941db7)
Previously we copied / creates files in the remote dir by running
complex shell commands like:
cat > file <<'EOF'
## the file was copied in here
EOF
This was a little hairy, but in particular it doesn't allow us to set
the ssh session to cooked mode (so we can send ^C to cancel a
conversion).
A cleaner way to do it is to use 'scp' to copy the files over.
The messages that come "through" from virt-v2v are already colourized.
However the other messages are not. This colourizes the ones
generated during kernel-mode conversions.
Note that because of the way journal/syslog works we have to write the
ansi_restore before the end of line, thus code like this:
ansi_magenta (stdout);
printf ("%s: %s", guestfs_int_program_name, data);
ansi_restore (stdout);
putchar ('\n');
This also adds a "Conversion finished successfully" message (in green).
Thanks: Ming Xie
Add powerpc64 (ppc64) and powerpc64le (ppc64le) cases to the
normalize_arch function. Also remove a few duplicate cases which are
already caught in the catch-all case at the end of the match.
In kernel conversion mode, we must ensure messages are flushed to the
journal as soon as they are received from virt-v2v, otherwise (now
that debugging is not the default) we won't see any messages at all
until the last moment.
Thanks: Ming Xie
This allows you to install extra packages in the disk/ISO. The
implementation of this option in virt-p2v-make-disk was particularly
simple and followed naturally from the previous commit.
- sanitize the logs, removing terminal color codes and carriage returns
- add an empty "properties" node
- put the log for skipped tests as "message" attribute, hoping it is
read from there
- do not blacklist the log of test-virt-rescue.pl, which should not
cause issues now
If an ssh server is completely unresponsive (ie. it doesn't even send
back ICMP errors), then eventually ssh times out after ConnectTimeout
seconds [ConnectTimeout is an ssh option].
Unfortunately the miniexpect timeout (default 60 seconds) was less
than ConnectTimeout so we never saw the ssh error. Instead we would
see the internal error:
remote server closed the connection unexpectedly,
waiting for: password prompt
This commit sets the ssh ConnectTimeout to an explicit value, and sets
the miniexpect timeout to be 20 seconds larger than ConnectTimeout, so
that in most cases we will see the ssh error message before miniexpect
times out.
Thanks: Ming Xie for finding and diagnosing the problem
In the GUI disks block, instead of using separate "Size" and "Model"
columns, combine that information into a markup column under "Device".
To make the GUI look consistent I also had to change the removables
"Device" column to be a markup column and embolden the device name
there too.
This is just refactoring, but it reveals and fixes a bug too. The
size_gb field was left as NULL when the --test-disk option was used.
Apparently passing NULL to gtk_list_store_set is fine, but just in
case I replaced it with "".
The error you would see is:
virt-v2v: error: insufficient free space in the conversion server temporary
directory /var/tmp (853.8M).
Either free up space in that directory, or set the LIBGUESTFS_CACHEDIR
environment variable to point to another directory with more than 1GB of
free space.
See also the virt-v2v(1) manual, section "Minimum free space check in the
host".
Also adds some documentation.
Thanks: Ming Xie and Xiaodai Wang
There is a tiny functional change in this patch, since overlay_dir is
now always evaluated once (eg. even in --inplace mode), whereas
previously it was evaluated twice but only in copying mode.
As described in the comment, this solves a number of problems with
non-standard remote configurations.
I tested this with:
- tcsh
- zsh
- ksh
and all behaved correctly.
Print a better error message when the non-root user on the conversion
server requires a password to use sudo, and p2v is told to use sudo.
See also RHZ#1340809.
Library's counterpart of the daemon's internal_filesystem_walk command.
It writes the daemon's command output on a temporary file and parses it,
deserialising the XDR formatted tsk_dirent structs.
It returns to the caller the list of tsk_dirent structs generated by the
internal_filesystem_walk command.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The tests check whether the filesystem_walk command is able to retrieve
information regarding both existing and deleted files.
A NTFS image is used as Ext3+ filesystems deletion is more aggressive
in terms of metadata removal.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
- generator: Added tsk_dirent struct
The tsk_dirent struct contains the information gathered via TSK APIs.
The struct contains the following fields:
* tsk_inode: inode of a file
* tsk_type: type of file such as for dirwalk command
* tsk_size: file size in bytes
* tsk_name: path relative to its disk partition
* tsk_flags: bitfield containing extra information
* tsk_spare[1-5]: extra space for future usage
- configure: Added libtsk compile-time check
Ensure libtsk is available at compile time.
If not, daemon routines depending on it won't be available.
- API: internal_filesystem_walk
The internal_filesystem_walk command walks through the FS structures
of a disk partition and returns all the files or directories
which could be found.
The command is able to retrieve information regarding deleted
or unaccessible files where other commands such as stat or find
would fail.
The gathered list of tsk_dirent structs is serialised into XDR format
and written to a file by the appliance.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The previous code treated floppy disks and CD-ROMs as the same kind of
thing, resulting in malformed libvirt XML. You would see the
following error when importing a guest into libvirt:
error: Failed to define domain from /tmp/v2vlibvirt063486.xml
error: internal error: Invalid floppy device name: hdb
because we incorrectly generated this bogus libvirt XML fragment:
<disk device='floppy' type='file'>
<driver name='qemu' type='raw'/>
<target dev='hdb' bus='ide'/>
</disk>
This commit models floppy devices as a distinct type, occupying their
own bus ("/dev/fdX"). When writing to libvirt, we generate correct
XML fragments, looking like this:
<disk device='floppy' type='file'>
<driver name='qemu' type='raw'/>
<target dev='fda'/>
</disk>
Miscellaneous other changes were required in the code. There is also
a regression test (see following commit).
Note this ignores floppy disks in '-o qemu' mode.
aarch64 image isn't buildable at the moment because of
Anaconda bug RHBZ#1348980.
ppc64 & ppc64le images are not buildable because of delays updating
the Fedora mirrors.
This adds <BootOrder>1</BootOrder> to the first drive, which is
necessary to make the drive bootable (or would be necessary, if oVirt
hadn't implemented their own workaround for the OVF we were previously
generating).
https://bugzilla.redhat.com/show_bug.cgi?id=1308535#c11
This is not a full solution to the problem. Really we should be
copying the boot order over from the source hypervisor, but that
requires extensions to libvirt as discussed here:
https://bugzilla.redhat.com/show_bug.cgi?id=1308535#c7
Thanks: Shmuel Melamud
e2fsck returns 1 in case of "file system errors corrected".
We treat it as success in normal e2fsck, but fail if e2fsck
is run by resize2fs.
Change 'manual' execution of e2fsck to dedicated function call.
On distros not running NetworkManager, use the nm-online -x parameter
to exit quickly. The network connection will be checked just after
anyway when attempting to connect to the conversion server using ssh.
Kiwi is the tool used by openSUSE / SLES to generate many sort of
disk images. Add a virt-p2v-make-kiwi tool and his documentation to
geneate the p2v appliance kiwi configuration.
Use the common denominator for SLES and openSUSE in the dependencies.
For example most of NetworkManager pieces and metacity aren't provided
on SLES.
ifconfig is in the net-tools-deprecated package in openSUSE Factory,
which means after openSUSE Leap 42.2. Older versions have it in the
net-tools package. Adding this complexity only to add ifconfig because
sysadmins prefer it isn't too good: iproute2 is doing the job.
The password prompt is actually generated by the openssh client which
is under our control, so we can regexp match on the specific prompt
printed by openssh. This also avoids mistaking any server-side
issue.net message for a password prompt (eg. if the server prints a
message like "all users must change their passwords today!") which
would have prevented virt-p2v from logging in.
This can fail because the username is wrong. Also don't unnecessarily
reveal irrelevant implementation details in the error message, just
say the login failed.
Thanks: Juquin Zhou
Previously cancelling the conversion only set a flag, which was
checked when the run dialog displayed new output from virt-v2v. When
virt-v2v was showing hundreds of debugging messages, this wasn't a
problem, but now that we are hiding those messages, cancelling the
conversion might mean a wait of seconds or minutes.
By polling (albeit infrequently) we can make the cancel button more
responsive.
For some failures, 30 lines was not sufficient and only part of the
error was shown. Increase the number of lines shown to 50.
This also colourizes the failure message, and prominently displays the
location of the full log. The following message is now shown:
*** virt-v2v command failed ***
The full log is available on the conversion server in:
/tmp/virt-p2v-20160620-sga9rhk7/virt-v2v-conversion-log.txt
Only the last 50 lines are shown below.
[followed by up to 50 lines of log]
Updates and fixes commit 7447fe2478.
PLD Linux got /etc/os-release only in the recent 3.0 release; since
older versions have only /etc/pld-release, check for it to identify
the guest and get its version.
Previously we displayed the complete output of virt-v2v in the run
dialog. This output included all the debugging messages, and was very
long and confusing for users (especially we had false bug reports
about "errors" appearing in the debug output).
Only display stdout in the run dialog. However make sure everything
(stdout and stderr) is still logged to the conversion log.
Instead of constructing and directly executing a long virt-v2v command
line, build a wrapper script with the same command line and send it to
the remote server (stored in /<remote_dir>/virt-v2v-wrapper.sh).
This will make it a bit easier to construct more complex virt-v2v
wrappers.
Note this commit on its own is a simple refactoring and does not
change any functionality.
This option (alternately spelled: --color, --colour, --colors, or
--colours) enables ANSI colour sequences output even if that would be
disabled becaues the output is not a TTY.
The debug() function is already sending these to stderr, but in a few
places we were using printf. Change those to eprintf, except for one
informational message which should have been using info().
The virt-p2v slow test tested the local virt-p2v. However it ran
against the installed virt-v2v (and in fact would have failed if
virt-v2v was not installed).
This commit sets the environment up to run the locally built virt-v2v.
This is unfortunately quite a lot more complex than it should be.
There is no simple "set this environment variable" option in
sshd_config.
It makes no sense. By setting one field to non-sensitive when the
other field is populated, we can avoid this happening, and also make
tabbing between the fields simpler.
As a consequence of making this change, I also got rid of the now
unnecessary explanatory text telling you to leave one field blank.
In particular if you hit the 'Test Connection' button when the server
name field was empty, you would still see a stopped spinner.
This updates commit 036a11f379.
By changing the logging mode of the p2v.service unit, we can make sure
that the progress of virt-p2v is visible in non-GUI mode at the
console.
Since the output is now always shown on the console, there's no need
to print the journalctl command for following that output when the
user logs in.
This simulates doing a non-GUI conversion with virt-p2v being
downloaded over PXE. Conveniently we already had a test that was
doing this, and this rule just runs the same test.
Instead of throwing away the ssh error and printing the generic
message "unexpected end of file waiting for password prompt", we
capture the ssh error and print it. The user will see the ssh
diagnostic, eg. "No route to host".
For some reason I don't quite understand, in
commit fd82bb12fd if it got the password
prompt unexpectedly a second time then it would send the password
again, continually until ssh closed the connection because of too many
retries.
This obviously makes no sense. If we get the password prompt back,
then we must have sent an incorrect password so abort the connection
and report the error at once.
OCaml 4.02 introduced the 'bytes' type, a mutable string intended to
replace the existing 'string' type for those cases where the byte
array can be mutated. In future the 'string' type will become
immutable. This is not the default now, but it can be forced using
the '-safe-string' compile option.
This commit changes the code so that it could be compiled using
'-safe-string' (but does not actually make that change).
If we detect OCaml < 4.02, we create a dummy 'Bytes' compatibility
module ((nearly) an alias for the 'String' module). The only
significant difference from upstream OCaml is that you must write the
'bytes' type as 'Bytes.t' in interfaces, apart from that everything
else should work.
Removes this warning:
File "common_utils.ml", line 826, characters 24-28:
Warning 40: F_OK was selected from type Unix.access_permission.
It is not visible in the current scope, and will not
be selected if the type becomes unknown.
In commit ae6f726ecc we started to use
the virt-customize code to replace various virt-sysprep operations.
This had the effect of adding many more possible operations to
virt-sysprep, but some of them (specifically --install) did not work
unless the appliance network is enabled. It was not enabled in
virt-sysprep, so these operations never worked.
This change does NOT enable the network by default. However it adds a
--network flag which may be used in conjunction with --install etc to
make those commands work.
In addition we now emit a warning for certain customize operations if
they fail and if the network is not enabled. It will print:
[ 4.5] Installing packages: tcpdump
Error: Cannot retrieve repository metadata (repomd.xml) for repository: base. Please verify its path and try again
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os&infra=stock error was
virt-sysprep: warning: the command may have failed because the network is
disabled. Try either removing '--no-network' or adding '--network' on the
command line.
virt-sysprep: error: yum -y install 'tcpdump': command exited with an error
(If the network is already enabled, or if the command is successful,
then the warning is not printed.)
Thanks: Xianghua Chen
You will now see an error such as:
$ virt-customize -a centos-6.img --truncate-recursive /home/foo
[ 0.0] Examining the guest ...
[ 16.5] Setting a random seed
[ 16.5] Recursively truncating: /home/foo
virt-customize: error: libguestfs error: find0: /home/foo: No such file or
directory
Thanks: Xianghua Chen
Commit 82df768514 breaks the regression
test for RHBZ#1232192 because g#part_list can return an error when
called on a completely blank disk.
By rewriting and simplifying the code using higher-order functions we
can avoid the need for the double-nested imperative loop and calling
g#part_list at all.
Fixes commit 82df768514.
The 'semodule' command, called by the guest tools uninstaller, will
run out of memory with the default of 500MB
(https://bugzilla.redhat.com/426775).
Allocate a large amount of RAM, that will be at least 800MB, but scale
correspondingly on other architectures like ppc64 that have larger
memory requirements.
Thanks: Pavel Butsykin
This is a mostly mechanical renaming of the internal pointer types
used in the libxml2 bindings. Also I have added a longer comment to
make the system clearer.
Guest tools for Windows in Parallels / Virtuozzo Server 6 contain a
paravirtualized video driver, which wants to take full control of the video
mode. To avoid excessive video mode switches in UEFI VMs it executes
"bcdedit /set {current} graphicsmodedisabled true".
When such a VM is imported into a QEMU/KVM-based hypervisor, the
standard video drivers in Windows do not set the video mode as they
expect it to have already been configured by the bootloader. As a result,
the VM runs with black screen.
So patch the BCD store (which is a Windows registry hive with a special
structure, located on the EFI system partition) to clear this setting.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
Store the list of EFI system partitions on the inspect object in order to be
able to tune their contents later in the process.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
Guest tools for Linux in Parallels / Virtuozzo Server 6 come with a script
that can be used to uninstall them.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
'tc_410_close_event.rb' was not being run. You could prove this by
simply inserting "exit 1" into that test.
The reason is unclear, but by renaming every class and method in the
tests to be unique, this ensures the tests are run.
Parallels proprietary hypervisor uses RDPMC as the hypercall
instruction. As this instruction is supported since early P6 family,
the drivers didn't even bother to check for the presence of the
corresponding feature in CPUID.
In QEMU/KVM, however, this instruction triggers #GP unless the VM is run
with PMU (performance monitoring unit) enabled, which is often not the
case (due to its impact on perfromance and migratability).
So, to prevent crashes upon conversion, stop respective drivers from
loading by disabling the corresponding services. Note that the services
are being disabled ("Start" value set to 4) rather than their node
removed, to avoid confusing the uninstaller which is scheduled to run
from a firstboot script.
In addition, prl_strg (storage filter driver) is unlinked from the
storage subsystem following the DelReg directive from its .inf file,
otherwise Windows crashes with BSOD 0x7b due to missing dependency of
the storage subsystem.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Some virtio-win drivers contain more files than just *.{cat,inf,sys}.
They are filtered out currently, which prevents the drivers from being
installed by PnP.
Stop filtering driver files by extension, and copy all of them instead.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Use this command as a starting point:
git shortlog -s -- p2v v2v | awk -F'\t' '{print $2}' | sort -f
However note the above will include people who made only incidental
code changes, and also misses out QA team members, so the actual list
still has to be curated by hand.
It's better to have this overview of how the code is arranged before
any details about modifying the code, since most people will be
reading code before writing.
Recent updates of Virtuozzo Server 6 (fka Parallels Cloud Server 6) have
"Virtuozzo tools" instead of "Parallels tools" as their display name in
the corresponding Uninstall registry key.
Recognize those, too, and schedule their uninstallation on first boot.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
On Linux 4.7.0 the old code was giving -EINVAL errors. This rewrites
the code based on online sources, and it works on at least RHEL 7,
Linux 4.4 and 4.7. Also checked with valgrind.
Because EXTRA_DIST included $(TESTS) this ended up distributing the
binary file v2v_unit_tests.
Unfortunately to fix this we need to list all the test scripts and
data files explicitly a second time, but there we are.
If this flag is not set (ie. we are doing local testing) then we
disable some dangerous features. Currently this is just the "Reboot"
button, but may include other features in future.
Currently virt-p2v requires Gtk 2. This commit changes virt-p2v so it
can be built with either Gtk 2 or 3.
By careful use of macros, this code should compile on both recent
Gtk 2 and Gtk 3.
With no other options, ./configure will now prefer Gtk 3 if it finds
it, or fall back to Gtk 2. But you can control this by setting
'./configure --with-gtk=2|3|check|no' where the options mean:
* --with-gtk=2 - Only test for Gtk 2
* --with-gtk=3 - Only test for Gtk 3
* --with-gtk=check - Check for Gtk 3 then Gtk 2 (default)
* --with-gtk=no - Don't build virt-p2v
In the ./configure output you will see something like this:
checking for --with-gtk option... 2
checking for GTK... yes
checking if we can build virt-p2v... yes, with Gtk 2
Gtk 3 deprecates the GDK thread synchronization locks (functions such
as gdk_threads_enter/gdk_threads_leave), and requires that you issue
all Gtk calls in the main thread. To do this you have to restructure
any code that calls Gtk from other threads so it is wrapped in an idle
task, and so is run from the main thread.
This commit transforms the code in this way.
I found through experimentation that libxcb crashes noisily if you
issue any X11 calls (and hence any significant Gtk calls) when the GDK
thread sychronization initializer has been removed, so if there are
any remaining calls that are missed by this patch we should find out
about them quickly.
Use, for example:
./run virt-p2v --test-disk=$(pwd)/test-data/phony-guests/windows.img
to test conversions using a file of test data instead of the real host
/dev/sda.
But allow the warning to be turned off using --no-warn-if-partition.
Ming Xie tried to create a bootable USB key using
'virt-p2v-make-disk -o /dev/sdX1'. That works, but doesn't create a
bootable key because it puts everything into the first partition.
Emit a warning if someone tries to do this to try to prevent user
error:
virt-builder: warning: output device (/dev/sdb1) is a partition. If you
are writing to a USB key or external drive then you probably need to write
to the whole device, not to a partition. If this warning is wrong then you
can disable it with --no-warn-if-partition
Returns true if the device is a host partition.
The only tedious bit of this patch is that now Common_utils depends on
Dev_t so we have to add extra objects every time something links with
Common_utils.
For virt-p2v-make-kickstart this doesn't do very much. But for
virt-p2v-make-disk it enables verbose mode in virt-builder which is
extremely useful for debugging problems.
Thanks: Ming Xie.
Run sudo with -n (non-interactive), so it will fail right away when not
configured to not require a password. This will avoid the connection to
time out.
You can run `xterm -display :0' at the virtual console in order to get
a shell in the graphical environment.
This is useful for debugging and doesn't have much overhead unlike
other terminals.
Fetch the full version from the remote virt-v2v, including the
--with-extra field, since that tells us which distro virt-v2v was
compiled for which is useful information.
Display this information in all debug output and dialogs.
Multiple cell alignment issues in the treeviews on the right of the
conversion dialog. This particularly showed up when you had multiple
heterogeneous disks or network cards because the description cells did
not align nicely.
Thanks: Ming Xie.
Use KVM, fallback to TCG.
Add more network devices so we can test the layout of that
dialog box.
Move the documentation into guestfs-hacking(1).
Updates commit f8cfdf274f.
The kickstart version of virt-p2v has SELinux disabled, so this is at
least consistent. Mainly it avoids the very long relabel when the
disk boots first time.
Since Void Linux provides only an /etc/os-release with no VERSION_ID
field, then special-case it to avoid that the os-release parsing ignore
it.
This provides basic distro identification, and icon.
When parsing lsb-release, allow a version number in the format "X" other
than "X.Y" for the DISTRIB_RELEASE key.
This fixes version parsing in Mageia 4 guests.
Add a simple helper to run a command from a sequence of arguments,
without using a shell: this should help reducing the amount of quoting
ineeded, since arguments are passed straight as such.
Make use of it in the places currently using shell_command, and which
don't assume they can run anything (so no shell redirections, `env`,
etc).
Move the make_dib_args helper function to Common_utils as
stringify_args, so it can be used also within Common_utils itself.
This is mostly code motion.
Some ova files like those produced by SUSE Studio have their ovf, mf
and other files inside a folder rather than at the root of the
tarball. Consider the paths relative to the ovf and mf files to cover
this case too.
To add this support, two things are needed:
* make the existing code searches for either the viostor
or the SUSE VMDP (Virtual Machine Driver Pack) files.
* add a firstboot script setting up VMDP.
Note that 2 firstboot scripts are intentionally added for the VMDP
setup. This is due to windows potentially rebooting after loading the
virtio block driver. It may happen that this reboot interrupts the VMDP
setup in the firstboot script, we thus make sure the setup is run a
second time in case it needs to finish the previous run.
Sync the windows firstboot script with its linux brother. Also change
the main redirection to append to the log rather than overwriting it.
With this change, the firstboot script will resist reboots in the
executed scripts.
Add a common debug function for printing debugging messages. It only
emits the debug message when the verbose (-v) flag is used on the
command line.
It sends the output to stderr, which is flushed immediately after the
message is printed (to help with debugging unexpected crashes). There
are good arguments for sending the debug to either stdout or stderr,
and almost all existing debug messages replaced by this change went to
stdout. However using stderr is consistent with libguestfs's own
debug messages which also go to stderr.
I only made simple changes to code of the form 'if verbose () then
printf ...'. There are more places which could be changed in future.
In a few places I removed gettext calls since we probably should not
translate debug messages.
It is still only emitted if we are debugging in order not to cause
unnecessary alarm. Note this code needs a better long term fix, this
is still a hack.
Now that UEFI is fully open source the UEFI firmware can be included
in Fedora. However the location is slightly different from the
location that was used by kraxel's out of distro firmware. This
commit searches for UEFI files in the new location, falling back to
the old location (until the end of this year when we will drop it
entirely).
Thanks: Cole Robinson, Gerd Hoffmann (kraxel) & Microsoft for
relicensing the FAT code.
Since Linux 4.7, the process umask is available in /proc/<pid>/status.
See:
3e42979e65
Use this value if available, else fall back to the existing codepath
for Linux <= 4.6 and other Unix.
Newer versions of e2fsprogs refuse modifying journal devices, causing
the test #1 of get_e2uuid to fail with:
libguestfs: error: set_e2uuid: Cannot modify a journal device.
Since the test case just needs to verify get_e2uuid runs without
failing, run it without expecting a certain UUID set earlier.
Followup of commit da4812ab6b.
Only in very recent OCaml versions (or in Fedora) the Arg module
supports the --foo=bar syntax for options with arguments. Do not
require it yet.
Fixes commit 64bb9edd52.
To make sure we can also find the initrd on openSUSE and SLES, we need two improvements:
* the initrd filename may not end with '.img'
* don't use the version + release from the RPM data, rather from the
/lib/modules/<version>/ path as we need to find it out anyway.
When possible, make the disk image format explicit when invoking tools
or using add-drive. This avoids warnings from qemu about the unspecified
format for the image, and also makes qemu slightly faster (skipping the
disk image probing).
Tests checking the image probing are not touched.
This changes also:
- old-style invocations of tools (`$tool $filename`) into new style
(`$tool -a $filename`)
- add-drive-ro/add-drive-with-if guestfish commands into add/add-drive
with explicit readonly/iface arguments
There should be no change in the tests results.
Turn the snippet reading user information from /etc/passwd in a slightly
more generic function, so there is no need to copy&paste for other
details.
Mostly code motion.
At least on openSUSE and SLES, the /etc/mtab file is already existing.
Skipping the symlink creation in init removes one error message during
the appliance boot.
SUSE Linux Entreprise Server doesn't have dhcpcd and the hivex package
is not in the default repositories. Better use dhcp-client and
libhivex0.
openSUSE needs ntfs-3g and ntfsprogs packages.
To allow saving space in the package shipping the windows virtio
drivers, we can use symlinks between the drivers folders. For example
SUSE VMDP drivers are the same for Win8.1 and Win2012r2, one folder
is a symlink to the other.
To take advantages of this, find must use the -L flag.
qemu feature detection takes about 95ms on my laptop. The overhead is
almost all due to the time taken by the glibc link loader opening the
170+ libraries that qemu is linked to (×2 because we need to run qemu
twice).
Fixing that is seriously hard work.
Therefore memoize the results of guestfs_int_test_qemu.
This is keyed on the size and mtime of the qemu binary, so if the user
changes the qemu binary (eg. setting LIBGUESTFS_HV) we discard the
memoized result and rerun the qemu commands. There is also a
generation number so we can bump the generation in future versions of
libguestfs to invalidate all previously cached data.
The memo is stored in the supermin cache directory (eg. /var/tmp/.guestfs-*)
in the files:
qemu.stat Result of 'stat(2)' of the qemu binary
qemu.help qemu -help output
qemu.devices qemu -devices ? output
Note these files are only stored when using the 'direct' backend. For
the libvirt backend, libvirt itself memoizes this data in its own
place.
This is code motion, but I have cleaned up and formalized the
interface between this module and other parts of the library.
Also this adds documentation to the interface.
The maxrss is described in getrusage(2) as "This is the maximum
resident set size used (in kilobytes).". This lets us find out how
much memory qemu really used, and is useful for determining if memory
minimization techniques such as DAX are having any effect.
When creating the dummy scratch disk for a /dev/null drive, force its
format as raw if it was not specified already. After all, raw was
already the only format allowed for this kind of drives, so it makes
sense to specify the format explicitly and avoid the autodetection in
qemu.
Use the version struct in inspect_fs to hold the version of a
filesystem, adapting the inspection code to that.
Also, move the parts of the version parsing to helper functions of the
version struct, so common bits like parsing "X.Y" or "X" version strings
is done only once.
Introduce a new struct version with few helper functions for it: this
allows to "atomically" represent a version number, without different
variables to be used and checked together.
Add a initialization function from a libvirt-style version number, and
apply it for the qemu and libvirt versions in the direct and libvirt
backends.
Previously I disabled ACPI because it was not necessary and was slow.
However measuring it again now, I can see no significant difference in
performance, and it is necessary if we are going to use DAX.
It's also useful to keep the appliance kernel state as similar to the
ordinary state, and setting acpi=off is a massive change to the way
that the kernel boots.
Therefore, reenable it.
Versions older than 4.3.1 output also the version string after the
output. Ignore such line, since it isn't relevant to what
btrfs_filesystem_show needs.
In /etc/fstab the UUID= or LABEL= field may be quoted. Augeas returns
the field including the quotes, and we passed this directly to
guestfs_findfs_uuid or guestfs_findfs_label. It happens that this
works on upstream findfs, although it doesn't work in RHEL 7.2. The
correct thing to do is to remove the quotes before passing the UUID or
label to these functions.
Thanks: Thom Carlin for reporting the bug, Karel Zak for identifying
the change in behaviour in util-linux.
This function is useful outside virt-v2v so move it into the core
utilities library.
The function has been rewritten from OCaml into C, but it should be
functionally identical.
I suspected data corruption (but didn't prove it). This commit just
adds some magic numbers to the structs so we will see data corruption
quickly if it happens again.
Previously we were running ldconfig to create /etc/ld.so.cache.
This is required, at least on Fedora, if we need to run any binary
that uses a library with a weird path. libiscsi (a dependency of
qemu-img, used by virt-dib) is an example of such a weird library,
since it puts its single library into /usr/lib64/iscsi for no readily
understandable reason, and drops a configuration file into
/etc/ld.so.conf.d/ so that this new directory gets picked up.
By copying the /etc/ld.so.cache from the host we get an already
configured cache which should contain every library on the host, so
there is no need to run ldconfig.
Running ldconfig adds about 100ms to the boot time. I would prefer
that we understood which libraries need ldconfig to be run, and fix
that. We could also consider running ldconfig in parallel, but since
it might be required by just about any binary that the init script
runs it's not clear what benefit that gives.
This reverts commit 66aa98265d.
If the guest uses SELinux, then make sure to run a relabel (or at least
schedule one) after the image build: this way the template is
relabelled, or at least it will do that at the next boot, without the
need for the user to ask for a relabel.
This just covers the case of building a new image with no additional
operations on it though.
Add unit tests for the following internal functions:
- guestfs_int_getumask
- guestfs_int_new_command and other src/command.c functions
- guestfs_int_qemu_escape_param
- guestfs_int_timeval_diff
- guestfs_int_match, match1, match2
- guestfs_int_add_string and other src/stringsbuf.c functions
Because we now need to use a libguestfs handle, we have to link the
unit tests to the library. But because we also need to access the
internal functions (to test them) we need to link the test statically
to the objects making up the library. So this requires a small change
to the linking of this test too.
Use /etc/hosts as alternative of /etc/fstab to detect whether a partition
represents the root of a Linux installation; the latter might not exist
in smaller/special installations like Docker images.
Improve the error messages produced by C-based tools in case of issues
with the command line options:
- explicitly mention to use -a/-d (and -A/-D in virt-diff)
- when extra arguments are found, mention the correct way to pass
options to certain command line switches (like --format)
- in virt-inspector, give a cleaner error message when neither -i nor
any -m is specified
In all the cases, keep the extra notice to use 'TOOL --help' to get more
help with it.
python/guestfs.py:136:37: E712 comparison to True should be 'if cond is True:' or 'if cond:'
python/t/tests_helper.py:42:8: E713 test for membership should be 'not in'
No functional changes, as the new versions follow the suggested Python
coding style to do the same things.
Reindent Python scripts to make sure lines are not longer than 80
columns.
Regarding autogenerated code (guestfs.py and bindtests.py): add an
helper function to make sure comma-separated lists are wrapped at the
wanted length.
This produces only differences in the indentation of long Python lines,
with no behaviour changes.
Add (after comma) or remove (before opening round bracket, and around
'=' in arguments) whitespaces according to the PEP 8 specification.
This is just code reformatting, with no behaviour changes; no content
changed beside whitespaces, so "git diff -w" gives an empty diff.
For Windows, we now print:
$ virt-customize -a ./test-data/phony-guests/windows.img --install MSSQL
[ 0.0] Examining the guest ...
[ 14.2] Setting a random seed
virt-customize: warning: random seed could not be set for this type of
guest
[ 14.2] Installing packages: MSSQL
virt-customize: error: cannot use '--install' because no package manager
has been detected for this guest OS.
If this guest OS is a common one with ordinary package management then this
may have been caused by a failure of libguestfs inspection.
For OSes such as Windows that lack package management, this is not
possible. Try using one of the '--firstboot*' flags instead (described in
the manual).
In cases where parsing the release file failed and so we have an
obviously incorrect major version number, don't try to infer the
package manager from the major version number.
In the bug report, parsing the /etc/redhat-release file of a CentOS
7.1 guest failed, so major version was set to 0, and the package
manager was inferred as "up2date". virt-customize then failed with a
peculiar error:
virt-customize: sorry, don't know how to use --install with the 'up2date' package manager
Instead this sets it to "unknown" which will cause virt-customize to
fail with:
virt-customize: --install is not supported for this guest operating system
which is (only very slightly) better.
Problem reported by novegin on IRC.
All developer documentation should go in guestfs-hacking(1) and the
"internal documentation" (ie. documentation about internal functions
and interfaces) belongs here, not in the coincidentally synonymous
guestfs-internals(1).
Since moving boot-benchmark-range.pl out from under a tests/
subdirectory, it is now being added to po/POTFILES. This causes the
following error:
/usr/bin/xgettext: Non-ASCII string at utils/boot-benchmark/boot-benchmark-range.pl:205.
Please specify the source encoding through --from-code.
Add --from-code parameter as instructed.
The Arg module of OCaml does not support hiding options from the --help
output: hence, mark --short-options and --long-options as internal
options, since we need them only for the bash completion scripts.
Create a new top level directory called 'utils' and move the
following programs there:
tests/qemu/boot-analysis -> utils/boot-analysis/
tests/qemu/boot-benchmark -> utils/boot-benchmark/
tests/qemu/qemu-boot -> utils/qemu-boot/
tests/qemu/qemu-speed-test -> utils/qemu-speed-test/
Also we only build the boot-analysis program on x86-64 and aarch64,
since it requires custom porting to each architecture.
Recently Debian switched to btrfs-progs, so almost all the distributions
(except openSUSE) have btrfs-progs which is then moved as common
package.
The old btrfs-tools name is left there, to support Debian Jessie and
older, and Ubuntu Xenial and older.
When (in --verbose mode) we dump the pass data, dump the times in
milliseconds (instead of nanoseconds) so they are consistent with
other output. Also dump the time difference from the previous event.
Useful for detailed debugging/analysis of problems.
When we are debugging use the verbose (debug messages enabled) version
of the AAVMF (UEFI) firmware. This is useful for diagnosing what is
causing failures to boot and what inside UEFI is causing slow booting.
Use an array of structs for the UEFI firmware instead of an array of
strings.
This change is hopefully just refactoring and there is no functional
difference.
It is safe to use /dev/urandom to seed the guest. In libvirt 1.3.4
this silly restriction has been lifted.
This restores commit b2c845333f.
See also commit 9423c16607.
Thanks: Cole Robinson for doing the libvirt implementation.
The old mapping code was directly copied from old virt-v2v, translated
from Perl to OCaml.
The new mapping code does a few things more accurately:
- Use the i_product_variant field (Windows InstallationType) if available.
- Simplify rules, so there is only one special case needed for RHEL 3/4.
- Don't assume Fedora == Desktop.
- Don't assume all later Windows variants are server.
- Works for Windows > 7.
Since commit b23e149774, we have been
strict about POD errors. An error in the Japanese translation caused
"L</guestfish" (ie. an unclosed L<>) to appear in the POD.
This commit fixes the problem, but when building you may need to do:
rm po-docs/ja/guestfish.pod
rm po-docs/podfiles; make -C po-docs update-po
in order to update the broken po-docs/ja/guestfish.pod file.
Unfortunately I have been unable to upload this change back to the
Zanata servers because of a variety of client problems.
PHP (5?) renamed the PHP_EXECUTABLE variable to TEST_PHP_EXECUTABLE.
As a result of that if you enabled debugging, the tests broke because
we no longer used our custom PHP wrapper to filter out debugging
environment variables before running the tests, so debug output was
mixed with the expected output.
This commit also updates an old comment telling you how to debug PHP
tests.
Prevents us from accidentally "losing" a disk during conversion. I
believe from code inspection that this assertion is always true for
the current code, so this should have no effect.
It's a large feature / function with its own set of nested functions,
so move the inspect_source function into its own module. It also lets
us specify and document the interface explicitly.
Also: Define a root_choice type in Types module. I turned it into a
non-polymorphic variant type for extra type safety.
This is just code motion.
- Add headings, further documentation to the Types module.
- Move two type definitions in Types module. Not a functional change.
- Add more comments and clean up comments throughout v2v/v2v.ml.
- Refactor guestcaps / conversion in v2v/v2v.ml. Not a functional
change, just hides local variables from the rest of the code.
- In --in-place mode, change inspection message to 'Inspecting the
source VM'. This matches the later message 'Closing the source VM'.
Enable a few POD options:
- Don't generate an errata section in the output.
- Send errors and warnings to stderr.
- Die if any errors or warnings are seen while generating the outputs.
The drive used for the appliance is a raw (sparse) disk: specify that
explicitly in its -drive qemu command line options, so qemu can skip the
autodetection of its format and save a tiny bit of time.
Allow kernel options such as p2v.o=libvirt to override internal
defaults, even for GUI configuration.
The main change is to split up the kernel conversion into two steps:
reading the kernel command line configuration, and performing the
conversion. The kernel command line can then be read by the main
program and used to initialize the config structure for either kernel
conversion or GUI conversion.
A small adjustment is required in the test because p2v.pre no longer
runs before kernel command line parsing. (The aim is to have
p2v.pre/post/fail still only run when doing a kernel conversion, not
in the GUI case.)
Since we started to use the parallel tests framework in automake,
'make check-slow' has been broken. This is because parallel tests
doesn't allow you to run 'make check TESTS=...' with a set of test
scripts which do not also appear in the static list of tests in the
Makefile.am. We would like to list and run only "fast" tests in the
Makefile.am, and have other scripts for slow tests.
The solution is to add the slow tests to Makefile.am, but condition
those tests on an environment variable SLOW=1 being set.
This commit fixes all the existing slow tests in this way, and updates
the documentation (guestfs-hacking(1)) to document how slow tests
should be written in future.
Valgrind has a weird hack where it invokes a glibc function called
__libc_freeres on exit. See:
http://valgrind.org/docs/manual/faq.html#faq.exit_errors
This is intended to free up memory that glibc won't normally free
(since glibc doesn't free everything on exit for efficiency reasons).
More importantly, valgrind runs __libc_freeres even if the process
calls _exit, resulting in this bug:
https://bugs.kde.org/show_bug.cgi?id=361810
(either a bug in valgrind, or in glibc, or in both, depending on your
point of view).
In any case we don't want this behaviour, so disable it.
Also we have to add suppressions for new "leaks" in glibc found by
valgrind because __libc_freeres no longer runs. In fact there is only
one such suppression needed, for TLS allocation in multithreaded
tests.
When we are valgrinding we don't really care about the child
processes, which might be qemu, libvirtd, etc. So disable tracing
into children (at least, as far as is possible with valgrind, which is
not entirely disabling it, but suppressing it).
This is convenient because it allows you to do:
rm -f docs/C_SOURCE_FILES
make docs/C_SOURCE_FILES
if you ever wanted to recreate that file (without doing 'make dist').
Note that the rules depend on configure.ac so that the files get
updated on each new version (otherwise the files would never get
updated).
On Linux this will load the whole file into the page cache. However
the output file is empty and zero sized just after it is opened, so
this has no effect. Note that the advice is not persistent, so this
really does nothing.
I considered adding the call back after the file has been written,
just before the close, but:
- If we do a virt-resize next then we will open and read the file mostly
sequentially, so readahead will deal with any missing pages.
- If we do a virt-customize next then we will only access a small
part of the disk image, so loading it all into the page cache adds
extra work.
- In any case, since we have just written the file it's likely to
still be in the page cache.
This commit adds guestfs_int_fadvise_normal, but it's not enabled
since nothing calls it.
It also documents what Linux actually does, which is a bit different
from what I thought these settings did. Note this is for Linux 4.6.0rc3
and may change in future.
This updates commit 83e92b4a97.
Setting POSIX_FADV_RANDOM makes no measurable difference, but at least
it's the right thing to do.
POSIX_FADV_WILLNEED makes no measurable difference either.
Moving the calls to posix_fadvise to just after the open() makes no
measurable difference, but does make the code a bit clearer.
Changing POSIX_FADV_NOREUSE on the input file descriptor to
POSIX_FADV_DONTNEED slows things down by about 10%.
Add wrappers around posix_fadvise and use them in places we were
calling posix_fadvise directly before.
Also in virt-builder we were doing this (and ignoring the result):
posix_fadvise (fd, 0, 0, POSIX_FADV_RANDOM|POSIX_FADV_DONTNEED);
However the POSIX_FADV_* flags are _not_ bitmasks! In fact
POSIX_FADV_RANDOM|POSIX_FADV_DONTNEED == POSIX_FADV_NOREUSE so we were
giving a completely different hint from what we thought we were
giving.
The current implementation of getumask involves writing a file with
mode 0777 and then testing what mode was created by the kernel. This
doesn't work properly if the user set a per-mount umask (or fmask/
dmask).
This alternative method was suggested by Josh Stone. By forking, we
can use the thread-unsafe method (calling umask) and pass the result
back over a pipe.
This change also fixes another problem: mode_t is unsigned, so cannot
be used to return an error indication (ie. -1). Return a plain int
instead.
Thanks: Josh Stone, Jiri Jaburek, Eric Blake.
Exclude more files from the documented sources in C_SOURCE_FILES:
- sources generated by XDR definitions
- sources generated by gperf
- errnostring.c (generated by the generator, and copied around from src
to daemon during build)
Also update C_SOURCE_FILES as well, so libguestfs builds again from
scratch.
Virtio-SCSI offers a number of advantages over virtio-blk, in
particular, it supports SCSI UNMAP (aka trim) which is crucial for
keeping the virtual drive from wasting host disk space.
This patch adds support for virtio-scsi as the virtual disk connection
type both on input and on output of v2v.
Virtio-blk remains the default, so for now virtio-scsi-based guests can
only be produced in --in-place mode.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
There's no particular reason that we have to update these files in
every 'make'. It's slow, and it doesn't really matter if a single
file is accidentally omitted from translations on a developer's build
machine.
On the other hand generating it from 'make dist' has advantages:
- We only do it once per release.
- The maintainer can check the changes carefully.
- If a developer adds some random files in their local build tree,
these won't be picked up accidentally.
- Changes to POTFILES etc won't leak into general commits.
Add the real list of C source files to the Makefile, instead of trying
to calculate it. This (will, in the next commit) fix a problem with
the build on Koji where we created a subdirectory (for python3/) but
the find command picked up files from that directory for the
documentation (and then failed).
This fixes commit 04229c68d6.
As in libvirt, allow internal documentation comments in C files.
These are marked using '/**' comments. Unlike libvirt which has an
ill-defined and inconsistent markup, the markup we use is Perl POD.
These comments are added to guestfs-internals(1), but are most likely
to be read in-place.
This commit changes some existing comments in src/launch.c to
demonstrate how this can be used.
Refactor registry patching to share common parts between Win <= 7 and
Win >= 8. Also branch between the two based on the presence of the
DriverDatabase subtree rather than on the OS version, which allows not
to pass the inspect handle into the functions.
OTOH make the patching functions accept the driver name and device PCI
id as arguments; this will allow to easily expand the code to support
other drivers (e.g. vioscsi) later on.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
It turned out (by trial and error) that for Windows >= 8 / 2012, even
though the PnP system uses names related to the original driver's .inf
file for certain entries under DriverDatabase registry subtree, this
doesn't have to hold for the storage driver to make it boot.
So use our own artificial names there and simplify the surrounding code.
This will also make sure the added entries don't confilct with those
generated by PnP proper once the driver is installed, and that the
entries are easy to look up in the eventual registry.
The name chosen to refer to the driver -- "guestor" -- is intentionally
different from the original "viostor" both to avoid collisions and to
naturally expand to other relevant drivers (e.g. "vioscsi") later on.
Tested on
Win 8 x32
Win 8 x64
Win 8.1 x32
Win 8.1 x64
Win 2012R2 x64
Win 10 x64
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
The patch to the Windows registry allowing it to boot off a virtio-blk
drive was initially conceived by comparing the state with virtio-blk
driver properly installed, to that without.
However, we don't want to replicate the Windows PnP system; rather we
need to apply just enough edits to make the system boot, and then let
the Windows PnP manager figure out the rest.
It turned out (by trial and error) that for Windows >= 8 / 2012 the
"just enough" set includes only the service entry for the driver, and a
few entries under DriverDatabase.
Tested on
Win 8 x32
Win 8 x64
Win 8.1 x32
Win 8.1 x64
Win 2012R2 x64
Win 10 x64
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
The patch to the Windows registry allowing it to boot off a virtio-blk
drive was initially conceived by comparing the state with virtio-blk
driver properly installed, to that without.
However, we don't want to replicate the Windows PnP system; rather we
need to apply just enough edits to make the system boot, and then let
the Windows PnP manager figure out the rest.
It turned out that for Windows <= 7 / 2008R2 the "just enough" set
includes only the service entry for the driver, and a
CriticalDeviceDatabase entry matching the PCI id of the device and
pointing at that service entry.
While at this, also drop CriticalDeviceDatabase entries for older and
no longer relevant PCI ids of the virtio-blk device.
Tested on
Win XP SP3 x32
Win XP SP2 x64
Win 2003R2 SP2 x32
Win 2003R2 SP2 x63
Win Vista SP2 x32
Win Vista SP2 x64
Win 2008 SP2 x32
Win 2008R2 SP1 x64
Win 7 SP1 x32
Win 7 SP1 x64
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
Adapt to the API changes in the newer versions of the Zend framework, in
particular regarding:
- strings handling
- resources handling (used for the guestfs_h pointer)
- iterating in hash maps
- data types for function arguments
Introduce helper macros to reduce greatly the amount of #if's all around
the generated C code.
In the generated description of the guestfish commands, wrap and indent
the help text, so it is a multiline string instead of a very long single
one.
This has no behaviour changes, only makes cmds.c more readable (and
easier to diff when the description of actions change).
Like with the previous commit, this replaces instances of:
if (something_bad) {
fprintf (stderr, "%s: error message\n", guestfs_int_program_name);
exit (EXIT_FAILURE);
}
with:
if (something_bad)
error (EXIT_FAILURE, 0, "error message");
(except in a few cases were errno was incorrectly being ignored, in
which case I have fixed that).
It's slightly more complex than the previous commit because we must be
careful to:
- Remove the program name (since error(3) prints it).
- Remove any trailing \n character from the message.
Candidates for replacement were found using:
pcregrep --buffer-size 10M -M '\bfprintf\b.*\n.*\bexit\b' `git ls-files`
Wherever we had code which did:
if (something_bad) {
perror (...);
exit (EXIT_FAILURE);
}
replace this with use of the error(3) function:
if (something_bad)
error (EXIT_FAILURE, errno, ...);
The error(3) function is supplied by glibc, or by gnulib on platforms
which don't have it, and is much more flexible than perror(3). Since
we already use error(3), there seems to be no downside to mandating it
everywhere.
Note there is one nasty catch with error(3): error (EXIT_SUCCESS, ...)
does *not* exit! This is also the reason why error(3) cannot be
marked as __attribute__((noreturn)).
Because the examples can't use gnulib, I did not change them.
To search for multiline patterns of the above form, pcregrep -M turns
out to be very useful:
pcregrep --buffer-size 10M -M '\bperror\b.*\n.*\bexit\b' `git ls-files`
It was confusing to have a mix of seconds and milliseconds. For all
upstream testing (eg. with SeaBIOS) we are discussing everything in
milliseconds, so use those exclusively.
Add a new test program called 'boot-benchmark'. This is similar to
'boot-analysis' but it simply boots and shuts down the appliance
several times in a row and measures how long it takes, calculating
mean and standard deviation.
When a struct containing FChar fields is defined
in generator/structs.ml, the Python code generator will generate
code referencing a dirent struct instead of the correct one
thus breaking at compile time.
This bug was left unnoticed because the dirent struct is the only
one using FChar fields. Yet it prevents any other struct
from using such fields.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Look for /lib/os-release in the /usr partition and try to use it, if
present, before using lsb-release later on. This should not change the
final result of the inspection, while using the os-release detection
method also for CoreOS.
Update the phony CoreOS image to use os-release instead, uploading the
version found in the current stable version.
The "icat" name comes from the employed command line tool which might be
replaced at any time with a different implementation.
The command name is a bit confusing because it's similar to "cat" but
act as "download".
download_inode is more clear and descriptive.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Use the proper length modifier for asprintf, instead of a padding
modifier. This fixes the parsing of versions in VERSION_ID which are a
single number (i.e. "X" and not "X.Y", etc).
Bug introduced with commit 32d19e3289, but
uncovered recently with the switch away from VLAs, i.e.
commit 07c496c53c.
In order to support the new features I am renaming the file with a better name.
The file sleuthkit.c will contain the code depending on the sleuthkit package.
The original tsk.c file will contain the logic built using libtsk
which is the sleuthkit core library.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Some of the systemd-tmpfiles snippets need the machine ID of the running
system; the current lack of this file produces warning messages during
the appliance boot like:
[/usr/lib/tmpfiles.d/systemd.conf:26] Failed to replace specifiers: /run/log/journal/%m
[/usr/lib/tmpfiles.d/systemd.conf:28] Failed to replace specifiers: /run/log/journal/%m
[/usr/lib/tmpfiles.d/systemd.conf:29] Failed to replace specifiers: /run/log/journal/%m
Thus create a new randomly-generated /etc/machine-id on boot.
These options allow you to control the appliance memory size, number
of vCPUs, and extra kernel options respectively.
Note that using --smp is not usually a good idea. Not only does it
slow down the appliance, but it tends to break the boot analysis
program because it makes runs (more) non-deterministic.
the description of the --selinux-relabel option suggests that it
perform an immediate relabel, when in fact it may (and probably will)
instead simply touch /.autorelabel on the image, which schedules a
relabel operation for the next time the image boots. This can be
surprising because it results both in an extended initial boot time
*and* results in an automatic reboot (on some distributions).
It seems that the documentation for --in-place mode of v2v wasn't clear
enough, so try to explain it better.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
In in-place mode, the decisions on which interfaces to use are made and
the source VM configuration is created by the outside entity. So in
that case v2v needs to look it up in the source configuraion, and try to
follow.
For that, the source VM configuration is used to populate requested caps
object which is then passed to the convert routine.
It's assumed that the configuration has exactly one type of storage
devices and no more than one type of network and video adapters;
anything else is rejected.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
Give the caller certain control over what kind of interface to use for
virtual disks, network and video cards upon conversion.
For that, make convert functions accept additional rcaps parameter
containing an object with optional capabilities similar to the ones
produced on output, with None indicating that the decision is left to
the convert function itself.
To facilicate review, this patch unconditionally passes rcaps with no
preferences; populating it with more sensible values is done in a
followup patch.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
Introduce a type to contain the guestcaps that are to be put in effect
in the converted VM: options of the block type, net type, and video.
It'll be populated by the caller and passed into convert function to
affect its choice of devices and drivers.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
Those will be useful when making decisions about what configuration to
set on output.
The data is also included in --print-source so the tests are adjusted
accordingly.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
This tedious workaround avoids a 0.26 second pause when using sgabios
(the Serial Graphics Adapter). It's basically a workaround for buggy
code in sgabios, but much easier than fixing the assembler.
Since commit bb5d30ab2a, we don't
require any external programs like grep to parse the command line. We
only use bash intrinsics.
Therefore we can do it early (but after /proc is mounted).
This allows verbose mode to enable set -x early on, so we can trace
most things that the init script does.
This function was kind of like debug(), except that it didn't check
the g->verbose flag and it only worked in the library (it would crash
if used after fork).
It also wasn't very useful. The sort of boot analysis done by
[forthcoming program] tests/qemu/boot-analysis.c means that
timestamping (some) messages is even less interesting than before.
Remove it and replace calls with debug() instead.
Dr. David Gilbert pointed out to me that the first line of the
qemu -help output includes the qemu version, so we don't need
to run qemu -version at all.
This saves about 0.04s on the launch time when using the direct
backend.
Read the content of /proc/cmdline using bash features, and use its
[[ ... ]] expression to find texts in a variable.
This shaves off 5 grep invocations.
Instead of hardcoding "make" in run-php-tests.sh, pass the actual name
of make from the Makefile; the default is still "make", mostly to use
the script without having to set $MAKE.
sha1sum(1) does not exist everywhere, so wrap it in an own function so
the right implementation can be chosen on each OS. Also, wrapping it
avoid using awk everytime.
Move it to the test-data directory, giving it a more generic name so it
can be used also for more than just md5; adjust qemu tests accordingly.
This is just code motion, no behaviour change.
The quiet option suppresses kernel messages. On my laptop it improves
appliance boot times by about 40% (3.5s -> 2.5s).
The emulated UART is slow and has a fixed, small FIFO (16 bytes). But
it has the advantage of being a simple ISA device which is available
very early in boot, thus enabling us to diagnose early boot problems.
So the aim is to reduce our usage of this UART on fast paths.
Of course when we are in verbose mode, we should not add this flag
because we want to see all the messages.
This change is not entirely invisible:
(1) Progress messages use the "Linux version ..." string from kernel
output in order to determine part of where we are in the boot process.
This string will no longer be detected. We should probably use a BIOS
message or maybe drop this altogether. I have added a comment to the
code.
(2) It is possible for programs to be listening for
GUESTFS_EVENT_APPLIANCE events, and they will see fewer messages now
(although what kernel messages programs see is never defined).
In case /usr is a symlink to /usr/bin, then we cannot rely on /usr/bin
to exist, since /usr might be in a different partition. Thus, in case
/bin is a symlink, check it points to "usr/bin".
Provide new <define> for the following elements: "distro",
"package_format", "package_management", and "format", adding all the
values so far possible for each.
Update the system at the end of the installation, so the generated image
is up-to-date. This also removes the need to manually update dnf on
i686 and x86_64.
The list of filesystems that is printed when there was an error prints
the internal mountable string even for the btrfs subvolumes. Let's
printing a valid -m option value instead.
Add command line options to optionally disable the output of the
installed application, and the guest icon.
This makes the inspection slightly faster, producing a smaller XML to
parse/process in case there is no interest in these details.
GCC has two warnings related to large stack frames. We were already
using the -Wframe-larger-than warning, but this reduces the threshold
from 10000 to 5000 bytes.
However that warning only covers the static part of frames (not
alloca). So this change also enables -Wstack-usage=10000 which covers
both the static and dynamic usage (alloca and variable length arrays).
Multiple changes are made throughout the code to reduce frames to fit
within these new limits.
Note that stack allocation of large strings can be a security issue.
For example, we had code like:
size_t len = strlen (fs->windows_systemroot) + 64;
char software[len];
snprintf (software, len, "%s/system32/config/software",
fs->windows_systemroot);
where fs->windows_systemroot is guest controlled. It's not clear what
the effects might be of allowing the guest to allocate potentially
very large stack frames, but at best it allows the guest to cause
libguestfs to segfault. It turns out we are very lucky that
fs->windows_systemroot cannot be set arbitrarily large (see checks in
is_systemroot).
This commit changes those to large heap allocations instead.
Implement the helper function for guestfs_inspect_list_applications2 to
be able to parse the list of installed applications with the APK package
manager (used on Alpine Linux).
Update the check for the first line (the one containing the version) to
match also the message with OpenJDK; switch to awk so it is easier to
extract the version inside the double quotes.
Introduce and use a new inspect_mount_root function to mount all the
mountpoints of a root in the guest, replacing the same code doing that
in different tools.
inspect_mount_root_ro is inspect_mount_root with readonly mount option.
If there are no labels set for the disks, the directory with the
symlinks will not even exists, causing list-disk-labels to fail with
ENOENT. In this situation, act as if the directory was there, but
empty.
This makes a bit easier to diagnose failures on mkfs, without the need
to restart the filesystem creation with verbose output (which will
produce a lot more output).
Use PRIi64 as format string for int64_t, so it builds and works fine
also on 32bit. Also switch from asprintf_nowarn to asprintf, since no
custom formats (eg %Q, %R) are used.
Adding ntfscat_i command for downloading files based on their inode number.
This allows the dowload of files unaccessible otherwise from a NTFS guest disk image.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Add a comment to appliance/Makefile.am describing the purpose of this
directory.
Add a 'make clean-supermin-appliance' rule which forces the supermin
appliance to be reprepared from scratch. See the discussion here:
https://www.redhat.com/archives/libguestfs/2016-February/msg00264.html
Document which feature, if any, is needed for a function; this should
help users in properly checking feature availability when using certain
functions.
Old virt-v2v changed the <graphics> to VNC if the guest was using the
Cirrus hardware and Spice if the guest used QXL.
In commit 3d1cb74b3e I got the logic
backwards, using Spice if the guest used Cirrus and VNC if the guest
used QXL, which obviously makes no sense.
In this commit, I preserve the original <graphics> type from the
source guest. This has the advantage that the user can use the same
method to access the guest after conversion. If the source guest had
no <graphics> element, then we force VNC (a safe choice), and if the
source guest is a local disk that we use SDL, but this should only be
used for testing.
Thanks: Xiaodai Wang for the original bug report here:
https://bugzilla.redhat.com/show_bug.cgi?id=1225789#c10
Set the (libguestfs handle) program name correctly when running
programs like virt-copy-in, so the program name is not "guestfish" but
"virt-copy-in".
Note this feature cannot be tested using the ./run script, since
libtool (buggily) resets the program name to "lt-guestfish". However
I tested it on the installed copy and it worked there.
When listing the subvolumes of a btrfs filesystem, ignore the default
subvolume: we get the content of it when mounting the whole device
(without specifying any particular subvolume), so avoid listing it
twice.
Extend the generator to generate a source (and the header for it) with
functions that print the content of a guestfs struct. The code is
modelled after the code for it in fish.ml, although made a bit more
generic (destination FILE*, line separator) so it can be used also in
the library, when tracing.
This just introduces the new code and builds it as part of the helper
libutils.la.
Since the daemon links to pcre and use regular expressions, and since
the COMPILE_REGEXP macro doesn't depend on any aspects of the
library-side code (eg. the guestfs_h handle etc), we can allow the
daemon to use the COMPILE_REGEXP macro. Move the macro to
"guestfs-internal-all.h" to permit this.
It seems the default behaviour of rake is to run tests sequentially, and
not in parallel (there are separate gems to achieve that behaviour).
Hence just invoke "rake test" to run all the available tests at once.
This reverts commit 8f30c3c3f8.
Since we enabled parallel tests, you can no longer run tests
under valgrind merely by doing:
TESTS_ENVIRONMENT = $(top_builddir)/run --test $(VG)
check-valgrind:
$(MAKE) check VG="@VG@"
The reason is that the parallel tests framework doesn't run
``$(TESTS_ENVIRONMENT) <test>''. It inserts some other processes in
between the environment and the test, so you end up valgrinding some
unrelated process (currently the 'env' program).
In run.in, remove out of date documentation for using $(VG).
In guestfs-hacking(1), document how to do it properly.
Instead of running all the tests manually, the unittest module has a
'discovery' mode to run tests by importing them from a directory: this
requires the tests to have different filenames, since they need to be
imported as modules now (hence an empty __init__.py is added), and the
current naming does not match the convention.
Using unittest as loader/runner brings another change: tests skipped as
whole cannot be done anymore with exit(77), since they are not run but
imported: thus introduce an helper module with decorators applied to the
test classes to skip them according to the current checks. This also
gets us nicer recordings in the unittest log.
Due to the relative imports (needed for the helper code), it is no more
possible to execute tests anymore by invoking them manually; although
it is possible to run single tests, still using unittest's runner:
$ cd python
python$ ../run python -m unittest discover -v t test010Load.py
This does not change anything in what the tests do/check.
If you ran 'make installcheck' when libguestfs-devel was not
installed, then the installcheck would fail in many places with:
gcc: error: ../../src/.libs/libguestfs.so: No such file or directory
We should stop the installcheck script immediately if the .so file was
not found.
Also this commit ensures that src/.libs/libguestfs.so* not existing
before installcheck is not fatal.
Make this error message useful and actionable by suggesting commands
that the end user can run to find available pools, etc.
I also had to extend the fixed size buffer we use for messages since
the message got truncated.
Introduce a new kind of bindings tests, 090-retvalues, to check all the
possible return values in bindings; start implementing them for
scripting languages such as GObject introspection, Perl, PHP, Python,
and Ruby, reusing existing implementations where existing.
Thanks Joshua Pincus for finding the bug.
I checked back with the original W2K8R2 guest which I had used for
getting these registry entries, and there was a transcription error
with two of the CompatibleIDs.
- use assertIsInstance, assertNotEqual, and assertIsNotNone as more
specific checks (will produce better logging)
- use assertRaises when expecting exceptions being thrown
- when testing internal_test_rhashtable, instead of checking type and
elements of the return values just check the return value as a whole
(easier and already getting all the work needed by unittest)
When listing logical volumes, ignore the ones which don't get
activated automatically. No /dev/VG/LV device node is created for
these ones which confuses APIs that attempt to do 'guestfs_lvs'
followed by opening the device node. Note that 'guestfs_lvs_full' is
unaffected by this change.
Rename the existing tests according to the naming/numbering described in
guestfs-hacking(1), and improve the current ones:
- guestfs_php_001.phpt: rename to guestfs_020_create.phpt
- guestfs_php_003.phpt: rename to guestfs_070_optargs.phpt
- guestfs_php_bindtests.phpt: rename to guestfs_090_bindtests.phpt
- guestfs_090_version.phpt: new, checks taken from the former
guestfs_php_002.phpt
- guestfs_100_launch.phpt: new, modelled after the equivalent in e.g.
OCaml/Perl/Python
- guestfs_php_002.phpt: remove, as what it did is now covered by
090_version and 100_launch
And that READ_ONLY access is insufficient. I believe this is a bug in
libvirt, but it requires further investigation. If it is resolved in
libvirt then we can update this documentation again.
Thanks: Ulhas Surse
Now that all the stuff related to Windows virtio drivers has been moved
into a dedicated module, it makes sense to move the definition of
virtio_win there, too, and stop passing it around as a parameter.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Move to lazy_make_tmpdir the logic for making world-readable (but only
for root) newly-created temporary directories, removing the non-fatal
code doing that in guestfs_impl_launch.
Followup of commit 772f649e59.
When running as root libvirt will launch qemu as qemu.qemu, which will
not be able to read inside the socket directory in case it is set as
XDG_RUNTIME_DIR under /run/user/0.
Since normal users don't need particular extra access permissions for
their sockdirs, restrict /tmp as only possible sockdir for root,
changing the permissions only for this user (and making this operation
fatal).
Fixes commit 55202a4d49 and
commit 7453952d24.
1 GB should be enough to create a btrfs filesystem, even with 64K page
size; hence, make the /dev/sda and /dev/sdb test devices smaller so
there is less space taken during the test run.
Followup of commit 8ffad75e5b and
commit 9e9b648770.
This patch updates the guestfs_inspect_get_drive_mappings API call to
also return drive letters for GPT paritions. Previously this worked
only for MBR partitions. This is achieved by matching the GPT partition
GUID with the info stored in the blob from
HKLM\SYSTEM\MountedDevices\DosDevices keys. For GPT partions this blob
contains a "DMIO:ID:" prefix followed by a 16 byte binary GUID.
Support globbing in paths passed to --delete, telling glob to not
return directories with leading slash.
This re-adds back globbing for --delete in virt-sysprep, which was
available before the integration with common code from virt-customize.
Add a new optional bool "directoryslash" to indicate whether the caller
wants trailing slashes in names of directories, defaulting to true (the
current behaviour); this helps with interoperability with other tools
(such as rm).
Related to RHBZ#1293271.
The current VDSM .meta file parser is incorrect:
https://code.engineering.redhat.com/gerrit/gitweb?p=vdsm.git;a=blob;f=vdsm/storage/fileVolume.py;h=20573bd2a3fe6909a8474bd7c53c8d0fea79091c;hb=c1289fd70edcde0a0b3bc8e37ecad1e2a5babe1a#l285
When parsing the key=value fields in the .meta file, it uses the
Python expression:
key, value = l.split("=")
which throws an exception "ValueError: too many values to unpack" if
the value part also contains an "=" character (because split returns
an n-tuple with n > 2). The correct way to split the string in this
situation is to use l.split("=", 1).
In commit 399e031c25 I changed the
DESCRIPTION field in .meta so that in RHEL it would look like this:
DESCRIPTION=generated by libguestfs 1.32.2rhel=7,release=1.el7,libvirt
thus containing an "=" character in the value and throwing an
exception in VDSM.
Although this is a bug in VDSM which needs to be fixed separately,
change any occurrence of "=" in the value to "_" to avoid this
problem.
Thanks: Derrick Ornelas for helping to diagnose this bug and finding
the offending code in VDSM.
One of the mkfs-btrfs tests used two 200 MB partitions. That isn't
enough to create a btrfs filesystem on aarch64 (with 64K page size).
Since we now have a 10 GB disk, we can make the test partition 10
times larger.
In btrfs-progs 4.4, the --leafsize parameter is deprecated. It's now
just an alias for --nodesize.
On aarch64, --nodesize 4096 does not work because it's smaller than
the page size (64K). Luckily we can test a 64K nodesize on any
platform, so use that instead.
Previously these were rather small - just 500 MB. This is too small
to create a btrfs device on aarch64 (where page size may be 64K), and
barely enough even on x86-64. This change makes both these devices
10 GB, and adjusts a few tests so they continue to pass.
Introduce a new read-only API to get a path where to store temporary
sockets: this is different from tmpdir, as we need short paths for
sockets (due to sockaddr_un::sun_path), and it is either
XDG_RUNTIME_DIR if set, or /tmp; adapt guestfs_int_create_socketname
to create sockets in that location.
Furthermore, print sockdir and XDG_RUNTIME_DIR in test-tool for
debugging.
Introduce an internal helper to create paths for sockets -- will be
useful for changing later the logic for placing sockets.
Futhermore, check that the length of sockets won't overflow the buffer
for their filenames.
Extract the bit of code for lazy creation of a temporary directory, so
it can be used for more directories as well.
This is just code motion, with no behaviour changes.
Move the paths of the sockets used, from the libvirt_xml_params struct
to backend_libvirt_data; this way, they will be usable also outside the
launch callback.
Simply code motion.
I find the code slightly easier to read if we get rid of the ~write
flag, and instead split the with_hive function into two functions, one
for readonly access, and one for write access.
This is just code refactoring.
Add a common module `Windows_virtio' and move all the code related to
finding and installing Windows virtio drivers to this module.
This is just code motion and code refactoring, there is no change in
functionality.
Instead of pass 0x3fffffff as mask value, pass a simplier 0xfff, which
is the value of the IN_ALL_EVENTS define. This will still catch all
the inotify events, and avoid a EINVAL error with Linux 4.4.
Actually these are all static functions, so they don't really need the
'guestfs_' prefix at all, but using a prefix avoids any possibility of
a collision with a standard C function.
This is a mechanical refactoring.
For functions linked to virt tools, make sure the name of the function
actually matches the virt tool. Because of the history of moving
functions across tools, some names no longer matched.
For mllib, use `guestfs_int_mllib_' prefix.
For OCaml bindings, use `guestfs_int_ocaml_' prefix.
This commit is by no means complete. There are many other C functions
in other language bindings which could do with being more consistently
named.
There is no functional change, this is just refactoring.
lvm2 2.02.107 adds the -S/--select option used in lvs to filter out only
public LVs (see RHBZ#1278878). To make this work again with versions
of lvm2 older than that, only on old versions filter out thin layouts
and compose the resulting device strings ourselves.
The filtering done is much simplier than what "-S lv_role=public" will
do, but should be good enough for our need.
In Fedora 23, dracut moved to /usr/bin. However we only looked for
/sbin/dracut, so we didn't find it. You would have seen a failure
like this:
virt-v2v: error: unable to rebuild initrd
(/boot/initramfs-4.2.3-300.fc23.x86_64.img) because mkinitrd or dracut was
not found in the guest
Look for dracut in /usr/bin as well.
Thanks: Arnaud Bougeard.
print "" is no more available in Python 3, and print(...) is available
since Python 2.7; as one of the tests was using print(), use this form
everywhere so the tests can run fine with Python 3.
The warning (see below) is fairly useless. This modification to the
code avoids it.
inotify.c: In function 'do_inotify_read':
inotify.c:219:32: error: logical 'or' of equal expressions [-Werror=logical-op]
if (errno == EWOULDBLOCK || errno == EAGAIN) /* End of list. */
^~
The btrfs implementation of vfs_minimum_size requires
btrfs-progs >= 4.2, erroring with ENOTSUP otherwise; thus mark the btrfs
test case of the vfs_minimum_size tests as TestRunOrUnsupported, so it
will not cause failures if the available btrfs-progs is not enough.
Create a new TestRunOrUnsupported test type, which represents a test
sequence where a failure with ENOTSUP in the last command only marks the
test as skipped. To be used mainly when testing features available only
with some versions of helper tools used in the appliance, for example.
Having to choose the os-version of the virt-p2v disk was confusing.
Users thought it had something to do with the physical machine being
converted.
In some cases, virt-p2v-make-disk can now choose a suitable os-version
for the user. Usually it's the latest numeric version of Fedora or
Debian, depending on the host distro. If we don't recognize the host
distro, we bail and ask the user to choose, but this is still an
improvement.
Actually the OCaml compiler rewrites all printf functions at compile
time to a series of primitive string operations, so the use of sprintf
here doesn't really matter. Still, it's a bit tidier not to use
sprintf where it's not needed.
Also this commit contains some code reformatting.
This is just refactoring.
When the user clicks on the second column of the list of network
interfaces, run 'ethtool --identify <if_name> 10', which (on supported
cards) flashes a light on the physical interface for 10 seconds,
allowing it to be identified by the operator.
Resolve the paths of the binaries we are looking in the guest to
identify the architecture. This way we can identify also busybox-based
guests such as Alpine Linux, where all the "base" binaries in /bin are
symlinks to busybox.
Apparent newer versions of that report everything on stdout, including
error messages; since we only print something on failure, fold stdout to
stderr so we can see everything on failure.
This flag cannot be disabled (yet) in V5 xfs filesystems; since 2 out
of the current 3 tests of xfs_admin check other results than that flag,
avoid setting it when not needed.
Apparently with newer btrfs-progs (seen with 4.4) 100M are not enough
for a btrfs filesystem; hence double the size of the partitions
created in the test of btrfs_image, so now 200M are enough for btrfs.
In commit af762e4342 (2015-03-05)
I stopped libguestfs from using the supermin -dtb parameter when
building appliances.
It has not been necessary for quite a long time to specify a DTB,
since qemu itself creates a device tree structure in the guest
(appliance) memory from the devices presented to the guest. Qemu does
this on the fly each time the guest starts.
As I promised in the commit mentioned above I would remove this code,
and this commit does that (for libguestfs only).
This allows you to build a 32 bit virt-p2v USB key, even on a 64 bit
host.
Because of problems with the Fedora build system it's unfortunately
not easy to build a 32 bit virt-p2v binary, so this patch punts on
this issue and assumes that the user can supply the binary themselves.
This action moves second(backup) GPT header to the end of the disk.
It is usable in in-place image expanding, since free space after
second GPT header is unusable. To use additional space, we have
to move second header. This is what sgdisk -e does.
However, sgdisk -e may perform additional actions if the partition
table has unexpected params (e.g. if we call sgdisk -e /dev/sda1,
it may fix partition table thus destroying the filesystem).
To prevent such cases, we do a dry-run at first and fail if
additional actions are scheduled.
If present, Parallels Tools may stand in the way of proper running the
windows guests in non-Parallels hypervisors, so we're better off
uninstalling them on the first boot into the new environment.
With this patch, the uninstall records for Parallels Tools are looked up
in the registry and, if found, corresponding firstboot actions are
registered, taking special care that those actions are run fully
unattended.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
This test would fail on machines with around 4 GB of free memory.
Adjust the test so it would now work on such machines (it will however
still fail on a machine with around 2 GB of free memory).
Some OSes (e.g. Windows Server 2012 R2) fail to boot if the disk
GPT GUID has changed. To preserve disk guid e.g. during virt-resize,
we need a way to get/set disk GUIDs.
Libvirt has a fixed 15 second timeout for qemu to exit. If qemu is
writing to a slow USB key, it can hang (in D state) for much longer
than this - many minutes usually.
The solution is to check specifically for the libvirt EBUSY error when
this happens, and retry the virDomainDestroyFlags operation
(indefinitely).
See also the description here:
https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html
Similar to the following OpenStack Nova commit:
http://git.openstack.org/cgit/openstack/nova/commit/?id=3907867
Thanks: Kashyap Chamarthy and Daniel Berrange.
The err->domain field is essentially random from the point of view
of libguestfs.
The err->int1 field has some useful information in some circumstances:
it has the errno value from the failed system call.
Resolve in the guest the given path, so absolute symlinks can be listed
using appliance tools without resolution errors.
Also remove the note about the possibility to escape the sysroot using
ll and llz, since realpath won't return paths outside sysroot.
Fixes part of RHBZ#1293276.
Fedora/i686 does not have a package called 'kernel', so the command
'rpm -q kernel' fails. In any case the method used wasn't exactly
very robust.
The new method is based on how febootstrap used to do this.
This updates commit d723b352f8.
Sometimes the user wants to know minimum size
for dirty (e.g. mounted) filesystems. In this case,
resize2fs -P will require calling e2fsck -f, while
"in general, it is not safe to run e2fsck on mounted filesystems".
Since resize2fs -P does not modify filesystem, we force it
to display (probably approximate) minimum size.
Allow the user to specify a template in --list mode, which will be the
only result in the resulting output (instead of all the available
templates).
This makes it easier to find out the details of a specific template.
For every user command line tool (eg. guestfish or virt-cat) require
that the manual page has a DESCRIPTION section. This doesn't apply to
non-CLI man pages (eg. guestfs-perl, guestfs-faq).
Require qemu >= 1.3.0, the first version that supported
`qemu-img --output=json'.
This means we require yajl (for parsing the JSON output of qemu-img),
and that in turn has consequences elsewhere.
Move the creation of iscsi URIs away from make_uri, composing them
manually: this is needed because libxml assumes colons (':') to separate
user and password for the authority part, while with iscsi URIs the
separator is percentage ('%'), which would be percent-encoded by libxml.
It's a bitmask, so unsigned is the right choice. eg. We can more
easily print it using %x.
This patch changes the size of the fd mask and the values of the
COMMAND_FLAG_* constants, but since these are internal definitions
that doesn't matter.
Commit e6f7d23782 added new po files for
pt_BR to po/ and po-docs/ (they are only very minimal). However they
were not added to the list of linguas, so amongst other things the
files were not included in EXTRA_DIST automatically.
./uk.po:1049: 'msgid' and 'msgstr' entries do not both end with '\n'
./uk.po:9638: format specifications in 'msgid' and 'msgstr' for argument 2 are not the same
This updates commit e6f7d23782.
Two tests opened the phony Fedora image in parallel without the
readonly flag. This occasionally resulted in strange errors
(especially observed in test-inspect.sh). Add the --readonly flag to
avoid this.
- add a flag to request chroot for the process, which is done only as
very last (before chdir) operation before exec'ing the process in the
child: this avoids using CHROOT_IN & CHROOT_OUT around command*
invocations, and reduces the code spent in chroot mode
- add failure checks for dup2, open, and chdir done in child, not
proceeding to executing the process if they fail
- open /dev/null without O_CLOEXEC, so it stays available for the
exec'ed process, and thus we don't need to provide an own fd for stdin
Followup of commit fd2f175ee7, thanks also
to the notes and hints provided by Mateusz Guzik.
Prior to this commit:
$ virt-builder --help
[...]
--sm-attach SELECTOR Attach to a subscription-manager pool
--sm-credentials SELECTOR credentials for subscription-manager
--sm-register Register using subscription-manager
Really the generator ought to check this.
'sfdisk --part-type' used to merely set the MBR ID byte in the
partition. However since sfdisk was rewritten, it now "helpfully"
corrupts the first sector of the partition if you change the type byte
from an ordinary partition to an extended partition.
So we need to change the order in which we sets the partition type
byte, to do it before copying the partition content.
This fixes a major regression in the way that virt-resize handles any
guest with logical partitions (specifically Ubuntu guests).
If uncompressing a template to a filesystem which is nearly full,
virt-builder displays an error which points to the wrong (source) file:
$ virt-builder fedora-22
[ 1.2] Downloading: http://libguestfs.org/download/builder/fedora-22.xz
[ 1.7] Planning how to build this image
[ 1.7] Uncompressing
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
(The error message is usually printed once by each thread, so it is
printed several times.)
Change the error message to point to the output file, which might be a
temporary or the final file depending on the build plan:
$ virt-builder fedora-22
[ 1.1] Downloading: http://libguestfs.org/download/builder/fedora-22.xz
[ 1.6] Planning how to build this image
[ 1.6] Uncompressing
fedora-22.img: No space left on device
fedora-22.img: No space left on device
fedora-22.img: No space left on device
fedora-22.img: No space left on device
On ARM machines with vGICv3, we need to use the extra -M
parameter gic-version=3 (or gic-version=2 for the old vGIC).
qemu can detect this feature for us if we pass gic-version=host,
so add that to the -M option. Unfortunately we cannot detect if
qemu itself supports this parameter, but this change only affects
aarch64.
Thanks: Wei Huang
Exit with 3 as return value when fusermount fails, because the specified
mount point is not considered mounted for the user. This is in line
with what the guestunmount documentation says.
Adapt the test-guestunmount-fd test to the updated return value.
Thanks to: Maxim Perevedentsev.
When running commands in the mounted guest (using the "command" API, and
APIs based on it), provide the /dev/null from the appliance as open fd
for stdin. Commands usually assume stdin is open if they didn't close
it explicitly, so this should avoid crashes or misbehavings due to that.
Older GCC versions (e.g. < 4.9) may complain about inline functions that
cannot be inlined (growing the generated code). It is not really a
problem for us, so just suppress again the warning.
Followup of commit c37c554fc5.
This allows certain guests which virt-v2v cannot access to be copied
off the remote hypervisor and converted. Essentially this just
automates the process of copying the guest's disks and adjusting the
libvirt XML.
This also adds a test of the new tool.
Split the map_source_to_uri function up so that we can get at just the
https://... URI.
This is almost refactoring. It doesn't quite handle the case where we
are passed a source path from the libvirt VMware driver that doesn't
match the expected "[datastore] path" format, but probably if we hit
that case it's a bug anyway.
If you try to convert a guest that has a zero-sized disk, it
will currently fail in a rather strange way. Usually you will
see errors in the debug output:
[ 0.562714] sd 2:0:0:0: [sda] Read Capacity(16) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 0.563884] sd 2:0:0:0: [sda] Sense Key : Not Ready [current]
[ 0.564587] sd 2:0:0:0: [sda] Add. Sense: Logical unit not ready, manual intervention required
followed by virt-v2v failing with:
libguestfs: trace: v2v: inspect_os = []
virt-v2v: error: inspection could not detect the source guest (or physical
machine).
Additionally, because of a problem with the ssh driver in qemu (or
perhaps, with sftp-server on the remote side) it is not possible to
use the ssh driver to open a block device path on the remote server.
The drive will appear as zero-sized, triggering the above error.
Therefore detect this situation, and emit an error (see below). Also
add a section to the manual describing the workaround required for
converting RHEL 5 Xen guests which are located on block devices.
virt-v2v: error: guest disk sda appears to be zero bytes in size.
There could be several reasons for this:
Check that the guest doesn't really have a zero-sized disk. virt-v2v
cannot convert such a guest.
If you are converting a guest from an ssh source and the guest has a disk
on a block device (eg. on a host partition or host LVM LV), then
conversions of this type are not supported. See "XEN OR SSH CONVERSIONS
FROM BLOCK DEVICES" in the virt-v2v(1) manual for a workaround.
If the user tries to close the running dialog during a running
conversion, act as if the cancel button was pressed (cancelling the
conversion).
If the conversion has finished/been cancelled, then this closes the
dialog and exits virt-p2v, as now.
Cancelling the conversion in this way ensures that qemu-nbd and ssh
subprocesses are cleaned up.
If the virt-p2v ISO doesn't contain the correct drivers for the
hardware on the physical server, that can cause peculiar conversion
problems. This was hard to diagnose because we could not see the
'dmesg' output from the physical server.
Upload 'dmesg' output from the physical server (more precisely: from
the virt-p2v ISO's kernel running on the physical server) to the
virt-v2v conversion server debugging directory, so that this
information is available for debugging later.
Unfortunately this involves yet another copy of the 'read_whole_file'
function.
Instead of creating a special 'Exported by ...' string just for this
module, use the Common_utils.generated_by string. Since the string is
just descriptive, this should make no difference.
Adding this causes 'make dist' to fail with:
make[1]: *** No rule to make target 'ChangeLog', needed by 'distdir'. Stop.
This updates commit a2edda266e.
See the description in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1281578
Thanks: Richard Tollerton for alerting me to this change in the way
that Windows version numbers are stored in the Registry starting with
Windows ≥ 10.
The separator character (currently comma) was picked arbitrarily to
allow us to parse the output of commands like 'lvs' for APIs such as
'lvs-full'. Unfortunately the choice of comma conflicts with the 'lvs
-o modules' column, since the list of modules is separated by commas,
breaking our parser.
Change the separator to another arbitrary character (colon) which
hopefully is not used by any column.
See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1278878#c11
When a disk image uses LVM thinp (thin provisioning), the guestfs_lvs
API would return the thinp pools. This confused other APIs because
thinp pools don't have corresponding /dev/VG/LV device nodes.
Filter the LVs that are returned using "lv_role=public".
Thanks: Fabian Deutsch
I enabled the -Wshadow warning temporarily in order to do these fixes,
but had to disable it again afterwards. The reason is that this warns
about shadowing globals, which is sort of a good thing, but because we
have a global called "verbose" just about everywhere, and at the same
time we baked a function argument called "verbose" into several
unchangable APIs, well, we're stuck without being able to use this
warning.
This seemingly redundant change avoids a gcc warning/error:
error: cannot optimize possibly infinite loops
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=34114#c3 and
following for an explanation.
Of course gcc is completely wrong and stupid here, because there's no
possibility on current or future hardware that an array of size
SSIZE_MAX could be allocated since it would by definition occupy at
least all addressible memory (if it was an array of bytes, which this
isn't), leaving no room for the code that is being compiled.
'virt-inspector --xpath' can segfault.
When run under valgrind, it shows this error:
==2254== Invalid free() / delete / delete[] / realloc()
==2254== at 0x4C29D6A: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==2254== by 0x53BA198: xmlFreeNodeList (tree.c:3690)
==2254== by 0x53B9F65: xmlFreeDoc (tree.c:1247)
==2254== by 0x405BFA: do_xpath (inspector.c:808)
==2254== by 0x405BFA: main (inspector.c:250)
==2254== Address 0x1030a037 is 311 bytes inside a block of size 1,048 alloc'd
==2254== at 0x4C28C50: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==2254== by 0x545DE86: xmlDictAddString.isra.0 (dict.c:270)
==2254== by 0x545E961: xmlDictLookup (dict.c:923)
==2254== by 0x539C6DC: xmlDetectSAX2 (parser.c:1067)
==2254== by 0x53B0B92: xmlParseDocument (parser.c:10725)
==2254== by 0x53B1276: xmlDoRead (parser.c:15295)
==2254== by 0x40587D: do_xpath (inspector.c:772)
==2254== by 0x40587D: main (inspector.c:250)
The cause appears to be that when copying the matching node(s) found
by the xpath expression, we have to copy them into the new document
(using xmlDocCopyNode instead of xmlCopyNode).
This bug has existed since this functionality was originally added in
commit d1ee71782a.
Extend open_guestfs to take an optional ?identifier parameter.
Use this parameter in virt-v2v which is currently the only place where
we use the handle identifier.
This higher order function encapsulates the following pattern:
match x with
| None -> ()
| Some x -> f x
(replaced with: `may f x`)
This is taken from OCaml lablgtk (Gtk bindings) where this pattern is
used frequently.
See also: https://ocaml.org/learn/tutorials/labels.html
Instead of creating Guestfs handles and manually apply common options
(e.g. debug and trace), use the open_guestfs in Common_utils.
This also applies the common options to handles which didn't set them
before, so we can inspect also their messages if needed.
Resolve also symlinks when checking whether a path passed to --edit is a
file: the file editing code supports symlinks anyway (properly changes
the target file, instead of overwrite the symlink with a real file), so
it is safe to allow this operation in customize/builder.
Check that a path provided to --edit exists already; while the is_file
call later will fail for non-existing files, with an explicit check a
better error message can be provided.
With commit bc2b417784
HAVE_LIBVIRT_BACKEND is defined based on the libvirt version (using its
version macro), although libvirt.h is included only after that check:
because of this, variables in the guestfs_h struct after the
HAVE_LIBVIRT_BACKEND block would be used wrongly if libvirt.h was not
included before guestfs-internal.h, like in the recently added
available.c (all the other places using libvirt features in the handle
already happened to do so).
Considering guestfs-internal.h already includes libvirt.h, move its
inclusion up, right before the libvirt version check.
Replace the custom XSLT + script to turn output from $RUN_OUTPUT_FILE
(from previous run.in versions) with a OCaml script to analyze .trs and
.log files of the automake test framework.
The final result should be the same, just with classname= for each test
set to the "path" (subdirectories) to that test.
Use (implicitly) parallel tests on new enough automake. The tests run
a bit quicker with this change. On my laptop, I measured 27 mins down
to 18 mins.
Change the ./run function so that ./run --test no longer spools the
test output to a file. That is not necessary when using parallel
tests, since the test harness does the same thing.
Note: This commit removes the $RUN_OUTPUT_FILE functionality. We will
change the CI integration in a future commit so it uses the .trs and
.log files.
The user did not have the 'file' command installed, and when they
attempted to do a v2v conversion of a Windows guest, inspection
succeeded, but the Types.i_arch field was found to contain "unknown".
virt-v2v conversion failed to locate the correct virtio drivers as a
result.
The 'file' command is required, and there's no other reason we should
not be able to detect the architecture of any Windows guest, so fail
hard in this situation.
We want to test if virt-v2v can find drivers using the *.inf files.
For this to work, one field is important:
[Version]
DriverVer = 07/17/2013,6.1.0.10018
Copy the [Version] section header and DriverVer field from the real
*.inf files to our fake *.inf files, which is enough for testing.
See also:
https://www.redhat.com/archives/libguestfs/2015-October/msg00352.html
Note that the fields are case insensitive and may or may not contain
spaces.
The virtio-win ISO has a different layout and superset of drivers
compared to the virtio-win RPM directory. Previously we simply put
all the RPM files into the fake ISO. Instead create a fake ISO which
has the same layout and (fake) drivers as the real ISO.
Refactor the internal_feature_available to return the result for just
one group, so it is easier to know on the library side what was the
actual error, and which group refers to; drop internal_available, as no
more needed after this.
On the library side, implement in available and feature_available the
real logic to iterate through the requested group, and error out or
return whether the groups are available. This also introduces caching
for the features, so each needs to be queried just once in each
appliance run.
The result of this is that there should be much less communication with
the daemon to know about available features; the downside is that
queries for more than one group at once, not already cached, will be
cause more queries to the daemon.
Rename the current available and feature_available into internal daemon
functions, and provide non-daemon functions wrapping them at library
side. This will make it possible to e.g. add caching for them.
Should be only refactoring, no actual behaviour change.
The error messages changes from:
$ virt-v2v -i disk /tmp/blank.img -o null
[...]
[ 14.6] Inspecting the overlay
virt-v2v: error: no root device found in this operating system image.
to:
$ virt-v2v -i disk /tmp/blank.img -o null
[...]
[ 15.2] Inspecting the overlay
virt-v2v: error: inspection could not detect the source guest (or physical
machine).
Assuming that you are running virt-v2v/virt-p2v on a source which is
supported (and not, for example, a blank disk), then this should not
happen. You should run 'virt-v2v -v -x ... >& log' and attach the complete
log to a new bug report (see http://libguestfs.org).
No root device found in this operating system image.
Various tests cannot be run in parallel just because they happen to
use conflicting names for temporary output files (eg. "test.out").
Change these tests to use unique temporary names, so the tests could
be run in parallel.
When virt-sparsify receives a user quit signal (eg from ^C) it
currently kills virt-sparsify and qemu instantly, meaning any mount +
fstrim in progress is uncleanly stopped. The (minor) side effect of
this is that the guest filesystem may require a journal replay or fsck
on boot.
Let virt-sparsify capture the user quit signal and shut down
gracefully.
It is not thought that the previous behaviour could cause guest
corruption; see
https://lists.nongnu.org/archive/html/qemu-devel/2015-11/threads.html#00402
for discussion.
- include <sys/un.h> for sockaddr_un, on any OS (removing the Mac OS X
blocks)
- include <fcntl.h> for O_* flags
This commits reverts a couple of changes in these files done in
commit 3b17152ec3.
Remove man pages and other pages which 'make clean' did not remove
before.
To evaluate which pages could be removed, I did a full build and
check, and then ran 'make clean' followed by 'git clean -xdf'. By
examining the output of the git clean command I could see which files
were being missed.
Files that are _not_ removed by make clean or make distclean:
- generator-built files
- Makefile, Makefile.in, .deps, .depend
- any ./configure output files (maybe they should be?)
It's a nice idea, but has never actually found a problem. After using
afl to find bugs in qemu-img, it's pretty clear to me that no
successful approach could be run in the short time demanded by the
libguestfs test suite. Afl tests run for days, weeks or even months.
A better approach may be to use a proven fuzz tester (such as afl) on
the inspection code; or to run this code for days on end on a separate
testing machine.
This reverts commit c2dcd10964.
Also move guestfs-performance to the right place on the website.
This updates commit 2eb01004bf
and commit d07515b947
and commit 74fba6e692
and commit 15b8b6453d.
Move the random set of HTML files we build from html/ into
the website/ directory.
Also in the website/ directory, put the index.html file from
http://libguestfs.org, which was previously not under version control.
It is generated from index.html.in so we can automatically add the
current version and release date.
Also in the website/ directory, put various CSS file, images, etc.
which are required by the website and were also previously not under
version control.
Change the 'make website' rule to 'make maintainer-upload-website'.
As the name suggests, it is only useful for the maintainer, and will
fail with an error for anyone else.
When I conceived commit dc1d0880b0, I
thought it would be a good idea to build test-data files, including
the phony guest images, during the 'make' phase. Previously these
were built early in the 'make check' phase.
However this has two disadvantages: (a) We run the appliance
unnecessarily during builds. For example, it is not necessary to
build the phony guests if we are going to do an 'installcheck'.
(b) Even if we are testing, the critical first time we run the
appliance is not in 'quickcheck' or 'tests/qemu', but some random
guest build.
Reorder the build sequence to fix this, and also a few other things.
This updates commit dc1d0880b0.
This test failed during release testing. It turned out to be because
the tests are run with LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 which
causes other messages to be printed.
Updates commit b5a6f899cd.
Create a new top-level directory called test-data, which will carry
all the test data which is large and/or shared between multiple tests.
There are actually several new subdirectories created:
test-data/binaries: The pre-built binary and library files for random
architectures that we use to test various architecture detection
features (was part of tests/data).
test-data/blank-disks: The blank disks which are used for disk format
detection (was part of tests/data).
test-data/files: Other miscellaneous test files from tests/data that
are not included in the above.
test-data/phony-guests: The phony guests (was tests/guests).
test-data: The top-level directory builds the 'test.iso' image file
that is used for testing the C API and in miscellaneous other tests.
Create a Windows module, analogous to the Linux module we already
have. Move utility functions related to Windows into this module.
In particular:
- detect_antivirus function moves here, and the Detect_antivirus
module goes away entirely
- copy_virtio_drivers / virtio_iso_path_matches_guest_os moves here,
and the virtio_iso_path_matches_guest_os can now be hidden
This is just refactoring, but allows us to make deeper changes to
copy_virtio_drivers.
The three utility functions xml_quote_attr, xml_quote_pcdata and
uri_escape were only used in a single module. Move them close to
where they are used.
This is just code refactoring.
Unfortunately Coccinelle places a Config module in the ocaml libdir,
and this confuses ocamlfind (only when Coccinelle is installed).
Since this is a private module that only libguestfs tools use
internally, just rename it from Config -> Guestfs_config.
Do this early since these tests generally don't depend on anything
else (the external program is either there or its not), and because
other things depend on some of these tests.
The error was:
libguestfs: trace: pread_device = "NTFS "
libguestfs: trace: part_to_dev "/dev/sda"
guestfsd: main_loop: new request, len 0x34
guestfsd: error: device name is not a partition
guestfsd: main_loop: proc 272 (part_to_dev) took 0.00 seconds
libguestfs: trace: part_to_dev = NULL (error)
virt-v2v: error: libguestfs error: part_to_dev: device name is not a
partition
This way, we can make sure that if a template needs to be expanded, the
result will have all the filesystems correctly expanded, fitting into
the new sizes of their partitions.
Introduce a new option to control how virt-resize behaves when asking to
expand a filesystem, either unknown to libguestfs or that virt-resize
cannot expand. The default keeps the current behaviour, i.e. just warn.
Replace ExtUtils::MakeMaker with Module::Build.
'perllocal.pod' and 'bindtests.pl' are no longer incorrectly
installed.
This change also removes the following phony deps: appliance src_deps
test_images. No other language binding needs explicit dependencies
for their tests, they just rely on the top level build order being
correct (ie. SUBDIRS in /Makefile.am).
'ocamlfind install' doesn't like to install the same package if it is
already installed. It will give errors like:
ocamlfind: Package guestfs is already installed
or:
ocamlfind: Conflict with file: /usr/lib64/ocaml/guestfs/mlguestfs.cmxa
Fix this by removing the old installed files before running
'ocamlfind install'. An alternative would be to use
'ocamlfind uninstall', but this seems more direct.
The error is:
ln -s xdg/virt-builder /tmp/p/etc/virt-builder
ln: failed to create symbolic link '/etc/virt-builder/virt-builder': File exists
There is no 'LN_SF' macro, but according to the documentation at
least, it should be safe to add a '-f' flag since it will work for
both the 'ln' and 'cp' variants of the macro.
When running in verbose mode, dump the XML we created for libvirt:
this way it is easier to debug mismatches between what we want to setup
in libvirt, and what actually gets defined.
Create the correct tags for <video> in the libvirtxml, so all the
attributes are in a <model> child of <video> instead of <video> itself.
Adapt the XML of test-v2v-i-ova to the different expected XML.
Move most of the content of either po-docs/ja/Makefile.am or
po-docs/uk/Makefile.am to po-docs/language.mk, and use it exclusively
instead of the former contents of the languange-specific Makefile.am.
This way, either adding a new documentation or enabling a new language
will not require copying over the same make code.
Move most of open_guestfs to Types, so a common function to open a
Guestfs handle is available. Since it does not do all the things the
old open_guestfs did, the two operations have been moved to the only
place requiring them.
This function has been placed in Types and not Utils, as will be needed
by other functions in Types; Utils uses Types, so this avoids a circular
dependency.
Use FilenameList as type for lists of file names, as used in some
listing-alike APIs. This way we can ensure anything different than just
file names in those lists is rejected outright.
As a consequence, test-big-dirs.pl does not need to prepend the
directory name anymore before calling listing-alike APIs: previously
they didn't fail, but the returned lists contained only invalid
elements (and only their size was checked).
Furthermore, add a new regression test for it.
We have phony tools for v2v tests, introduced with
commit c1af1ba32f126b7be275bcc8cd4ea4e5af27a88e; make use of it also in
tests related to v2v outside the v2v directory.
I have also removed the text equivalent of this file. Originally I
added this because it was thought good to have the release notes
available in a format that doesn't require any special tools to read.
But:
- POD files are basically text.
- Debian tooling generates the text file in a slightly different way
from Fedora tooling, resulting in git marking the file as being
updated when it isn't really.
- github can format and display POD files.
In this mode, converting of the VM configuration, setting up the
rollback path for error cases, transforming the VM storage and so on is
taken care of by a third-party toolset, and virt-v2v is only supposed to
tune up the guest OS directly inside the source VM, to enable it to boot
and run under the input hypervisor.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Factor out perfoming the actual convertion of the guest, which includes
determinig the appropriate guest os specific conversion module and
running its conversion routine.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Iterating over source disks and creating temporary overlays for easy
rollback fits nicely into a separate function. In addition, determining
their size doesn't need to wait until the guestfs is launched: the size
can be obtained via disk_virtual_size() method.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Opening the source VM and amending the properties in its internal
representation in accordance with command-line options fit nicely into
two isolated functions.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
If Augeas fails to parse a file, more information is available in at
least these fields:
><fs> aug-ls /augeas/files/etc/fstab/error
/augeas/files/etc/fstab/error/char
/augeas/files/etc/fstab/error/lens
/augeas/files/etc/fstab/error/line
/augeas/files/etc/fstab/error/message
/augeas/files/etc/fstab/error/pos
Pull out some of these fields and add them to the error message.
The new error message looks like this:
virt-v2v: error: libguestfs error: /etc/fstab:17:4: augeas parse failure:
Iterated lens matched less than it should
This updates commit 279addeb7d.
In the case where libvirt was installed, but is too old to run the
backend, the libvirt backend would be compiled out but still
referenced by the library.
Fix this by splitting the definition of HAVE_LIBVIRT (libvirt is
installed at all) and HAVE_LIBVIRT_BACKEND (libvirt is new enough to
run the libvirt backend).
Reported by: Efstratios Skleparis.
During the inspection phase, check for errors after aug_load: if any of
the errors happened in any of the requested files, then report the error
straight away; ignoring the error means that information would be
silently lost. For example, a malformed /etc/fstab would have caused
the inspection to not handle any of the additional mount points, giving
cryptic errors later on when trying to access files in any of the mount
points.
Now guests with invalid files such as /etc/fstab, /etc/mdadm.conf, and
/etc/sysconfig/network will cause the inspection to fail, instead of
being reported with a single mount point ('/').
Now that virtio driver copying is localized in a single function and the
auxiliary guestfs handle doesn't leak outside of it (and thus an extra
qemu instance doesn't survive beyond that function's runtime) there's no
need in the forced GC which used to trigger closing of that handle and
termination of that qemu instance.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Refactor copying of virtio windows drivers into the guest so that the
matching of the drivers to the guest os flavor and copying the files
happens one next to the other in a single function, and no guestfs
handle (nor any other irrelevant info) is leaked outside.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Copy the appropriate driver files from the virtio-win storage into
%SYSTEMROOT%\Drivers\VirtIO once they are discovered, and stick to using
those copies later on.
This way the knowledge of where the drivers come from originally is
consolidated in one place, so the lifetime of the associated entities
becomes easier to control (to be implemented in followup patches).
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Move this function to the VCenter module. This is easier since it
doesn't have to do dcPath calculation (see previous commit).
The readhead parameter is no longer labelled.
In libvirt >= 1.2.20, the VMware libvirt driver supplies the correct
dcPath to use via <vmware:datacenterpath> in the libvirt XML. If
libvirt passes us this element, use it.
This code still allows the user to override dcPath using the --dcPath
option on the command line, but that's mainly for safety so we can fix
any problems in virt-v2v or libvirt in the field. As we get more
confident in libvirt and as libvirt 1.2.20 is more widely adopted, we
will be able to deprecate this parameter entirely.
Thanks: Matthias Bolte for adding the <vmware:datacenterpath> element
to libvirt in
https://libvirt.org/git/?p=libvirt.git;a=commit;h=636a99058758a0447482f3baad94de8de3ab1151
libguestfs: trace: v2v: read_lines "/etc/os-release"
[...]
libguestfs: trace: v2v: read_lines = ["NAME=Fedora", "VERSION="20 (Heisenbug)"", "ID=fedora", "VERSION_ID=20", "PRETTY_NAME="Fedora 20 (Heisenbug)"", "ANSI_COLOR="0;34"", "CPE_NAME="cpe:/o:fedoraproject:fedora:20"", "HOME_URL="https://fedoraproject.org/"", "BUG_REPORT_URL="https://bugzilla.redhat.com/"", "REDHAT_BUGZILLA_PRODUCT="Fedora"", "REDHAT_BUGZILLA_PRODUCT_VERSION=20", "REDHAT_SUPPORT_PRODUCT="Fedora"", "REDHAT_SUPPORT_PRODUCT_VERSION=20"]
At this point libguestfs segfaulted.
It seems as if 'major' is not used here, so don't try to free it.
This fixes commit 32d19e3289.
On 32 bit platforms, reading the memory values can cause some numbers
to be read as negative numbers. Fix this by treating memory values as
64 bit integers throughout the parsing and calculation.
The log file was not being closed at the right time, resulting in the
process running out of open files. Also the log filename buffer was
not freed.
This fixes commit 7bbf5cab28.
Set the handle identifier of the main handle to "v2v".
Whenever we open a second handle for any substantial operation, set
the handle identifier for that handle too.
This should make traces clearer.
For that, add autofoo machinery to generate the iso out of the fake
virtio-win drivers introduced in the previous patches, and pass the
resulting image in $VIRTIO_WIN to v2v in a new test otherwise similar to
test-v2v-windows-conversion.sh.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
In order to test the copying of virtio-win drivers into the guest during
v2v, create a set of fake virtio-win drivers and make use of them in the
corresponding v2v tests.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
This option can be used to direct output from each individual
boot test to a different file. Usual usage is:
qemu-boot --log /tmp/output.% -v -x -n 1000
Add a per-handle identifier. The main effect of this is to change
trace output from:
libguestfs: trace: foo
to:
libguestfs: trace: ID: foo
which makes it simpler to follow traces from multi-threaded programs.
v2v/test-v2v-windows-conversion.sh used to query if the expected
directories and files were present in the VM upon conversion; however it
would ignore the results of that query.
That lead to the test passing even though the checks failed.
To fix it, parse the output of guestfish and verify that all is-file and
is-dir commands report the expected "true".
The approach taken is slightly more elaborate than what is being used in
other components, but it allows to generate comprehensible reports in
case of test failure, and facilitates future expansion of the set of
checks including negative ones.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Without rhev-apt.exe the code path where firstboot scripts are created
isn't taken, so it isn't tested and v2v/test-v2v-windows-conversion.sh
is supposed to fail (it actually doesn't but for another reason,
addressed in a followup patch).
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
The tests that depend on rhsrvany.exe being available, actually don't
need it to be the real one: all they want is the file being present.
So, instead of skipping the tests if the real rhsrvany.exe is not found,
store a fake one in the repository, and point $VIRT_TOOLS_DATA_DIR at
its location.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Make sure to close the open files even when test_fuse() returns earlier
because of failures, otherwise the mountpoint is still in use by open
files when later guestunmount is called on it.
** NB: This is an API break for OCaml programs using Guestfs.event_callback. **
Because of the way I implemented Guestfs.event_callback which had the
Guestfs.t handle as the first parameter, we had to store the (OCaml)
Guestfs.t handle in the C handle's private data area. To do that, we
had to create a global root pointing to the handle.
This of course meant that the handle could not be garbage collected
(thanks Roman Kagan for spotting this).
This changes the API of Guestfs.event_callback so that a handle is no
longer passed. The OCaml handle can now be garbage collected again.
For programs that need the Guestfs.t handle in the callback function
(which turns out to be *none* of the OCaml programs we have written),
you can do:
g#set_event_callback (callback_fn g) [Guestfs.EVENT_FOO];
In this case, since the closure passed to Guestfs.set_event_callback
is still registered as a global root, that will capture a reference to
the handle, so the handle won't be able to be garbage collected until
you delete the callback.
These are considerably more efficient than ordinary global roots, but
with the caveat that the program is not allowed to modify them without
calling a special function. We don't modify them, so this change is
safe.
This requires OCaml >= 3.11, but we have that on RHEL 6
(since we dropped support for RHEL 5).
See also:
http://caml.inria.fr/pub/ml-archives/caml-list/2008/03/c3bf86990088236ceeb9a0f0f4c35390.en.html
This is just a straight refactoring. Various ad hoc string_*
functions that appeared in Common_utils have been renamed and placed
in the String.* namespace. The old vs "new" functions are:
string_prefix -> String.is_prefix
string_suffix -> String.is_suffix
string_find -> String.find
replace_str -> String.replace
string_nsplit -> String.nsplit
string_split -> String.split
string_lines_split -> String.lines_split
string_random8 -> String.random8
In Common_utils, override the Char and String modules from stdlib.
This hides the original (stdlib) modules, and means that whenever you
use Char.foo or String.foo you are in fact calling the
Common_utils.Char.foo or Common_utils.String.foo function.
As it stands, this change does nothing, but it will allow us to drop
unsafe functions (like String.lowercase) and add extra functions.
Calling _exit(2) in the child process has the side effect that
tmp/libguestfsXXXXXX is not cleaned up. Clean it up by ensuring the
handle is properly closed before _exit.
This test has been broken for a while. It is meant to test that when
the appliance cachedir is empty, two simultaneous runs of libguestfs
(both rebuilding the full appliance) will not cause conflicts, because
(eg) the locking in either supermin or libguestfs is not working.
However the test only set $TMPDIR, but the ./run script sets
$LIBGUESTFS_CACHEDIR which overrides $TMPDIR, so it was simply reusing
the existing appliance, and hence not testing anything.
Fix this by clearing $LIBGUESTFS_CACHEDIR.
Note the test now takes a lot longer to run since it does a full
appliance rebuild.
Accoding to its documentation [1], it takes as second parameter a
PangoWrapMode and not a GtkWrapMode. While the change should reflect
the actual value intended, it should slightly change the label wrap
mode: GTK_WRAP_WORD is 2, which corresponds (as int value) to
PANGO_WRAP_WORD_CHAR.
[1] https://developer.gnome.org/gtk2/stable/GtkLabel.html#gtk-label-set-line-wrap-mode
There is GUESTFSD_EXT_CMD defining a string for udevadm (so it is marked
as "used tool" in the appliance), but it is not actually used when
starting udevadm.
There should be no behaviour change.
Instead of hardcoding the location of perl (assuming it is installed in
/usr), use /usr/bin/env to run it, and thus picking it from $PATH.
This makes it possible to run these scripts also on installations with
perl in a different prefix than /usr.
Also, given that we want enable warnings on scripts, turn the -w
previously in shebang to explicit "use warnings;" in scripts which
didn't have it before.
Apparently implicitly pulled by something else on Linux, it is not when
building on e.g. FreeBSD.
This reverts the change to this file of
commit 3b17152ec3.
We don't support RHEL 5 upstream (see the 'oldlinux' branch for a
version that works with RHEL 5). Therefore remove a bunch of hacks
that were only needed on RHEL 5.
Because of previous automated commits, such as changing 'guestfs___'
-> 'guestfs_int_', several function calls no longer lined up with
their parameters, and some lines were too long.
The bulk of this commit was done using emacs batch mode and the
technique described here:
http://www.cslab.pepperdine.edu/warford/BatchIndentationEmacs.html
The changes suggested by emacs were then reviewed by hand.
When searching for virtio drivers, it makes no sense to fail if we
encounter a file without extension.
Ignore such files instead, just like we do for the files whose extension
doesn't match that of a driver-related file.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Our documentation of set-label says that labels for btrfs are limited
to 256 characters, while btrfs-filesystem(1) says:
Note
the maximum allowable length shall be less than 256 chars
which is indeed true, as trying to set a label with 256 characters
fails. Hence, fix our documentation to say that the limit is actually
255 characters.
Related to RHBZ#1164708.
Previously we used (on arm): -M virt -machine accel=...
After this change, we use: -machine virt,accel=...
On x86 there is no change.
The -M option is an abbreviation for -machine, and we can combine the
two options together. (Qemu combines the two options internally, but
it's more predictable if we do it ourselves).
Upstream qemu has supported the -machine option for over 5 years, and
in fact we implicitly relied on it (despite what the comment in the
code said).
- fix the syntax of hyperlinks
- replace the deprecation text with a simplier named list item, so it's
more visible
- use a named list item for the pointer to the C documentation of each
API
- add a named list item for the version of each API
If you've ever tried to use this option, you'll know that it didn't
work well. It broke random things (probably RHBZ#1020216, definitely
RHBZ#1023630), and caused random failures generally, while often not
actually failing when valgrind itself found problems.
Commit f6a790b67d changed the Makefile
so it only builds the tests when you run 'make check'.
If you didn't run 'make check' then (in particular) bindtests won't be
built, and so when you install it cannot be removed.
The simple fix for this is to make the removal of bindtests.*
conditional.
This fixes commit f6a790b67d.
Use the generator_built variable to list any files which are built by
the generator. This ensures they get automatically rebuilt.
Three files were missing from EXTRA_DIST, so they were not included in
the tarball, meaning you would have needed OCaml to build from the
tarball. You would have seen this when building:
cd .. && /builddir/build/BUILD/libguestfs-1.31.9/generator/generator
written gobject/docs/guestfs-title.sgml
written customize/customize-synopsis.pod
written customize/customize-options.pod
generated 476921 lines of code
Miscellaneous improvements to the ocamldoc:
- Generate more sub-headings.
- Document the object-oriented API.
- Use a common function to generate the doc for module and OO APIs.
Using 'ocamlfind ocamldoc' is much faster than running 'ocamldoc'
directly, because ocamlfind will run the native code program
'ocamldoc.opt' if it is available.
This change approximately halves the time taken to compile the ocaml
bindings.
Modify the copy-in/copy-out APIs to use the new guestfs_int_cmd_pipe
internal API.
This new API allows us to capture the stderr from the tar subprocess
if it fails, fixing RHBZ#1267032. The user will now see errors like
this:
$ guestfish -N fs -m /dev/sda1 copy-in '/tmp/d/*' / : ll /
libguestfs: error: tar subprocess failed: tar: *: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
Anything that doesn't involve pipes [parent-child communication] is
moved into the run_child function, so we can reuse the same code when
writing the new pipe APIs (see following commit). Anything involving
setting up pipes stays in the existing run_command function.
This is just refactoring.
Allow this useful function to be called from elsewhere.
This commit is almost, but not completely refactoring. I made a minor
change to the function so that it \0-terminates the returned data,
which is convenient for some callers.
When building from git, you need OCaml to compile the generator to
build the generated files. Previously configure didn't detect that
situation, resulting in failed builds. This commit causes the
configure script to error out if the generated files are not present
and OCaml is not installed.
Note that you can still build without OCaml, if you get the generated
files from somewhere else. The source tarballs distributed on the
website contain the generated files.
As in commit e102bcf3cf (for Perl), any
Python program has the handle program name field set to 'python'. Set
it to the true name (derived from sys.argv[0]).
When compiling the libguestfs bindings as a pip module, it's helpful
if you can compile the bindings with a different version of libguestfs
(eg. installed 1.28, pip module 1.30).
That broke the previous assumption that we were always compiling
against precisely the same version of libguestfs (which made sense
because the python bindings are part of the libguestfs source tree).
Python distutils has no way to check a C library to find out if
functions are defined. Therefore we have to rely on GUESTFS_HAVE_*
macros from <guestfs.h>.
Previously we only defined GUESTFS_HAVE_* macros for functions that
were not deprecated, test or debug functions. The reasoning for that
is given in this commit message [note that 'LIBGUESTFS_HAVE_' is the
old/deprecated macro]:
commit 2d8fd7dacd
Author: Richard Jones <rjones@redhat.com>
Date: Thu Sep 2 22:45:54 2010 +0100
Define LIBGUESTFS_HAVE_<shortname> for C API functions.
The actions each have a corresponding define, eg:
#define LIBGUESTFS_HAVE_VGUUID 1
extern char *guestfs_vguuid (guestfs_h *g, const char *vgname);
However functions which are for testing, debugging or deprecated do
not have the corresponding define. Also a few functions are so
basic (eg. guestfs_create) that there is no point defining a symbol
for them.
This wasn't in fact carried through very consistently, since when we
replaced s/LIBGUESTFS_HAVE_/GUESTFS_HAVE_/, we kept the old
LIBGUESTFS_HAVE_* macros, but defined them for every function. Oops.
This commit defines GUESTFS_HAVE_* for every function in <guestfs.h>,
making it easier to write the Python language bindings (see following
commits).
Allow the tests to be run on the installed libguestfs.so library and
on installed programs (but not installed language bindings, yet).
To run the tests, you have to have a copy of the libguestfs source
tree with exactly the same version as the installed libguestfs. You
have to build the libguestfs source tree. Then you do:
make installcheck
I investigated making automake 'make installcheck' work, but it's
basically broken (no surprise there).
This is derived from the old 'make-check-on-installed.pl'
script (which is also removed in this patch).
If either zcat or cpio fails when spawned in initrd-list, pclose will
return the actual return value of it, but reply_with_perror still uses
errno regardless; thus, the reported error is:
libguestfs: error: initrd_list: pclose: Success
which is not much helpful.
Instead, when pclose returns > 0, extract the actual return value of the
subprocess, and print that. Thus now we get for example:
libguestfs: error: initrd_list: pclose: command failed with return code 1
It's currently impossible to correctly predict the dcPath parameter
from the data that libvirt gives us. So allow the user to override
--dcpath themselves.
Eventually we will get better support in libvirt for this, and this
option will no longer be needed:
https://www.redhat.com/archives/libvir-list/2015-September/thread.html#00201
This enhances commit 51bc573d0c
and commit 20f1eb400b.
If a Linux guest has /etc/os-release (mandatory for systemd, so all the
recent Linux distributions have it), try to parse it for the distro, its
version, and its product name. If we cannot get all of them, the
detection as done so far continues.
Other than changes in the product_name string (which is free-form
anyway, and already documented to not be parseable), there should be no
regressions in the detected distro and version.
Filter out from the git ls-files output the files we don't care they
appeared (gnulib and intltools stuff), and use comm to only show the
remaining added files (i.e. files not being distributed).
The output of comm is redirected to file and then printed, so we can
check it is empty (failing the target if not).
Add a new "simplestreams" repository type, and a simple parser for
fetching and reading the JSON indexes of the Simple Streams v1.0
format.
Read only datatype=image-downloads contents, and only the latest
versions of each content available as disk image (disk.img or
disk1.img).
Add a simple test, using the "released" images from the CirrOS project.
Check if the Windows guest has Group Policy Objects installed, or one
of several popular pieces of anti-virus software. If we are
installing a virtio block driver, then experience has shown this may
cause a 7B boot failure.
Print a warning when this combination happens.
The warnings look like this:
[ 19.9] Converting Windows Server 2008 R2 Enterprise to run on KVM
virt-v2v: warning: this guest has Windows Group Policy Objects (GPO) and a
new virtio block device driver was installed. In some circumstances, Group
Policy may prevent new drivers from working (resulting in a 7B boot error).
If this happens, try disabling Group Policy before doing the conversion.
virt-v2v: warning: this guest has Anti-Virus (AV) software and a new virtio
block device driver was installed. In some circumstances, AV may prevent
new drivers from working (resulting in a 7B boot error). If this happens,
try disabling AV before doing the conversion.
virt-v2v: This guest has virtio drivers installed.
Change the handling of -ic, -if, -oc, -of, -on, -os, --password-file,
--vdsm-vm-uuid, --vdsm-ovf-output, --vmtype options, so that if any
appear multiple times on the command line we error out:
$ virt-v2v -i disk /tmp/centos-6.img -o null -of qcow2 -of raw
virt-v2v: error: -of option used more than once on the command line
Thanks: Juquin Zhou for finding the bug.
Turns out that some of the Windows guests in the test suite try to
reactivate if the date is more than some time delta (maybe 1 year?)
after the previous activation. Just disabling the network wasn't
enough to prevent this.
qemu allows us to boot guests with the clock set to a particular date
in the past, and libvirt exposes this (albeit very clumsily):
https://docs.fedoraproject.org/en-US/Fedora/18/html/Virtualization_Administration_Guide/sect-Virtualization-Tips_and_tricks-Libvirt_Managed_Timers.html
Allow guests to be booted with the date set to a particular past date,
by setting the test_plan.guest_clock field.
This required adding some more libxml2 bindings.
When calling test_connection to test the connection to a conversion
server, we also query for the remote server's version and driver
information (using 'virt-v2v --version' and
'virt-v2v --machine-readable').
However we didn't clear the existing information from previous runs.
One consequence of this was that if you used the back button in the
GUI, the list of output drivers would be repopulated two or more
times. https://bugzilla.redhat.com/show_bug.cgi?id=1227609
Another is that connecting to a different server could confuse
test_connection into thinking the connection was successful (because
of data from the previous run) when in fact it was not.
Thanks: Tingting Zheng for finding the original bug.
Split in an own function the code dong the mounting, the inspection of
the kernel files, and the downloading, including a per-OS function for
the actual kernel files. This gives few advantages:
- the download phease is not repeated for all the files to fetch
- it is easier to eventually support multi-root disk images
- it is possible to support OSes different than Linux; virt-get-kernel
now will just report the unsupported OS, instead of a generic
"no kernel found" message
This is mostly code refactoring, with (on Linux) no actual behaviour
change.
Make print_version_and_exit, long_options, display_short_options, and
display_long_options private, as set_standard_options now takes care of
handling the job for the common command line options.
Move --debug-gc as common option for all the OCaml-based tools, even a
couple of them which didn't have it previously.
As implementation note, make set_debug_gc private to
set_standard_options, as it needed to be moved otherwise, and it is no
more required as public function.
Introduce a new common helper to add the common options for libguestfs
tools (short/long options, version, verbose, trace), and sort them.
All the OCaml-based tools had these options already, so there are no
functional changes in the interface they provide.
The only difference is that now the options are always sorted, while
some tools didn't had them like that previously: because of this, a
couple of ditto markers (as descriptions) don't match what's above them
anymore, and thus their full description is put instead.
When running (eg) dnf on a 32 bit i686 guest when the host is 64 bit
x86_64, dnf believes it is running on a 64 bit machine and so tries to
install x86_64 packages. We can 'trick' dnf into believing it's a 32
bit machine using the setarch program.
$ virt-builder fedora-22 --arch i686 --install 'gperf'
...
[ 27.4] Installing packages: gperf
...
Running transaction test
Error: Transaction check error:
package libgcc-5.1.1-4.fc22.x86_64 is intended for a different architecture
...
The use of a heredoc to solve quoting issues comes from:
http://stackoverflow.com/a/3435460
Thanks: Jan Sedlák for finding the solution.
In -i libvirtxml, -i ova and -o libvirt drivers, replace the ad hoc
xpath_to_* functions with use of the new xpath convenience functions
introduced in the previous commit.
This is not entirely refactoring because I fixed a few bugs found by
type safety.
Previously given a path such as:
vpx://vcenter/Folder/Datacenter/esxi
we calculated dcPath=Folder. However this is obviously wrong. We
should chop the path at the final (esxi) element to give
dcPath=Folder/Datacenter.
In vCenter, the datacenter is identified by a path, and the parameter
used for this is called 'dcPath'. Rename the function to avoid any
confusion about what we're getting here.
This is just renaming function/variable names and has no other effect.
Several tools handle --debug-gc command-line option, by explicitly
forcing GC on every exit path. This is tedious and prone to forgetting
some of the exit paths.
Instead, add a generic handler for --debug-gc, which installs an at_exit
hook to do the GC consistency check, and which can be called right in
the command-line parser. Also adjust all users of --debug-gc to use
that handler.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Build the P2V disk image and boot it. We don't actually use PXE
specifically, but we do test the whole PXE / kernel command line path
much more thoroughly.
This is a 'check-slow' test because it takes ages to run.
When using the virt-p2v ISO in command line mode, we did not wait for
the network to come online before starting virt-p2v. Therefore
virt-p2v could exit with an error when testing the ssh connection (or
on the other hand, it might work randomly). If the user logs in and
runs 'launch-virt-p2v' by hand, then it would usually work because the
network had been brought online in the meantime.
Fix this by waiting for NetworkManager to bring the connection online
before calling test_connection(). Note that the obvious way to fix
this (changing the systemd service to wait for network-online.target)
does *not* work - I added a comment to the service about this.
Thanks: Tingting Zheng
Raise a warning for each kernel specified in grub which does not
actually exist, keep going on with the conversion using the remaining
(existing) kernels.
Put in a list the errnos to expose, filling the content of the
Guestfs.Errno submodule from that.
Also, generate a separate guestfs-c-errnos.c with the implementations of
the functions returning the errno codes.
Only code motion and refactoring, no actual changes on the content of
the ocaml Guestfs module.
pcre < 8 has limitations on the types of partial matching
possible. As noted in the pcrepartial(3) man page, these can
be worked around by adding non-capturing parentheses around
various subexpressions, as I have done here.
Apparently gtk_init_check doesn't (always?) print an error message
when it returns false, so virt-p2v just exits without any message.
This was observed on RHEL 6 when $DISPLAY was not set.
File input_libvirt_other.ml, line 1, characters 0-1:
Error: The implementation input_libvirt_other.ml
does not match the interface input_libvirt_other.cmi:
Class declarations do not match:
class virtual input_libvirt :
'a ->
string option ->
string ->
object
method adjust_overlay_parameters : Types.overlay -> unit
method as_options : string
method virtual source : unit -> Types.source
end
does not match
class virtual input_libvirt :
string option ->
string option ->
string ->
object
method adjust_overlay_parameters : Types.overlay -> unit
method as_options : string
method virtual source : unit -> Types.source
end
A parameter has type 'a but is expected to have type string option
When the output format is qcow2 and -oa preallocated is used,
previously it would only allocate the metadata. That was a regression
over previous behaviour of virt-v2v 0.9. Change it so it allocates
the full file size in this case.
For raw, this allows "off" as a synonym for "sparse" (to make it
consistent with qcow2).
For qcow2, this allows "sparse" as a synonym for "off".
It also adds qcow2 "full" preallocation, which is actually mapped to
the qemu option "falloc" (see arguments about this on the qemu-devel
mailing list, which we lost).
This also updates the test.
Normalize the target architecture, and also each architecture when
checking for a compatible image.
This sort of reverts the effects of
commit 8864c47b94, but at least it is
possible to build e.g. Debian-based amd64 images on any x86_64 system
without being considered as foreign architecture.
Small helper to normalize an architecture string, so it is easier to
compare them and check for compatibilities.
Make use of it in guest_arch_compatible, simplifying it.
If a function name, with its trailing parentheses, is in the
environment , trying to unset it will error out with a message of "not
a valid identifier". Handle this as a special case, and unset it with
the -f option which can handle the parentheses in the supplied
identifier.
It turns out that -- some of the time -- systemd ignores unit files
which are not placed in one of the official unit directories. This
caused the service to be timed out and killed when the systemd daemon
was reloaded, as could happen for long-running package installs at
first boot.
Thanks: Marius Vollmer for diagnosing the problem and finding the
solution.
If the format of the disk image of the template is not specified in the
index, virt-builder would print
Converting auto to qcow2
when asking to generate a qcow2 image. Instead, in this case print only
the destination format, like
Converting to qcow2
I discovered that 'dnf upgrade' doesn't actually upgrade to the latest
versions of packages unless you also supply the '--best' flag.
This also changes update -> upgrade, since apparently 'dnf update'
is deprecated.
Because this 'feature' is broken (since 2013):
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=13928
we have to replace all instances of $(top_srcdir) in *_SOURCES lines
with a relative path. According to what I read, this shouldn't break
split builds, but I didn't test it.
The only things automake moans about now are:
* Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/\${ <-- HERE ([^ \t=:+{}]+)}/ at /usr/bin/automake line 3936.
- This is another bug in automake
* autoreconf: configure.ac: AM_GNU_GETTEXT is used, but not AM_GNU_GETTEXT_VERSION
- This is a feature, not a bug.
Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
554829 NEW https://bugzilla.redhat.com/show_bug.cgi?id=554829
SELinux handling could be done better.
572337 NEW https://bugzilla.redhat.com/show_bug.cgi?id=572337
libguestfs should support gptsync
@@ -56,9 +53,6 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
824021 NEW https://bugzilla.redhat.com/show_bug.cgi?id=824021
inspection cannot recognize guest which uses btrfs subvolumes for root
824867 NEW https://bugzilla.redhat.com/show_bug.cgi?id=824867
Windows guest with dynamic disk can not boot on rhev after conversion by virt-p2v
832602 NEW https://bugzilla.redhat.com/show_bug.cgi?id=832602
"error in chunked encoding" when trying to extract (tar-out) a truncated ISO image
@@ -74,9 +68,6 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
848926 NEW https://bugzilla.redhat.com/show_bug.cgi?id=848926
virt-v2v should verify that the RHEV-M domain is an export storage domain
855058 NEW https://bugzilla.redhat.com/show_bug.cgi?id=855058
RFE: virt-p2v: display more information about storage devices
857763 NEW https://bugzilla.redhat.com/show_bug.cgi?id=857763
libguestfs 'file-architecture' returns 'ARM' for arm binaries
@@ -98,12 +89,6 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
903987 NEW https://bugzilla.redhat.com/show_bug.cgi?id=903987
Command "acl-set-file" is inconvenient to use
906766 NEW https://bugzilla.redhat.com/show_bug.cgi?id=906766
virt-resize cannot resize RHEL 5 guest with GPT partition table type
906777 NEW https://bugzilla.redhat.com/show_bug.cgi?id=906777
In tests, set_label sometimes fails
910115 NEW https://bugzilla.redhat.com/show_bug.cgi?id=910115
RFE: support ntfsresize --bad-sectors option in virt-resize
@@ -125,9 +110,6 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
921604 NEW https://bugzilla.redhat.com/show_bug.cgi?id=921604
RFE: virt-v2v: add option to specify spice or VNC and whether to add USB connections
927447 NEW https://bugzilla.redhat.com/show_bug.cgi?id=927447
[suse] virt tools hang on disk image, but libguestfs-test-tool runs OK
963232 NEW https://bugzilla.redhat.com/show_bug.cgi?id=963232
RFE: v2v: a --no-cleanup would be useful for debugging of disk images that guestfs has issues converting
@@ -137,9 +119,6 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
982516 NEW https://bugzilla.redhat.com/show_bug.cgi?id=982516
sh "rpm -Va" fails because apparently /dev/shm is not created/bind-mounted correctly
983969 NEW https://bugzilla.redhat.com/show_bug.cgi?id=983969
RFE: virt-sysprep should be SELinux-aware
986242 NEW https://bugzilla.redhat.com/show_bug.cgi?id=986242
golang bindings cannot be compiled if golang-guestfs package is installed
@@ -152,12 +131,6 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
991830 NEW https://bugzilla.redhat.com/show_bug.cgi?id=991830
virt-resize throws ambiguous error when /dev/mapper/VG-LV path is used
994965 NEW https://bugzilla.redhat.com/show_bug.cgi?id=994965
libguestfs doesn't work on armv7hl
1019388 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1019388
firstboot scripts (virt-builder, virt-sysprep) don't work for Debian 6 & 7 guests
1023630 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1023630
enable-valgrind-daemon option breaks UML
@@ -176,9 +149,6 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
1040912 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1040912
[ARM] mount-local / FUSE support does not seem to work at all
1051694 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1051694
libguestfs FTBFS on Ubuntu saucy
1056047 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1056047
virt-df generates no output for Solaris 11 x86_64 guest.
@@ -191,54 +161,33 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
1061040 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1061040
RFE: Enable ZFS support
1064041 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1064041
virt-sparsify fails if a btrfs filesystem contains readonly snapshots
1075594 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1075594
RFE: python-libguestfs should be available on pypi
1079734 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1079734
Tiny Core Linux is not recognized by libguestfs inspection
1089100 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1089100
NetworkManager avc unlink denied for resolv.conf after using --selinux-relabel
1092583 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1092583
guestfish removes query string from URI
1097272 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1097272
Ruby bindings should use RB_GC_GUARD (instead of volatile) for enhanced safety
1099292 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1099292
sfdisk failed by "Device or resource busy" when using one command line
1099475 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1099475
sfdisk: BLKRRPART: Device or resource busy
1099976 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1099976
virt-builder gives GPG warning message with gnupg2
1100138 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1100138
some operation cause "lvremove" can not find the object
1100140 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1100140
some operation cause "lvremove" can not find the object
1102241 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1102241
[RFE] libguestfs should detect OSTree (project-atomic) qcow2 disk image
1102619 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1102619
lvm-clear-filter failed when has two same name but different uuid's VG
1102620 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1102620
lvm-clear-filter failed when has two same name but different uuid's VG
1103444 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1103444
virt-v2v of RHEL AS release 3(Taroon) VMware VM fails because rpm binary segfaults
1107317 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1107317
Ubuntu package fails to depend on linux-image-extra (causes missing virtio-serial)
1108171 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1108171
RFE: virt-builder should be able to build PXE images for baremetal installation
@@ -251,41 +200,20 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
1129110 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1129110
Libguestfs NTFS not setting ACLs
1132652 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1132652
virt-sysprep firstboot doesn't work with RHEL 7 guest
1134726 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1134726
[RFE] virt-builder should support args with the --run command
1136371 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1136371
add apt-rpm support to supermin
1139785 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1139785
virt-sysprep should change UUID fields in /etc/sysconfig/network-scripts/ifcfg-* files
1142184 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1142184
virt-sparsify --help print description of a option which is hard to get common understanding
1144137 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1144137
virt-inspector fails on Minix 3 guest
1146477 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1146477
virt-v2v sometimes hangs with printing: Add. Sense: No additional sense information
1150298 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1150298
ARM 32 bit on Ubuntu: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
1151902 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1151902
virt-v2v:Network interface disappear after conversion of guest with interface type direct
RFE: virt-v2v should do something with <interface type='direct'> (either virtualize it or give an error)
1151903 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1151903
virt tools --csv output should have a meaningful output
1151905 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1151905
virt tools --csv output should have a meaningful output
1152533 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152533
virt-v2v fail to convert guest with virtio lun disk
RFE: virt-v2v should do something with <disk device='lun'> (either virtualize it or give an error)
1152818 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152818
llz give ? for the security context of a file, which is wrong
@@ -293,18 +221,6 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
1152819 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152819
Can not end a running command in virt-rescue by press ^C or other keys, the only way is to exit virt-rescue
1152821 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152821
Can not end a running command in virt-rescue by press ^C or other keys, the only way is to exit virt-rescue
1152825 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152825
virt-rescue --selinux can not work well, when enable selinux in the command line the value of 'getenforce' is still Disabled in virt-rescue appliance
1152832 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152832
virt-rescue --selinux can not work well, when enable selinux in the command line the value of 'getenforce' is still Disabled in virt-rescue appliance
1152833 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152833
llz give ? for the security context of a file, which is wrong
1153702 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1153702
Using "upload - XXX" doesn't work with remote
@@ -315,28 +231,7 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
RFE: Allow uploading virt-v2v converted images directly to Cinder
1161019 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1161019
Remove rhev related files(rhsrvany.exe and rhev-apt.exe) after conversion of windows guests to kvm
1164729 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164729
set-label can only set <=63 bytes for btrfs and ntfs filesystem which not meet the help message. Also for btrfs and ntfs it should give a warning message when the length exceed the limited length
1164765 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164765
[RFE] Add a new API to scan devices for a btrfs filesystem
1164766 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164766
[RFE] Add a new API to scan devices for a btrfs filesystem
1166057 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1166057
btrfs filesystem will not work well if you create the filesystem with multiple disks at the same time, such as: mkfs-btrfs "/dev/sda1 /dev/sdb1"
1167916 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1167916
P2V error:unexpected end of file waiting for password prompt.
1168626 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1168626
Tracker: Build RHEL 7.2 - based virt-p2v ISO
1168865 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1168865
warning: /files/boot/grub/device.map/hd0 references unknown device "vda" when convert a local guest with virtio disk to openstack server (-i disk))
RFE: Only install RHEV-APT if virt-v2v -o rhev/-o vdsm option is used
1170771 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1170771
virt-sparsify fails to sparsify CentOS 7 SWAP
@@ -347,162 +242,48 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
1172425 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1172425
[RFE]virt-v2v failed to convert VMware ESX VM with snapshot
1172659 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1172659
virt-resize --expand fails on ubuntu-14.04.img image (regression)
1173462 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1173462
virt-resize on RHEL 5.7 fails to resize the raw format disk image
1173695 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1173695
RFE: allow passing in a pre-opened libvirt connection from python
1174551 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1174551
"lstatnslist" and "lstatlist" don't give an error if the API is used wrongly
1176526 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1176526
virt-resize cannot shrink ext filesystem by a small amount
1176801 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1176801
File /etc/sysconfig/kernel isn't updated when convert XenPV guest with regular kernel installed
1177460 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1177460
virt-customize does not work with fedora atomic
1179576 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1179576
virt-v2v prints warning: /files/etc/fstab/8/spec references unknown device "hdb"
1180170 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1180170
virt-builder gives GPG warning message with gnupg2
1180769 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1180769
Security context on image file gets reset
1183493 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1183493
RFE: virt-rescue: add a flag to autorun the suggested mount commands and chroot, suggested with --suggest
1186935 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1186935
libguestfs cannot inspect recent Fedora / RHEL >= 7 when /usr is a separate partition
1190666 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1190666
virt-builder should get RHEL images from RHN
1193238 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1193238
Specifying a --source option doesn't overwrite the built-in templates
1195278 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1195278
libguestfs-test-tool fails in nested environment
1195881 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1195881
virt-builder can't docker pull
1197786 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1197786
virt-builder fails --get-kernel on F21 PPC64 iso
1203595 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1203595
No error messages output if append '--format qcow2' after '-a guest.img', guest.img is a raw format image file
1203816 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1203816
virt-sysprep should error if a command line flag is not used because the operation is disabled
1203898 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1203898
Support inspecting docker images without /etc/fstab
1204063 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1204063
virt-resize should warn if it cannot resize filesystems because optional support is missing
1204131 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1204131
1272119 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1272119
virt-v2v on W2K3 32-bit image fails
1277744 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1277744
When VirtualBox is running, libguestfs fails with ioctl(KVM_CREATE_VM) failed: 16 Device or resource busy
1284191 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1284191
virt-builder cgroup error when running docker
1291339 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1291339
[RFE] Add support for layered/composable images in virt-builder repos
1293271 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1293271
virt-sysprep --delete fails to delete a symbolic link to a dir
1293516 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1293516
rm-rf can not delete files and dirs in symbolic link dir
1308535 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1308535
RFE: v2v: Copy bios.hddOrder to target
1311890 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1311890
virt-v2v: error: libguestfs error: hivex_open: hivex failed to open /Windows/System32/config/SOFTWARE: Operation not supported
1314739 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1314739
The X server of Xen rhel5.11 can't start successfully after converted by virt-p2v
1317180 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1317180
virt-sysprep fails with immutable files
1318922 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1318922
network name mangling after virt-v2v conversion causes old-style network scripts to fail
1322512 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1322512
Libguestfs error: filesize: /var/lib/rpm/Name: No such file or directory
1322837 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1322837
Setting LIBGUESTFS_HV causes security context to be reset
1336627 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1336627
Convert rhel6.8 x86 iscsi-qla4xxx host by virt-p2v failed with control conversion closed by remote
1337052 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1337052
Can't see iscsi-be2iscsi storage on the p2v client
1337865 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1337865
Part-init and part-add failed to create partition with mac parttype on aarch64
1341619 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1341619
Sendkey"ctrl+alt+detele" is not useful at p2v client GUI mode
1346818 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1346818
RFE: virt-get-kernel should be able to extract the kernel command line too
1347830 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1347830
virt-edit on domains over remote transport attempts to edit a local path
1350762 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1350762
Xterm's scroll bar can't work in p2v client if connect machine via virtual console
1351000 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1351000
virt-customize --commands-from-file will fail if there is an extra blank in the file
1354507 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1354507
virt-v2v conversions from vCenter do not consistently obey the proxy environment variables
1354509 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1354509
v2v: Map Windows 8.1 client to Windows 8.1 in oVirt
1356550 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1356550
QXL driver can't be installed automatically when convert win7 or win2008r2 from kvm to rhev by virt-v2v
1357427 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1357427
virtio driver can't be installed for netwrok automatically in win2012 guest sometimes when convert the guest using virt-v2v by auto
1359086 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1359086
Rebase libguestfs in RHEL 7.4
1362497 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1362497
[RFE] Add support for importing Debian/Ubuntu guests with virt-v2v
1363764 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1363764
Permission problems on disk images result in confusing behavior
1363796 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1363796
[RFE] virt-v2v: add ability to identify source templates and indicate it in output to RHEV
1364058 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1364058
[RFE] virt-v2v should provide an option to store the traces to log file
1364518 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1364518
libguestfs-java: mismatch class version vs java dependency
1365473 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1365473
Can't convert rhel6.8 x86-lpfc host by virt-p2v
1368599 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1368599
pip install fails with ImportError: /usr/lib64/python2.7/site-packages/libguestfsmod.so: undefined symbol: put_isoinfo
1369010 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1369010
v2v UEFI conversions fail if the source has a bus='ide' CD-ROM
1370816 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1370816
virt-builder rhel-6.5 --selinux-relabel fails
1371379 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1371379
Fail to inspect ppc64le ISO file
1372037 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1372037
txz-in hangs on btrfs filesystem
1372671 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1372671
Graphics type of guest become sdl after converted by virt-v2v -i disk
1373879 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1373879
virt-resize --expand fails with win2008 x86 image on aarch64 uefi host
1374232 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1374232
selinux relabel fails on RHEL 6.2 guests with "libguestfs error: selinux_relabel: : Success"
1374254 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1374254
Failed to boot up Win7sp1 for first time which installed on fc hardware after converting to rhv4.0
1374629 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1374629
[virt-p2v]virt-p2v recognize LCDRIVE & virtual floppy as hard disks
1379579 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1379579
guestfish mistakely parse nbd exportname
1382275 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1382275
virt-v2v is pulling in the X11 stack
1383048 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1383048
Device.map which is /dev/sdb or /dev/mapper/mpatha can't be updated to vda after p2v conversion
1383665 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1383665
mount: unknown filesystem type 'xfs' error while converting centos VMware VM to glance image.
1387213 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1387213
[RFE] Add support for importing Debian/Ubuntu guests with virt-v2v
1393747 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1393747
RFE: Libguestfs should support LUKS partition in guest image
1398191 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1398191
RFE: virt-v2v should support converting vm with LUKS disk encryption format
1400332 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1400332
RFE: virt-builder should support templates with encrypted filesystems
1405273 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1405273
RFE: virt-builder should support a URL as the output diskimage
1406906 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1406906
Segmentation fault when reading corrupted path with Python 3 bindings
1409024 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1409024
[Debian] Missing db_dump abort inspection
1409637 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1409637
0x0000007B BSOD after converting Windows 2008 R2 using p2v
1410102 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1410102
Make install error for libguestfs-1.35.19
1412090 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1412090
Virt-v2v will use the latest kernel instead of the default kernel.
1413489 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1413489
Inspection doesn't find any OS in cirros-d161201-x86_64-disk.img
1414505 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1414505
virt-p2v needs RHEL 5 build to support LSI Logic / Symbios Logic MegaRAID SAS 8208ELP/8208ELP
1415935 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1415935
virt-v2v auto running convert esx guests failed with error "qemu-img: error while reading sector 24942336: Protocol error"
1415955 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1415955
Ubuntu 14 doesn't generate ssh host keys during first boot
1417306 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1417306
QEMU image file locking (libguestfs)
1417549 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1417549
/usr/bin/x86_64-linux-gnu-ld.bfd.real: ../common/progress/.libs/libprogress.a(libprogress_la-progress.o): undefined reference to symbol 'UP@@NCURSES_TINFO_5.0.19991023'
1420103 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1420103
Remove files in package libguestfs-bash-completion, these files are bash completion files, some of the virt tool completion are already implement in another file, so can remove its completion file
set-label can only set <=127 bytes for btrfs and <=126 bytes for ntfs filesystem which not meet the help message. Also for ntfs it should give a warning message when the length >128 bytes
error(f_"cannot find os-version '%s' with architecture '%s'.\nUse --list to list available guest types.")
argarchin
letitem=selected_cli_itemcmdlineindexin
letarg=fstitemin
letentry=snditemin
letsigchecker=entry.Index.sigcheckerin
@@ -264,12 +288,21 @@ let main () =
(* --- If we get here, we want to create a guest. --- *)
(* Warn if the user might be writing to a partition on a USB key. *)
(matchcmdline.outputwith
|Somedevicewhenis_partitiondevice->
ifcmdline.warn_if_partitionthen
warning(f_"output device (%s) is a partition. If you are writing to a USB key or external drive then you probably need to write to the whole device, not to a partition. If this warning is wrong then you can disable it with --no-warn-if-partition")
device;
|Some_|None->()
);
(* Download the template, or it may be in the cache. *)
error(f_"%s checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!")
error(f_"no guest operating systems or multiboot OS found in this disk image\nThis is a failure of the source repository. Use -v for more information.")
in
Customize_run.rungrootops;
Customize_run.rungrootcmdline.customize_ops;
(* Collect some stats about the final output file.
* Notes:
@@ -715,7 +731,7 @@ let main () =
* and therefore bypasses the host cache). In general you should not
* use cache=none.
*)
ifsyncthen
ifcmdline.syncthen
Fsync.fileoutput_filename;
(* Now that we've finished the build, don't delete the output file on
error(f_"GPG failure: could not verify digital signature of file\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!");
This Fedora image contains only unmodified @Core group packages.
It is thus very minimal. The kickstart and install script can be
found in the libguestfs source tree:
builder/website/fedora.sh
Fedora and the Infinity design logo are trademarks of Red Hat, Inc.
Source and further information is available from http://fedoraproject.org/
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
