Make the location of the Windows virtio drivers overridable with the
environment variable VIRTIO_WIN_DIR, in the same vein as is done for
virt-tools.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
Add p_mbr_p_type as member of type partition
to describe mbr partition type.
Currently we use:
List.filter (fun p -> parttype <> MBR || p.G.part_num <= 4_l)
to filter out logical partitions.
Commit 0c396a4bce
introduce API part_get_mbr_part_type,
we could use this to know the part_type.
Furthermore, we could also use p_mbr_p_type for resizing
logical partitions.
Signed-off-by: Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
Rebuilding the ruby documentation takes 51 seconds on my laptop, and
that's a significant fraction of the entire build. Make sure we only
rebuild the documentation when necessary.
RHEL 5 was unusual in having separate sub-repositories (Server, VT,
Cluster etc) within the main RHEL-5-Server directory. Thankfully no
other version of RHEL does this. Previously we set up the repository
baseurl incorrectly so it didn't include .../Server in the path.
In commit 1a1cb1ec3c, I tried to fix the
RHEL 5 repo location to include .../Server. However that broke the
virt-install --location parameter.
Use a separate baseurl path, which should fix both problems.
This fixes commit 1a1cb1ec3c and
commit 3bc9ba6c84.
The error you would have seen before was:
File "v2v_test_harness.ml", line 1:
Error: The files ../../ocaml/guestfs.cmi and v2v_test_harness.cmi
make inconsistent assumptions over interface Guestfs
btrfs-prog commit:
58a39524619f38d193b8adc3d57888ec07b612aa
change the default output to binary prefix,
and introduced a new option '--raw'
to keep the traditional behaviour.
This patch will add a check function to determine
whether to add '--raw' option to 'btrfs show qgroup'.
Signed-off-by: Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
Properly skip empty lines before the first section, otherwise they will
trigger a syntax error in parsing.
Add a simple test for it, so it doesn't regress.
Instead of save every time there's a new element in the cache, batch the
saving to disk every 100 changes, saving the unsaved remainder at the
exit.
While not a big optimization, this reduces a bit the disk usage during
generator run.
The --lcontext option was removed from coreutils ls (in 2007, cough).
Fedora carried a patch re-adding these options [I believe
unintentionally], but that patch was dropped in Fedora 23, so the
--lcontext option is gone for good.
This reverts commit f9c5b4aa2a.
Since wallclock time differs between boots, we cannot just compare two
screenshots by checking if they are identical -- if the screenshot
contains any time information, then that will be different, and it
turns out that VMs print the current time at boot.
Therefore parse out and use the "similarity" metric printed by the
ImageMagick "compare" command when it compares the screenshots, and
allow a small epsilon for the case where a few oixels are different.
block.header_size is uint32_t so cannot be compared directly with n (ssize_t).
pxzcat-c.c: In function 'worker_thread':
pxzcat-c.c:602:21: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
if (n >= 0 && n != block.header_size-1) {
^
This API converts the C guestfs_h pointer to a 64 bit integer,
which should always be possible on reasonable architectures.
Therefore silence a GCC warning (which only occurs on 32 bit ARM).
Fix the 'make check-valgrind' rule in the ocaml subdirectory so it
doesn't run '$VG run-bindtests', effectively running valgrind on bash.
Bash in Fedora Rawhide has added a new memory leak, resulting in a
failure, but we don't want to test bash anyway.
Instead of doing all firstboot actions in a single script, take the
advantage of the firstboot infrastructure and store and run unrelated
actions as individual steps.
This facilitates troubleshooting and fault recovery; besides it makes
adding more actions easier.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
The firstboot infrastructure used to give the firstboot scripts some
cryptic names which were impossible to relate to the actions they were
supposed to take.
This patch reworks the scheme such that the caller registering a
firstboot script has to provide a descriptive name for the action. That
name, with non-alphanumeric characters replaced with dashes, prefixed by
the serial number, is then used as the name of the script, e.g.
0004-install-gcc
or
0002-msiexec--i-foo-msi.bat
OTOH the numbering becomes internal to the API, i.e. the scripts are
numbered and executed in the order they are registered.
This greatly facilitates debugging and troubleshooting in case when
there are multiple firstboot scripts.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
This patch is an attempt to enhance the firstboot driver script for
Windows, and make it somewhat closer in functionality to what is done
for Linux guests.
Specifically, for every firstboot script it now will log its exit
status, and, if the script reported success, move it to -done directory.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
This patch moves line ending conversion for windows scripts into a
separate helper function.
This simplifies code a bit, and fixes the problem that actual firstboot
scripts used to remain with unix-style line endings.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
Use commandrvf() instead of commandvf() to execute e2fsck. A non-zero
exit status does not always indicate a failure.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
When the user chooses to not verify the signatures on the indexes
(using --no-check-signature), there is no point in requiring as many
--fingerprint as --source (or even just one), as they will not be used
anyway.
In this case just ignore all the values of the specified --fingerprint
arguments.
Process the sources specified via the --source command line arguments
before sources read from repositories, so the former have a chance to
"override" the latter.
Related to RHBZ#1193238.
Ensure the arch string is sane before using it in the following XPath
expression. Since the arch string can be derived from untrusted guest
data [see src/filearch.c], this prevents a possible XPath injection
vulnerability.
We also need to be more careful about PyString_FromString and similar
functions returning NULL on failure. Currently we don't check this
every time. This commit adds more checks, but is still not complete.
libguestfs has used double and triple underscores in identifiers.
These aren't valid for global names in C++.
The second step is to replace all guestfs__* (2 underscores) with
guestfs_impl_*.
These functions are used where a libguestfs API call is implemented on
the library side. The generator creates a wrapper function which
calls guestfs_impl_* to do the work. (Libguestfs APIs which are
passed directly by the daemon work differently and don't require a
guestfs_impl_* function).
This is an entirely mechanical change done using:
git ls-files | xargs perl -pi.bak -e 's/guestfs___/guestfs_impl_/g'
Reference: http://stackoverflow.com/a/228797
libguestfs has used double and triple underscores in identifiers.
These aren't valid for global names in C++.
The first step is to replace all guestfs___* (3 underscores) with
guestfs_int_*. We've used guestfs_int_* elsewhere already as a prefix
for internal identifiers.
This is an entirely mechanical change done using:
git ls-files | xargs perl -pi.bak -e 's/guestfs___/guestfs_int_/g'
Reference: http://stackoverflow.com/a/228797
* fusermount is Linux-only; on BSD and OS X umount should be used
* fuser has no -v flag on BSD/OSX, and -c is the
POSIX-compatible equivalent of -m
* Does not solve the lack of pretty output of fuser -v, but does make it
work on Mac OS X.
Add a bash function 'prepend' for intelligently prepending elements to
paths. eg:
prepend PYTHONPATH "/foo"
would set PYTHONPATH to "/foo" or "/foo:<previous-contents-of-PYTHONPATH>"
Tested by:
(1) Building and testing libguestfs twice: first without libguestfs
installed as a system library, and then with it installed.
(2) Examining the output of './run printenv' by hand and comparing
environment variables to the expected values.
As advised by Daniel P. Berrange, the third parameter can be passed on all
platforms rather than specifically Mac.
Quoting a libvirt commit rationale after Daniel:
commit 9fa3a8ab6fd82ad2f5a14b490696085061418718
Author: Doug Goldstein <cardoe@cardoe.com>
Date: Wed Oct 30 11:22:58 2013 -0500
MacOS: Handle changes to xdrproc_t definition
With Mac OS X 10.9, xdrproc_t is no longer defined as:
typedef bool_t (*xdrproc_t)(XDR *, ...);
but instead as:
typedef bool_t (*xdrproc_t)(XDR *, void *, unsigned int);
For reference, Linux systems typically define it as:
typedef bool_t (*xdrproc_t)(XDR *, void *, ...);
The rationale explained in the header is that using a vararg is
incorrect and has a potential to change the ABI slightly do to compiler
optimizations taken and the undefined behavior. They decided
to specify the exact number of parameters and for compatibility with old
code decided to make the signature require 3 arguments. The third
argument is ignored for cases that its not used and its recommended to
supply a 0.
Generate a simple bindtests test for the PHP binding, so it is possible
to easily test all the argument types.
Unlike the bindtests for other languages, optional arguments are not
tested, due to the limitations of optional arguments in PHP (or maybe
they way we implement them).
Make sure to copy the strings we add to the char** array, otherwise they
are stale pointers which we'll try to free later.
Also, properly destruct the temporary zval.
OptString maps to a "s!" argument, which makes zend_parse_parameters not
touch the variables (char* and length) when NULL is passed as parameter.
Hence, set both to NULL/0, and check for non-NULL char* variable before
checking its length.
- use <p>..</p> for text paragraphs, instead of just using <p> to
separate them
- slightly improve the metadata in eventToString and set_event_callback
- fix the textual @see in set_event_callback, so it is accepted also in
JDK 8
- escape the doc text, so &, <, and > will not be considered as HTML
tags but actual text
- use the @deprecated tag instead of adding the customary deprecation
note to the doc text
The gnulib 'error' module uses 'program_name'. On some platforms --
but not Linux / glibc -- it references it as:
extern char *program_name;
This means when you compile libguestfs on non-glibc (eg. Mac OS X)
gnulib requires 'program_name' as an external string reference, which
we don't provide.
This change doesn't define this string reference for gnulib, but it
does change the name of the macro we use to avoid conflicts if we
eventually need to export 'program_name' as a string.
Thanks: Margaret Lewicka
When copying disks that use EFI, we created a new partition table,
randomizing the GPT GUID of the first partition. Since EFI may store
the GUID in its NVRAM variables, this could make the guest unbootable.
In GPT, each partition has a GUID assigned randomly. Allow this GUID
to be written and read.
Note this is different from the GUID type code which is used to
identify the type of the partition.
Commit 9135129b0f changed
two stack buffers to pointers:
- uint8_t buf[BUFFER_SIZE];
- unsigned char outbuf[BUFFER_SIZE];
+ CLEANUP_FREE uint8_t *buf = NULL;
+ CLEANUP_FREE uint8_t *outbuf = NULL;
but we were still using sizeof buf to calculate the size of the
buffer. sizeof buf == 8 so the original code which used large buffers
for reading/writing the file changed to using 8 byte buffers.
Currently implemented as guestfish commands, provide them instead as
single source -> destination functions for the library, so they can be
used also in other places.
These functions are not added to guestfish, since guestfish has its own
implementation (which will soon switch to call copy-in and copy-out for
multiple paths).
test-dlopen needs to be added only when libdl is available, and it is
like that already (within if HAVE_LIBDL); hence don't unconditionally
add it as well.
Check whether --part-type is supported (in the rewritten sfdisk in
util-linux >= 2.26), and use it instead of --print-id & --change-id.
The actual result should be the same, just not using a (recently)
deprecated command line API.
Unlike -o rhev which have only one data domin, -o vdsm can and usually
does have multiple data domain.
The path to vdsm is pre mounted so no need to assume nfs path with -os
Example:
-o vdsm -os /rhev/data-center/<data-center-uuid>/<data-domain-uuid>
Bug-Url: https://bugzilla.redhat.com/1176591
Signed-off-by: Shahar Havivi <shaharh@redhat.com>
Kernel 3.19 has problems uncompressing the RAM disk with <= 500 MB.
(This is likely to be a kernel bug)
64 KB pages are common on aarch64, so treat this case the same as ppc,
and use a larger default appliance memory size.
Thanks: Laszlo Ersek for help and reproducing the bug.
Since libguestfs >= 1.26 / supermin 5, the appliance has lived in a
subdirectory of $TMPDIR/.guestfs-$UID. As a result using 'ls -l'
didn't display any information about the files in the appliance.
Using '-R' causes this information to be displayed again.
ls -Z stops ls from displaying the size field. --lcontext uses longer
lines but has the advantage of still displaying the size field as well
as the SELinux context.
It is documented as 'p2v', but in fact this was not implemented
so there is no way to log in as root on the console.
On RHEL we also have to add the 'passwd' package, else you get:
Error creating Live CD : Unable to set unencrypted password due to lack of /usr/bin/passwd
Covert common_utils_tests to use oUnit as testing framework, replacing
the hand-made assert in favour of structured unit tests and better
error reporting.
common_utils_tests is now built only when the oUnit module has been
found.
Pass to --commands-from-file the name of a file containing customization
commands in each line, as if they were specified as command line
arguments.
This eases the reuse of commands among different
builder/customize/sysprep invocations.
When libguestfs is running using TCG on aarch64, we need to pass the
-cpu cortex-a57 parameter to qemu. Libvirt doesn't let us do this,
complaining "Unable to find CPU definition".
As a temporary workaround only, use <qemu:commandline> to pass this
argument directly to qemu. When libvirt is fixed we can remove this
hack.
This is a workaround for libvirt bug RHBZ#1184411.
See:
https://www.redhat.com/archives/libvirt-users/2014-August/msg00043.htmlhttps://bugzilla.redhat.com/show_bug.cgi?id=1184411
AAVMF is an open source UEFI implementation for aarch64 based on OVMF.
As aarch64 is heading for requiring UEFI even inside guests, if the
AAVMF firmware is installed on the host, use it as a hint that we
should boot the guest using AAVMF instead of the default "empty
machine".
Note this requires very recent AAVMF, libvirt, qemu. However that's
OK since it's only applicable to aarch64. On non-aarch64, this patch
does nothing.
Thanks: Laszlo Ersek for a lot of help getting this right.
If "strdup (d->d_name)" fails with "i" > 0, then both "p" and
"ret->guestfs_int_dirent_list_val" are non-null pointers, but the latter
is no more valid (since "p" is the new realloc'ed buffer). Hence, trying
to free both will access to invalid memory.
Make sure to free only one of them, "p" if not null or
"ret->guestfs_int_dirent_list_val" otherwise.
wait_qemu_nbd would fail in Rawhide where /etc/hosts contains:
127.0.0.1 localhost [...]
::1 localhost [...]
This peculiar mapping caused qemu-nbd to always select an IPv6
address. However since we used the obsolete inet_pton function and
forced AF_INET, we were always trying to connect to the IPv4
localhost.
As this was never going to work anyway, fix the code to use
getaddrinfo to resolve "localhost", and use a loop to connect to all
possible addresses.
Also: Add a short sleep in the reconnection loop so that we don't
hammer the qemu-nbd server while it is starting up.
This test launches a dummy guest using IDE. Aarch64 doesn't support
IDE, and because it also doesn't support PCI (virtio-mmio instead)
it's hard to formulate libvirt XML that will work on both x86 and
virtio-mmio.
Since this bug is not architecture specific (RHBZ#912499), skip it on
non-x86. The test will print:
skipping test: arch is not x86 and does not support IDE
Since systemd 209, all the functionalities of the former libsystemd-*
(including libsystemd-journal) have been merged into a single
libsystemd, with the former libraries left as compatibility ones.
Thus, first look for libsystemd, and if not found try again with the
libsystemd-journal as used before.
With commit a594b7f90a the checks for
input and output files with ':' were removed; while the input file is
handled correctly by the disk_create API, an output filename with ':'
could still be handled as specifying a transport, failing the last
"qemu-img convert" call.
Use the new qemu_input_filename helper to handle the output filename
correctly.
It mimics a bit in OCaml the logic used in
commit a95214b198 (for the main library)
and commit 588af1953e (in disk-create),
so in OCaml tools direct calls to qemu/qemu-img with filenames provided
by the user can properly handle filenames with e.g. ':'.
Since mkfifo, mknod_b, and mknod_c add the correct file type to the
modes of the resulting file, make sure the specified mode contains only
permissions bits.
blkid(1) (or actually, libblkid) seems to handle filesystem labels up
to 127 characters. Considering that btrfs labels can be up to 128
characters, this means long labels are not read correctly (i.e. get
truncated) by blkid. Furthermore, ntfs labels are actually unicode,
and libblkid seems to not decode them correctly.
Hence, if ntfsprogs is available invoke `ntfslabel` to get the label
of ntfs filesystems.
Related to RHBZ#1164708.
blkid(1) (or actually, libblkid) seems to handle filesystem labels up
to 127 characters. Considering that btrfs labels can be up to 255
characters, this means long labels are not read correctly (i.e. get
truncated) by blkid.
Get the filesystem type, and if btrfs is available invoke
`btrfs filesystem` to get the label of btrfs filesystems.
Related to RHBZ#1164708.
Add the 'label' optional argument to the mkfs action, so it is possible
to set a filesystem label direct when creating it. There may be
filesystems not supporting changing the label of existing filesystems
but only setting it at creation time, so this new optarg will help.
Implement it for the most common filesystems (ext*, fat, ntfs, btrfs,
xfs), giving an error for all the others, just like set_label does.
Obviously no one reads the FAQ, so make the error message
even clearer.
The new error messages are variation on this:
libguestfs: error: /bin/false exited with error status 1.
To see full error messages you may need to enable debugging.
Do:
export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again. For further information, read:
http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.
The test assumed (for no particular reason) that it could always run a
KVM guest. When run on a machine without KVM, or in a VM without
nested KVM, you get this error:
libvirt: Domain Config error : internal error: no supported architecture for os type 'hvm'
Traceback (most recent call last):
File "./t/820-rhbz912499.py", line 77, in <module>
dom = conn.createXML (xml, libvirt.VIR_DOMAIN_START_AUTODESTROY)
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3523, in createXML
if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
libvirt.libvirtError: internal error: no supported architecture for os type 'hvm'
FAIL: ./t/820-rhbz912499.py
Use a qemu guest instead, which should have a better chance of working
on all hosts.
Builds of bison-generated files currently fail with:
index-parse.y: In function 'yyparse':
index-parse.c:1256:6: error: assuming pointer wraparound does not occur when comparing P +- C1 with P +- C2 [-Werror=strict-overflow]
if (yyss + yystacksize - 1 <= yyssp)
^
This error only occurred on one machine, and it was unclear why. It
turned out to be because the other build machines I use had:
CFLAGS="-Wall" ./configure ...
whereas on the failing machine I was just using ./configure w/o CFLAGS.
This caused the compile flags of index-parse.c to be changed from:
gcc -Wstrict-overflow ... index-parse.c
to:
gcc -Wstrict-overflow -Wall ... index-parse.c
but since -Wstrict-overflow is short for -Wstrict-overflow=2, and
-Wall is short for -Wstrict-overflow=1, and since the error only
occurs at level >= 2, using CFLAGS="-Wall" was effectively turned off
the warning.
Anyway, I believe the optimizations that GCC does when
-fstrict-overflow is enabled to be dubious, frequently dangerous, and
useless. They are only done because we want C99 to work on
architectures that don't exist in reality (and which even if they did
exist, libguestfs would probably not support).
The Linux kernel uses -fno-strict-overflow, and we should too, for the
same reasons.
Normally this requires the program to have at least one C source file.
virt-sysprep doesn't have this, but it uses C sources from other
directories, and therefore we can make it work.
This returns the C pointer to the guestfs_h handle (ie. guestfs_h *).
The purpose of this API is to interwork with other non-C libraries.
See RHBZ#1075164 for the rationale when applied to another library.
This allows the Perl binding of guestfs_add_libvirt_dom to work.
This relies on the internal Perl representation of virDomainPtr, but
Dan assures me this isn't going to change.
Commit 96158d42f5 enabled the previously
private guestfs_add_libvirt_dom API. It also tried to enable the
existing test for this API, but failed to do that correctly. Also the
test was broken. Fix all of this.
This fixes commit 96158d42f5.
YACC index-parse.c
index-parse.y:97.34-41: warning: type <fields> is used, but is not associated to any symbol [-Wother]
%destructor { field_free ($$); } <fields>
^^^^^^^^
index-parse.y:95.36-45: warning: type <sections> is used, but is not associated to any symbol [-Wother]
%destructor { section_free ($$); } <sections>
^^^^^^^^^^
We had code (added for RHBZ#912499) which labels overlay files
correctly so that libvirt can read them. Unfortunately this code was
broken by subsequent commits: the new backend setting for the
imagelabel is only copied to the 'data' struct during launch, but the
create_cow_overlay callback is called before launch (when adding
drives).
The fix is easy: ensure create_cow_overlay_libvirt checks for the
backend setting and initializes the 'data' struct.
This change also means we need to free (data->selinux_imagelabel)
before setting it in launch (and we do the same for
data->selinux_label, although that's not strictly necessary).
For background on this, see:
https://bugzilla.redhat.com/show_bug.cgi?id=912499#c10
This allows the Python binding of guestfs_add_libvirt_dom to work.
There is a regression test to ensure this keeps working.
Note this requires a patched libvirt-python, supporting the
c_pointer() method.
This API already existed (as guestfs___add_libvirt_dom), and was used
by a few tools.
This commit changes it to a public API.
Note that for reasons outlined in the previous commit message, it is
impossible to call this from guestfish or from non-C language
bindings.
This implements Pointer arguments properly, at least for certain
limited definitions of "implements" and "properly".
'Pointer' as an argument type is meant to indicate a pointer passed to
an API. The canonical example is the following proposed API:
int guestfs_add_libvirt_dom (guestfs_h *g, virDomainPtr dom, ...);
where 'dom' is described in the generator as:
Pointer ("virDomainPtr", "dom")
Pointer existed already in the generator, but the implementation was
broken. It is not used by any existing API.
There are two basic difficulties of implementing Pointer:
(1) In language bindings there is no portable way to turn (eg.) a Perl
Sys::Virt 'dom' object into a C virDomainPtr.
(2) We can't rely on <libvirt/libvirt.h> being included (since it's an
optional dependency).
In this commit, we solve (2) by using a 'void *'.
We don't solve (1), really. Instead we have a macro
POINTER_NOT_IMPLEMENTED which is used by currently all the non-C
language bindings. It complains loudly and passes a NULL to the
underlying function. The underlying function detects the NULL and
safely returns an error. It is to be hoped that people will
contribute patches to make each language binding work, although in
some bindings it will always remain impossible to implement.
This didn't work at all because the regular expression did not match
the returned Augeas path.
In future if the regular expression doesn't match, this will give an
internal error instead of continuing with a bogus value.
Thanks: Junqin Zhou for providing the test case and debug information.
Regular EFI disks have /EFI on the first (VFAT) partition, but they
are not installers.
Fix this by only considering something to be an installer if it has a
single partition (or whole disks like ISOs).
Implementing this is quite complex since when checking a filesystem,
we don't have information about whether we are even looking at a
partition, nor about whether it's the first partition out of how many.
The majority of the commit is changing the code to collect that
information.
Because of the size of an extended partition reported by Linux is always 1024
bytes, so it will always fail to copy to an extended partition.
This patch fixes this problem by copying to the offset of an extended
partition in the destination disk.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Wait up to 10 seconds for qemu-nbd to start up and respond to clients.
Otherwise the conversion server may attempt to connect before qemu-nbd
is ready to serve it.
After a very long and trying episode with a Windows guest that refused
to boot after conversion, we managed to successfully boot it by
disabling Windows Group Policy. It appears that Group Policy
prevented the virtio driver from being used.
Document this in the manual.
Prior to this commit the distro for a FreeBSD or a NetBSD system, in
conjuction to what happened for OpenBSD, was shown as 'unknown'.
*BSDs are complete OSes, not a kernel like Linux, but theoritically you
could have FreeBSD as ostype and PC-BSD as distro.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Take into granted that partition 'c' has a special purpose. It has
always the same size as the enclosing slice and is not mapped under
Linux.
This is a best effort try. The mapping will be incorrect if there is a
gap in the disklabel partitions sequence, e.g. 'b' (swap) partition is
missing but 'd' partition is defined.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Make a best effort try to map NetBSD disklabel partitions to Linux
partitions. The mapping will be incorrect if there is a gap in the
disklabel partitions sequence, e.g. 'b' (swap) partition is missing but
'e' partition is defined.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
This patch lets guestfish show command synopsis if the syntax of command issued
by user is wrong, rather than telling user that the number of parameters is wrong.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
The assumption that Linux will map the MBR partition to /dev/sda1
and the BSD 'a' partition to /dev/sda5 is not always correct.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Because get-uuid looks more like a counterpart to set-uuid, and is
more likely come to mind if one knows set-uuid.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Just like --long-options, it makes it possible to know which short
options are supported by each tool; this can help improving the bash
completion, for example.
As contrast to shortdesc which is forced to begin with lowercase, this
patch forces longdesc to begin with uppercase.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Commit 588af1953e started with a very
conservative list of permitted backing formats (just "raw" or
"qcow2"). We can allow almost any format permitted by qemu, but this
commit just adds "vmdk" to this whitelist.
Include the version of virt-p2v and its MD5 checksum in the source
kickstart. This is useful for working out which version of virt-p2v
the kickstart corresponds to.
This relies on the remote to keep sending us data. If it hangs, then
the cancel button won't work. This could also be fixed by introducing
a timeout to the read syscall.
I was fetching cancel_button from the wrong dialog (conv_dlg, not
run_dlg), and therefore getting the wrong button.
This explains why the back button on the conversion dialog was always
insensitive. It's because cancel_button was assigned to this button,
and then I was setting cancel_button to insensitive.
It did nothing and was left over from an earlier attempt to add
fullscreen mode. That is not needed any longer since we now run
virt-p2v under the matchbox window manager.
The <target dev> attribute in libvirt isn't very informative. What we
really want to know is whether the source disk used IDE or SCSI, as
that allows us to remap block devices accurately during conversion.
For example, if the source was IDE and RHEL 5, and after conversion
virtio will be supported, then we know that we need to remap
"/dev/hda" to "/dev/vda".
Drop the s_target_dev and s_removable_target_dev fields and replace
them with s_controller and s_removable_controller.
For -i libvirt, use the <target bus> attribute to get this information.
For -i ova, use the OVF disk controller's ResourceType.
(http://blogs.vmware.com/vapp/2009/11/virtual-hardware-in-ovf-part-1.html)
If /etc/favicon.png is a symbolic link, follow it.
Unfortunately RHEL 7 and Fedora have crappy 16x16 /etc/favicon.png
symlinks in the base distro. It would be nice to ignore this symlink,
but it's almost impossible to determine if the symlink is part of the
base distro or was added by the user. (This is a bug in those
distros.) virt-inspector and virt-mananger both ignore favicons.
Commit fd70cdc3c5 passes the -annot flag
to the compiler, generating *.annot files (containing type information
used by emacs and IDEs). Remove these files when doing 'make clean'.
Use AS_CASE to properly map from the host cpu to the qemu cpu suffix;
this avoids doing multiple string replacements using sed, and it is
possible to use case-like syntax.
Hardcoding -ltinfo breaks on distros that do not build the sep library
(which is the default ncurses behavior). Use pkg-config to look up the
right libraries regardless of how the distro built things.
The labels 'hw_disk_bus' and 'hw_vif_model' were swapped. This
happened to work for virtio guests. For non-virtio guests you ended
up with absurdities like requiring an rtl8139 driver for the disk.
Thanks: Junquin Zhou
Previously we used to try to find the 'vmlinuz' file by running 'rpm
-ql kernel' and looking for any file called 'vmlinuz-*'.
If there were multiple 'kernel' packages installed, the rpm command
would list files from all of them, resulting in a random 'vmlinuz-*'
being chosen, not necessarily the one corresponding to the kernel
package we were looking at.
Use 'rpm -ql kernel-<VERSION>-<RELEASE>' instead so that we only look
for files in the right kernel package.
Thanks: James Mighion
In tools except guestfish, we don't use ncurses, or even curses, just
2 termcap functions and a global variable. These are provided by
libtinfo, so just link to that.
In guestfish we use readline which needs ncurses. Leave that alone.
md5sum(1) does not exist everywhere, so wrap it in an own function so
the right implementation can be chosen on each OS. Also, wrapping it
avoid using awk everytime.
Query RbConfig::CONFIG for "libdir", and use that when trying to link to
the ruby library. This fixes the libruby detection when it is installed
in a non-standard library directory.
Create an autoconf macro to deal with the possible combinations of names
and versions of the libdb tools. This also eases the addition of a libdb
version, or a new pattern for the names of tools.
There is no functional change.
Use a %top block for including <config.h>, so it is included prior to
any other include, even for system ones. This makes sure its definitions
are properly used in gnulib replacement headers.
Do a configure check to see whether UNIX_PATH_MAX is defined; if it is
not defined, provide it with the size of sun_path in the struct
sockaddr_un, thus with the proper size for any libc.
Use CLEANFILES to remove the generated symlinks, and just ignore the
errors when copying the scripts in srcdir!=builddir situations.
Followup of commit 40ac54829d.
If a script does not finish, hangs, etc, it would be executed again at
the next boot, since the injected firstboot.sh helper removes it only
after it finished.
Before executing a script, move it to another internal subdirectory
(scripts-done) and execute it from there, so it is not run again by
firstboot.sh. The downside is that now scripts are executed only once
at all, so if a script fails it will not be attempted at the next boot.
Also, remove all the files found in scripts-done, as they have been run
(or at least attempted) in a previous boot.
This fixes RHBZ#1159651.
This adds a customize option:
virt-customize --ssh-inject USER
virt-customize --ssh-inject USER:string:KEY_STRING
virt-customize --ssh-inject USER:file:FILENAME
(ditto for virt-builder and virt-sysprep)
In each case this injects into the guest user USER
a) the current (host) user's ssh pubkey
b) the key specified as KEY_STRING
c) the key in FILENAME
adding it to ~USER/.ssh/authorized_keys in the guest.
For example:
virt-builder fedora-20 --ssh-inject root
will add the local user's ssh pubkey into the root account of the
newly created guest. Or:
virt-customize -a disk.img \
--ssh-inject 'mary:string:ssh-rsa AAAA.... mary@localhost'
adds the given ssh pubkey to mary's account in the guest.
This doesn't set the SELinux labels correctly on newly created files
and directories, so you have to use --selinux-relabel (probably we
should fix this as part of the general effort to fix SELinux
relabelling). However it should preserve the labels if the
~/.ssh/authorized_keys file already exists.
Most of this work is based on a patch sent to the mailing list by
Richard W.M. Jones <rjones@redhat.com>:
https://www.redhat.com/archives/libguestfs/2014-November/msg00000.html
Since the daemon has long since used the same build system as the rest
of libguestfs, it no longer needs its own gnulib.
This arcane bit of code was left over from
commit e05ddc70f1 (added 2009-08-03 !)
Implement the GUESTFS_ADD_DRIVE_OPTS_SECRET argument of
guestfs_add_drive_opts. For libvirt we have to save the secret in
libvirtd first, get a UUID, and then pass the UUID back through the
domain XML.
On filesystems whose dirent.d_type is DT_UNKNOWN or some unknown value,
manually check whether an entry is a directory, thus completing in the
proper way.
Instead of passing the (uri, key, proxy) tuple around, pass the whole
Sources.source record; this requires creating proper Sources.source also
for uri+fingerprint passed via command line.
No functional changes.
Instead of making up an ordinary curl command line, write a temporary
config file and use 'curl --config tmpfile' to pass the arguments.
The advantage is that it's more secure if we want to supply passwords
to curl, since a '--user username:password' parameter on the command
line could be read (eg. by 'ps ax'), but the temporary file has mode
0600 and cannot be read by other users.
This is mostly code motion, but it also passes the '-q' option to curl
to stop it from reading default configuration files.
Must NOT use:
- One of the interim versions of virt-v2v (1.27.x) that we published
during development.
- Old virt-v2v (0.9.x).
Also use the --version option (instead of -V) since old virt-v2v
required it:
$ virt-v2v -V
Option v is ambiguous (version, vmtype)
Usage:
[...]
$ virt-v2v --version
0.9.1
Include the usb-storage kernel module, to permit booting from USB
keys.
Rerun dracut to rebuild the initramfs. This is complicated because by
default dracut will try to build an initramfs for the running
(ie. appliance) kernel, which is completely useless. We need to find
the latest installed kernel and rebuild the initramfs for that
instead, which is hairy.
This also does a small refactoring of the distro-specific code,
removing the $selinux_relabel variable and replacing it with a generic
$extra_args variable.
I only added this to the virt-builder script, since one assumes that
livecd-creator/whatever should make the kickstart ISO bootable by
installing whatever drivers and modules are necessary.
Running 'dracut -f' installs a dracut for the running kernel (of the
appliance) not the installed kernel. Stupid stupid stupid.
This reverts commit 7a6d44ebfe.
I only added this to the virt-builder script, since one assumes that
livecd-creator/whatever should make the kickstart ISO bootable by
installing whatever drivers and modules are necessary.
Add common error/warning/info functions.
Examine every existing call to printf/eprintf and change where
necessary so that:
- error is used instead of eprintf + exit 1
- warning no longer needs ~prog argument (it is added by Utils module)
- any verbose output should go to stdout, not stderr
- info is used to print general informational messages
Also, don't indent summary of changes.
Add a Customize_utils module. This contains common error/warning/info
functions, and also quote = Filename.quote.
Examine every existing call to printf/eprintf and change where
necessary so that:
- error is used instead of eprintf + exit 1
- warning no longer needs ~prog argument (it is added by Utils module)
- any verbose output should go to stdout, not stderr
- info is used to print general informational messages
Also, don't pass ~prog parameter around. Instead we just get it from
the executable name.
Add a Utils module. This contains common error/warning/info functions,
and also quote = Filename.quote.
Examine every existing call to printf/eprintf and change where
necessary so that:
- error is used instead of eprintf + exit 1
- warning no longer needs ~prog argument (it is added by Utils module)
- any verbose output should go to stdout, not stderr
- info is used to print general informational messages
get_current_dir_name exists on GNU libc only, so not portable.
On the other hand, while POSIX leaves a null buffer argument for getcwd
as unspecified behaviour, basically the most used/important Unix
implementations (GNU libc, FreeBSD's libc, etc) allow such value,
returning a new allocated buffer with the current directory.
In any case, the change just affects two tests, so even if it hits a
libc not implementing this behaviour for getcwd, only tests are
affected.
uri-c.c and perl_edit-c.c end up (directly or indirectly) including
guestfs-internal-frontend.h, which in turns includes libvirt/libvirt.h;
hence, make sure to use the libvirt CFLAGS.
Add a IfNotCrossAppliance prereq for tests, so a test using it can only
be run when the appliance is a copy of the running host system; this can
help marking tests which run stuff (usually built in the host) inside
the appliance.
Since the signature of iconv() changes between implementations (the
constness of the second parameter, in particular), make use of the iconv
module of gnulib to handle these potential differences, including an
external (out of libc) iconv implementation.
Previously I modified the parse_libvirt_xml function to get rid of the
awkward 'map_source*' functions, and have the callers map over and
modify the source disks afterwards. However this broke the case where
an "nbd:..." URL was returned by parse_libvirt_xml, since the callers
might try to map this URL (eg. turning it into an absolute path).
This broke virt-p2v specifically.
This commit changes parse_libvirt_xml to return a list of tuples
containing disk information, giving the callers more information with
which to do the mapping.
This fixes commit 3596165282.
Take the opportunity presented by the refactoring in the previous
commit to inline and simplify functions in these input subclasses.
This finally removes the map_source* functions.
This refactors the code into more logical units, based on the previous
commits. So Xen + the input_libvirt Xen code is grouped and moved
into a new module called Input_libvirt_xen_ssh. Similarly VCenter +
input_libvirt vCenter code is moved to Input_libvirt_vcenter_https.
There is no change here, purely code motion.
Previously we fixed RHBZ#1151033 by increasing the cURL readahead
parameter to a large value. Unfortunately this is too large -- and
hence slow -- for the conversion phase, which broke on slow vCenter
servers (RHBZ#1153589).
What we do now is to perform the conversion phase with the default
readahead (2MB) to ensure stability, since performance of the
conversion phase is not critical. Then before copying we change the
readahead to the larger value (64MB) to ensure efficient copying.
This is called before the copying phase to allow the input method to
adjust overlay/backing parameters.
The default implementation does nothing, so this commit has no effect.
Note that it doesn't matter what the number is, as long as it's
different for each disk.
The unique number lets us store extra data in input objects which is
associated with each disk.
Get rid of the awkward map_source* functions, and replace it with
equivalent code which modifies the source disks objects coming back
from Input_libvirtxml.parse_libvirt_xml.
This is just code refactoring. Apart from the order in which certain
tests are done, this should be equivalent to the previous code.
Previously we had one class ('input_libvirt') which handled all three
cases. Now we have one superclass ('input_libvirt') and three
subclasses of that ('input_libvirt_vcenter_https',
'input_libvirt_xen_ssh', 'input_libvirt_other') which handle the three
cases separately.
This is just code motion, and should be functionally equivalent to
what was here before.
Unfortunately udev removed the udevtimeout setting on the kernel
command line. This means the default timeout was being set back to 30
seconds, which is way too low for remote disks.
However it was re-added (2014-07-29) as udev.event-timeout. Set both
to be sure.
Note this requires systemd >= 216.
In full_path, skip the trailing slash in the base directory when
different than "/", as the slash will eventually be added when building
the resulting path.
Make sure to output the field for checksum even for non-regular files,
as empty, in CSV output mode. This ensures each line has the same number
of fields, regardless of the file type.
OVA is not a particularly well-specified format. The specification
allows a directory to be an OVA, so enable that. The spec doesn't
mention that ZIP can be used in place of tar, but since we have seen
these in the wild, allow that too.
The way to fix these bugs -- for future reference -- is as follows:
(1) Install the corresponnding Windows version with virtio enabled.
See:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/chap-KVM_Para_virtualized_virtio_Drivers.html
(the ".vfd" method)
(2) Download the SYSTEM and SOFTWARE registries from the
virtio-enabled guest.
(3) Examine each registry using 'hivexsh', especially the keys which
we modify during virt-v2v (see v2v/convert_windows.ml). Remember to
use both the 'ls' and 'lsval' commands in hivexsh.
(4) Download the SYSTEM and SOFTWARE registries from the guest that
fails to boot.
(5) Examine the same registry keys from each registry using 'hivexsh'.
(6) Work out what the differences are, paying particular attention to
quoting and the type of each key (eg. str(1), str(2), etc).
Thanks: Matthew Booth.
There were two large bugs before: (1) It only found one manifest file
(meaning that multi-volume OVAs would be broken). (2) The regular
expression didn't work so it wouldn't check the hash. Bugs (1) + (2)
combined so that the check just didn't do anything at all, which is
why we didn't notice the multi-volume problem.
Actually I suspect we shouldn't be doing this check at all, since I
bet there are OVAs out there with broken SHA-1's.
Ignore an environment variable:
- if an environment variable is an empty string, AND
- if an empty string would have no meaning for that environment
variable (eg. LIBGUESTFS_BACKEND can only take on certain values,
and an empty string is not one of them).
Note that LIBGUESTFS_BACKEND_SETTINGS is meaningful as an empty
string, and for this case I have removed the STRNEQ (str, "") test.
Previously:
$ LIBGUESTFS_BACKEND="" guestfish -a /dev/null run
libguestfs: error: invalid backend:
After this commit:
$ LIBGUESTFS_BACKEND="" guestfish -a /dev/null run
(no error message is printed)
Remove the common 'add_resource' function.
Duplicate that function into code for adding disks, and separate code
for adding removables (floppies & CDROMs).
The removables code should look for all removable devices
(ResourceType 14, 15, 16) in a single query, so that the order doesn't
depend on the order in which we do the queries.
If the user does ^C then this leaves temporary overlay files around
(possibly a bug?). Offer an 'exit' choice to the user which exits
cleanly.
The new message looks like this:
Dual- or multi-boot operating system detected. Choose the root filesystem
that contains the main operating system from the list below:
[1] /dev/sda3 (Fedora release 20 (Heisenbug))
[2] /dev/sdb3
Enter a number between 1 and 2, or 'exit': exit
Improve the homebrew JSON writer:
- add more types (including also nested dictionaries and lists)
- format in a compact way (single line), or indented (multilines)
When copying from file to file, don't leave the destination file
around if the copy fails.
However in the same code don't try unlinking the destination device on
failure.
We never updated the no_trim variable in the case where this option
was supplied, so it never did anything, proving once again that unless
you have a test case, the feature won't work.
When translating the original virt-v2v code, I misunderstood how image
directories are created. There is one directory under images/ per
disk image, not one for all the disks in a single guest.
Note this requires that -o vdsm uses multiple --vdsm-image-uuid
options, one per disk image of the guest.
Reported by: Alain Vondra
Thanks: Tingting Zheng
virt-log uses /dev/stdout as output file for guestfs_download, which
will open the output file in truncate mode, truncating the output of
test-virt-log.sh.
Make sure to redirect the output of virt-log so it does not clobber the
test output.
Thanks to Richard for the hint.
Additional information about URIs, which may contain cluster and/or
folder names (thanks: James Mighion).
Replace esx.example.com with vcenter.example.com to reduce confusion
about vCenter vs ESXi.
Since procps 3.3.10, free does not output the "-/+ buffers/cache" line
anymore. On the other hand, the data from it can be calculated from the
memory values, so just do the calculation manually (with awk).
This effectively demotes UML to unsupported status. This has happened
because UML has regressed quite a lot upstream. There are multiple
grave bugs revealed by running the test suite and I don't have the
inclination to fix them.
When using the libvirt backend, don't use the SLIRP. Use
qemu-bridge-helper via libvirt to give us a full network connection.
One consequence of this is that 'ping' works in
'virt-builder --run-command'.
A less useful consequence is that the host firewall will prevent
connections on non-standard ports to the host. So you can't (eg)
connect to a rsync daemon on the host listening on an arbitrary port,
which worked before.
The default bridge is 'virbr0', but you can override this by setting
LIBGUESTFS_BACKEND_SETTINGS=network_bridge=<some_bridge>
Note: this does not fix virt-rescue (since it overrides the default
backend and uses 'direct' for various reasons).
qemu in SLIRP mode offers DHCP services to the appliance. We don't
use them, but use a fixed IP address intead. This changes the
appliance to get its IP address using DHCP.
Note: This is only used when the network is enabled. dhclient is
somewhat slower, but the penalty (a few seconds) is only paid for
network users.
On SuSE or other distros, dhcpcd could be used if available.
The previous tests were very limited, and ran the same two tests
on every run.
Randomly test:
- MBR vs GPT
- Primary, extended partitions
- Expanding vs shrinking
- Use of multiple --resize parameters
- Source and target raw vs qcow2 formats
- Resizing various different filesystem types
- LV expand
- extra partition or no extra partition
To run the test over and over again (useful for finding regressions in
virt-resize), do:
while (cd resize; LIBGUESTFS_TRACE=1 ../run ./test-virt-resize.pl) >&/tmp/log ; do echo -n .; done
If it fails, you can see the failure in /tmp/log.
To rerun a specific test, search for the 'seed:' in the output of the
original test, and specify it on the command line:
(cd resize; ../run ./test-virt-resize.pl --seed=<SEED>)
When saving a configuration file, Augeas creates a new file and
replaces the old one with it; this creates a /etc/shadow file without
the SELinux xattrs, since they are missing.
Thus, create a temporary file with all the attributes of /etc/shadow, so
all the attributes of it (permissions and xattrs, among others) can be
restored properly on the new /etc/shadow.
As side effect, if a guest is already properly SELinux-labelled, then
there should be no more need to relabel it to make sure /etc/shadow
still has the right SELinux xattrs.
This should be equivalent to the previous code, but it makes it easier
for the compiler to pick up errors (eg. missing cases).
This updates commit fc34e2d16c.
Function mbr_part_type returns one of "primary", "extended" and
"logical". The type is used by parted when adding partitions.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
And introduce parameter create_surplus to indicate whether to
create surplus partition or not. Later this parameter will be
used by when calculating positions for target logical partitions.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Old virt-v2v probably emitted the same warning.
However it seems unlikely that /dev/cdrom needs to be remapped, as
udev or the user should be able to supply a suitable symlink from
/dev/cdrom to the removable device. Therefore suppress this warning.
Switch from acl_to_text to acl_to_any_text, so it is possible to specify
options, like forcing the numeric IDs for users/groups. This ensure the
resulting string has always numberic IDs, so the comparison can always
succeed.
Adapt the comparison string to the lack of final endline now.
You can use this to accelerate testing:
$ virt-v2v --no-copy --debug-overlays [etc]
[...]
[ 82.0] Converting Windows 7 Ultimate to run on KVM
This guest has virtio drivers installed.
[ 86.0] Closing the overlay
[ 91.0] Creating output metadata
Overlay saved as /var/tmp/win7-test-sda.qcow2 [--debug-overlays]
[ 91.0] Finishing off
No disk format conversion/copying is done, so it's very fast.
The overlay file is preserved and can be tested directly (it uses the
source file as backing).
It's not really necessary: we have never needed this for debugging
inspection for example, and if hive open failures did occur we would
ask the user to send us the hive files. And it causes huge amounts of
debugging messages to be printed which fills up the log.
Old virt-v2v did not remap this device, so after conversion from Xen
the file would have something like:
boot=/dev/xvda
Although this does not appear to affect anything, better to remap it.
Otherwise you get lots of errors like:
rm: cannot remove ‘/var/tmp/mm/42f5e571-21af-44bd-8164-3ebe6bbc9f5a/images/7b28b0e6-79ce-4d78-a6cb-769760a3d373/31e2b06b-85cb-46ed-aac3-8bcef93dc704.meta’: Permission denied
rm: cannot remove ‘/var/tmp/mm/42f5e571-21af-44bd-8164-3ebe6bbc9f5a/images/7b28b0e6-79ce-4d78-a6cb-769760a3d373/31e2b06b-85cb-46ed-aac3-8bcef93dc704’: Permission denied
because the code is running as root.
For '-o rhev' this is likely to fail (although for obscure reasons it
*doesn't* fail on a real RHEV server, I don't quite understand why).
The reason is because virt-v2v may be running as root, and with root
squashing we don't have permissions to write into the storage domain
directory.
For '-o vdsm', VDSM is supposed to make sure we are running as the
correct user, so there's no reason to doubt things will work.
Make the input/output ordering of command line arguments clearer by
changing:
virt-v2v -i disk -o local -os /var/tmp disk.img
to:
virt-v2v -i disk disk.img -o local -os /var/tmp
We need to write files and directories as user:group 36:36, else
RHEV-M cannot import the VM. Doing this in the presence of NFS is
difficult. See v2v/kvmuid.mli for how it is done in this commit.
Also we allow the output mode to actually create the disk image. This
lets the output mode set ownership and permissions correctly if
required.
The default method just calls Guestfs#disk_create, so there is no
actual change in this commit.
Change Lib_ovf.create_meta_files so that instead of writing the
.meta files, it just returns the content. Output_rhev and
Output_vdsm are correspondingly changed so that they write the
content to the .meta files.
This is just code refactoring, there is no functional change.
The old virt-v2v code would either edit the grub config file (grub
legacy) or use grubby --set-default (grub2).
When translating this code, I figured we could use grubby in both
cases, since grubby exists for grub legacy guests.
However it doesn't appear to work for grub legacy guests. Use the
configuration file editing method for these, same as old virt-v2v.
Thanks: Tingting Zheng
The existing APIs guestfs_stat, guestfs_lstat and guestfs_lstatlist
return a stat structure that contains atime, mtime and ctime fields
that store only the timestamp in seconds.
Modern filesystems can store timestamps down to nanosecond
granularity, and the ordinary glibc stat(2) wrapper will return these
in "hidden" stat fields:
struct timespec st_atim; /* Time of last access. */
struct timespec st_mtim; /* Time of last modification. */
struct timespec st_ctim; /* Time of last status change. */
with the following macros defined for backwards compatibility:
#define st_atime st_atim.tv_sec
#define st_mtime st_mtim.tv_sec
#define st_ctime st_ctim.tv_sec
It is not possible to redefine guestfs_stat to return a longer struct
guestfs_stat with room for the extra nanosecond fields, because that
would break the ABI of guestfs_lstatlist as it returns an array
containing consecutive stat structs (not pointers). Changing the
return type of guestfs_stat would break API. Changing the generator
to support symbol versioning is judged to be too intrusive.
Therefore this adds a new struct (guestfs_statns) and new APIs:
guestfs_statns
guestfs_lstatns
guestfs_lstatnslist
which return the new struct (or array of structs in the last case).
The old APIs may of course still be used, forever, but are deprecated
and shouldn't be used in new programs.
Because virt tools are compiled with -DGUESTFS_WARN_DEPRECATED=1, I
have updated all the places calling the deprecated functions. This
has revealed some areas for improvement: in particular virt-diff and
virt-ls could be changed to print the nanosecond fields.
FUSE now returns nanoseconds in stat calls where available, fixing
https://bugzilla.redhat.com/show_bug.cgi?id=1144891
Notes about the implementation:
- guestfs_internal_lstatlist has been removed and replaced by
guestfs_internal_lstatnslist. As the former was an internal API no
one should have been calling it, or indeed can call it unless they
start defining their own header files.
- guestfs_stat and guestfs_lstat have been changed into library-side
functions. They, along with guestfs_lstatlist, are now implemented
as wrappers around the new functions which just throw away the
nanosecond fields.
This gives us finer control over how system calls are done,
and also potentially lets us test more.
Currently two tests are disabled:
- utimens because of https://bugzilla.redhat.com/show_bug.cgi?id=1144766
- utimes because our stat call does not return the nanosecond fields
This file causes the test ISO to be rebuilt every time.
Instead of including it as a Makefile dependency, create it just
before making the ISO, which has the same effect.
A test in fish/ also used this file. Change it so it doesn't need it.
Add support for aarch64 to the file_architecture API, so we can
detect aarch64 guests correctly.
Add a test binary and library. These were generated on an aarch64
machine by doing:
echo 'main(){}' > bin.c
gcc bin.c -o bin-aarch64-dynamic
strip --strip-all bin-aarch64-dynamic
echo '' > lib.c
gcc -shared lib.c -o lib-aarch64.so
strip --strip-all lib-aarch64.so
Previously we required that the source hypervisor knew the
architecture of the guest, and passed that through to KVM.
However this was error-prone for several reasons:
- OVF (-i ova) doesn't define the architecture
- Disk images (-i disk) don't have an associated architecture
- Libvirt XML sometimes lacks the <type arch=...> field, especially
for RHEL 5-era libvirt.
- It might not be set correctly for the guest.
We know the real architecture from inspection of the guest, so use
that instead.
The escape sequence \e[0;30m sets the colour to black. This is fine
if you use a black-on-white terminal, but fails rather badly if you
use a white-on-black terminal.
Instead use the escape sequence \e[0m which restores the terminal to
the defaults, and works in both cases.
If the RPM database in the source guest is corrupt, then subsequent
operations will fail randomly.
Old virt-v2v appears to have avoided this by cleaning the RPM database
very early on. When translating the code I inadvertantly moved this
cleanup to a later stage -- in particular, after inspection of kernels
was done which requires running rpm commands. Move this back to the
start.
When chowning the final directories, instead of chowning them to 36.36
(ie. hard-coding those UID/GIDs), use a --reference chown so it should
take the ownership from the parent directory.
Unfortunately chowning is very complex because of:
- NFSv3 vs NFSv4
- root squash
- idmap
which creates approximately 8 different ways for this to fail.
When presented with a RHEL 7 guest, virt-v2v would print the following
warning message:
virt-v2v: warning: could not update grub2 console: aug_get: no matching
node (ignored)
This happened because on RHEL 7, /etc/sysconfig/grub is a symlink to
/etc/default/grub, and the Augeas grub2 lens returns entries under
/files/etc/default/grub/*, but the code was checking for
/files/etc/sysconfig/grub/*.
The fix is to check all the possible Augeas paths until one matches.
(Only the first match gets updated however).
Also, don't rebuild the grub2 config unless we've actually changed it.
This avoids doing a potentially failure-prone operation when it's not
necessary.
I suspect that the old virt-v2v code did not work correctly for RHEL 7
guests, but old virt-v2v didn't print a warning here, it just failed
to update silently.
All tests run under the ./run binary. For a long time the ./run
binary has set the $PATH environment variable to contain all of the
directories with binaries in them.
Therefore there is no reason to use ../fish/guestfish instead of just
plain guestfish (and the same applies to other built binaries).
If do_part_get_parttype returns NULL, then return from the function
instead of dereferencing the NULL pointer and crashing. Note that
do_part_get_parttype has already called reply_with_error in this case.
This fixes commit 0316d24770.
Found by: Lingfei Kong
Old virt-v2v rebuilt the initrd late. Currently we rebuild the initrd
right after choosing a kernel, but it's best to update the initrd
at the end since mkinitrd may depend on other configuration files
that we are changing (especially see next commit).
The old virt-v2v code had:
my $prefix;
if ($virtio) {
$prefix = 'vd';
} elsif ($libata) {
$prefix = 'sd';
} else {
$prefix = 'hd'
}
The translated code dropped the test for virtio by accident, which
meant that device names would be mapped to sd* or hd* even if the
virtio-blk driver was enabled for the guest.
Thanks: Tingting Zheng
For Xen guests, a device can be presented twice as (eg) xvda and sda,
and therefore we should map any otherwise unmapped xvdX.
For non-Xen guests, this should have no effect.
I made a mistake while translating the Perl conversion code and
transcribed --set-default as --set-kernel. grubby has no --set-kernel
option so it failed.
Thanks: Tingting Zheng
Previously only "running" domains were blocked from conversion.
However "blocked" is essentially the same as running (on Xen it just
means the domain is running but waiting on an I/O resource). And
"paused" domains probably shouldn't be converted either since their
disks are only crash-consistent.
Don't rely on libvirt's implicit print-to-stderr. Instead make sure
that libvirt error messages are raised in the exception message.
Also allow error strings to be translated.
When you call close on any block device, udev kicks off a rule which
runs blkid to reexamine the device. We need to wait for this rule to
finish running since it holds the device open and can cause other
operations to fail, notably mkfs.
(cherry picked from commit a9c8123c72)
Estimate the amount of space required on the target by each source
disk after conversion. The estimate is approximately a ceiling on
this, and actual use should be considerably lower.
Provide an extra output method (#check_target_free_space) which output
modules may use to check that there is sufficient free space to
proceed, before conversion starts. None of the output modules
actually implement this at the moment.
In most C tools, virt-sysprep and virt-customize, you have to put the
--format parameter before the corresponding -a parameter. ie. The
following is correct:
guestfish --format qcow2 -a disk1 -a disk2
But the following is incorrect. The --format parameter is dangling
and prior to this commit would have been silently ignored:
guestfish -a disk1 -a disk2 --format qcow2
After this change, dangling --format parameters now lead to an error:
guestfish: --format parameter must appear before -a parameter
In virt-customize, also check that --attach-format parameter appears
before --attach parameter.
Thanks: Lingfei Kong
If there is an authentication error when getting the session cookie,
emphasize in the error message that the username can be specified in
the URI.
Unfortunately libvirt asks for the username previously, but we cannot
access that.
In theory this discards the output. Unfortunately in practice we have
to write the output to a temporary file and delete it (because of
limitations in qemu-img convert).
You will see an error like this:
$ virt-sysprep --remove-user-accounts foo,bar -a /dev/null
virt-sysprep: error: user-accounts: --remove-user-accounts parameter was
used, but the "user-account" operation is not enabled
The --user-accounts option, with its double-negative '-' prefix on
user names, is confusing. Replace it with '--remove-user-accounts'
and '--keep-user-accounts' options.
This updates commit 128d474095.
-o rhev: This is "traditional" output to a RHEV-M Export Storage Domain.
It requires no cooperation from RHEV itself, and in all other respects
works the same way as old virt-v2v did.
-o vdsm: This is the new mode which requires VDSM cooperation. VDSM
manages the conversion, creating the directories, suggesting UUIDs,
and cleaning up on failure. This allows direct import into a Data
Domain.
The --rhev* command arguments now only apply to -o vdsm, and therefore
have been renamed to --vdsm*
The --vmtype command line argument may be used with -o rhev or -o vdsm.
Also give an error if users use the incorrect one:
$ virt-builder --list --format json
virt-builder: virt-builder --list: use '--list-format', not '--format'.
The default charset for these filesystems depends on how the kernel was
configured with, so explicitly set the default one (as specified in
Linux sources), eventually setting later a new one for the tests.
Previously the core virt-v2v code had to know which Convert_* module
to call for each guest type (based on inspection). After this change
the core code doesn't need to know that, but instead Convert_* modules
register their interest in particular guest types.
Fedora patches OCaml's Arg module to support --option=arg. Since this
won't work on other distros or upstream OCaml, don't include such
examples in the documentation.
The two adjacent sections can be read separately, and the second
section doesn't mention not using make install / using ./run, so
repeat it a second time.
Before trying to perform the conversion, estimate if there is
sufficient free space on the guest filesystem(s) for the conversion to
be successful.
If not, you will see an error similar to this one:
[ 6.0] Checking for sufficient free disk space in the guest
virt-v2v: error: not enough free space for conversion on filesystem '/'.
4354048 bytes free < 20000000 bytes needed
virt-p2v can run without a window manager. However it then sits in
the top left corner of the screen. Add a lightweight window manager
to the ISO so that position and geometry hints work.
Two scripts are added, replacing the make targets from the
previous commit:
virt-p2v-make-disk: Builds a virt-p2v disk or USB key using
virt-builder.
virt-p2v-make-kickstart: Builds a kickstart file (only) which can be
fed to livecd-creator or similar tools on Red Hat-derived distros.
Kickstart generation is slightly different: Instead of requiring the
downstream 'libguestfs-pvhelper' package, the kickstart now embeds the
virt-p2v binary(!) making it self-contained and downstream packaging
simpler.
In addition there are documentation changes.
As part of reworking the scripts that are used to build the P2V ISO,
first remove the separate p2v-iso directory. Similar scripts and
documentation will be introduced back into the p2v/ directory in
following commits.
Import the upstream lens for the shadow file, just with a different
identifier and not matching /etc/shadow by default. Instead, apply a
transformation to have it match /etc/shadow only if the version of
augeas is at least 1.2.1 [1].
[1] While the last upstream version is 1.2.0, all the development seems
to happen in master, so whatever the next version is going to be
numbered (e.g. 1.2.1 or 1.3.0), the check will be fine anyway.
It will print an error like this:
virt-v2v: error: internal error: invalid argument: libvirt domain 'windows'
is running, it must be shut down in order to perform virt-v2v conversion
This only works for libvirt domains, and is explicitly disabled for
test:/// URIs since these domains are always "running", but we use
these URIs for testing.
Unhelpfully, Filename.chop_extension throws an exception if the
filename has no extension (instead of just returning the filename).
Therefore the -i disk option would fail if you passed any filename
that didn't have an extension.
When setting the password for a locked account, make sure to still write
the password after the "!!" marker, otherwise the account will have no
password.
Previously we only looked for <driver name=qemu type=...>. However
for Xen imports, the name field could be name=tap (or other values in
fact). Since there can only be one <driver/> element under <disk/>,
look for any <driver type=...> in order to get the disk format.
A further complication for Xen is that type=aio means "raw" (for
obscure historical reasons), so map that.
Thanks: Dan Berrangé.
The empty_element macro was not safe when used in a "naked" if
statement such as:
if (config->flags & FLAG_ACPI) empty_element ("acpi");
Fix the macro so it uses do { ... } while (0) to make it safe here.
The new output of virt-v2v --machine-readable now lists the input and
output drivers available in the binary:
$ virt-v2v --machine-readable
virt-v2v
libguestfs-rewrite
input:disk
input:libvirt
input:libvirtxml
output:glance
output:libvirt
output:local
output:rhev
With this information, we can now pre-populate the virt-p2v GUI
combo box.
Instead of having all interfaces connect to the "default" network on
the hypervisor target, you can now create a map of interface to target
network, either on the kernel command line, eg:
p2v.network=em1:rhevm
p2v.network=em1:rhevm,em2:management,other
or through the GUI.
Enhance the user-account sysprep operation, so it is possible to select
which user accounts should be removed only (instead of all), or which to
eventually keep.
Name these the same as the virt-v2v options, so for example
'-oa' -> 'p2v.oa'.
Also add the missing p2v.oc (-oc) option.
This updates commit c516d2d07d.
On Ubuntu, exclude the Ubuntu-only 85-lvm2.rules since it causes the
automatic activation of volume groups on addition or change. This
interferes with libguestfs, as vgs which are disables suddently get
enabled.
ocamldep -all (introduced in commit a3881445ef) creates a dependency
rule
optgroups.cmi ... : utils.cmx ...
Because we never build a native code version of the generator,
utils.cmx can never be remade, and so this results in optgroups.cmi
being always rebuilt and hence rebuilding the whole directory.
It's unclear how to fix this, but reverting the ocamldep -all change
in this directory works around it.
Partially reverts commit a3881445ef.
In v2v, 'stringMap.ml' has no corresponding *.mli file. ocamldep does
not generate a *.cmi dependency for such modules, and this can result
in parallel builds failing rarely because:
(1) stringMap.ml is built (which has the side-effect of generating
stringMap.cmi)
(2) at the same time, types.mli is compiled. types.mli depends on
stringMap.cmi, but it is half way through being created by (1).
There is no dependency to serialize (1) and (2).
The resulting error is:
File "types.mli", line 1:
Error: Corrupted compiled interface
stringMap.cmi
Using 'ocamldep -all' generates extra deps for the *.cmi files, and
otherwise appears to be safe, so use it.
There are currently a few upstream OCaml bugs with dependencies and
parallel builds:
http://caml.inria.fr/mantis/view.php?id=3190http://caml.inria.fr/mantis/view.php?id=4991http://caml.inria.fr/mantis/view.php?id=5000
It's not clear which of these apply here.
The parsing of p2v.disks, p2v.removable and p2v.interfaces did not
correctly end at the first space, and therefore any other parameters
that happened to be on the command line afterwards would be consumed
as a list of disks or interfaces.
In commit 53e0d3ea5f I fixed a cause of
disk corruption in the overlay file during conversion.
To ensure this doesn't occur again unexpectedly, add a simple test to
the virt-v2v code and to the tests.
When saving, resolve the path of the file being edited and use that as
real target to write to. Otherwise, if the file name is a symlink then
it will be replaced by a regular file with the new content, leaving the
old file untouched.
Extend test-edit.sh to check for this situation.
Allow null as value for the editor parameter of edit_file_editor, which
will then get it from the EDITOR envvar (falling back on vi).
This is basically code motion from the two edit_file_editor users to it.
To speed up conversions by storing everything we read during
conversion in the overlay instead of having to reread it from the
remote server during copying.
Thanks: Matthew Booth.
When drives are added with this flag, reads from the backing file are
copied into the overlay, improving performance of reads to the same
area of disk (at the expense of local storage). This is excellent for
reading remote / slow sources like HTTP.
Add the perl file editing, mostly based in the virt-edit implementation.
This introduces a mild code duplication with edit_file_editor; will deal
with it in a later commit.
Do not unconditionally exit if guestfs_case_sensitive_path, but let
windows_path still return null. Make virt-edit then check for that, and
eventually exit on its own.
Move the code handling Windows paths from virt-edit to a common file,
so that can be shared by various tools.
Mostly code motion, with a minimum touch (the additional guestfs_h*
parameter in mount_drive_letter) to make it build and work.
This seems* to cause disk corruption in the overlay files. A
manifestation of this is that the qcow2 header is not written back
correctly to disk, resulting in the backing file of the overlay
"disappearing" and hence hardly any data being copied over to the
target.
* Note this could be for a variety of reasons and doesn't necessarily
indicate a fault in qemu or qcow2.
Previously if you used `-i libvirt -ic [a remote URI]' then the remote
URI would be opened during object creation, which occurs during
command line parsing. The practical effect was that virt-v2v could
appear to hang or give an error (eg if the remote URI is inaccessible)
before printing any log messages. This could be very confusing for
the user.
Split out -i libvirtxml into a separate module, containing the libvirt
XML to source parsing code as a function called
Input_libvirtxml.parse_libvirt_xml.
Modify -i libvirt so it does all initialization and calls the helper
function during the #source method, and not at object creation time.
This was based an incorrect translation of what the old virt-v2v was
doing. %systemroot%\Drivers is not expected to exist already, and
indeed does not exist in a real Windows 7 guest.
Add a mini-binding to libvirt virStoragePoolGetXMLDesc to get the pool
XML directly using the libvirt API. Change existing external calls to
`virsh pool-dumpxml' to use this API instead.
The `virsh dumpxml' command doesn't work properly when the libvirt
source requires authentication. This is because the authentication
prompts are sent to stdout, but stdout is also the place where we are
reading the output XML from.
Add a mini-binding to libvirt virDomainGetXMLDesc which avoids this,
getting the XML directly using the libvirt API. Change existing
external calls to `virsh dumpxml' to use this API instead.
The way to set the filter for lvm devices was to open lvm.conf, look
for uncommented "filter =" lines and replace the configuration there.
This had the issue that if there is no uncommented filter line, then the
filter cannot be changed at all; considering newer lvm2 releases ship a
sample configuration with no uncommented filter lines, then the old way
became wrong and not sufficient.
Instead, take a copy of the upstream lvm.aug lens, with a simple change
to allow parsing also negative values in configuration, and install it
in the daemon. When asking to change the lvm filter, use augeas making
sure to use this custom lens for the lvm.conf copy used within the
appliance.
The C code didn't have -Werror enabled (when configured) and didn't
enable the full set of warnings we would want.
However OCaml-C bindings commonly declare functions which are
called directly from OCaml, eg:
external parse_memory : string -> doc = "v2v_xml_parse_memory"
value v2v_xml_parse_memory (value xmlv)
These do not require prototypes as no other C code will call them, so
we have to switch off the GCC -Werror=missing-prototypes flag for
these files.
Also fixes some confusion between _CPPFLAGS and _CFLAGS.
Move the fields from Types.overlay which were related to the target
file out. Now Types.overlay is only concerned with the actual qcow2
overlay files, not with a mixture of both files.
For every guest disk, there is one chain which looks like this:
source_disk overlay target
+------------+ +----------------+ +---------------+
| s_qemu_uri |<------ ov_source |<------ target_overlay|
| etc | | ov_overlay_file| | target_file |
+------------+ | ov_sd | | target_format |
| ov_virtual_size| +---------------+
+----------------+
describes source describes temp. describes target
disk file overlay (qcow2) file
This is just refactoring.
This function had got smaller and smaller until now we might as well
just inline it. Having it after the main function just obscured the
order that the code runs.
No functional change, just refactoring.
The ov_preallocation field was essentially an implementation detail of
the V2v module. There was no need to carry it around in the overlay
struct, and nothing else used it. This also allows us to simplify
other code.
If the <memory> or <vcpu> elements was missing from the source
metadata then we would choose useless defaults, resulting in (for
example) <vcpu>0</vcpu> being written to the output.
Choose sensible defaults for these integer values.
When we map the network or bridge names (using --network or --bridge
command line options), this commit keeps the original name and adds it
as a comment to the output metadata.
The comment will look something like this:
<interface type='bridge'>
<!-- bridge mapped from "VM Network" to "bridge1" -->
<source bridge='bridge1'/>
<model type='rtl8139'/>
</interface>
Previously Types.input and Types.output were variants, resulting in
the main v2v.ml code having to have knowledge of all the input and
output types. So you would end up with code in v2v.ml like:
match output with
| OutputLibvirt -> (* call some libvirt-specific code *)
| OutputRHEV -> (* call some RHEV-specific code *)
This changes these types to be abstract superclasses, with specific
implementations provided by (and hidden inside) each input/output
module. The above code in v2v.ml now becomes:
output#call ... (* no backend-specific knowledge needed *)
Although this commit is very large, it is just refactoring. There is
no functional change in the code.
This allows `-ic esx://...' to work because we map the strange source
URI used by ESX to a remote HTTPS connection.
Example disk section that this handles:
<disk type='file' device='disk'>
<source file='[datastore1] Windows/Windows.vmdk'/>
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
Instead of the ?dir parameter which was essentially a hack, allow
callers of create_xml to provide functions that map
<source file="..."> or <source dev="..."> to qemu URIs.
The default for these mappings is the identity function.
For -i libvirtxml we replace the old absolute_path_of_disk function
with a source file mapping function which does the same thing.
There is no functional change in this patch.
<source dev=...> in the libvirt XML should always refer to a block
device, eg. /dev/sda1. Don't attempt to make these paths absolute
(they should already be absolute paths, and if they are not it's wrong
to try to make them).
There is already a partial binding for this in fish/uri.c &
mllib/uri-c.c. However it is specialized to parsing the -a parameter
on the command line and we want access to the full underlying
functionality of xmlParseURI.
These are now collected from the source metadata and (where
possible) passed through to the destination.
For reference, see the metadata structure of the old virt-v2v (roughly
equivalent to Types.source in the new code):
https://git.fedorahosted.org/cgit/virt-v2v.git/tree/metadata-format.txt
As you can see the new code now covers all the metadata features of
the old code.
Mostly complete and making most of the features work.
The notable lack is ntfsprogs and ntfs-3g, which due to the way
mount.ntfs is handled (i.e. using alternatives) do not work properly.
This is simpler to use and more convenient than -i libvirtxml, because
users don't need to bother with writing libvirt XML. However it is
less expressive because many source capabilities such as memory and
number of vCPUs cannot be specified this way.
As initializing the target (-o ...) is a distinct operation, which can
fail if the user supplied the wrong command line arguments, make sure
we display an explicit message before we perform this step.
It prints something like this:
[ 5.0] Initializing the target -o libvirt -os default
Since `-o local' and `-o libvirt' both need to generate libvirt XML
they should share the same code to generate it.
Also generate it using the DOM module instead of printing the XML
directly. This is cleaner and avoids quoting issues.
No functional change here.
Notes:
- Drop the acpi feature for RHEL 3. Old virt-v2v says there is
a problem presenting ACPI to RHEL 3 guests although it is not
specific about what this is.
- Old virt-v2v does not pass features through to OVF at all.
This seems to be the new way [1] in Archlinux to specify, as symlink,
the default version of Java.
Thanks to Peter Wu for the notice about it.
[1] https://wiki.archlinux.org/index.php/Java
This warning was meant for the case where /dev/kvm is inaccessible and
the user must add themselves to a special group.
However we didn't take into account whether /dev/kvm was openable in
this test. If it's openable, no point issuing the warning.
Move the is_openable ("/dev/kvm") test earlier, and don't issue the
warning if that was successful.
The virt-v2v tests here are not meant to be thorough tests of guest
conversion. There is a test suite used to test conversion which is
kept outside libguestfs because it contains lots of proprietary
unredistributable guests like Windows.
The tests here instead are testing:
- virt-v2v command line arguments
- virt-v2v reads and writes files to the correct places
- valgrind & memory leaks
- any conversion properties that can be tested using just the
phony guests (but we don't go out of our way here)
- that up to date guests don't crash virt-v2v
Fix the Name and Packages database so it looks enough like a modern
RPM database to fool inspection. I'm not sure whether or not the old
version ever worked, but it doesn't work now.
The format is not documented, but in brief:
The Name database contains (Name, link) pairs. The link is an
arbitrary 32 bit integer.
The Packages database contains (link, RPM-blob) pairs.
The RPM-blob is a binary blob formatted like this:
* Number of fields 4 bytes, big endian
* Size in bytes of the store 4 bytes, big endian
* Field descriptions:
list of (RPMTAG, 0, offset in store, 0)
4 * 4 * nr_fields bytes, big endian
* The store binary data containing ASCIIZ strings
For the values of RPMTAG_* that we understand, see src/inspect-apps.c.
The calling code allows Fedora guests to be used when calling
Convert_linux. However Convert_linux itself would fail with an
assertion error when you did this.
Previous virt-v2v would write to a temporary destination file then
atomically rename everything at the end.
However this appears to be unnecessary since we write the OVF file
after copying the disks, so (I assume) RHEV-M won't see the disks
before then.
In any case nothing used ov_target_file_tmp so we can remove this
feature to simplify the code.
Currently various UUIDs used by `-o rhev' are generated randomly.
This means they are different on every run of the program.
Allow some of them to be specified on the command line. (If not
specified they are still generated randomly).
This allows RHEV-M to run virt-v2v several times (eg to examine the
OVF output) while having paths to files and OVF be stable across runs.
At the moment there is just one "miscellaneous RHEV command line
option" (--vmtype), but we may add more in future. Put them in a
single struct for convenience.
This is just code motion.
If the environment variables LIBGUESTFS_TRACE=1 or LIBGUESTFS_DEBUG=1
were set, then the current code would end up disabling trace/debug if
the command line arguments did not contain -x/-v.
Fix this so the effect of enabling trace/debug is cumulative.
Previously it was called 'libvirt.conf' which is more than a little
bit confusing.
This is only an internal name, so there is no visible change in
functionality.
It seems that Automake cannot handle properly the files generated by
flex and bison, leaving them in the dist tarball.
For now, add a dist-hook to manually prune them from distdir.
When using optional arguments of type OStringList, the code free'ing
the member in the optargs_s struct corresponding to that optional
argument would just check for a non-PyNone PyObject for that argument.
If before that optional argument there are other arguments which can
cause an earlier error return from that binding function, the free'ing
code will then act on garbage values.
Enhance the check by also checking whether the optargs struct has the
bitmask with the element for that argument, meaning that the
corresponding struct member was initialized.
The error is:
../src/.libs/libutils.a(
libutils_la-cleanup.o): In function `guestfs___cleanup_xmlBufferFree':
cleanup.c:(.text+0x79): undefined reference to `xmlBufferFree'
../src/.libs/libutils.a(libutils_la-cleanup.o): In function
`guestfs___cleanup_xmlFreeDoc':
cleanup.c:(.text+0x99): undefined reference to `xmlFreeDoc'
[etc.]
Thanks: Arnaud Gaboury
If pthread_cond_wait fails and the thread worker ends, make sure to
unlock retire_mutex, which has been locked for the condition.
The return value of pthread_mutex_unlock is ignored, as there's nothing
that can be done with it at that point.
Fix the memory issues in the implementation that uses no -p nor -i:
- use add_string_nodup to add results from get_blkid_tag (which returns
new strings), so those strings are not leaked
- use free_stringslen to clean the hash on error, as in such case the
stringbuf will not be terminated with a null entry, thus causing
free_strings to crash
When dealing with DeviceList parameters, the generator produces code
similar to the following:
CLEANUP_FREE_STRING_LIST char **devices = NULL;
[...]
devices = malloc (sizeof (char *) * (args.devices.devices_len+1));
{
size_t i;
for (i = 0; i < args.devices.devices_len; ++i)
RESOLVE_DEVICE (args.devices.devices_val[i], devices[i],
, goto done);
devices[i] = NULL;
}
The block hidden within the RESOLVE_DEVICE macro is supposed to
assign something to devices[i]; on the other hand, the code in
RESOLVE_DEVICE can cause to just end (with an error) the current RPC,
which would cause the cleanup of the "devices" array... whose members
from the i-th to the (args.devices.devices_len-1)-th would be garbage
pointers, causing random memory to be free'd (and thus crashing the
daemon).
Avoid the access to garbage memory just by having a cleaned "devices"
array, so there will be always a NULL element after the initialized
members.
Add a test for vgcreate which passes a wrong device path causing the
situation above, to test that vgcreate would fail gracefully.
Use %d to parse them as int (since the variables for them as int)
instead of %u, even if they both need to be at least > 0; the library
will check for the validity of them anyway.
Move the minimum memory check from the environment parsing to
set_memsize, so the limit is actually enforced also when using the API.
Adapt the rhbz557655.sh test to the invalid memsize values being
rejected now, and add a new test for checking invalid parameters
explicitly.
Even if luaL_error is a "no return" function for the Lua runtime, adopt
also in action functions the "return" idiom recommeded for it.
This also helps code analyzers in not thinking that "g" might still be
null after the null check followed by luaL_error.
Handy macro to reply the right way for an unavailable feature.
While generally used so far in generated code, it can shorten that a
bit, and avoid copy&paste when wanting to do manual feature checking.
This refactors existing code to close file descriptors in the recovery
process, and also adds code to close file descriptors between the
fork() and exec() of QEMU or User-Mode Linux.
The reason is to avoid leaking main process file descriptors where the
main process (or other libraries in the main process) are not setting
O_CLOEXEC at all or not setting it atomically. Python is a particular
culprit.
See also this OpenStack Nova bug report:
https://bugs.launchpad.net/nova/+bug/1313477
Thanks: Qin Zhao for identifying and characterizing the problem in Nova.
This is version 2 of this commit. This commit is identical to the
reverted commit 115fcc3432 except that
we don't close stderr.
This attempted fix for RHBZ#1123007 causes the qemu command line to be
lost when verbose mode is enabled. Since this is essential for
debugging many problems, I am reverting the patch.
This reverts commit 115fcc3432.
This is useful when analysing with `perf' since it allows us to
increase the amount of time in the test versus the amount of time
spent starting the appliance.
This refactors existing code to close file descriptors in the recovery
process, and also adds code to close file descriptors between the
fork() and exec() of QEMU or User-Mode Linux.
The reason is to avoid leaking main process file descriptors where the
main process (or other libraries in the main process) are not setting
O_CLOEXEC at all or not setting it atomically. Python is a particular
culprit.
See also this OpenStack Nova bug report:
https://bugs.launchpad.net/nova/+bug/1313477
Thanks: Qin Zhao for identifying and characterizing the problem in Nova.
We were mounting different filesystems on top of each other. In fact
this is not a problem, but it's nicer to unmount each filesystem
properly after examining it.
This test worked by uploading a 100MB file into a 100MB container and
seeing the inevitable crash. Unfortunately virtio-mmio (used on
aarch64) is quite slow. Since this test is not testing the speed of
virtio-mmio, use a smaller test file so the test finishes in a
reasonable time.
This library replaces Win::Hivex::Regedit, or at least enough for us
to be able to make the simple Registry modifications needed for
installing firstboot scripts.
To fix RHBZ#1056290, I prevented virt-sparsify being used if the
output device is a block device.
I have now retested this scenario and it does work (in both copying
and in-place mode), and does not delete the output device, and
therefore we can relax this restriction so only char devices are
banned.
This is useful for oVirt which uses a qcow2 formatted block device to
store virtual machines.
For example:
$ time ./run ./tests/qemu/qemu-boot -n 100
real 1m19.794s
user 0m10.001s
sys 0m5.928s
will run 100 appliance start/stops, from multiple threads.
This test has not been run since 2012 as it depends on specifics of
how supermin and new-style appliances work (see
commit 2d89aef897).
This commit removes the test entirely.
In OCaml 4.02, the 'string' type can be made immutable. All String.*
functions that are concerned with creating or mutating strings now
give a warning like this:
Warning 3: deprecated feature: String.unsafe_set
Disable this warning since we don't want to turn on immutable strings
yet.
This still only prints early console messages. Output stops
after:
bootconsole [earlycon0] disabled
but I have not found any way to get later kernel messages.
This option allows oVirt to pass a prebuilt qcow2 file to use as the
temporary overlay. The file must be qcow2, and must have indisk as a
backing file - the code does minimal checks to ensure this is correct.
Example usage:
qemu-img create -f qcow2 -b indisk overlay.qcow2
virt-sparsify indisk --tmp prebuilt:overlay.qcow2 outdisk
Note this only applies in copying mode.
Run ldconfig early in the init script, so libraries outside standard
library paths but with a proper ld.so configuration file pointing at
them can be found.
Just like no /etc/fstab is not an error, having one with no entries
shouldn't be an issue either.
With systemd, this could be a valid setup, with mount points set its own
way.
We were passing an int64_t which happens to be the same as time_t, but
only on 64 bit. This didn't work on 32 bit (or rather, it works on
little endian, but only by accident).
This fixes commit fb546eaee0.
sd_journal_get_realtime_usec is defined as:
int sd_journal_get_realtime_usec(sd_journal* j, uint64_t* usec);
Unfortunately we passed size_t* as the second argument. This happened
to work on 64 bit machines because size_t is 64 bit there, but failed
on 32 bit machines, where we didn't pass a sufficiently large result
buffer.
This fixes commit 2d996df66c.
These log files can get pretty big, and that causes problems
like this one:
http://linuxforum.ru/viewtopic.php?id=35381
and this one:
https://bugzilla.redhat.com/show_bug.cgi?id=1099172
In both cases the problem was /var/log/lastlog which is apparently a
large sparse file. Supermin cannot cope with sparseness, so it copies
the whole file non-sparse. But since this could affect other log
files this commit covers various common log file names and patterns.
Thanks: semperN
Sort the structs when generating code. Since the structs are
logically indepedent of each other, this should have no effect except
to make generated files list the structs in a different order.
However this also fixes the following build problem:
File "convert_linux.ml", line 322, characters 43-50:
Error: This expression has type G.stat = Guestfs.stat
but an expression was expected of type G.dirent = Guestfs.dirent
It turns out the OCaml bindings don't like the fact that we have
two structs with a common field name (dirent.ino and stat.ino).
In OCaml < 4.01, this means that any attempt to reference stat.ino
would fail because dirent.ino appears second in the file, overriding
stat.ino.
Sorting the structs has the side effect of making stat.ino appear
second, thus resolving the build failure above.
In OCaml >= 4.01 the compiler now uses some disambiguation rules based
on the known types to resolve this problem, so accessing either field
would work no matter what order they are listed in.
See:
http://ocaml.org/releases/4.01.0.htmlhttp://www.lexifi.com/blog/type-based-selection-label-and-constructorshttp://www.lexifi.com/blog/ocaml-extensions-lexifi-overidding-record-labels-and-constructors
virt-customize:
virt-sparsify:
virt-sysprep:
virt-v2v:
- These tools consistently used -v to mean verbose/debugging
and -x to mean enable libguestfs tracing.
virt-builder:
virt-resize:
- These two tools did not recognize -x at all, and used -v
to enable libguestfs tracing and general debugging.
- This commit changes these two tools to consume -v/-x
consistently with the other tools.
Unfortunately this has a cascade of effects through the code.
It turned out that Common_utils was exporting the 'G' module (an alias
for Guestfs). We want any code that uses G as a shortcut to declare:
module G = Guestfs
at the top, since that avoids confusion for newbie (or experienced)
OCaml programmers.
Old Xen PV guests had:
<driver name='tap' type='aio'/>
The previous xpath expression matched any driver type attribute in
order to pick up the format. However we only want to match:
<driver name='qemu' type='raw'/>
so we need to check the name attribute as well.
I'm not sure if this indicates that the xpath expression is wrong, but
in any case it can be that xpathobj->nodesetval is NULL. In this
case, return 0 from xpathobj_nr_nodes instead of segfaulting.
Commit c4dc70f8c4 broke the man pages
for the Perl scripts in the tools/ directory.
It inserted the =encoding line at the top of the file, instead of in
front of the first =head1 entry, and this meant that the #!/usr/bin/perl
and the prologue became a part of the POD, which was not intended.
Add the virt-sparsify --tmp option.
This works in two ways. Either you can specify a temporary directory,
in which case it's just the same as setting $TMPDIR before:
virt-sparsify indisk outdisk --tmp /var/tmp
Or, as a new feature, you can specify a block device which is directly
used (and OVERWRITTEN):
virt-sparsify indisk outdisk --tmp /dev/sdX
This is useful for oVirt nodes, where there is limited temporary
space, but a block device can be assigned to the node.
In both cases it is only used in copying mode. In-place
sparsification doesn't require large amounts of temporary space.
Previously if you were root, and you tried to change directory into a
directory which was not owned by you and not readable (eg. 0700
bin:bin), it would fail.
This doesn't fail on regular directories because when you are root the
kernel just ignores permissions.
Although libguestfs in general tries not to duplicate kernel code, in
the case where we emulate the FUSE access(2) system call,
unfortunately we have to do it by stat-ing the object and performing
some (half-arsed) heuristics.
This commit modifies the FUSE access(2) system call, so root is now
able to chdir to any directory.
It also adds some debugging so we can debug these complex permissions
checks in the field if some other problem arises in future.
This commit also removes the disk image signature. These have not
been needed since virt-builder 1.24.1. No one should be running that
version of virt-builder since there are newer 1.24 branch versions.
See:
https://www.redhat.com/archives/libguestfs/2014-June/thread.html#00045
Notes:
(1) Edit dracut configuration to make sure the change is permanent, in
case kickstart, virt-builder or something else installs a kernel or
runs dracut.
(2) We need to run dracut on the just-installed kernel, not on the
running kernel. The snippet to do this was copied from the Fedora
cloud kickstart.
Thanks: Olaf Hering, George Dunlap, Dario Faggioli.
This device is not present at least on ARM. Unfortunately libvirt
doesn't give us any way to query when the device is present.
This fixes commit bed592498d.
Setting environment variables such as PS1 for bash before starting it
might not be effective when the startup scripts provided by the
distribution unconditionally change it.
Hence, set PS1 and TERM in a ~/.bashrc, which will be source'd last and
thus be able to set them the way we want.
Augeas has been a mandatory dependency of libguestfs for quite some
time, so the aug_* functions could have been always called (without even
checking the augeas feature). Thus retire the feature, marking it as
always available with no more functions depending on it.
It seems recent (?) NetBSD versions have their kernel as /netbsd, so
also check for it to detect NetBSD installations.
The current detection so far basically relied on generic files and
directories which can potentially be in every UNIX system, misdetecting
them if a /etc/release file is present in them.
If the image is located on NFS, and the NFS mount is performing a
large copy, then all other NFS operations stop. This delay can be
large enough that we hit the current 10 minute timeout in udev.
Increasing this to 100 minutes is reported to cure the problem (in
reality we want udev to wait indefinitely).
This implements the guestmount --fd option to allow you to run
guestmount captive under another process (typically using
`guestmount --fd=<FD> --no-fork').
See: https://bugzilla.redhat.com/show_bug.cgi?id=1100498
Resolve the given path within the chroot, so scrub can be invoked
outside the chroot on an already-resolved path.
Add few tests in scrub-file for this and other similar issues.
commit a86eb0e0d2 made it an optional
feature, as on Windows it was not available; on the other hand, realpath
has been used unconditionally already for quite some time, so having
just the "realpath" command conditional on the presence of it does not
make much sense.
Drop the configure/build system handling of it, make the "realpath"
command no more optional, and keep the "realpath" feature as
unconditionally available now.
Support the possibility to have optional groups always enabled (e.g.
because they were present in the past, and they need to be kept for
users).
Add and use few helper optgroups-related functions to deal also with
them.
console=ttyS0 used to work. Now it's broken again, and the correct
setting appears to be console=hvc0. Since we can pass multiple
console settings, do that to try and catch the right console.
This is just a whitespace change. The new output looks like this:
Output file: fedora-20.img
Output size: 4.0G
Output format: raw
Total usable space: 5.2G
Free space: 4.5G (86%)
Previously it wasn't installed at all. However it makes sense to
install it somewhere so that it can be placed in a package (a
requirement if we're going to create a kickstart). By placing it in
libexecdir it is hopefully clear that it should not be used.
Note on Red Hat distros, libexecdir is usually /usr/libexec. On
Debian it is usually /usr/lib/libguestfs.
Bump to >= v196 the minimum version of libsystemd-journal needed for the
journal integration, since that version provides
sd_journal_get_data_threshold and sd_journal_set_data_threshold.
It is not possible to have the ISO be built automatically when the
user runs 'make'. The README file describes how you should go about
building the ISO or disk image.
This is a graphical standalone front-end to virt-v2v which can be run
on physical machines (usually linked into a ISO or PXE boot image) to
convert the physical machine to a virtual machine.
Copy the information about the proxy of a source in all the entries of
that source; this way it is possible to use it later when accessing to
the actual image of an entry.
Root partition is on /dev/sda2:
$ virt-filesystems -a rhel-5.10 --all --long -h
Name Type VFS Label MBR Size Parent
/dev/sda1 filesystem ext2 /boot - 510M -
/dev/sda2 filesystem ext4 / - 4.5G -
/dev/sda3 filesystem swap SWAP-hda3 - 1.0G -
/dev/sda1 partition - - 83 510M /dev/sda
/dev/sda2 partition - - 83 4.5G /dev/sda
/dev/sda3 partition - - 82 1.0G /dev/sda
/dev/sda device - - - 6.0G -
It would be better if we could mount by label in guestfish ...
There is another problem with RHEL 5 guests in that the
yum URL is slightly wrong.
Get rid of --enable-gcc-warnings and just add all the warnings in any
case; they are tested anyway, so only warnings supported by the compiler
will be used.
Regarding -Werror, add a new --enable-werror which enables its use.
Allow to skip testing the filesystem "foo" if the environment variable
SKIP_TEST_CHARSET_FIDELITY_foo=1 is set. This way it possible to not
test one or more filesystems without disabling the test altogether.
So far, passwords in URLs (eg http://user:password@host..) have been
handled as part of the username, and thus passing
add-drive path username:username:password ...
instead of
add-drive path username:username secret:password ...
Fix the parsing of URLs to handle passwords as separate elements,
properly passing it as "secret" parameter for add-drive, and properly
readd it when building URLs in the direct backend.
Furthmore, to keep curl- and ssh-based qemu drivers working with
authenticated resources, make sure they can accept secrets.
Reported in comment #1 of RHBZ#1092583.
This is useful for generating Linux initramfses from other types of
filesystems. For example:
guestfish --ro -a disk.img -i cpio-out / - | gzip -9 > initrd.img
Switch from xmlParse* to xmlRead* so we can explicitly specify parsing
flags, including no network usage and no entity resolution.
The two behaviours mentioned above were not done before either, so this
should not introduce behaviour changes in libguestfs.
qemu-system-arm 2.0 as currently available through Debian/unstable
doesn't want to tell us anything about available devices unless we
specify a machine type.
When parallel builds are enabled it was possible for index-parse.c to
be compiled before the file had been completely written by bison. The
usual symptom was that the 'do_parse' symbol was missing -- this
simply happened because that symbol is defined at the end of this
file, and the compiler compiled the file before the end part was
written out. But you could also see other strange & random compile
failures as you would expect.
This was tested by running this script on an 8 core server:
cd builder/
export MAKEFLAGS=-j9
while make clean >/dev/null; make all V=1 >& /tmp/log; do echo -n . ; done
Previously it would fail after <= 5 iterations. With this change it
runs for at least 100 iterations.
Parsing sources .conf files is not a fatal error (that file would just
be ignored), so explicitly state that such parsing errors are ignored.
This should address the last bit in RHBZ#1077817.
Convert the tests to the MiniTest test framework, but keeping the usage
of the old Test::Unit as fallback in case MiniTest is not available.
In the latter case, use a bit of "glue" to make the old API look like
the new API, so we can just rely on the newer MiniTest API.
Isolate in a common file all the standard boilerplate in tests, i.e. the
import of the test framework and the guestfs module (including the
import path hack needed for the latter).
Thanks to Vít Ondruch for the precious hints and suggestions.
Use parted to get the name of partitions in GPT layouts instead of
sgdisk, to reduce the possible discrepancy between output of tools.
The actual case here is that recent parted versions fixed/improved their
UTF-16 handling of partition names in GPT, and sgdisk seems to not be
properly handling them, returning also unicode control bytes.
Since parted can provide partition names already, just make use of it.
Since sgdisk is no more needed for part_get_name, the function is no
more optional on it.
Add the possibility to configure the proxy in each repository .conf
file, specifying whether use no proxy at all, follow the system
configuration or use a specific proxy.
Previous commit 97c57a8887 added a
potential bug in the sparsify build. If virt-sparsify had used a
*.mli file then it would have tried to run a command like:
ocamlc mlguestfs.cmxa -c foo.mli -o foo.cmi
This would fail because ocamlc doesn't know about native code
libraries (*.cmxa). In fact virt-sparsify was not affected by this
because it doesn't use any *.mli files.
Fix this by adjusting the automake variables.
If a disk has type 0x42 partition (which would indicate LDM), but LDM
is not available then try parsing the partition anyway. It might be
parseable as plain old NTFS.
There are two observations which make this possible: (1) virt-sparsify
contains at least one C unit, so automake thinks it is compiling a C
program. (2) Automake lets us add a *_LINK rule to override linking.
We list the C sources of virt-sparsify as virt_sparsify_SOURCES, so
automake thinks it is building a C program as normal.
We override virt_sparsify_LINK to make it actually link an OCaml
program.
We add virt_sparsify_DEPENDENCIES listing the OCaml objects so that
they get built before linking.
We need a small linker script (link.sh) which adds some options at the
end of the linker command line which are impossible to add using pure
automake.
Inspired by the same idea in supermin:
https://github.com/libguestfs/supermin/blob/master/src/Makefile.am
As long as over ~300MB is recovered, declare sparsification to be
successful in this test.
It looks as if the journal or other metadata takes up a variable
amount of space after sparsification.
We must avoid touching appliance/supermin.d unless it is necessary, so
that we avoid unnecessary rebuilds of the full appliance.
Unfortunately since we created temporary files there, even if we
didn't decide to keep those temporary files they would still end up
touching supermin.d. To stop this, move the temporary files out.
Exclude the arch-independent pkg-config files (no pkg-config available
in the appliance).
Exclude also the doc-base and reportbug files, typically found in Debian
systems.
Add a new lexer token, unused in the grammar, for the unknown lines in
index files; this should allow to better handle such kind of parsing
errors, removing the need to exit() directly (and leave things in an
unclean state).
Since HPET is specific to x86, we can safely add it its option only on
x86 and x86_64 when creating the libvirt XML (no more hitting the
launching failures due to that on other architectures).
Regarding the direct qemu launch, since qemu 1.1 (which is our current
minimum) "-ho-hpet" appears in the help only where actually supported,
so we could just checking for it and adding it only if present. This
should fix the architecture issues on this backend as well.
The boot partition name is /dev/ubda1 in this case, so use a small sed
script to canonicalize it for the test comparison.
This fixes commit f6a37740f2.
61323fa introduced a mandatory arch field within index files to
identify the architecture of the OS installed within a given template.
This change simply documents the requirement within the relevant
section of the virt-builder man page.
Signed-off-by: Lee Yarwood <lyarwood@redhat.com>
This removes the internal APIs internal_set_libvirt_selinux_label and
internal_set_libvirt_selinux_norelabel_disks. The communication
between the libvirt domain and the backend now uses the backend
settings.
Currently the backend settings are treated as a list of strings. You
can set the whole list (clearing any strings there previously), but
you cannot search for an individual string or replace an individual
string.
This adds further APIs allowing you to do that. We treat the backend
settings as a list of environment-like strings (ie. name=value), and
add the following functions:
- set-backend-setting (name, value)
Set name=value. Any previous settings of name are cleared.
- get-backend-setting (name)
Search for name or name=value and return the value.
- clear-backend-setting (name)
Remove any name or name=value settings.
This also adds a regression test.
This is not used by libguestfs right now, but it does allow you to
build a python "distribution" of libguestfs, like this:
make -C python sdist
The distribution tarball will be in python/dist/
You can copy the distribution tarball somewhere, unpack it, and use
regular 'python setup.py' commands such as:
python setup.py build
python setup.py install [--prefix=...]
python setup.py --name
In future we hope to be able to upload the distribution tarball to
PyPi, but licensing issues prevent this at present.
guestfs___validate_guid was a new function added to utils.c in
commit beef77403c.
However utils.c should not include <guestfs-internal.h> since the
other functions in this file can be used by all front-end code, not
just libguestfs.so.
This function is only needed in libguestfs.so, so move it to another
source file, and remove include of <guestfs-internal.h> from utils.c.
Also: use 'size_t' for counting, not 'int'.
This fixes commit beef77403c.
Breaks Python distutils which doesn't use C99:
utils.c: In function ‘guestfs___copy_string_list’:
utils.c:79:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
for (size_t j = 0; j < i; ++j)
^
utils.c:79:7: note: use option -std=c99 or -std=gnu99 to compile your code
[Gnulib tries to replace getopt because of the bug shown below.
However because -lgnu was not used in the Makefile, the replacement
failed. -- RWMJ]
from config.log:
/* This code dumps core on glibc 2.14. */
{
static char program[] = "program";
static char w[] = "-W";
static char dummy[] = "dummy";
char *argv[] = { program, w, dummy, NULL };
optind = opterr = 1;
if (getopt (3, argv, "W;") != 'W')
result |= 64;
}
gdb --quiet --readnow ./conftest -ex r -ex bt -ex detach -ex q
Reading symbols from /usr/src/packages/BUILD/libguestfs-1.26.0/conftest...expanding to full symbols...done.
Starting program: /usr/src/packages/BUILD/libguestfs-1.26.0/conftest
Missing separate debuginfo for /lib64/ld-linux-x86-64.so.2
Try: zypper install -C "debuginfo(build-id)=5d1a12e6f0e95331cc4e39df74ecbd5adb76a1f7"
Missing separate debuginfo for /lib64/libc.so.6
Try: zypper install -C "debuginfo(build-id)=a06caa12df8f953a453befa827c3145adaf6269a"
main(323) result 0
main(344) result 0
main(357) result 0
main(370) result 0
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b2dbf8 in _getopt_internal_r () from /lib64/libc.so.6
Detaching from program: /usr/src/packages/BUILD/libguestfs-1.26.0/conftest, process 31042
This change actually fixes build for me on sles11sp3 and 11.4.
The OCaml compiler isn't happy when two modules have the same name in
different directories. Since customize/ contains a library of modules
used in other directories, and since those directories may have
main.ml files too, we need to rename 'customize/main.ml' to
'customize/customize_main.ml'.
This adds the following options to virt-customize:
--attach
--attach-format
-m / --memsize
--network / --no-network
--smp
This fixes commit 2b208d84db.
The customize module includes an --selinux-label option which can run
'fixfiles restore' (on most SELinux guests) or set /.autorelabel (on
some older ones).
Commit 49014f81f3 renamed the old
--selinux-label option to --autorelabel, but note this was not
included in a stable version of libguestfs.
Note this change leaves a bunch of now redundant code for detecting if
we created a new file in the guest.
Split virt-builder into build and customize steps, so that we can spin
off a separate tool called 'virt-customize'. This commit does not in
fact create such a tool, but it moves all the common code into a
library, in the customize/ subdirectory of the source.
Although this is mostly refactoring, it does change the order in which
virt-builder command line arguments are processed, so they are now
processed in the order they appear, not the inflexible fixed order
used before.
Previously we ran them in essentially a random order, although it
might have looked alphabetical in some cases because the modules are
loaded in order.
Some tests might spawn an appliance, which will take time on slower
architectures and on some virtualized environments.
Hence raise the per-test timeout from the default of 60s to 300s (which
should be hopefully enough for now).
Commit f75142c577 disabled parallel
builds in the ocaml subdirectory (which was the correct thing to do).
This made building the OCaml tests in particular very slow. Therefore
fix things so that parallel builds can be used again.
See also discussion here about different approaches:
https://www.redhat.com/archives/libguestfs/2014-March/thread.html#00223
This reintroduces commit dce94f3e26.
When parsing the repository configuration files, track the file names
parsed, and in case of same file name in multiple location consider only
the one in the directory coming before others in the prioritised list of
paths.
This way it is possible to "shadow" a configuration file in a system
path with one with the same name in a local directory, for example.
Always close the file (ignoring its result) after a parsing, and cleanup
the parse_context object before any exit().
This eases the debugging of memory issues in the actual parser.
This changes podwrapper so that the input (POD) files should not
contain an =encoding directive. However they must be UTF-8.
Podwrapper then adds the '=encoding utf8' directive back during final
generation.
This in particular avoids problems with nested =encoding directives in
fragments. These break POD, and are undesirable anyway.
Switch the lex/yacc parser into reentrant mode, to ease the handling of
parsing-specific data; introduce a new parser_context struct for that,
which is added as extra data to the parser.
This should cause no behaviour changes in the parsing, just no more
global variables used for getting data in/out the parser.
So far the failure of some test would have not reported a non-zero
return value by run-tests.php. Since now all the PHP tests pass, we can
ask for failures to be fatal, by exporting REPORT_EXIT_STATUS=1 for
run-tests.php.
We have generally forbidden declaration-after-statement in our code
(although there are plenty of places that do it). In any case this
warning is not in the default set, so we don't need to turn off the
warning.
RHEL 6-era bash cannot handle the multi-line $(...) expression in the
original code. Rewrite the test so it is semantically the same but
doesn't have the expression that old bash can't handle.
The output of test-virt-rescue.pl contains non-ASCII chars which give
troubles when being XSLT-transformed, so just avoid its output to leave
the rest of the resulting XML well-formed.
If the RUN_OUTPUT_FILE environment variable is set, "run" will output
to the file indicated by that snippets of XML with the results and
outputs of the tests run.
Together with the run-xml-to-junit.sh (and its associated
run-xml-to-junit.xsl style sheet) it is possible to convert that output
file to a jUnit-like XML file, which could be used in CI systems.
It's reasonable to expect megabytes will be freed up and given back to
the host. Previously we just tested that at least 1 block was
released. This test checks that at least 1000 blocks are released
(since stat(2) returns blocks of 512 bytes: 512 bytes * 1000 = approx 0.5 MB).
Properly initialize the variables for them to NULL, so there is no crash
because of trying to use rubbish pointers.
Kind of followup of commit 289fd29f0b.
Since the default PHP test runner ignores a good number of environment
variables to potentially tampering the test suite execution, create a
custom php-for-tests.sh script which does nothing more than sourcing the
custom environment that our run-php-tests.sh outputs and running the
actual "php" executable (the one found by configure).
This fixes the loading of the guestfs_php.so module in the test suite,
as the libguestfs.so.0 library can be found by that module.
Commit 71dfd73042 broke 'make clean' in
the appliance subdirectory, so it would remove all the supermin.d/*
files, but without removing stamp-supermin so you couldn't rebuild
supermin.d/ with a 'make' afterwards.
In any case, 'make clean' is not supposed to delete the appliance
(which is expensive to rebuild).
Fix this so that 'make clean' doesn't remove supermin.d/* files.
'make distclean' can be used to delete this, and since it also removes
stamp-supermin, it will be recreated properly by 'make'.
Since virt-sysprep tends to delete a lot of files, adding discard
support to it makes some sense.
Note that this probably won't have any effect for most filesystems
since:
(a) ext4 mounts also need to use -o discard,
(b) ext4, and maybe others, require you to call fstrim explicitly,
they don't discard automatically (except for userspace tools like
mkfs.ext4 but that doesn't apply in this case).
Allow callers to specify that all the disks from a domain are added
with a specific cachemode (instead of always having to use the
default, writeback).
This adds a discard parameter to guestfs_add_drive_opts which
approximately maps to the discard=ignore|unmap parameter supported by
qemu.
If discard is set to "enable" then we force discard=unmap (and try to
fail if it is not possible). If discard is set to the more useful
"besteffort" option, then we enable discard if possible. The default
is "disable".
Add the possibility to choose which architecture use to build the wanted
image (--arch). Since this implies that running commands on the guest is
usually not possible when the architecture is different than the host
one, virt-builder will nevertheless try to check whether the host is
compatible with the guest, allowing to run commands only in that case.
The caching scheme is adapted to account for the architecture (with
--print-cache showing the architecture as well).
Group together all the various parameters (except the guestfs handle)
passed to the create_drive_*, to avoid passing all of them at every
function call.
This is mostly an internal refatoring with no behaviour change.
RWMJ:
- Patch taken from Debian downstream package.
- The default in the original patch was to disable the tests. I have
changed this to default to enable them instead.
- Removed AC_SUBST as the variable was not being used.
If appliance is disabled, the whole appliance/ subdirectory is
skipped.
If appliance is enabled, we can assume that the daemon is enabled too
(see previous commit) so we don't need the conditional code. That
conditional code didn't work since the supermin 5 rewrite.
See top of configure.ac:
> # NB: AC_CHECK_PROG(S) or AC_PATH_PROG(S)?
> # Use AC_CHECK_PROG(S) for programs which are only used during build.
> # Use AC_PATH_PROG(S) for program names which are compiled into the
> # binary and used at run time. The reason is so that we know which
> # programs the binary actually uses.
/proc/sys/kernel/hotplug is an obsolete interface for processing
hotplug events. It is not used by modern kernels, which should set
this path to the empty string (because they have
CONFIG_UEVENT_HELPER_PATH="").
Unfortunately our udev initialization code did:
echo -e '\000\000\000\000' > /proc/sys/kernel/hotplug
which was intended to set this file to \0 bytes. However on Debian,
where /bin/sh is "dash", not "bash", this writes "-e" to this file,
which breaks udev events.
Thanks: Peter Rajnoha for solving this.
https://bugzilla.redhat.com/show_bug.cgi?id=1020806#c10
Update the documentation according to the new .conf files structure, the
drop of VIRT_BUILDER_SOURCE and VIRT_BUILDER_FINGERPRINT, the drop of
hardcoded source and its key, etc.
Sigchecker can still import keys from the user's keyring, so there is no
need to hardcode fingerprint and key of the index hosted on
libguestfs.org (which is now pointed to using a .conf file).
Drop these two environment variables, and the implicit hardcoded source
hosted at libguestfs.org.
This means all the sources must be provided as .conf files, or at each
invocation with --source.
Introduce and use simple .conf files to configure the sources of indexes
for virt-builder. The location of these files is in XDG_CONFIG_DIRS /
XDG_CONFIG_HOME, so it can be easily overridden.
There are three .conf(.in) files shipped with this commit:
- "test-index.conf.in" (in "test-config"), which points to the
"test-index" index (used in tests only); the tests are adapted to
point to the hierarchy containing this .conf
- "libguestfs.conf.in" (in "test-website"), which points to the local
"index.asc" (i.e. the offline copy of the libguestfs.org index);
run(.in) will point to the hierarchy providing this .conf
- "libguestfs.conf.in" (directly among the other sources), which points
to the online "index.asc" and it is installed in sysconfdir, along
with the key of this repository
The tests are adapted, other than to the different way to pick sources,
to the different output of --list, as "test-index" is not signed.
Use Sigchecker.gpgkey_type instead of just string as type in the
sources list; adapt the listing code (and its expected output) to that.
No behaviour change which eases a bit the addition of new sources with
other key types.
Move in an own module the code which calls the C
virt_builder_parse_index and does the array -> list conversion of the
result. This way this code can be easily called also in places different
than Index_parser without the need to copy the types mapping, etc.
Just code motion, no actual behaviour changes.
Introduce a mandatory arch= key in all the entries of index files, to
identify which architecture is each. Adapt the long and JSON list
outputs to print also this new field.
This introduces an incompatibility with index files created with
virt-builder < 1.26, as they will be rejected until entries will have
the arch= key added (which is ignored by older virt-builder, so adding
it will not create backward-compatibility issues).
Extend Sigchecker so it allows both fingerprints (to be imported from
user's keyring, as before) and keys stored in files. To simplify this
process (and have the fingerprint always around), the key is imported
on Sigchecker.create time, instead of lazily at the first verification.
Create a temporary directory and tell gpg to use it as homedir, so
imported keys do not get into the user's keyring. This also avoid
importing the default key when a different one is needed to check the
signature.
The only exception is when a non-default fingerprint is used: in this
case, that key is read from the user's keyring, since it is where it is.
Even though so far there is just one possible setting to read, isolate
in an own function the code to parse a configuration file and read the
settings out of it.
Now there's a new config_t handle used every time, but since config_read
would reset an handle completely, there is no behaviour change.
1151902 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1151902
virt-v2v:Network interface disappear after conversion of guest with interface type direct
1151903 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1151903
virt tools --csv output should have a meaningful output
1151905 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1151905
virt tools --csv output should have a meaningful output
1152369 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152369
[RFE]virt-v2v failed to convert guest with UEFI firmware
1152533 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152533
virt-v2v fail to convert guest with virtio lun disk
1152818 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152818
llz give ? for the security context of a file, which is wrong
1152819 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152819
Can not end a running command in virt-rescue by press ^C or other keys, the only way is to exit virt-rescue
1152821 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152821
Can not end a running command in virt-rescue by press ^C or other keys, the only way is to exit virt-rescue
1152825 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152825
virt-rescue --selinux can not work well, when enable selinux in the command line the value of 'getenforce' is still Disabled in virt-rescue appliance
1152832 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152832
virt-rescue --selinux can not work well, when enable selinux in the command line the value of 'getenforce' is still Disabled in virt-rescue appliance
1152833 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152833
llz give ? for the security context of a file, which is wrong
1153702 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1153702
Using "upload - XXX" doesn't work with remote
1154408 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1154408
libguestfs does not allow registering a qemu+tcp backend
1155229 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1155229
RFE: Allow uploading virt-v2v converted images directly to Cinder
1159030 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1159030
libvirt backend does not set RBD password
1161019 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1161019
Remove rhev related files(rhsrvany.exe and rhev-apt.exe) after conversion of windows guests to kvm
1164708 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164708
set-label can only set <=127 bytes for btrfs and <=126 bytes for ntfs filesystem which not meet the help message. Also for ntfs it should give a warning message when the length >128 bytes
1164729 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164729
set-label can only set <=63 bytes for btrfs and ntfs filesystem which not meet the help message. Also for btrfs and ntfs it should give a warning message when the length exceed the limited length
1164765 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164765
[RFE] Add a new API to scan devices for a btrfs filesystem
1164766 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164766
[RFE] Add a new API to scan devices for a btrfs filesystem
1166057 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1166057
btrfs filesystem will not work well if you create the filesystem with multiple disks at the same time, such as: mkfs-btrfs "/dev/sda1 /dev/sdb1"
1167623 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1167623
Remove "If reporting bugs, run virt-v2v with debugging enabled .." message when running virt-p2v
1167916 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1167916
P2V error:unexpected end of file waiting for password prompt.
1167921 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1167921
p2v: No Network Connection dialog
1167922 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1167922
p2v: No virtual console
1168144 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1168144
warning: fstrim: fstrim: /sysroot/: FITRIM ioctl failed: Operation not supported (ignored) when convert win2003 guest from xen server
1168626 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1168626
Tracker: Build RHEL 7.1 - based virt-p2v ISO
1168865 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1168865
warning: /files/boot/grub/device.map/hd0 references unknown device "vda" when convert a local guest with virtio disk to openstack server (-i disk))
1170771 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1170771
virt-sparsify fails to sparsify CentOS 7 SWAP
1171654 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1171654
Modify a file in virt-rescue with vi on some linux terminal such as yakuake, can lead to abnormal display in virt-rescue shell
1172425 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1172425
[RFE]virt-v2v failed to convert VMware ESX VM with snapshot
1172659 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1172659
virt-resize --expand fails on ubuntu-14.04.img image (regression)
1172660 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1172660
virt-resize --expand fails on ubuntu-14.04.img image
1173462 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1173462
virt-resize on RHEL 5.7 fails to resize the raw format disk image
1173695 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1173695
Error occurred when install a rhel guest with disk pool
1174073 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1174073
The listen address for vnc is changed after conversion by virt-v2v
1174200 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1174200
New virt-v2v failure: CURL: Error opening file: NSS: client certificate not found (nickname not specified): Invalid argument
1174551 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1174551
"lstatnslist" and "lstatlist" don't give an error if the API is used wrongly
1176493 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1176493
virt-v2v ignores sound device when convert xen guest to local kvm
1176526 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1176526
virt-resize cannot shrink ext filesystem by a small amount
1176801 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1176801
File /etc/sysconfig/kernel isn't updated when convert XenPV guest with regular kernel installed
1177460 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1177460
virt-customize does not work with fedora 21 atomic
1177910 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1177910
Libguestfs fails to locate the qemu-kvm on POWER8(BE) host.
1179576 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1179576
virt-v2v prints warning: /files/etc/fstab/8/spec references unknown device "hdb"
1180170 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1180170
virt-builder gives GPG warning message with gnupg2
1180769 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1180769
Security context on image file gets reset
1183493 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1183493
RFE: virt-rescue: add a flag to autorun the suggested mount commands and chroot, suggested with --suggest
1183780 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1183780
guestfs needs README.fixed to actually work
1184762 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1184762
fuse test deadlocks on Rawhide
1186800 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1186800
virt-v2v should support gzip format ova as input
1186850 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1186850
Update RHEV APT.exe
1186935 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1186935
libguestfs cannot inspect recent Fedora / RHEL >= 7 when /usr is a separate partition
1187231 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1187231
rhsrvany crashes when run as a service
1188553 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1188553
Remove files in package libguestfs-bash-completion, these files are bash completion files, some of the virt tool completion are already implement in another file, so can remove its completion file
eprintf(f_"%s: no plan could be found for making a disk image with\nthe required size, format etc. This is a bug in libguestfs!\nPlease file a bug, giving the command line arguments you used.\n")prog;
exit1
error(f_"no plan could be found for making a disk image with\nthe required size, format etc. This is a bug in libguestfs!\nPlease file a bug, giving the command line arguments you used.");
eprintf(f_"%s: no guest operating systems or multiboot OS found in this disk image\nThis is a failure of the source repository. Use -v for more information.\n")prog;
exit1in
(* Set the random seed. *)
msg(f_"Setting a random seed");
ifnot(Random_seed.set_random_seedgroot)then
eprintf(f_"%s: warning: random seed could not be set for this type of guest\n%!")prog;
(* Set the hostname. *)
(matchhostnamewith
|None->()
|Somehostname->
msg(f_"Setting the hostname: %s")hostname;
ifnot(Hostname.set_hostnamegroothostname)then
eprintf(f_"%s: warning: hostname could not be set for this type of guest\n%!")prog
eprintf(f_"%s: %s: command exited with an error\n")progdisplay;
exit1
error(f_"no guest operating systems or multiboot OS found in this disk image\nThis is a failure of the source repository. Use -v for more information.")
@@ -146,28 +137,24 @@ and verify_detached t filename sigfile =
ift.check_signaturethen(
matchsigfilewith
|None->
eprintf(f_"virt-builder: error: there is no detached signature file\nThis probably means the index file is missing a sig=... line.\nYou can use --no-check-signature to ignore this error, but that means\nyou are susceptible to man-in-the-middle attacks.\n");
exit1
error(f_"there is no detached signature file\nThis probably means the index file is missing a sig=... line.\nYou can use --no-check-signature to ignore this error, but that means\nyou are susceptible to man-in-the-middle attacks.\n")
eprintf(f_"virt-builder: error: GPG failure: could not verify digital signature of file\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!\n");
exit1
);
ifr<>0then
error(f_"GPG failure: could not verify digital signature of file\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!");
eprintf(f_"virt-builder: error: fingerprint of signature does not match the expected fingerprint!\n found fingerprint: %s\n expected fingerprint: %s\n")
eprintf(f_"virt-builder: error: could not run sha512sum command to verify checksum\n");
exit1
);
ifr<>0then
error(f_"could not run sha512sum command to verify checksum");
letcsum_actual=read_whole_filecsum_filein
@@ -230,8 +194,6 @@ let verify_checksum t (SHA512 csum) filename =
else
csum_actualin
ifcsum<>csum_actualthen(
eprintf(f_"virt-builder: error: checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!\n")
csum_actualcsum;
exit1
)
ifcsum<>csum_actualthen
error(f_"checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
