Since systemd 209, all the functionalities of the former libsystemd-*
(including libsystemd-journal) have been merged into a single
libsystemd, with the former libraries left as compatibility ones.
Thus, first look for libsystemd, and if not found try again with the
libsystemd-journal as used before.
With commit a594b7f90a the checks for
input and output files with ':' were removed; while the input file is
handled correctly by the disk_create API, an output filename with ':'
could still be handled as specifying a transport, failing the last
"qemu-img convert" call.
Use the new qemu_input_filename helper to handle the output filename
correctly.
It mimics a bit in OCaml the logic used in
commit a95214b198 (for the main library)
and commit 588af1953e (in disk-create),
so in OCaml tools direct calls to qemu/qemu-img with filenames provided
by the user can properly handle filenames with e.g. ':'.
Since mkfifo, mknod_b, and mknod_c add the correct file type to the
modes of the resulting file, make sure the specified mode contains only
permissions bits.
blkid(1) (or actually, libblkid) seems to handle filesystem labels up
to 127 characters. Considering that btrfs labels can be up to 128
characters, this means long labels are not read correctly (i.e. get
truncated) by blkid. Furthermore, ntfs labels are actually unicode,
and libblkid seems to not decode them correctly.
Hence, if ntfsprogs is available invoke `ntfslabel` to get the label
of ntfs filesystems.
Related to RHBZ#1164708.
blkid(1) (or actually, libblkid) seems to handle filesystem labels up
to 127 characters. Considering that btrfs labels can be up to 255
characters, this means long labels are not read correctly (i.e. get
truncated) by blkid.
Get the filesystem type, and if btrfs is available invoke
`btrfs filesystem` to get the label of btrfs filesystems.
Related to RHBZ#1164708.
Add the 'label' optional argument to the mkfs action, so it is possible
to set a filesystem label direct when creating it. There may be
filesystems not supporting changing the label of existing filesystems
but only setting it at creation time, so this new optarg will help.
Implement it for the most common filesystems (ext*, fat, ntfs, btrfs,
xfs), giving an error for all the others, just like set_label does.
Obviously no one reads the FAQ, so make the error message
even clearer.
The new error messages are variation on this:
libguestfs: error: /bin/false exited with error status 1.
To see full error messages you may need to enable debugging.
Do:
export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again. For further information, read:
http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.
The test assumed (for no particular reason) that it could always run a
KVM guest. When run on a machine without KVM, or in a VM without
nested KVM, you get this error:
libvirt: Domain Config error : internal error: no supported architecture for os type 'hvm'
Traceback (most recent call last):
File "./t/820-rhbz912499.py", line 77, in <module>
dom = conn.createXML (xml, libvirt.VIR_DOMAIN_START_AUTODESTROY)
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3523, in createXML
if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
libvirt.libvirtError: internal error: no supported architecture for os type 'hvm'
FAIL: ./t/820-rhbz912499.py
Use a qemu guest instead, which should have a better chance of working
on all hosts.
Builds of bison-generated files currently fail with:
index-parse.y: In function 'yyparse':
index-parse.c:1256:6: error: assuming pointer wraparound does not occur when comparing P +- C1 with P +- C2 [-Werror=strict-overflow]
if (yyss + yystacksize - 1 <= yyssp)
^
This error only occurred on one machine, and it was unclear why. It
turned out to be because the other build machines I use had:
CFLAGS="-Wall" ./configure ...
whereas on the failing machine I was just using ./configure w/o CFLAGS.
This caused the compile flags of index-parse.c to be changed from:
gcc -Wstrict-overflow ... index-parse.c
to:
gcc -Wstrict-overflow -Wall ... index-parse.c
but since -Wstrict-overflow is short for -Wstrict-overflow=2, and
-Wall is short for -Wstrict-overflow=1, and since the error only
occurs at level >= 2, using CFLAGS="-Wall" was effectively turned off
the warning.
Anyway, I believe the optimizations that GCC does when
-fstrict-overflow is enabled to be dubious, frequently dangerous, and
useless. They are only done because we want C99 to work on
architectures that don't exist in reality (and which even if they did
exist, libguestfs would probably not support).
The Linux kernel uses -fno-strict-overflow, and we should too, for the
same reasons.
Normally this requires the program to have at least one C source file.
virt-sysprep doesn't have this, but it uses C sources from other
directories, and therefore we can make it work.
This returns the C pointer to the guestfs_h handle (ie. guestfs_h *).
The purpose of this API is to interwork with other non-C libraries.
See RHBZ#1075164 for the rationale when applied to another library.
This allows the Perl binding of guestfs_add_libvirt_dom to work.
This relies on the internal Perl representation of virDomainPtr, but
Dan assures me this isn't going to change.
Commit 96158d42f5 enabled the previously
private guestfs_add_libvirt_dom API. It also tried to enable the
existing test for this API, but failed to do that correctly. Also the
test was broken. Fix all of this.
This fixes commit 96158d42f5.
YACC index-parse.c
index-parse.y:97.34-41: warning: type <fields> is used, but is not associated to any symbol [-Wother]
%destructor { field_free ($$); } <fields>
^^^^^^^^
index-parse.y:95.36-45: warning: type <sections> is used, but is not associated to any symbol [-Wother]
%destructor { section_free ($$); } <sections>
^^^^^^^^^^
We had code (added for RHBZ#912499) which labels overlay files
correctly so that libvirt can read them. Unfortunately this code was
broken by subsequent commits: the new backend setting for the
imagelabel is only copied to the 'data' struct during launch, but the
create_cow_overlay callback is called before launch (when adding
drives).
The fix is easy: ensure create_cow_overlay_libvirt checks for the
backend setting and initializes the 'data' struct.
This change also means we need to free (data->selinux_imagelabel)
before setting it in launch (and we do the same for
data->selinux_label, although that's not strictly necessary).
For background on this, see:
https://bugzilla.redhat.com/show_bug.cgi?id=912499#c10
This allows the Python binding of guestfs_add_libvirt_dom to work.
There is a regression test to ensure this keeps working.
Note this requires a patched libvirt-python, supporting the
c_pointer() method.
This API already existed (as guestfs___add_libvirt_dom), and was used
by a few tools.
This commit changes it to a public API.
Note that for reasons outlined in the previous commit message, it is
impossible to call this from guestfish or from non-C language
bindings.
This implements Pointer arguments properly, at least for certain
limited definitions of "implements" and "properly".
'Pointer' as an argument type is meant to indicate a pointer passed to
an API. The canonical example is the following proposed API:
int guestfs_add_libvirt_dom (guestfs_h *g, virDomainPtr dom, ...);
where 'dom' is described in the generator as:
Pointer ("virDomainPtr", "dom")
Pointer existed already in the generator, but the implementation was
broken. It is not used by any existing API.
There are two basic difficulties of implementing Pointer:
(1) In language bindings there is no portable way to turn (eg.) a Perl
Sys::Virt 'dom' object into a C virDomainPtr.
(2) We can't rely on <libvirt/libvirt.h> being included (since it's an
optional dependency).
In this commit, we solve (2) by using a 'void *'.
We don't solve (1), really. Instead we have a macro
POINTER_NOT_IMPLEMENTED which is used by currently all the non-C
language bindings. It complains loudly and passes a NULL to the
underlying function. The underlying function detects the NULL and
safely returns an error. It is to be hoped that people will
contribute patches to make each language binding work, although in
some bindings it will always remain impossible to implement.
This didn't work at all because the regular expression did not match
the returned Augeas path.
In future if the regular expression doesn't match, this will give an
internal error instead of continuing with a bogus value.
Thanks: Junqin Zhou for providing the test case and debug information.
Regular EFI disks have /EFI on the first (VFAT) partition, but they
are not installers.
Fix this by only considering something to be an installer if it has a
single partition (or whole disks like ISOs).
Implementing this is quite complex since when checking a filesystem,
we don't have information about whether we are even looking at a
partition, nor about whether it's the first partition out of how many.
The majority of the commit is changing the code to collect that
information.
Because of the size of an extended partition reported by Linux is always 1024
bytes, so it will always fail to copy to an extended partition.
This patch fixes this problem by copying to the offset of an extended
partition in the destination disk.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Wait up to 10 seconds for qemu-nbd to start up and respond to clients.
Otherwise the conversion server may attempt to connect before qemu-nbd
is ready to serve it.
After a very long and trying episode with a Windows guest that refused
to boot after conversion, we managed to successfully boot it by
disabling Windows Group Policy. It appears that Group Policy
prevented the virtio driver from being used.
Document this in the manual.
Prior to this commit the distro for a FreeBSD or a NetBSD system, in
conjuction to what happened for OpenBSD, was shown as 'unknown'.
*BSDs are complete OSes, not a kernel like Linux, but theoritically you
could have FreeBSD as ostype and PC-BSD as distro.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Take into granted that partition 'c' has a special purpose. It has
always the same size as the enclosing slice and is not mapped under
Linux.
This is a best effort try. The mapping will be incorrect if there is a
gap in the disklabel partitions sequence, e.g. 'b' (swap) partition is
missing but 'd' partition is defined.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Make a best effort try to map NetBSD disklabel partitions to Linux
partitions. The mapping will be incorrect if there is a gap in the
disklabel partitions sequence, e.g. 'b' (swap) partition is missing but
'e' partition is defined.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
This patch lets guestfish show command synopsis if the syntax of command issued
by user is wrong, rather than telling user that the number of parameters is wrong.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
The assumption that Linux will map the MBR partition to /dev/sda1
and the BSD 'a' partition to /dev/sda5 is not always correct.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Because get-uuid looks more like a counterpart to set-uuid, and is
more likely come to mind if one knows set-uuid.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Just like --long-options, it makes it possible to know which short
options are supported by each tool; this can help improving the bash
completion, for example.
As contrast to shortdesc which is forced to begin with lowercase, this
patch forces longdesc to begin with uppercase.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Commit 588af1953e started with a very
conservative list of permitted backing formats (just "raw" or
"qcow2"). We can allow almost any format permitted by qemu, but this
commit just adds "vmdk" to this whitelist.
Include the version of virt-p2v and its MD5 checksum in the source
kickstart. This is useful for working out which version of virt-p2v
the kickstart corresponds to.
This relies on the remote to keep sending us data. If it hangs, then
the cancel button won't work. This could also be fixed by introducing
a timeout to the read syscall.
I was fetching cancel_button from the wrong dialog (conv_dlg, not
run_dlg), and therefore getting the wrong button.
This explains why the back button on the conversion dialog was always
insensitive. It's because cancel_button was assigned to this button,
and then I was setting cancel_button to insensitive.
It did nothing and was left over from an earlier attempt to add
fullscreen mode. That is not needed any longer since we now run
virt-p2v under the matchbox window manager.
The <target dev> attribute in libvirt isn't very informative. What we
really want to know is whether the source disk used IDE or SCSI, as
that allows us to remap block devices accurately during conversion.
For example, if the source was IDE and RHEL 5, and after conversion
virtio will be supported, then we know that we need to remap
"/dev/hda" to "/dev/vda".
Drop the s_target_dev and s_removable_target_dev fields and replace
them with s_controller and s_removable_controller.
For -i libvirt, use the <target bus> attribute to get this information.
For -i ova, use the OVF disk controller's ResourceType.
(http://blogs.vmware.com/vapp/2009/11/virtual-hardware-in-ovf-part-1.html)
If /etc/favicon.png is a symbolic link, follow it.
Unfortunately RHEL 7 and Fedora have crappy 16x16 /etc/favicon.png
symlinks in the base distro. It would be nice to ignore this symlink,
but it's almost impossible to determine if the symlink is part of the
base distro or was added by the user. (This is a bug in those
distros.) virt-inspector and virt-mananger both ignore favicons.
Commit fd70cdc3c5 passes the -annot flag
to the compiler, generating *.annot files (containing type information
used by emacs and IDEs). Remove these files when doing 'make clean'.
Use AS_CASE to properly map from the host cpu to the qemu cpu suffix;
this avoids doing multiple string replacements using sed, and it is
possible to use case-like syntax.
Hardcoding -ltinfo breaks on distros that do not build the sep library
(which is the default ncurses behavior). Use pkg-config to look up the
right libraries regardless of how the distro built things.
The labels 'hw_disk_bus' and 'hw_vif_model' were swapped. This
happened to work for virtio guests. For non-virtio guests you ended
up with absurdities like requiring an rtl8139 driver for the disk.
Thanks: Junquin Zhou
Previously we used to try to find the 'vmlinuz' file by running 'rpm
-ql kernel' and looking for any file called 'vmlinuz-*'.
If there were multiple 'kernel' packages installed, the rpm command
would list files from all of them, resulting in a random 'vmlinuz-*'
being chosen, not necessarily the one corresponding to the kernel
package we were looking at.
Use 'rpm -ql kernel-<VERSION>-<RELEASE>' instead so that we only look
for files in the right kernel package.
Thanks: James Mighion
In tools except guestfish, we don't use ncurses, or even curses, just
2 termcap functions and a global variable. These are provided by
libtinfo, so just link to that.
In guestfish we use readline which needs ncurses. Leave that alone.
md5sum(1) does not exist everywhere, so wrap it in an own function so
the right implementation can be chosen on each OS. Also, wrapping it
avoid using awk everytime.
Query RbConfig::CONFIG for "libdir", and use that when trying to link to
the ruby library. This fixes the libruby detection when it is installed
in a non-standard library directory.
Create an autoconf macro to deal with the possible combinations of names
and versions of the libdb tools. This also eases the addition of a libdb
version, or a new pattern for the names of tools.
There is no functional change.
Use a %top block for including <config.h>, so it is included prior to
any other include, even for system ones. This makes sure its definitions
are properly used in gnulib replacement headers.
Do a configure check to see whether UNIX_PATH_MAX is defined; if it is
not defined, provide it with the size of sun_path in the struct
sockaddr_un, thus with the proper size for any libc.
Use CLEANFILES to remove the generated symlinks, and just ignore the
errors when copying the scripts in srcdir!=builddir situations.
Followup of commit 40ac54829d.
If a script does not finish, hangs, etc, it would be executed again at
the next boot, since the injected firstboot.sh helper removes it only
after it finished.
Before executing a script, move it to another internal subdirectory
(scripts-done) and execute it from there, so it is not run again by
firstboot.sh. The downside is that now scripts are executed only once
at all, so if a script fails it will not be attempted at the next boot.
Also, remove all the files found in scripts-done, as they have been run
(or at least attempted) in a previous boot.
This fixes RHBZ#1159651.
This adds a customize option:
virt-customize --ssh-inject USER
virt-customize --ssh-inject USER:string:KEY_STRING
virt-customize --ssh-inject USER:file:FILENAME
(ditto for virt-builder and virt-sysprep)
In each case this injects into the guest user USER
a) the current (host) user's ssh pubkey
b) the key specified as KEY_STRING
c) the key in FILENAME
adding it to ~USER/.ssh/authorized_keys in the guest.
For example:
virt-builder fedora-20 --ssh-inject root
will add the local user's ssh pubkey into the root account of the
newly created guest. Or:
virt-customize -a disk.img \
--ssh-inject 'mary:string:ssh-rsa AAAA.... mary@localhost'
adds the given ssh pubkey to mary's account in the guest.
This doesn't set the SELinux labels correctly on newly created files
and directories, so you have to use --selinux-relabel (probably we
should fix this as part of the general effort to fix SELinux
relabelling). However it should preserve the labels if the
~/.ssh/authorized_keys file already exists.
Most of this work is based on a patch sent to the mailing list by
Richard W.M. Jones <rjones@redhat.com>:
https://www.redhat.com/archives/libguestfs/2014-November/msg00000.html
Since the daemon has long since used the same build system as the rest
of libguestfs, it no longer needs its own gnulib.
This arcane bit of code was left over from
commit e05ddc70f1 (added 2009-08-03 !)
Implement the GUESTFS_ADD_DRIVE_OPTS_SECRET argument of
guestfs_add_drive_opts. For libvirt we have to save the secret in
libvirtd first, get a UUID, and then pass the UUID back through the
domain XML.
On filesystems whose dirent.d_type is DT_UNKNOWN or some unknown value,
manually check whether an entry is a directory, thus completing in the
proper way.
Instead of passing the (uri, key, proxy) tuple around, pass the whole
Sources.source record; this requires creating proper Sources.source also
for uri+fingerprint passed via command line.
No functional changes.
Instead of making up an ordinary curl command line, write a temporary
config file and use 'curl --config tmpfile' to pass the arguments.
The advantage is that it's more secure if we want to supply passwords
to curl, since a '--user username:password' parameter on the command
line could be read (eg. by 'ps ax'), but the temporary file has mode
0600 and cannot be read by other users.
This is mostly code motion, but it also passes the '-q' option to curl
to stop it from reading default configuration files.
Must NOT use:
- One of the interim versions of virt-v2v (1.27.x) that we published
during development.
- Old virt-v2v (0.9.x).
Also use the --version option (instead of -V) since old virt-v2v
required it:
$ virt-v2v -V
Option v is ambiguous (version, vmtype)
Usage:
[...]
$ virt-v2v --version
0.9.1
Include the usb-storage kernel module, to permit booting from USB
keys.
Rerun dracut to rebuild the initramfs. This is complicated because by
default dracut will try to build an initramfs for the running
(ie. appliance) kernel, which is completely useless. We need to find
the latest installed kernel and rebuild the initramfs for that
instead, which is hairy.
This also does a small refactoring of the distro-specific code,
removing the $selinux_relabel variable and replacing it with a generic
$extra_args variable.
I only added this to the virt-builder script, since one assumes that
livecd-creator/whatever should make the kickstart ISO bootable by
installing whatever drivers and modules are necessary.
Running 'dracut -f' installs a dracut for the running kernel (of the
appliance) not the installed kernel. Stupid stupid stupid.
This reverts commit 7a6d44ebfe.
I only added this to the virt-builder script, since one assumes that
livecd-creator/whatever should make the kickstart ISO bootable by
installing whatever drivers and modules are necessary.
Add common error/warning/info functions.
Examine every existing call to printf/eprintf and change where
necessary so that:
- error is used instead of eprintf + exit 1
- warning no longer needs ~prog argument (it is added by Utils module)
- any verbose output should go to stdout, not stderr
- info is used to print general informational messages
Also, don't indent summary of changes.
Add a Customize_utils module. This contains common error/warning/info
functions, and also quote = Filename.quote.
Examine every existing call to printf/eprintf and change where
necessary so that:
- error is used instead of eprintf + exit 1
- warning no longer needs ~prog argument (it is added by Utils module)
- any verbose output should go to stdout, not stderr
- info is used to print general informational messages
Also, don't pass ~prog parameter around. Instead we just get it from
the executable name.
Add a Utils module. This contains common error/warning/info functions,
and also quote = Filename.quote.
Examine every existing call to printf/eprintf and change where
necessary so that:
- error is used instead of eprintf + exit 1
- warning no longer needs ~prog argument (it is added by Utils module)
- any verbose output should go to stdout, not stderr
- info is used to print general informational messages
get_current_dir_name exists on GNU libc only, so not portable.
On the other hand, while POSIX leaves a null buffer argument for getcwd
as unspecified behaviour, basically the most used/important Unix
implementations (GNU libc, FreeBSD's libc, etc) allow such value,
returning a new allocated buffer with the current directory.
In any case, the change just affects two tests, so even if it hits a
libc not implementing this behaviour for getcwd, only tests are
affected.
uri-c.c and perl_edit-c.c end up (directly or indirectly) including
guestfs-internal-frontend.h, which in turns includes libvirt/libvirt.h;
hence, make sure to use the libvirt CFLAGS.
Add a IfNotCrossAppliance prereq for tests, so a test using it can only
be run when the appliance is a copy of the running host system; this can
help marking tests which run stuff (usually built in the host) inside
the appliance.
Since the signature of iconv() changes between implementations (the
constness of the second parameter, in particular), make use of the iconv
module of gnulib to handle these potential differences, including an
external (out of libc) iconv implementation.
Previously I modified the parse_libvirt_xml function to get rid of the
awkward 'map_source*' functions, and have the callers map over and
modify the source disks afterwards. However this broke the case where
an "nbd:..." URL was returned by parse_libvirt_xml, since the callers
might try to map this URL (eg. turning it into an absolute path).
This broke virt-p2v specifically.
This commit changes parse_libvirt_xml to return a list of tuples
containing disk information, giving the callers more information with
which to do the mapping.
This fixes commit 3596165282.
Take the opportunity presented by the refactoring in the previous
commit to inline and simplify functions in these input subclasses.
This finally removes the map_source* functions.
This refactors the code into more logical units, based on the previous
commits. So Xen + the input_libvirt Xen code is grouped and moved
into a new module called Input_libvirt_xen_ssh. Similarly VCenter +
input_libvirt vCenter code is moved to Input_libvirt_vcenter_https.
There is no change here, purely code motion.
Previously we fixed RHBZ#1151033 by increasing the cURL readahead
parameter to a large value. Unfortunately this is too large -- and
hence slow -- for the conversion phase, which broke on slow vCenter
servers (RHBZ#1153589).
What we do now is to perform the conversion phase with the default
readahead (2MB) to ensure stability, since performance of the
conversion phase is not critical. Then before copying we change the
readahead to the larger value (64MB) to ensure efficient copying.
This is called before the copying phase to allow the input method to
adjust overlay/backing parameters.
The default implementation does nothing, so this commit has no effect.
Note that it doesn't matter what the number is, as long as it's
different for each disk.
The unique number lets us store extra data in input objects which is
associated with each disk.
Get rid of the awkward map_source* functions, and replace it with
equivalent code which modifies the source disks objects coming back
from Input_libvirtxml.parse_libvirt_xml.
This is just code refactoring. Apart from the order in which certain
tests are done, this should be equivalent to the previous code.
Previously we had one class ('input_libvirt') which handled all three
cases. Now we have one superclass ('input_libvirt') and three
subclasses of that ('input_libvirt_vcenter_https',
'input_libvirt_xen_ssh', 'input_libvirt_other') which handle the three
cases separately.
This is just code motion, and should be functionally equivalent to
what was here before.
Unfortunately udev removed the udevtimeout setting on the kernel
command line. This means the default timeout was being set back to 30
seconds, which is way too low for remote disks.
However it was re-added (2014-07-29) as udev.event-timeout. Set both
to be sure.
Note this requires systemd >= 216.
In full_path, skip the trailing slash in the base directory when
different than "/", as the slash will eventually be added when building
the resulting path.
Make sure to output the field for checksum even for non-regular files,
as empty, in CSV output mode. This ensures each line has the same number
of fields, regardless of the file type.
OVA is not a particularly well-specified format. The specification
allows a directory to be an OVA, so enable that. The spec doesn't
mention that ZIP can be used in place of tar, but since we have seen
these in the wild, allow that too.
The way to fix these bugs -- for future reference -- is as follows:
(1) Install the corresponnding Windows version with virtio enabled.
See:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/chap-KVM_Para_virtualized_virtio_Drivers.html
(the ".vfd" method)
(2) Download the SYSTEM and SOFTWARE registries from the
virtio-enabled guest.
(3) Examine each registry using 'hivexsh', especially the keys which
we modify during virt-v2v (see v2v/convert_windows.ml). Remember to
use both the 'ls' and 'lsval' commands in hivexsh.
(4) Download the SYSTEM and SOFTWARE registries from the guest that
fails to boot.
(5) Examine the same registry keys from each registry using 'hivexsh'.
(6) Work out what the differences are, paying particular attention to
quoting and the type of each key (eg. str(1), str(2), etc).
Thanks: Matthew Booth.
There were two large bugs before: (1) It only found one manifest file
(meaning that multi-volume OVAs would be broken). (2) The regular
expression didn't work so it wouldn't check the hash. Bugs (1) + (2)
combined so that the check just didn't do anything at all, which is
why we didn't notice the multi-volume problem.
Actually I suspect we shouldn't be doing this check at all, since I
bet there are OVAs out there with broken SHA-1's.
Ignore an environment variable:
- if an environment variable is an empty string, AND
- if an empty string would have no meaning for that environment
variable (eg. LIBGUESTFS_BACKEND can only take on certain values,
and an empty string is not one of them).
Note that LIBGUESTFS_BACKEND_SETTINGS is meaningful as an empty
string, and for this case I have removed the STRNEQ (str, "") test.
Previously:
$ LIBGUESTFS_BACKEND="" guestfish -a /dev/null run
libguestfs: error: invalid backend:
After this commit:
$ LIBGUESTFS_BACKEND="" guestfish -a /dev/null run
(no error message is printed)
Remove the common 'add_resource' function.
Duplicate that function into code for adding disks, and separate code
for adding removables (floppies & CDROMs).
The removables code should look for all removable devices
(ResourceType 14, 15, 16) in a single query, so that the order doesn't
depend on the order in which we do the queries.
If the user does ^C then this leaves temporary overlay files around
(possibly a bug?). Offer an 'exit' choice to the user which exits
cleanly.
The new message looks like this:
Dual- or multi-boot operating system detected. Choose the root filesystem
that contains the main operating system from the list below:
[1] /dev/sda3 (Fedora release 20 (Heisenbug))
[2] /dev/sdb3
Enter a number between 1 and 2, or 'exit': exit
Improve the homebrew JSON writer:
- add more types (including also nested dictionaries and lists)
- format in a compact way (single line), or indented (multilines)
When copying from file to file, don't leave the destination file
around if the copy fails.
However in the same code don't try unlinking the destination device on
failure.
We never updated the no_trim variable in the case where this option
was supplied, so it never did anything, proving once again that unless
you have a test case, the feature won't work.
When translating the original virt-v2v code, I misunderstood how image
directories are created. There is one directory under images/ per
disk image, not one for all the disks in a single guest.
Note this requires that -o vdsm uses multiple --vdsm-image-uuid
options, one per disk image of the guest.
Reported by: Alain Vondra
Thanks: Tingting Zheng
virt-log uses /dev/stdout as output file for guestfs_download, which
will open the output file in truncate mode, truncating the output of
test-virt-log.sh.
Make sure to redirect the output of virt-log so it does not clobber the
test output.
Thanks to Richard for the hint.
Additional information about URIs, which may contain cluster and/or
folder names (thanks: James Mighion).
Replace esx.example.com with vcenter.example.com to reduce confusion
about vCenter vs ESXi.
Since procps 3.3.10, free does not output the "-/+ buffers/cache" line
anymore. On the other hand, the data from it can be calculated from the
memory values, so just do the calculation manually (with awk).
This effectively demotes UML to unsupported status. This has happened
because UML has regressed quite a lot upstream. There are multiple
grave bugs revealed by running the test suite and I don't have the
inclination to fix them.
When using the libvirt backend, don't use the SLIRP. Use
qemu-bridge-helper via libvirt to give us a full network connection.
One consequence of this is that 'ping' works in
'virt-builder --run-command'.
A less useful consequence is that the host firewall will prevent
connections on non-standard ports to the host. So you can't (eg)
connect to a rsync daemon on the host listening on an arbitrary port,
which worked before.
The default bridge is 'virbr0', but you can override this by setting
LIBGUESTFS_BACKEND_SETTINGS=network_bridge=<some_bridge>
Note: this does not fix virt-rescue (since it overrides the default
backend and uses 'direct' for various reasons).
qemu in SLIRP mode offers DHCP services to the appliance. We don't
use them, but use a fixed IP address intead. This changes the
appliance to get its IP address using DHCP.
Note: This is only used when the network is enabled. dhclient is
somewhat slower, but the penalty (a few seconds) is only paid for
network users.
On SuSE or other distros, dhcpcd could be used if available.
The previous tests were very limited, and ran the same two tests
on every run.
Randomly test:
- MBR vs GPT
- Primary, extended partitions
- Expanding vs shrinking
- Use of multiple --resize parameters
- Source and target raw vs qcow2 formats
- Resizing various different filesystem types
- LV expand
- extra partition or no extra partition
To run the test over and over again (useful for finding regressions in
virt-resize), do:
while (cd resize; LIBGUESTFS_TRACE=1 ../run ./test-virt-resize.pl) >&/tmp/log ; do echo -n .; done
If it fails, you can see the failure in /tmp/log.
To rerun a specific test, search for the 'seed:' in the output of the
original test, and specify it on the command line:
(cd resize; ../run ./test-virt-resize.pl --seed=<SEED>)
When saving a configuration file, Augeas creates a new file and
replaces the old one with it; this creates a /etc/shadow file without
the SELinux xattrs, since they are missing.
Thus, create a temporary file with all the attributes of /etc/shadow, so
all the attributes of it (permissions and xattrs, among others) can be
restored properly on the new /etc/shadow.
As side effect, if a guest is already properly SELinux-labelled, then
there should be no more need to relabel it to make sure /etc/shadow
still has the right SELinux xattrs.
This should be equivalent to the previous code, but it makes it easier
for the compiler to pick up errors (eg. missing cases).
This updates commit fc34e2d16c.
Function mbr_part_type returns one of "primary", "extended" and
"logical". The type is used by parted when adding partitions.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
And introduce parameter create_surplus to indicate whether to
create surplus partition or not. Later this parameter will be
used by when calculating positions for target logical partitions.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Old virt-v2v probably emitted the same warning.
However it seems unlikely that /dev/cdrom needs to be remapped, as
udev or the user should be able to supply a suitable symlink from
/dev/cdrom to the removable device. Therefore suppress this warning.
Switch from acl_to_text to acl_to_any_text, so it is possible to specify
options, like forcing the numeric IDs for users/groups. This ensure the
resulting string has always numberic IDs, so the comparison can always
succeed.
Adapt the comparison string to the lack of final endline now.
You can use this to accelerate testing:
$ virt-v2v --no-copy --debug-overlays [etc]
[...]
[ 82.0] Converting Windows 7 Ultimate to run on KVM
This guest has virtio drivers installed.
[ 86.0] Closing the overlay
[ 91.0] Creating output metadata
Overlay saved as /var/tmp/win7-test-sda.qcow2 [--debug-overlays]
[ 91.0] Finishing off
No disk format conversion/copying is done, so it's very fast.
The overlay file is preserved and can be tested directly (it uses the
source file as backing).
It's not really necessary: we have never needed this for debugging
inspection for example, and if hive open failures did occur we would
ask the user to send us the hive files. And it causes huge amounts of
debugging messages to be printed which fills up the log.
Old virt-v2v did not remap this device, so after conversion from Xen
the file would have something like:
boot=/dev/xvda
Although this does not appear to affect anything, better to remap it.
Otherwise you get lots of errors like:
rm: cannot remove ‘/var/tmp/mm/42f5e571-21af-44bd-8164-3ebe6bbc9f5a/images/7b28b0e6-79ce-4d78-a6cb-769760a3d373/31e2b06b-85cb-46ed-aac3-8bcef93dc704.meta’: Permission denied
rm: cannot remove ‘/var/tmp/mm/42f5e571-21af-44bd-8164-3ebe6bbc9f5a/images/7b28b0e6-79ce-4d78-a6cb-769760a3d373/31e2b06b-85cb-46ed-aac3-8bcef93dc704’: Permission denied
because the code is running as root.
For '-o rhev' this is likely to fail (although for obscure reasons it
*doesn't* fail on a real RHEV server, I don't quite understand why).
The reason is because virt-v2v may be running as root, and with root
squashing we don't have permissions to write into the storage domain
directory.
For '-o vdsm', VDSM is supposed to make sure we are running as the
correct user, so there's no reason to doubt things will work.
Make the input/output ordering of command line arguments clearer by
changing:
virt-v2v -i disk -o local -os /var/tmp disk.img
to:
virt-v2v -i disk disk.img -o local -os /var/tmp
We need to write files and directories as user:group 36:36, else
RHEV-M cannot import the VM. Doing this in the presence of NFS is
difficult. See v2v/kvmuid.mli for how it is done in this commit.
Also we allow the output mode to actually create the disk image. This
lets the output mode set ownership and permissions correctly if
required.
The default method just calls Guestfs#disk_create, so there is no
actual change in this commit.
Change Lib_ovf.create_meta_files so that instead of writing the
.meta files, it just returns the content. Output_rhev and
Output_vdsm are correspondingly changed so that they write the
content to the .meta files.
This is just code refactoring, there is no functional change.
The old virt-v2v code would either edit the grub config file (grub
legacy) or use grubby --set-default (grub2).
When translating this code, I figured we could use grubby in both
cases, since grubby exists for grub legacy guests.
However it doesn't appear to work for grub legacy guests. Use the
configuration file editing method for these, same as old virt-v2v.
Thanks: Tingting Zheng
The existing APIs guestfs_stat, guestfs_lstat and guestfs_lstatlist
return a stat structure that contains atime, mtime and ctime fields
that store only the timestamp in seconds.
Modern filesystems can store timestamps down to nanosecond
granularity, and the ordinary glibc stat(2) wrapper will return these
in "hidden" stat fields:
struct timespec st_atim; /* Time of last access. */
struct timespec st_mtim; /* Time of last modification. */
struct timespec st_ctim; /* Time of last status change. */
with the following macros defined for backwards compatibility:
#define st_atime st_atim.tv_sec
#define st_mtime st_mtim.tv_sec
#define st_ctime st_ctim.tv_sec
It is not possible to redefine guestfs_stat to return a longer struct
guestfs_stat with room for the extra nanosecond fields, because that
would break the ABI of guestfs_lstatlist as it returns an array
containing consecutive stat structs (not pointers). Changing the
return type of guestfs_stat would break API. Changing the generator
to support symbol versioning is judged to be too intrusive.
Therefore this adds a new struct (guestfs_statns) and new APIs:
guestfs_statns
guestfs_lstatns
guestfs_lstatnslist
which return the new struct (or array of structs in the last case).
The old APIs may of course still be used, forever, but are deprecated
and shouldn't be used in new programs.
Because virt tools are compiled with -DGUESTFS_WARN_DEPRECATED=1, I
have updated all the places calling the deprecated functions. This
has revealed some areas for improvement: in particular virt-diff and
virt-ls could be changed to print the nanosecond fields.
FUSE now returns nanoseconds in stat calls where available, fixing
https://bugzilla.redhat.com/show_bug.cgi?id=1144891
Notes about the implementation:
- guestfs_internal_lstatlist has been removed and replaced by
guestfs_internal_lstatnslist. As the former was an internal API no
one should have been calling it, or indeed can call it unless they
start defining their own header files.
- guestfs_stat and guestfs_lstat have been changed into library-side
functions. They, along with guestfs_lstatlist, are now implemented
as wrappers around the new functions which just throw away the
nanosecond fields.
This gives us finer control over how system calls are done,
and also potentially lets us test more.
Currently two tests are disabled:
- utimens because of https://bugzilla.redhat.com/show_bug.cgi?id=1144766
- utimes because our stat call does not return the nanosecond fields
This file causes the test ISO to be rebuilt every time.
Instead of including it as a Makefile dependency, create it just
before making the ISO, which has the same effect.
A test in fish/ also used this file. Change it so it doesn't need it.
Add support for aarch64 to the file_architecture API, so we can
detect aarch64 guests correctly.
Add a test binary and library. These were generated on an aarch64
machine by doing:
echo 'main(){}' > bin.c
gcc bin.c -o bin-aarch64-dynamic
strip --strip-all bin-aarch64-dynamic
echo '' > lib.c
gcc -shared lib.c -o lib-aarch64.so
strip --strip-all lib-aarch64.so
Previously we required that the source hypervisor knew the
architecture of the guest, and passed that through to KVM.
However this was error-prone for several reasons:
- OVF (-i ova) doesn't define the architecture
- Disk images (-i disk) don't have an associated architecture
- Libvirt XML sometimes lacks the <type arch=...> field, especially
for RHEL 5-era libvirt.
- It might not be set correctly for the guest.
We know the real architecture from inspection of the guest, so use
that instead.
The escape sequence \e[0;30m sets the colour to black. This is fine
if you use a black-on-white terminal, but fails rather badly if you
use a white-on-black terminal.
Instead use the escape sequence \e[0m which restores the terminal to
the defaults, and works in both cases.
If the RPM database in the source guest is corrupt, then subsequent
operations will fail randomly.
Old virt-v2v appears to have avoided this by cleaning the RPM database
very early on. When translating the code I inadvertantly moved this
cleanup to a later stage -- in particular, after inspection of kernels
was done which requires running rpm commands. Move this back to the
start.
When chowning the final directories, instead of chowning them to 36.36
(ie. hard-coding those UID/GIDs), use a --reference chown so it should
take the ownership from the parent directory.
Unfortunately chowning is very complex because of:
- NFSv3 vs NFSv4
- root squash
- idmap
which creates approximately 8 different ways for this to fail.
When presented with a RHEL 7 guest, virt-v2v would print the following
warning message:
virt-v2v: warning: could not update grub2 console: aug_get: no matching
node (ignored)
This happened because on RHEL 7, /etc/sysconfig/grub is a symlink to
/etc/default/grub, and the Augeas grub2 lens returns entries under
/files/etc/default/grub/*, but the code was checking for
/files/etc/sysconfig/grub/*.
The fix is to check all the possible Augeas paths until one matches.
(Only the first match gets updated however).
Also, don't rebuild the grub2 config unless we've actually changed it.
This avoids doing a potentially failure-prone operation when it's not
necessary.
I suspect that the old virt-v2v code did not work correctly for RHEL 7
guests, but old virt-v2v didn't print a warning here, it just failed
to update silently.
All tests run under the ./run binary. For a long time the ./run
binary has set the $PATH environment variable to contain all of the
directories with binaries in them.
Therefore there is no reason to use ../fish/guestfish instead of just
plain guestfish (and the same applies to other built binaries).
If do_part_get_parttype returns NULL, then return from the function
instead of dereferencing the NULL pointer and crashing. Note that
do_part_get_parttype has already called reply_with_error in this case.
This fixes commit 0316d24770.
Found by: Lingfei Kong
Old virt-v2v rebuilt the initrd late. Currently we rebuild the initrd
right after choosing a kernel, but it's best to update the initrd
at the end since mkinitrd may depend on other configuration files
that we are changing (especially see next commit).
The old virt-v2v code had:
my $prefix;
if ($virtio) {
$prefix = 'vd';
} elsif ($libata) {
$prefix = 'sd';
} else {
$prefix = 'hd'
}
The translated code dropped the test for virtio by accident, which
meant that device names would be mapped to sd* or hd* even if the
virtio-blk driver was enabled for the guest.
Thanks: Tingting Zheng
For Xen guests, a device can be presented twice as (eg) xvda and sda,
and therefore we should map any otherwise unmapped xvdX.
For non-Xen guests, this should have no effect.
I made a mistake while translating the Perl conversion code and
transcribed --set-default as --set-kernel. grubby has no --set-kernel
option so it failed.
Thanks: Tingting Zheng
Previously only "running" domains were blocked from conversion.
However "blocked" is essentially the same as running (on Xen it just
means the domain is running but waiting on an I/O resource). And
"paused" domains probably shouldn't be converted either since their
disks are only crash-consistent.
Don't rely on libvirt's implicit print-to-stderr. Instead make sure
that libvirt error messages are raised in the exception message.
Also allow error strings to be translated.
When you call close on any block device, udev kicks off a rule which
runs blkid to reexamine the device. We need to wait for this rule to
finish running since it holds the device open and can cause other
operations to fail, notably mkfs.
(cherry picked from commit a9c8123c72)
Estimate the amount of space required on the target by each source
disk after conversion. The estimate is approximately a ceiling on
this, and actual use should be considerably lower.
Provide an extra output method (#check_target_free_space) which output
modules may use to check that there is sufficient free space to
proceed, before conversion starts. None of the output modules
actually implement this at the moment.
In most C tools, virt-sysprep and virt-customize, you have to put the
--format parameter before the corresponding -a parameter. ie. The
following is correct:
guestfish --format qcow2 -a disk1 -a disk2
But the following is incorrect. The --format parameter is dangling
and prior to this commit would have been silently ignored:
guestfish -a disk1 -a disk2 --format qcow2
After this change, dangling --format parameters now lead to an error:
guestfish: --format parameter must appear before -a parameter
In virt-customize, also check that --attach-format parameter appears
before --attach parameter.
Thanks: Lingfei Kong
If there is an authentication error when getting the session cookie,
emphasize in the error message that the username can be specified in
the URI.
Unfortunately libvirt asks for the username previously, but we cannot
access that.
In theory this discards the output. Unfortunately in practice we have
to write the output to a temporary file and delete it (because of
limitations in qemu-img convert).
You will see an error like this:
$ virt-sysprep --remove-user-accounts foo,bar -a /dev/null
virt-sysprep: error: user-accounts: --remove-user-accounts parameter was
used, but the "user-account" operation is not enabled
The --user-accounts option, with its double-negative '-' prefix on
user names, is confusing. Replace it with '--remove-user-accounts'
and '--keep-user-accounts' options.
This updates commit 128d474095.
-o rhev: This is "traditional" output to a RHEV-M Export Storage Domain.
It requires no cooperation from RHEV itself, and in all other respects
works the same way as old virt-v2v did.
-o vdsm: This is the new mode which requires VDSM cooperation. VDSM
manages the conversion, creating the directories, suggesting UUIDs,
and cleaning up on failure. This allows direct import into a Data
Domain.
The --rhev* command arguments now only apply to -o vdsm, and therefore
have been renamed to --vdsm*
The --vmtype command line argument may be used with -o rhev or -o vdsm.
Also give an error if users use the incorrect one:
$ virt-builder --list --format json
virt-builder: virt-builder --list: use '--list-format', not '--format'.
The default charset for these filesystems depends on how the kernel was
configured with, so explicitly set the default one (as specified in
Linux sources), eventually setting later a new one for the tests.
Previously the core virt-v2v code had to know which Convert_* module
to call for each guest type (based on inspection). After this change
the core code doesn't need to know that, but instead Convert_* modules
register their interest in particular guest types.
Fedora patches OCaml's Arg module to support --option=arg. Since this
won't work on other distros or upstream OCaml, don't include such
examples in the documentation.
The two adjacent sections can be read separately, and the second
section doesn't mention not using make install / using ./run, so
repeat it a second time.
Before trying to perform the conversion, estimate if there is
sufficient free space on the guest filesystem(s) for the conversion to
be successful.
If not, you will see an error similar to this one:
[ 6.0] Checking for sufficient free disk space in the guest
virt-v2v: error: not enough free space for conversion on filesystem '/'.
4354048 bytes free < 20000000 bytes needed
virt-p2v can run without a window manager. However it then sits in
the top left corner of the screen. Add a lightweight window manager
to the ISO so that position and geometry hints work.
Two scripts are added, replacing the make targets from the
previous commit:
virt-p2v-make-disk: Builds a virt-p2v disk or USB key using
virt-builder.
virt-p2v-make-kickstart: Builds a kickstart file (only) which can be
fed to livecd-creator or similar tools on Red Hat-derived distros.
Kickstart generation is slightly different: Instead of requiring the
downstream 'libguestfs-pvhelper' package, the kickstart now embeds the
virt-p2v binary(!) making it self-contained and downstream packaging
simpler.
In addition there are documentation changes.
As part of reworking the scripts that are used to build the P2V ISO,
first remove the separate p2v-iso directory. Similar scripts and
documentation will be introduced back into the p2v/ directory in
following commits.
Import the upstream lens for the shadow file, just with a different
identifier and not matching /etc/shadow by default. Instead, apply a
transformation to have it match /etc/shadow only if the version of
augeas is at least 1.2.1 [1].
[1] While the last upstream version is 1.2.0, all the development seems
to happen in master, so whatever the next version is going to be
numbered (e.g. 1.2.1 or 1.3.0), the check will be fine anyway.
It will print an error like this:
virt-v2v: error: internal error: invalid argument: libvirt domain 'windows'
is running, it must be shut down in order to perform virt-v2v conversion
This only works for libvirt domains, and is explicitly disabled for
test:/// URIs since these domains are always "running", but we use
these URIs for testing.
Unhelpfully, Filename.chop_extension throws an exception if the
filename has no extension (instead of just returning the filename).
Therefore the -i disk option would fail if you passed any filename
that didn't have an extension.
When setting the password for a locked account, make sure to still write
the password after the "!!" marker, otherwise the account will have no
password.
Previously we only looked for <driver name=qemu type=...>. However
for Xen imports, the name field could be name=tap (or other values in
fact). Since there can only be one <driver/> element under <disk/>,
look for any <driver type=...> in order to get the disk format.
A further complication for Xen is that type=aio means "raw" (for
obscure historical reasons), so map that.
Thanks: Dan Berrangé.
The empty_element macro was not safe when used in a "naked" if
statement such as:
if (config->flags & FLAG_ACPI) empty_element ("acpi");
Fix the macro so it uses do { ... } while (0) to make it safe here.
The new output of virt-v2v --machine-readable now lists the input and
output drivers available in the binary:
$ virt-v2v --machine-readable
virt-v2v
libguestfs-rewrite
input:disk
input:libvirt
input:libvirtxml
output:glance
output:libvirt
output:local
output:rhev
With this information, we can now pre-populate the virt-p2v GUI
combo box.
Instead of having all interfaces connect to the "default" network on
the hypervisor target, you can now create a map of interface to target
network, either on the kernel command line, eg:
p2v.network=em1:rhevm
p2v.network=em1:rhevm,em2:management,other
or through the GUI.
Enhance the user-account sysprep operation, so it is possible to select
which user accounts should be removed only (instead of all), or which to
eventually keep.
Name these the same as the virt-v2v options, so for example
'-oa' -> 'p2v.oa'.
Also add the missing p2v.oc (-oc) option.
This updates commit c516d2d07d.
On Ubuntu, exclude the Ubuntu-only 85-lvm2.rules since it causes the
automatic activation of volume groups on addition or change. This
interferes with libguestfs, as vgs which are disables suddently get
enabled.
ocamldep -all (introduced in commit a3881445ef) creates a dependency
rule
optgroups.cmi ... : utils.cmx ...
Because we never build a native code version of the generator,
utils.cmx can never be remade, and so this results in optgroups.cmi
being always rebuilt and hence rebuilding the whole directory.
It's unclear how to fix this, but reverting the ocamldep -all change
in this directory works around it.
Partially reverts commit a3881445ef.
In v2v, 'stringMap.ml' has no corresponding *.mli file. ocamldep does
not generate a *.cmi dependency for such modules, and this can result
in parallel builds failing rarely because:
(1) stringMap.ml is built (which has the side-effect of generating
stringMap.cmi)
(2) at the same time, types.mli is compiled. types.mli depends on
stringMap.cmi, but it is half way through being created by (1).
There is no dependency to serialize (1) and (2).
The resulting error is:
File "types.mli", line 1:
Error: Corrupted compiled interface
stringMap.cmi
Using 'ocamldep -all' generates extra deps for the *.cmi files, and
otherwise appears to be safe, so use it.
There are currently a few upstream OCaml bugs with dependencies and
parallel builds:
http://caml.inria.fr/mantis/view.php?id=3190http://caml.inria.fr/mantis/view.php?id=4991http://caml.inria.fr/mantis/view.php?id=5000
It's not clear which of these apply here.
The parsing of p2v.disks, p2v.removable and p2v.interfaces did not
correctly end at the first space, and therefore any other parameters
that happened to be on the command line afterwards would be consumed
as a list of disks or interfaces.
In commit 53e0d3ea5f I fixed a cause of
disk corruption in the overlay file during conversion.
To ensure this doesn't occur again unexpectedly, add a simple test to
the virt-v2v code and to the tests.
When saving, resolve the path of the file being edited and use that as
real target to write to. Otherwise, if the file name is a symlink then
it will be replaced by a regular file with the new content, leaving the
old file untouched.
Extend test-edit.sh to check for this situation.
Allow null as value for the editor parameter of edit_file_editor, which
will then get it from the EDITOR envvar (falling back on vi).
This is basically code motion from the two edit_file_editor users to it.
To speed up conversions by storing everything we read during
conversion in the overlay instead of having to reread it from the
remote server during copying.
Thanks: Matthew Booth.
When drives are added with this flag, reads from the backing file are
copied into the overlay, improving performance of reads to the same
area of disk (at the expense of local storage). This is excellent for
reading remote / slow sources like HTTP.
Add the perl file editing, mostly based in the virt-edit implementation.
This introduces a mild code duplication with edit_file_editor; will deal
with it in a later commit.
Do not unconditionally exit if guestfs_case_sensitive_path, but let
windows_path still return null. Make virt-edit then check for that, and
eventually exit on its own.
Move the code handling Windows paths from virt-edit to a common file,
so that can be shared by various tools.
Mostly code motion, with a minimum touch (the additional guestfs_h*
parameter in mount_drive_letter) to make it build and work.
This seems* to cause disk corruption in the overlay files. A
manifestation of this is that the qcow2 header is not written back
correctly to disk, resulting in the backing file of the overlay
"disappearing" and hence hardly any data being copied over to the
target.
* Note this could be for a variety of reasons and doesn't necessarily
indicate a fault in qemu or qcow2.
Previously if you used `-i libvirt -ic [a remote URI]' then the remote
URI would be opened during object creation, which occurs during
command line parsing. The practical effect was that virt-v2v could
appear to hang or give an error (eg if the remote URI is inaccessible)
before printing any log messages. This could be very confusing for
the user.
Split out -i libvirtxml into a separate module, containing the libvirt
XML to source parsing code as a function called
Input_libvirtxml.parse_libvirt_xml.
Modify -i libvirt so it does all initialization and calls the helper
function during the #source method, and not at object creation time.
This was based an incorrect translation of what the old virt-v2v was
doing. %systemroot%\Drivers is not expected to exist already, and
indeed does not exist in a real Windows 7 guest.
Add a mini-binding to libvirt virStoragePoolGetXMLDesc to get the pool
XML directly using the libvirt API. Change existing external calls to
`virsh pool-dumpxml' to use this API instead.
The `virsh dumpxml' command doesn't work properly when the libvirt
source requires authentication. This is because the authentication
prompts are sent to stdout, but stdout is also the place where we are
reading the output XML from.
Add a mini-binding to libvirt virDomainGetXMLDesc which avoids this,
getting the XML directly using the libvirt API. Change existing
external calls to `virsh dumpxml' to use this API instead.
The way to set the filter for lvm devices was to open lvm.conf, look
for uncommented "filter =" lines and replace the configuration there.
This had the issue that if there is no uncommented filter line, then the
filter cannot be changed at all; considering newer lvm2 releases ship a
sample configuration with no uncommented filter lines, then the old way
became wrong and not sufficient.
Instead, take a copy of the upstream lvm.aug lens, with a simple change
to allow parsing also negative values in configuration, and install it
in the daemon. When asking to change the lvm filter, use augeas making
sure to use this custom lens for the lvm.conf copy used within the
appliance.
The C code didn't have -Werror enabled (when configured) and didn't
enable the full set of warnings we would want.
However OCaml-C bindings commonly declare functions which are
called directly from OCaml, eg:
external parse_memory : string -> doc = "v2v_xml_parse_memory"
value v2v_xml_parse_memory (value xmlv)
These do not require prototypes as no other C code will call them, so
we have to switch off the GCC -Werror=missing-prototypes flag for
these files.
Also fixes some confusion between _CPPFLAGS and _CFLAGS.
Move the fields from Types.overlay which were related to the target
file out. Now Types.overlay is only concerned with the actual qcow2
overlay files, not with a mixture of both files.
For every guest disk, there is one chain which looks like this:
source_disk overlay target
+------------+ +----------------+ +---------------+
| s_qemu_uri |<------ ov_source |<------ target_overlay|
| etc | | ov_overlay_file| | target_file |
+------------+ | ov_sd | | target_format |
| ov_virtual_size| +---------------+
+----------------+
describes source describes temp. describes target
disk file overlay (qcow2) file
This is just refactoring.
This function had got smaller and smaller until now we might as well
just inline it. Having it after the main function just obscured the
order that the code runs.
No functional change, just refactoring.
The ov_preallocation field was essentially an implementation detail of
the V2v module. There was no need to carry it around in the overlay
struct, and nothing else used it. This also allows us to simplify
other code.
If the <memory> or <vcpu> elements was missing from the source
metadata then we would choose useless defaults, resulting in (for
example) <vcpu>0</vcpu> being written to the output.
Choose sensible defaults for these integer values.
When we map the network or bridge names (using --network or --bridge
command line options), this commit keeps the original name and adds it
as a comment to the output metadata.
The comment will look something like this:
<interface type='bridge'>
<!-- bridge mapped from "VM Network" to "bridge1" -->
<source bridge='bridge1'/>
<model type='rtl8139'/>
</interface>
Previously Types.input and Types.output were variants, resulting in
the main v2v.ml code having to have knowledge of all the input and
output types. So you would end up with code in v2v.ml like:
match output with
| OutputLibvirt -> (* call some libvirt-specific code *)
| OutputRHEV -> (* call some RHEV-specific code *)
This changes these types to be abstract superclasses, with specific
implementations provided by (and hidden inside) each input/output
module. The above code in v2v.ml now becomes:
output#call ... (* no backend-specific knowledge needed *)
Although this commit is very large, it is just refactoring. There is
no functional change in the code.
This allows `-ic esx://...' to work because we map the strange source
URI used by ESX to a remote HTTPS connection.
Example disk section that this handles:
<disk type='file' device='disk'>
<source file='[datastore1] Windows/Windows.vmdk'/>
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
Instead of the ?dir parameter which was essentially a hack, allow
callers of create_xml to provide functions that map
<source file="..."> or <source dev="..."> to qemu URIs.
The default for these mappings is the identity function.
For -i libvirtxml we replace the old absolute_path_of_disk function
with a source file mapping function which does the same thing.
There is no functional change in this patch.
<source dev=...> in the libvirt XML should always refer to a block
device, eg. /dev/sda1. Don't attempt to make these paths absolute
(they should already be absolute paths, and if they are not it's wrong
to try to make them).
There is already a partial binding for this in fish/uri.c &
mllib/uri-c.c. However it is specialized to parsing the -a parameter
on the command line and we want access to the full underlying
functionality of xmlParseURI.
These are now collected from the source metadata and (where
possible) passed through to the destination.
For reference, see the metadata structure of the old virt-v2v (roughly
equivalent to Types.source in the new code):
https://git.fedorahosted.org/cgit/virt-v2v.git/tree/metadata-format.txt
As you can see the new code now covers all the metadata features of
the old code.
Mostly complete and making most of the features work.
The notable lack is ntfsprogs and ntfs-3g, which due to the way
mount.ntfs is handled (i.e. using alternatives) do not work properly.
This is simpler to use and more convenient than -i libvirtxml, because
users don't need to bother with writing libvirt XML. However it is
less expressive because many source capabilities such as memory and
number of vCPUs cannot be specified this way.
As initializing the target (-o ...) is a distinct operation, which can
fail if the user supplied the wrong command line arguments, make sure
we display an explicit message before we perform this step.
It prints something like this:
[ 5.0] Initializing the target -o libvirt -os default
Since `-o local' and `-o libvirt' both need to generate libvirt XML
they should share the same code to generate it.
Also generate it using the DOM module instead of printing the XML
directly. This is cleaner and avoids quoting issues.
No functional change here.
Notes:
- Drop the acpi feature for RHEL 3. Old virt-v2v says there is
a problem presenting ACPI to RHEL 3 guests although it is not
specific about what this is.
- Old virt-v2v does not pass features through to OVF at all.
This seems to be the new way [1] in Archlinux to specify, as symlink,
the default version of Java.
Thanks to Peter Wu for the notice about it.
[1] https://wiki.archlinux.org/index.php/Java
This warning was meant for the case where /dev/kvm is inaccessible and
the user must add themselves to a special group.
However we didn't take into account whether /dev/kvm was openable in
this test. If it's openable, no point issuing the warning.
Move the is_openable ("/dev/kvm") test earlier, and don't issue the
warning if that was successful.
1151902 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1151902
virt-v2v:Network interface disappear after conversion of guest with interface type direct
1151903 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1151903
virt tools --csv output should have a meaningful output
1151905 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1151905
virt tools --csv output should have a meaningful output
1152369 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152369
[RFE]virt-v2v failed to convert guest with UEFI firmware
1152533 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152533
virt-v2v fail to convert guest with virtio lun disk
1152818 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152818
llz give ? for the security context of a file, which is wrong
1152819 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152819
Can not end a running command in virt-rescue by press ^C or other keys, the only way is to exit virt-rescue
1152821 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152821
Can not end a running command in virt-rescue by press ^C or other keys, the only way is to exit virt-rescue
1152825 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152825
virt-rescue --selinux can not work well, when enable selinux in the command line the value of 'getenforce' is still Disabled in virt-rescue appliance
1152832 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152832
virt-rescue --selinux can not work well, when enable selinux in the command line the value of 'getenforce' is still Disabled in virt-rescue appliance
1152833 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1152833
llz give ? for the security context of a file, which is wrong
1153702 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1153702
Using "upload - XXX" doesn't work with remote
1154408 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1154408
libguestfs does not allow registering a qemu+tcp backend
1155229 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1155229
RFE: Allow uploading virt-v2v converted images directly to Cinder
1159030 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1159030
libvirt backend does not set RBD password
1161019 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1161019
Remove rhev related files(rhsrvany.exe and rhev-apt.exe) after conversion of windows guests to kvm
1164708 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164708
set-label can only set <=127 bytes for btrfs and <=126 bytes for ntfs filesystem which not meet the help message. Also for ntfs it should give a warning message when the length >128 bytes
1164729 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164729
set-label can only set <=63 bytes for btrfs and ntfs filesystem which not meet the help message. Also for btrfs and ntfs it should give a warning message when the length exceed the limited length
1164765 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164765
[RFE] Add a new API to scan devices for a btrfs filesystem
1164766 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1164766
[RFE] Add a new API to scan devices for a btrfs filesystem
1166057 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1166057
btrfs filesystem will not work well if you create the filesystem with multiple disks at the same time, such as: mkfs-btrfs "/dev/sda1 /dev/sdb1"
1167623 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1167623
Remove "If reporting bugs, run virt-v2v with debugging enabled .." message when running virt-p2v
1167916 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1167916
P2V error:unexpected end of file waiting for password prompt.
1167921 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1167921
p2v: No Network Connection dialog
1167922 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1167922
p2v: No virtual console
1168144 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1168144
warning: fstrim: fstrim: /sysroot/: FITRIM ioctl failed: Operation not supported (ignored) when convert win2003 guest from xen server
1168626 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1168626
Tracker: Build RHEL 7.1 - based virt-p2v ISO
1168865 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1168865
warning: /files/boot/grub/device.map/hd0 references unknown device "vda" when convert a local guest with virtio disk to openstack server (-i disk))
1170771 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1170771
virt-sparsify fails to sparsify CentOS 7 SWAP
1171654 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1171654
Modify a file in virt-rescue with vi on some linux terminal such as yakuake, can lead to abnormal display in virt-rescue shell
1172425 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1172425
[RFE]virt-v2v failed to convert VMware ESX VM with snapshot
1172659 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1172659
virt-resize --expand fails on ubuntu-14.04.img image (regression)
1172660 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1172660
virt-resize --expand fails on ubuntu-14.04.img image
1173462 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1173462
virt-resize on RHEL 5.7 fails to resize the raw format disk image
1173695 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1173695
Error occurred when install a rhel guest with disk pool
1174073 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1174073
The listen address for vnc is changed after conversion by virt-v2v
1174200 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1174200
New virt-v2v failure: CURL: Error opening file: NSS: client certificate not found (nickname not specified): Invalid argument
1174551 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1174551
"lstatnslist" and "lstatlist" don't give an error if the API is used wrongly
1176493 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1176493
virt-v2v ignores sound device when convert xen guest to local kvm
1176526 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1176526
virt-resize cannot shrink ext filesystem by a small amount
1176801 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1176801
File /etc/sysconfig/kernel isn't updated when convert XenPV guest with regular kernel installed
1177460 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1177460
virt-customize does not work with fedora 21 atomic
1177910 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1177910
Libguestfs fails to locate the qemu-kvm on POWER8(BE) host.
1179576 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1179576
virt-v2v prints warning: /files/etc/fstab/8/spec references unknown device "hdb"
1180170 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1180170
virt-builder gives GPG warning message with gnupg2
1180769 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1180769
Security context on image file gets reset
1183234 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1183234
part_get_mbr_id test fails: sfdisk --change-id: sfdisk: change-id is deprecated in favour of ---part-type
1183493 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1183493
RFE: virt-rescue: add a flag to autorun the suggested mount commands and chroot, suggested with --suggest
Remove files in package libguestfs-bash-completion, these files are bash completion files, some of the virt tool completion are already implement in another file, so can remove its completion file
part-get-name give 'libguestfs: error: part_get_name: parted does not support the machine output (-m)' error message when run 'part-get-name /dev/sda 1'
eprintf(f_"%s: no plan could be found for making a disk image with\nthe required size, format etc. This is a bug in libguestfs!\nPlease file a bug, giving the command line arguments you used.\n")prog;
exit1
error(f_"no plan could be found for making a disk image with\nthe required size, format etc. This is a bug in libguestfs!\nPlease file a bug, giving the command line arguments you used.");
eprintf(f_"%s: no guest operating systems or multiboot OS found in this disk image\nThis is a failure of the source repository. Use -v for more information.\n")prog;
exit1in
error(f_"no guest operating systems or multiboot OS found in this disk image\nThis is a failure of the source repository. Use -v for more information.")
in
Customize_run.run~prog~verbose~quietgrootops;
Customize_run.run~verbose~quietgrootops;
(* Collect some stats about the final output file.
eprintf(f_"virt-builder: error: could not export public key\nUse the '-v' option and look for earlier error messages.\n");
exit1
);
ifr<>0then
error(f_"could not export public key\nUse the '-v' option and look for earlier error messages.");
ignore(import_keyfilegpgtmpdirverbosefilename);
fp
)else
@@ -148,8 +137,7 @@ and verify_detached t filename sigfile =
ift.check_signaturethen(
matchsigfilewith
|None->
eprintf(f_"virt-builder: error: there is no detached signature file\nThis probably means the index file is missing a sig=... line.\nYou can use --no-check-signature to ignore this error, but that means\nyou are susceptible to man-in-the-middle attacks.\n");
exit1
error(f_"there is no detached signature file\nThis probably means the index file is missing a sig=... line.\nYou can use --no-check-signature to ignore this error, but that means\nyou are susceptible to man-in-the-middle attacks.\n")
eprintf(f_"virt-builder: error: GPG failure: could not verify digital signature of file\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!\n");
exit1
);
ifr<>0then
error(f_"GPG failure: could not verify digital signature of file\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!");
eprintf(f_"virt-builder: error: fingerprint of signature does not match the expected fingerprint!\n found fingerprint: %s\n expected fingerprint: %s\n")
eprintf(f_"virt-builder: error: could not run sha512sum command to verify checksum\n");
exit1
);
ifr<>0then
error(f_"could not run sha512sum command to verify checksum");
letcsum_actual=read_whole_filecsum_filein
@@ -212,8 +194,6 @@ let verify_checksum t (SHA512 csum) filename =
else
csum_actualin
ifcsum<>csum_actualthen(
eprintf(f_"virt-builder: error: checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!\n")
csum_actualcsum;
exit1
)
ifcsum<>csum_actualthen
error(f_"checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!")
error~prog (f_"'%s' is missing. This file is required in order to install Windows firstboot scripts. You can get it by building rhsrvany (https://github.com/rwmjones/rhsrvany). Original error: %s")
error(f_"'%s' is missing. This file is required in order to install Windows firstboot scripts. You can get it by building rhsrvany (https://github.com/rwmjones/rhsrvany). Original error: %s")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.