For input methods which use nbdkit, we can cheaply add
nbdkit-rate-filter to control input-side network bandwidth. These
options control that filter. We can choose to set the bandwidth
statically and optionally change it dynamically:
--bandwidth 10M
# static bandwidth of 10 Mbps, no dynamic adjustment possible
--bandwidth 5M --bandwidth-file /tmp/bw
# initial static bandwidth of 5 Mbps, adjustable by writing to /tmp/bw
--bandwidth-file /tmp/bw
# no initial bandwidth cap, can be added later by writing to /tmp/bw
It only makes sense to control the input side since virt-v2v writes a
lot less data than it reads.
‘virt-v2v -i libvirtxml’ has a little-known feature where it can read
network disks over HTTP or HTTPS. This can be used to test VMware
conversions without needing VMware, although as far as I can tell the
current test suite does not use the feature. This commit changes this
functionality to use nbdkit-curl-plugin instead of the qemu curl
driver.
This change shouldn't affect functionality. The readahead size is
changed from a fixed 1M buffer to using the readahead filter which is
self-configuring.
See also commit 38bf2a0f97 which
originally introduced this functionality in 2017.
The readahead filter is a self-configuring filter that makes
sequential reads faster when the plugin is slow (and all of the
plugins we use here are always slow).
I observed the behaviour of the readahead filter with our qcow2
overlay when converting a guest from a vCenter source. Even when
doing random reads, qemu issues 64K reads which happen to also be the
minimum request size of the readahead filter, so there is no extra
overhead. When doing the sequential copy the readahead filter
performed better than qemu curl’s readahead because it scaled the
prefetched data appropriately on long contiguous stretches and then
shrunk it back to 64K around fragmented parts of the base image.
This uses the technique described in the nbdkit-probing(1) man page.
It should work with a wide range of versions of nbdkit, and is the one
which the nbdkit developers currently recommend.
Initially this is a like-for-like replacement, but in future commits
this will allow us to implement:
- password authentication (instead of SSH agent)
- bandwidth throttling
- readahead
Note this requires nbdkit >= 1.12.
In forthcoming commits we will be adding support for ssh, curl and
other features that require nbdkit >= 1.12.
As a prelude to that work, add generic code for querying ‘nbdkit
--dump-config’ and ‘nbdkit plugin --dump-plugin’ and checking the
minimum version number.
This changes the minimum version from 1.1.16 to 1.2, although that was
released about a year ago and is widely available, and in any case
we're going to require 1.12 in the next commit.
This refactoring takes the nbdkit-specific code from the ‘-it vddk’
mode and puts it into a separate module. The idea is to reuse this
module (in future commits) to support ssh, curl and rate limiting.
This refactoring is not quite neutral because it moves some of the
prechecking into the Nbdkit.create function. This means it will
happen a little bit later in the cycle (in input#source instead of
input#precheck - refer to the diagram in v2v/types.mli). However it's
still almost in the same place and importantly still happens before we
start copying.
cgo does not allow arbitrary CFLAGS to be used. Instead it contains a
list of flags (safelist) that are allowed to be passed to the compiler.
Sadly -U option (introduced in commit d8d8c856a1) is not among them.
See: https://github.com/golang/go/issues/23672
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
It makes little sense to require the oVirt certificate, especially when
the verification of the connection (-oo rhv-verifypeer) is disabled by
default. The only work done with the certificate in that case is
checking that it is a valid certificate file.
Hence, make -oo rhv-cafile optional, requiring it only when
-oo rhv-verifypeer is enabled.
By default the installer will set the hostname to what the DHCP returns,
exposing details of the machine where make-template.ml runs.
Instead, force "unassigned-hostname.unassigned-domain" as hostname, so
plays nicely with the hostname setting code in virt-customize.
According to the Erlang website:
The old legacy erl_interface library (functions with prefix erl_) is
deprecated as of OTP 22, and will be removed in OTP 23. This does
not apply to the ei library. Reasonably new gcc compilers will issue
deprecation warnings. In order to disable these warnings, define the
macro EI_NO_DEPR_WARN.
That's a shame and probably means we will have to drop the Erlang
bindings soon unless someone ports them to this new API (stable APIs
FTW people!). In the meantime add the flag to prevent warn-errors
about deprecation.
This way it is possible to override the UUIDs of the uploaded disks,
instead of letting RHV generate them.
This can be useful to force certain UUIDs, and to specify the disks in
--no-copy mode (which now can be used).
Check for the INITIALIZING state of the image transfer right away,
without waiting 5 seconds even before the first time: this way, if the
transfer is already in the right state then there is no need to wait.
Otherwise it complains about missing files that it has no rules for, for example
`builder/index-parser.c`.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Instead of waiting for the completion of the nbdkit transfers to get the
UUIDs of the disks, use the new #disk_copied hook to do that after each
disk is copied.
This has almost no behaviour on rhv-upload, except for the --no-copy
mode:
- previously it used to hit the 5 minute timeout while waiting for the
finalization of the first disk
- now it asserts on the different number of collected UUIDs vs the
actual targets; at the moment there is nothing else that can be done,
as this assumption is needed e.g. when creating the OVF file
Improve the way the precheck script checks for the specified resources:
- look directly for a data center with the specified storage domain
- get the storage domain object from the storage domains attached to the
data center found
- similarly, look for the specified cluster among the ones attached to
the data center found
When everything is found, return the UUID of the storage domain, and of
the cluster back to virt-v2v, which will store them.
Similarly, rework the createvm script to directly get the requested
cluster, instead of looking for it once again. Also, since the UUID of
the storage domain is available in virt-v2v already, use it directly
instead of using a placeholder.
This should fix a number of issues:
- unexisting/unattached storage domains are rejected outright
- the cluster is rejected if not part of the same data center of the
selected storage domain
- renaming the specified storage domain during the data copying will not
cause the conversion to fail (which will still use the specified
storage domain, no matter the new name)
Based on the hints by Daniel Erez in RHBZ#1612653.
This way it is possible to communicate data from the precheck script
back to virt-v2v.
For now there are no results, so the resulting JSON is discarded.
Split the VM existance check out of the precheck script to a new vmcheck
script, and invoke that in #prepare_targets. Invoke the precheck script
in #precheck, as now it can be run with only values of command line
options.
This does not change which checks are performed; however, an invalid
cluster name will make virt-v2v fail way earlier (even before connecting
to the source).
This removes only the tool itself, and all the bits strictly needed to
not break the build.
This is now available as separate tool in its own repository:
https://github.com/libguestfs/virt-p2v
Added in virt-p2v commit:
commit e83b6f50af34ce650063ecc520bfabab400e8e73
Author: Matthew Booth <mbooth@redhat.com>
Date: Fri Mar 26 09:40:20 2010 +0000
Add export to RHEV
Allow guests to be written to a RHEV NFS export storage domain.
Add 'rhev' output method and -osd command-line option.
Example command line:
virt-v2v -f virt-v2v.conf -ic 'esx://yellow.rhev.marston/' \
-o rhev -osd blue:/export/export RHEL3-32
This will connect to an ESX server and write the guest 'RHEL3-32' to
blue:/export/export, which is a pre-initialised RHEV export storage domain.
