Commit Graph

10857 Commits

Author SHA1 Message Date
Richard W.M. Jones
3a87c4bb44 test-data/phony-guests: Add comment to Makefile above Fedora btrfs image.
Fixes commit 754e819438.
2018-07-18 13:15:47 +01:00
Richard W.M. Jones
fe1a886612 v2v: Implement MAC address to network/bridge mapping.
This allows specific NICs (identified by their source MAC address) to
be mapped to networks or bridges on the target.  You can use the --mac
parameter to select this mapping, eg:

 $ virt-v2v ... \
    --mac 52:54:00:d0:cf:0e:network:mgmt \
    --mac 52:54:00:d0:cf:0f:network:clientdata

The old --network and --bridge mappings can also be used but --mac
takes precedence.

Note this does not adjust MAC addresses inside the guest which is a
hard problem to solve.  For this to work you must still carry over the
MAC addresses from the source to target hypervisor as that is how most
guests identify and associate functions with specific network
interfaces.
2018-07-17 16:32:44 +01:00
Pino Toscano
682d3f8b01 v2v: -o rhv-upload: check for ovirtsdk4 (RHBZ#1601943)
Check earlier whether the ovirtsdk4 Python can be imported correctly,
to avoid errors later on during the migration.
2018-07-17 17:12:38 +02:00
Pino Toscano
4167ee2173 uefi: add non-deprecated Fedora paths for OVMF w/ secboot
Add new paths for the Secure Boot variant of OVMF as found in Fedora.
2018-07-17 15:21:41 +02:00
Richard W.M. Jones
0521a2f9b7 Version 1.39.7. v1.39.7 2018-07-14 10:23:45 +01:00
Richard W.M. Jones
2280c501ff build: Disable warnings about attribute malloc functions. 2018-07-13 20:39:04 +01:00
Richard W.M. Jones
fa2107e49e maintainer: Update to latest gnulib. 2018-07-13 20:38:36 +01:00
Pino Toscano
aa9e18f6d1 v2v: change QXL ResourceType in OVirt flavour (RHBZ#1598715)
Due to a conflict with the IDs of the OVF standard, and the existing
implementation in ovirt-engine, the ID of QXL devices changed to a
different value.

As a consequence, change the ResourceType of QXL devices, but only in
OVirt flavour to avoid breaking vdsm mode.

See: https://bugzilla.redhat.com/show_bug.cgi?id=1598715#c5
2018-07-13 18:04:46 +02:00
Richard W.M. Jones
58f7043234 Revert "v2v: oVirt changed the ResourceType for QXL video devices (RHBZ#1598715)."
This fix is incorrect, for details see:
https://bugzilla.redhat.com/show_bug.cgi?id=1598715#c11

This reverts commit 296b2f66c7.
2018-07-13 13:32:38 +01:00
Richard W.M. Jones
296b2f66c7 v2v: oVirt changed the ResourceType for QXL video devices (RHBZ#1598715).
See:
https://bugzilla.redhat.com/show_bug.cgi?id=1598715#c5
2018-07-12 10:02:04 +01:00
Richard W.M. Jones
1bc9facfdd builder: templates: More RHEL templates. 2018-07-12 09:44:03 +01:00
Richard W.M. Jones
32202e3917 v2v: Remove old TODO file. 2018-07-12 08:32:08 +01:00
Richard W.M. Jones
aeabe622ac build: Disable -Wabi warning as it is broken with GCC 8.1. 2018-07-11 16:28:43 +01:00
Richard W.M. Jones
1f79f2531e builder: templates: Update RHEL virt-install osinfo version for Fedora 28. 2018-07-11 10:17:19 +01:00
Richard W.M. Jones
b11b870166 v2v: Preserve VM Generation ID (RHBZ#1598350).
virt-v2v moves guests, it doesn't clone them.  Therefore we should try
to preserve the VM Generation ID (genid) as much as possible.

This has the ability to read the genid from VMware VMX files and
libvirt XML (but note RHBZ#1598348).  It can also write the genid to
libvirt (‘-o libvirt’, ‘-o local’) and QEMU (‘-o qemu’).

We are missing support currently for all OVF-based formats (hence
‘-i ova’, and all oVirt/RHV output modes).  It's unclear where we
would store the genid in this format.  oVirt does not yet support
genid (see RHBZ#1118825).
2018-07-09 08:56:45 +01:00
Richard W.M. Jones
9a25ac6b4e v2v: Add s_mapping_explanation field to source NIC.
Instead of storing the original network name (s_vnet_orig) in the
source NIC struct and then trying to guess what happened much later
when we're writing target metadata, get our newly created Networks
abstract data type to store the precise mapping explanation.
2018-07-09 08:56:45 +01:00
Richard W.M. Jones
c3bec46355 v2v: Make Cmdline.NetworkMap into an abstract data type.
No change, just code refactoring.
2018-07-09 08:56:45 +01:00
Daniel Erez
4ed1bc5a79 v2v: rhv plugin - find suitable host (RHBZ#1596810) (RHBZ#1596851)
For direct upload, a suitable host must be in status 'Up'
and belong to the same datacenter as the created disk.
Added these criteria to the host search query.
2018-07-05 21:35:23 +01:00
Richard W.M. Jones
2894b8f392 v2v: Add output#prepare_metadata method to prepare metadata before copying.
Currently unused, this method will allow output methods to prepare the
target hypervisor to receive a guest before the actual copy/upload
begins.

For rhv-upload this is useful since RHV would prefer to have the
metadata for the guest before receiving disk data.  The only metadata
unavailable here is target_stats.target_actual_size, but in the RHV API
case this is never available as we stream the data directly into RHV
rather than writing it to a local file.
2018-07-03 19:45:05 +01:00
Richard W.M. Jones
7850e0bfb8 v2v: Add comment about where the actual copy is performed in the code. 2018-07-03 19:45:05 +01:00
Richard W.M. Jones
6ca5986e45 v2v: Move per-target size stats into a separate mutable sub-struct.
This change avoids us threading the targets structure awkwardly
through several functions, and also makes it clear from the code that
the targets struct does not change after its creation in init_targets
(but the stats change).

Just code refactoring.
2018-07-03 19:45:05 +01:00
Richard W.M. Jones
8fdc1d4852 v2v: types: Fix comment for #create_metadata method. 2018-07-03 19:45:05 +01:00
Richard W.M. Jones
186ac2bfb2 v2v: Single character whitespace change. 2018-07-03 19:20:27 +01:00
Nir Soffer
891b5a0ec0 v2v: rhv-upload-plugin: Optimize only direct upload
The optimization to start the transfer on the local host makes sense
only when using the rhv-direct=true option. When using a proxy, let the
engine choose a host.
2018-06-30 07:41:08 +01:00
Nir Soffer
f5442d2f04 v2v: rhv-upload-plugin: Improve error handling
When optimizing the connection using unix socket, we handle these cases:

- The local host is not an oVirt host (no /etc/vdsm/vdsm.id).
- The local host is an oVirt host, but is not registered with engine.
- Creating UnixHTTPConnection() fails. Unlikely and probably a bug in
  the plugin, but we can recover by using the https connection.

The current code handle these cases silently, making it harder to
understand why the unix socket optimization did no happen. Add debug
message to make this clear.

Also comment in the error handler why we take this path instead of
failing the operation.
2018-06-30 07:41:08 +01:00
Nir Soffer
d5f36bacf9 v2v: rhv-upload-plugin: Remove unneeded auth
Old imageio proxy was using Authorization header for GET and PUT
requests. Remove unneeded authorization when sending OPTIONS request.

Remove unneeded duplicated comments about authorization for old
imageio, and replace them with a comment when we set needs_auth.
2018-06-29 16:09:11 +01:00
Pino Toscano
8d33ff8ba4 v2v: improve -os documentation for rhv-upload
Thanks to: Ming Xie
2018-06-29 12:59:26 +02:00
Richard W.M. Jones
1d7b63447a Version 1.39.6. v1.39.6 2018-06-29 10:23:54 +01:00
Richard W.M. Jones
d3dded2208 Update AUTHORS. 2018-06-29 10:22:36 +01:00
Pino Toscano
5a990d77b5 bash: add a completion script for libguestfs-test-tool
Use the simple virt-v2v-copy-to-local script for it.
2018-06-28 10:10:10 +02:00
Pino Toscano
53de4aeec5 test-tool: add a documentation test
Add the podcheck.pl test, to make sure the documentation is in sync with
the available options.

Also fix the formats of options in the current POD, so it matches what
used already in other documentations, and what podcheck.pl expects.
2018-06-28 10:10:10 +02:00
Pino Toscano
5565cf767a test-tool: implement --short-options & --long-options
While not useful per-se, it will help checking the available options.
2018-06-28 10:10:10 +02:00
Pino Toscano
b61c98b9e1 v2v: -o local: print libvirt XML
In verbose mode, print the generated libvirt XML to stderr for
debugging, just like -o libvirt does.
2018-06-28 10:10:10 +02:00
Richard W.M. Jones
b7a2e6270d v2v: -o rhv-upload: Use Unix domain socket to access imageio (RHBZ#1588088).
In the case where virt-v2v runs on the same server as the imageio
daemon that we are talking to, it may be possible to optimize access
using a Unix domain socket.

This is only an optimization.  If it fails or if we're not running on
the same server it will fall back to the usual HTTPS over TCP
connection.

Thanks: Nir Soffer, Daniel Erez.
2018-06-28 08:37:11 +01:00
Richard W.M. Jones
a1e1f6ec88 v2v: -o rhv-upload: Always fetch server options when opening the connection.
Previously we lazily requested the server options in the can_*
callbacks.  The can_* callbacks are always called by nbdkit straight
after open, so this just adds complexity for no benefit.  This change
simply makes the code always fetch the server options during the open
callback.

This is — functionally at least — mostly just refactoring.  However I
also added a useful debug message so we can see what features the
imageio server is offering.
2018-06-26 08:49:37 +01:00
Nir Soffer
32ab53cecd v2v: rhv-upload-plugin: Remove unused variables
Remove some instances of "transfer" and "transfer_service" that are not
used in current code.
2018-06-26 08:42:06 +01:00
Nir Soffer
0ae61ce99c v2v: rhv-upload-plugin: Fix name error
"http" and "transfer" variables were missing in emulate_zero, so the
code would fail with NameError.  This can happen only when communicating
with old imageio versions not supporting the "zero" feature.

Testing with qemu-img 2.12 show that we never send emulated zero request
because of the highestwrite mechanism, but it can break with older
qemu-img-rhev used on RHEL.
2018-06-25 19:09:34 +01:00
Nir Soffer
f4e0a8342d v2v: rvh-upload-plugin: Always read the response
Python manual warns[1]:

    Note that you must have read the whole response before you can send
    a new request to the server.

The reason for this warning is exposed only when the server is using
keep alive connections. When the response is not read, sending a new
request will fail with:

    httplib.ResponseNotReady

Even if Content-Length was 0 or the request has no content. The failure
looks like this when using --verbose:

nbdkit: python[1]: debug: zero count=33554432 offset=0 may_trim=1 fua=0
nbdkit: python[1]: debug: zero count=33554432 offset=33554432 may_trim=1 fua=0
nbdkit: python[1]: error: /home/nsoffer/src/libguestfs/tmp/rhvupload.Au2B5I/rhv-upload-plugin.py: zero: error: Request-sent
nbdkit: python[1]: debug: sending error reply: Input/output error
qemu-img: error writing zeroes at offset 0: Input/output error

Change all requests to read the whole response.

Tested with imageio patch supporting keep alive connections:
https://gerrit.ovirt.org/#/c/92296/

[1] https://docs.python.org/3.8/library/http.client.html#http.client.HTTPConnection.getresponse
2018-06-25 19:09:22 +01:00
Richard W.M. Jones
264f7ae9dd docs: building: autotools and gettext must be installed separately.
Thanks: Nir Soffer
2018-06-25 08:48:21 +01:00
Yuri Chornoivan
adbf1bc320 Fix minor typos 2018-06-25 08:35:43 +01:00
Richard W.M. Jones
3886a113f2 p2v: Allow virt-v2v input and output drivers containing '-' (RHBZ#1590220).
The new ‘-o rhv-upload’ output mode contains a '-' character in the
name, but the regular expression which matched the output of the
virt-v2v command did not recognize '-' as a valid character.  It ended
up mapping this to just "rhv" meaning two "rhv" entries would appear
in the list of output drivers.

Thanks: Ming Xie.
2018-06-22 12:51:10 +01:00
Richard W.M. Jones
75e8b13867 v2v: <File ovf:size> changed to the actual size (if known).
Note that this attribute is optional.

Thanks: Arik Hadas
2018-06-21 15:16:53 +01:00
Richard W.M. Jones
7c2afc88fd v2v: Add <Disk ovf:capacity> attribute containing disk virtual size (RHBZ#1592468).
Virt-v2v transfers to newer versions of ovirt-engine can fail with:

  ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "failed to parse a given ovf configuration ovf error: [empty name]: cannot read '//*/disksection' with value: null". HTTP response code is 400.

This was caused by a change made to oVirt:

  https://gerrit.ovirt.org/#/c/91902/

so that it now requires the <Disk ovf:capacity> attribute.

Thanks: Arik Hadas
2018-06-21 15:16:53 +01:00
Richard W.M. Jones
8c3375d47c v2v: docs: Removed vsyscall support in Debian kernels requiring workaround (RHBZ#1592061).
Thanks: Haigang Li
2018-06-21 15:14:20 +01:00
Richard W.M. Jones
e2910eeaf8 v2v: -o libvirt: Don't write only <vendor> without <model> (RHBZ#1591789).
Avoids the libvirt error:

  error: XML error: CPU vendor specified without CPU model
2018-06-21 15:09:05 +01:00
Richard W.M. Jones
70bc83f893 lib: libvirt: Convert all socket parameters to an absolute path (RHBZ#1588451). 2018-06-21 13:19:04 +01:00
Nir Soffer
77a412c0a1 v2v: -o rhv-upload: Optimize http request sending
When sending request with small or no payload, it is simpler and
possibly more efficient to use the high level HTTPSConnection.request(),
instead of the lower level APIs.

The only reason to use the lower level APIs is to avoid copying the
payload, or on python 2, to use a bigger buffer size when streaming a
file-like object.
2018-06-18 11:56:00 +01:00
Pino Toscano
4f927e58a9 fuse: fix build when not available
The 'localmountpoint' variable in the handle is available only when
building with FUSE support, so guard it in a proper #ifdef block.

Fixes commit 296370fb86.

Thanks to: Corentin Labbe (both reporting, and testing)
2018-06-15 18:36:39 +02:00
Richard W.M. Jones
afd1c70601 gitignore: Don't ignore patch files. 2018-06-14 11:10:48 +01:00
Richard W.M. Jones
a804a6f205 v2v: tests: Stable-sort CPU features in --print-source option.
The test failed with:

  -   CPU features: pae,apic,acpi
  +   CPU features: acpi,apic,pae

Fixes commit 67145499ec.
2018-06-12 17:36:28 +01:00