Commit Graph

11148 Commits

Author SHA1 Message Date
Pino Toscano
497e04f97b RHEL 8: point to KB for supported v2v hypervisors/guests 2020-07-20 13:06:55 +02:00
Pino Toscano
152d998728 RHEL 8: use platform-python
Use the stable platform-python provided in BaseOS, instead of relying on
some arbitrary version installed by the user.
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
df499286f5 RHEL 8: v2v: rhv-upload: Remove restriction on -oa sparse.
See: https://bugzilla.redhat.com/show_bug.cgi?id=1565681
and the v2v-devel private thread "Do we already support migration using FC?"
2020-07-20 13:06:55 +02:00
Pino Toscano
422dbeb2dc RHEL 8: v2v: do not mention SUSE Xen hosts (RHBZ#1430203)
They are not supported in RHEL 8.
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
0fe5bafac4 RHEL 8: v2v: -i disk: force VNC as display (RHBZ#1372671)
The SDL output mode is not supported in RHEL 8's qemu-kvm.
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
38abe40ffa RHEL 8: v2v: Disable the virt-v2v --in-place option.
This disables the virt-v2v --in-place option which we do not
wish to support in RHEL.
(See commit d0069559a9).
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
76da9d0e3d RHEL 8: tests: Disable daemon tests that require the 'unix' backend. 2020-07-20 13:06:55 +02:00
Richard W.M. Jones
1e02333d39 RHEL 8: Fix tests for libguestfs-winsupport.
It doesn't let us use guestfish for arbitrary Windows edits.
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
eaae3564f0 RHEL 8: Reject use of libguestfs-winsupport features except for virt-* tools (RHBZ#1240276). 2020-07-20 13:06:55 +02:00
Richard W.M. Jones
4fe5014634 RHEL 8: Fix list of supported sound cards to match RHEL qemu (RHBZ#1176493). 2020-07-20 13:06:55 +02:00
Pino Toscano
7f6221bd32 RHEL 8: Disable alternate Augeas lenses.
These are included in the RHEL augeas package, and therefore not
required.

See:
https://www.redhat.com/archives/libguestfs/2014-October/msg00220.html
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
5104b5d514 RHEL 8: v2v: Disable the --qemu-boot option (RHBZ#1147313).
This cannot work because there is no Gtk or SDL output mode
in RHEL 8's qemu-kvm.

In addition you will have to edit the -display option in the
qemu script.
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
c5b1a209eb RHEL 8: v2v: Select correct qemu binary for -o qemu mode (RHBZ#1147313).
RHEL 8 does not have qemu-system-x86_64 (etc), and in addition the
qemu binary is located in /usr/libexec.  Encode the path to this
binary directly in the script.

Note that we don't support people running qemu directly like this.
It's just for quick testing of converted VMs, and to help us with
support cases.
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
73ed123912 RHEL 8: Remove User-Mode Linux (RHBZ#1144197).
This isn't supported in RHEL 8.
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
8ada76f335 RHEL 8: Disable unsupported remote drive protocols (RHBZ#962113).
This disables support for unsupported remote drive protocols:

 * ftp
 * ftps
 * http
 * https
 * tftp
 * gluster
 * iscsi
 * sheepdog
 * ssh

Note 'nbd' is not disabled, and of course 'file' works.

We hope to gradually add some of these back over the lifetime of RHEL 8.
2020-07-20 13:06:55 +02:00
Richard W.M. Jones
bdca6cb09e RHEL 8: Remove 9p APIs from RHEL (RHBZ#921710). 2020-07-20 13:06:55 +02:00
Richard W.M. Jones
00007f9d38 RHEL 8: Remove libguestfs live (RHBZ#798980).
This isn't supported in RHEL 8.
2020-07-20 13:06:55 +02:00
Pino Toscano
2125023a89 vCenter: fix parsing of HTTP status string (RHBZ#1837328)
vCenter 7 answers with an HTTP/2 status string, so we cannot extract
the status code from it by using fixed positions in that string.
Hence, pick the status code by reading what's after the whitespace.

Tested with vCenter 6.5 and 7.

(cherry picked from commit d2aa82317964d62fcc8dc7b6737773003d04b998
in virt-v2v)
2020-07-20 13:06:50 +02:00
Pino Toscano
7700915a65 sysprep: add Kerberos keytab file removal
This new operation removes the Kerberos /etc/krb5.keytab file from the
guest.

Thanks to Christian Heimes and François Cami for the hints.

Related to RHBZ#1789592.

(cherry picked from commit faa5d8507f)
2020-05-26 11:30:28 +02:00
Pino Toscano
fa3ff43b76 sysprep: add IPA offline unenrollment (RHBZ#1789592)
This new operation unenrolls the guest from a IPA server offline, by
removing the configuration files and certificates.

Thanks to Christian Heimes and François Cami for the hints.

(cherry picked from commit 0a53e2c7fc)
2020-05-26 11:30:24 +02:00
Pino Toscano
d78ec54de6 sysprep: ca-certificates: request system CA store update
In case any certificate is removed from the guest, regenerate the system
CA store.

(cherry picked from commit b9065fa7ad)
2020-05-26 11:30:19 +02:00
Pino Toscano
f3fa23fd9d sysprep: add a update_system_ca_store side effect
Add a simple side effect to make operation flag that a regeneration of
the system CA store is needed. In case it is flagged, regenerate the
system CA store directly, or using a firstboot script in case of
incompatible architectures.

This change is almost a no-op, since no operation requires the
regeneration of the system CA store yet.

(cherry picked from commit bb7fc6d0a1)
2020-05-26 11:30:14 +02:00
Pino Toscano
63d711246f customize: port do_run to run_in_guest_command
Make use of the new helper function in Tools_utils to run commands in
the guest.

(cherry picked from commit b25e3495f5)
2020-05-26 11:29:59 +02:00
Pino Toscano
1519dfe45d mltools: add run_in_guest_command helper
Add an helper function to run a command in the guest, checking for the
host/guest compatibility.  This is mostly extracted from the internal
do_run helper currently in the Customize_run module of virt-customize.

(cherry picked from commit e73eca3b73f7d0a54615c5dc55eadd09dc170035
in libguestfs-common)
2020-05-26 11:28:42 +02:00
Richard W.M. Jones
fabaf7328a daemon: xattr: Filter out user.WofCompressedData from xattrs (RHBZ#1811539).
See comment in code for justification.

Thanks: Yongkui Guo for finding the bug.
(cherry picked from commit c2c11382bb)
2020-05-26 11:16:14 +02:00
Richard W.M. Jones
1c976cc5d1 daemon: Add filter_list utility function.
For filtering lists of strings based on a predicate.

(cherry picked from commit af8ed266a2)
2020-05-26 11:16:09 +02:00
Richard W.M. Jones
052549a055 daemon: xattr: Refactor code which splits attr names from the kernel.
The kernel returns xattr names in a slightly peculiar format.  We
parsed this format several times in the code.  Refactor this parsing
so we only do it in one place.

(cherry picked from commit 5c175fe732)
2020-05-26 11:16:02 +02:00
Tomáš Golembiovský
34a8b1a711 windows: delay installation of qemu-ga MSI
Instead of running firstboot script during early boot schedule a task
delayed for 2 minutes.

During the first boot, after virt-v2v conversion, Windows installs the
drivers injected by virt-v2v. When this installation is finished
Windows enforces some kind of internal reboot. This unfortunately
terminates any running firstboot scripts thus killing the installation
of qemu-ga MSI.

This is just a best-effort mitigation. It can still happen (e.g. with
slow disk drives) that the drivers are not yet installed when the
delayed installation starts. On the other hand we cannot delay it too
much otherwise we risk that the users logs in and will be doing some
work when the MSI installation starts. After MSI installation finishes
the VM needs to be rebooted which would be annoying if that would happen
under users hands. Although this is not a best fix (that may come later
as it is more complex, e.g. introducing waiting mechanism), the delay as
it is defined works in most cases. And it dramaticaly improves the
situations -- originaly I experienced more than 90% failure rate.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>

(cherry picked from commit dc66e78fa37db33e3c7358b7f7c7fa809cf62f9d
in virt-v2v)
2020-03-16 18:18:18 +01:00
Tomáš Golembiovský
34305aa93b windows: fix detection of qemu-ga installer on RHV
The detection was incorrectly matching only 32-bit installer on all
architectures.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>

(cherry picked from commit 45acf8d0557bee948c035305a6bafc6c9963a467
in virt-v2v)
2020-03-16 18:17:39 +01:00
Tomáš Golembiovský
b5e6c21a82 windows: small tweaks of qemu-ga firstboot script
- match log file with script name
- restart manually only after successfull install, this also helps
  debugging because we can log the installer return code

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>

(cherry picked from commit 59f9ff40621240a6eed28c4425d3d69d8b21bc0e
in virt-v2v)
2020-03-16 18:16:43 +01:00
Tomáš Golembiovský
c664b50ab6 v2v: windows: install QEMU Guest Agent MSI
Use firstboot script to install MSI with QEMU-GA from virtio-win ISO or
oVirt/RHV guest tools ISO.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
(cherry picked from commit 00b4ed312b)
2020-03-16 18:12:43 +01:00
Tomáš Golembiovský
1d0a22aa3f v2v: fix path to source when copying files from guest tools directory
The debug message was slightly changed too to better match the similar
message for ISO case. It refers to the root directory instead of the
specific subdirectory inside guest tools.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
(cherry picked from commit f4bda5571a)
2020-03-16 17:32:31 +01:00
Tomáš Golembiovský
ebc6680ab1 add versioned directory for guest agent on EL8
There was no source directory for EL8 guest agent (only EL6 and EL7).

RHBZ#1791802

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>

(cherry picked from commit 79dd6a6bbb7e95691be18f54d601318d7d713702
in virt-v2v)
2020-02-24 13:50:39 +01:00
Richard W.M. Jones
03b67600b9 docs: Fix update-crypto-policies command (RHBZ#1791257).
The command as documented was wrong.  We need to use the --set option
to change the policy.

Fixes commit d5cbe7b4bee5dec9e28b1db03e933c97ef6d11e0.
Thanks: Xiaodai Wang

(cherry picked from commit 0edf419e983fe75daef9eaa7bd0578cbcada2e73
in virt-v2v)
2020-02-24 13:48:28 +01:00
Richard W.M. Jones
49623c945c v2v: -o rhv-upload: Make -oo rhv-cafile optional in all cases (RHBZ#1791240).
This is actually not required, because ovirtsdk4 will use the system's
global trust store if necessary.  Therefore we can make it optional in
all cases.

(cherry picked from commit 65ee9387d4be0e3c5cd214b967fef7a1a8841233
in virt-v2v)
2020-02-24 13:46:09 +01:00
Nir Soffer
8d39f6bcf2 v2v: Optimize convert for images with small holes
"qemu-img convert" detects zeroes in allocated areas and punch holes in
the destination image. This may save space on the destination image, but
slows down conversion when using outputs such as rhv-upload, which have
very large overhead per requests.

Using the -S flag, we can treat small areas filled with zeroes as data,
limiting the number of requests, and speeding the operation.

Here is an example converting Fedora 30 image:

$ virt-builder fedora-30 -o src.img
...

$ qemu-img map -f raw --output json src.img | wc -l
213

$ qemu-img convert -f raw -O raw -t none -T none src.img dst.img

$ qemu-img map -f raw --output json dst.img | wc -l
1443

$ ls -lhs *.img
1.2G -rw-r--r--. 1 nsoffer nsoffer 6.0G Nov  1 21:48 dst.img
1.2G -rw-r--r--. 1 nsoffer nsoffer 6.0G Nov  1 21:46 src.img

Qemu did 1443 writes instead of 213 (5.8X). Lets repeat this conversion
with the -S option:

$ qemu-img convert -f raw -O raw -t none -T none -S 64k src.img dst.img

$ qemu-img map -f raw --output json dst.img | wc -l
213

$ ls -lhs *.img
1.2G -rw-r--r--. 1 nsoffer nsoffer 6.0G Nov  1 21:48 dst.img
1.2G -rw-r--r--. 1 nsoffer nsoffer 6.0G Nov  1 21:46 src.img

Picking a good value for -S is not easy. Testing show that 64k is best
value for this test image for limiting the number of requests:

$ for size in 4k 8k 16k 32k 64k; do \
    printf "%5s: " $size; \
    qemu-img convert -f raw -O raw -t none -T none -S $size src.img dst.img; \
    qemu-img map -f raw --output json dst.img | wc -l; \
done
   4k: 1443
   8k: 731
  16k: 521
  32k: 387
  64k: 213

We need more testing with oVirt to measure the performance improvement
and pick a good value. This should probably be an option, but lets start
with a minimal change.

(cherry picked from commit 2aa78ade2d48e926b7b04050338ebd8a0c5e3f05
in virt-v2v)
2020-02-24 13:42:23 +01:00
Nir Soffer
d132157c9e rhv-upload: Fix waiting for transfer
We were not considering failures while initializing the transfer. In
this case the transfer phase can change to PAUSED_SYSTEM or
FINISHED_FAILURE, and transfer_url will be None, which failed the
upload with a misleading error:

    RuntimeError: direct upload to host not supported, requires
    ovirt-engine >= 4.2 and only works when virt-v2v is run within the
    oVirt/RHV environment, eg. on an oVirt node

Change the wait loop to consider all cases:
- Transfer failed and was removed
- Transfer failed and will be removed soon
- Transfer paused by the system (cancel required)
- Unexpected transfer phase (cancel required)
- Timeout waiting for TRANSFERRING state (cancel required)

Reported-by: Xiaodai Wang

(cherry picked from commit 40e1844827e4d096b1919a2159f9effc41915a73
in virt-v2v)
2020-02-24 13:34:54 +01:00
Nir Soffer
7fd1c3be6b rhv-upload: Show transfer id in error message
(cherry picked from commit d4ca9b6ca42d4ad3c717f5c59402ca6ff5d322bb
in virt-v2v)
2020-02-24 13:30:27 +01:00
Nir Soffer
200a365f20 rhv-upload: Check status more frequently
Checking status more frequently save a couple of seconds. Here is
an example flow tested with oVirt upload_disk.py example:

With 5 seconds wait:

Created disk in 11.085111 seconds
Created transfer in 1.857502 seconds

With 1 second wait:

Created disk in 4.991227 seconds
Created transfer in 1.961243 seconds

(cherry picked from commit 8816c5db220f518ef70beec7ac543290e3d5c0c7
in virt-v2v)
2020-02-24 13:28:46 +01:00
Pino Toscano
d02694e659 v2v: -o rhv-upload: check for a valid image transfer right away
Check for the INITIALIZING state of the image transfer right away,
without waiting 5 seconds even before the first time: this way, if the
transfer is already in the right state then there is no need to wait.
2020-02-24 13:25:51 +01:00
Richard W.M. Jones
a4433085bf openstack: Increase Cinder volume attach timeout to 5 minutes (RHBZ#1685032).
In some cases we have observed the time taken for a Cinder volume to
attach to the conversion appliance can be longer than the current 60
seconds.  Increase the timeout to 5 minutes.

Thanks: Ming Xie.

(cherry picked from commit e2ce290f6e366716f857eeaddc1dc680e5608c80
in virt-v2v)
2020-02-24 13:21:41 +01:00
Martin Kletzander
3dc8b808bd mlcustomize: Trim whitespaces from commands read from file (RHBZ#1351000)
The first split does not care about the whole string, it is just trying to get
the command name in front, so triml is just right.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2020-02-24 13:15:47 +01:00
Pino Toscano
34b2da9890 docs: remove paragraph about VMware tools on Windows (RHBZ#1785528)
Starting from libguestfs/virt-v2v 1.39.12, virt-v2v attempts to
uninstall the VMware tools from Windows guests, so there is no need to
remove them manually before the conversion.

Thanks to: Ming Xie.

(cherry picked from commit 397b4a90d16f4eb116d55605cbdf3bd844108315
in virt-v2v)
2020-01-13 13:31:44 +01:00
Pino Toscano
abb0a1dcc6 options: allow a UUID as identifier for --key
This way it is possible to specify the UUID of the LUKS device instead
of the libguestfs device name to decrypt a device during the inspection.

Make the usage of the new luks_uuid API conditional, so other projects
using the common submodule do not require a libguestfs version bump.

(cherry picked from commit bb4a2dc17a78b53437896d4215ae82df8e11b788
in libguestfs-common)

PT: the documentation was amended manually.
2019-12-06 13:02:56 +01:00
Pino Toscano
3afce3e84c options: rename key.device as key.id
In the future it will be also something else other than the device name.

(cherry picked from commit c863ee5e1df5e1eca7ad6821bd2db3796277a6bd
in libguestfs-common)

PT: the documentation was amended manually.
2019-12-06 12:59:27 +01:00
Richard W.M. Jones
d3c6f71eed options: Allow multiple --key parameters.
This allows multiple --key parameters on the command line to match a
single device.  For example:

  tool --key /dev/sda1:key:trykey1 --key /dev/sda1:key:trykey2

would try "trykey1" and "trykey2" against /dev/sda1.

(cherry picked from commit c10c8baedb88e7c2988a01b70fc5f81fa8e4885c
in libguestfs-common)
2019-12-06 12:53:48 +01:00
Richard W.M. Jones
d2ca3f2f9b options: Simplify selector parsing for --key options.
Refactor this code to use guestfs_int_split_string function which
slightly simplifies it.  This should have no effect.

(cherry picked from commit 530d0beef74d48617717463a5b585f21e2ed62be
in libguestfs-common)
2019-12-06 12:53:32 +01:00
Richard W.M. Jones
7f12a3ddad options: Fix segfault when multiple --key parameters given.
Easily reproducible using:

  $ guestfish --key dev1:key:key1 --key dev2:key:key2

causing this stack trace (or others depending on where the memory
corruption was caught):

  Program received signal SIGABRT, Aborted.
  0x00007ffff7905625 in raise () from /lib64/libc.so.6
  (gdb) bt
  #0  0x00007ffff7905625 in raise () from /lib64/libc.so.6
  #1  0x00007ffff78ee8d9 in abort () from /lib64/libc.so.6
  #2  0x00007ffff79494af in __libc_message () from /lib64/libc.so.6
  #3  0x00007ffff7950a6c in malloc_printerr () from /lib64/libc.so.6
  #4  0x00007ffff79528d0 in _int_free () from /lib64/libc.so.6
  #5  0x00005555555bdd6e in free_key_store ()
  #6  0x0000555555589027 in main ()
  (gdb) quit

(cherry picked from commit 8c42f772614b44a8cb974afa904ec9f518431ab2
in libguestfs-common)
2019-12-06 12:53:20 +01:00
Pino Toscano
bcd376966f New API: luks_uuid
Return the UUID of a LUKS device.

(cherry picked from commit 206ce8bbf1)
2019-12-06 12:52:46 +01:00
Richard W.M. Jones
70e2215cd1 v2v: Copy static IP address information over for Windows guests (RHBZ#1626503).
For Linux the guest itself remembers the IP address associated with
each MAC address.  Thus it doesn't matter if the interface type
changes (ie. to virtio-net), because as long as we preserve the MAC
address the guest will use the same IP address or the same DHCP
configuration.

However on Windows this association is not maintained by MAC address.
In fact the MAC address isn't saved anywhere in the guest registry.
(It seems instead this is likely done through PCI device type and
address which we don't record at the moment and is almost impossible
to preserve.)  When a guest which doesn't use DHCP is migrated, the
guest sees the brand new virtio-net devices and doesn't know what to
do with them, and meanwhile the right static IPs are still associated
with the old and now-defunct interfaces in the registry.

We cannot collect the required information from within the guest.
However we can collect it outside the tool by some other means
(eg. using VMware Tools APIs) and present this information to virt-v2v
which then writes it into the Windows guest at firstboot time.

This commit adds the --mac ..:ip:.. sub-option which creates a
Powershell script to set network adapters at firstboot.  An option
such as:

  --mac 00:0c:29:e6:3d:9d:ip:192.168.0.89,192.168.0.1,24,192.168.0.254

approximately turns into this script:

  # Wait for the netkvm (virtio-net) driver to become active.
  $adapters = @()
  While (-Not $adapters) {
      Start-Sleep -Seconds 5
      $adapters = Get-NetAdapter -Physical |
                     Where DriverFileName -eq "netkvm.sys"
  }
  $mac_address = '00-0c-29-e6-3d-9d'
  $ifindex = (Get-NetAdapter -Physical |
                 Where MacAddress -eq $mac_address).ifIndex
  if ($ifindex) {
      New-NetIPAddress -InterfaceIndex $ifindex
                       -IPAddress '192.168.0.89'
                       -DefaultGateway '192.168.0.1'
                       -PrefixLength 24
      Set-DnsClientServerAddress -InterfaceIndex $ifindex
                       -ServerAddresses ('192.168.0.254')
  }

Thanks: Brett Thurber for diagnosing the problem and suggesting paths
towards a fix.

(cherry picked from commit dfd9fac743)
2019-11-28 11:20:05 +01:00