We reimplemented some functions which can now be found in the OCaml
stdlib since 4.01 (or earlier). The functions I have dropped are:
- String.map
- |>
- iteri (replaced by List.iteri)
- mapi (replaced by List.mapi)
Note that our definition of iteri was slightly wrong: the type of the
function parameter was too wide, allowing (int -> 'a -> 'b) instead of
(int -> 'a -> unit).
I also added this new function to the Std_utils.String module as an
export from stdlib String:
- String.iteri
Thanks: Pino Toscano
Commit dbe0b69f24 transscribed the Perl
regexp incorrectly so that it only matched the impossible case of
‘resume=/dev/X’ for a single non-whitespace character X.
This fixes the regular expression, referencing back to the original
Perl code in virt-v2v.
In the original conversion of virt-v2v from Perl
(commit 0131d6f666), the Perl regular
expression was incorrectly transscribed. ‘\b’ was not converted to
‘\\b’ so the new regexp was looking for an ASCII BEL character, not a
word boundary. Also ‘|’ was used, but Str requires ‘\|’ (ie. "\\|"
in the final source).
To fix these problems I converted the code to use PCRE, and went back
to the original virt-v2v code (virt-v2v.git:
lib/Sys/VirtConvert/Converter/Linux.pm) to find out what the Perl
regular expression should have been.
Note I have also removed ‘.*’ at the beginning and end of the regexp
because PCRE regexps are not anchored.
This pure refactoring changes the convert () functions so that the
conversion operations are listed first, followed by the
implementations of those operations.
The ‘vmware-uninstall-tools.pl’ script tries to rebuild the initrd.
On SUSE, if kdump initrd has been enabled, this would use the wrong
root device because ‘mkdumprd’ doesn't know what root device to use.
Fix that by setting the ‘rootdev’ environment variable.
See also https://bugzilla.redhat.com/1465849 (this is not a fix).
Create a module ‘C_utils’ containing functions like ‘drive_name’ and
‘shell_unquote’ which come from the C utilities.
The new directory ‘common/mlutils’ also contains the ‘Unix_utils’
wrappers around POSIX functions missing from the OCaml stdlib.
The new module ‘Std_utils’ contains only functions which are pure
OCaml and depend only on the OCaml stdlib. Therefore these functions
may be used by the generator.
The new module is moved to ‘common/mlstdutils’.
This also removes the "<stdlib>" hack, and the code which copied the
library around.
Also ‘Guestfs_config’, ‘Libdir’ and ‘StringMap’ modules are moved
since these are essentially the same.
The bulk of this change is just updating files which use
‘open Common_utils’ to add ‘open Std_utils’ where necessary.
Complementary fix of commit 2d25872df3.
On SLES 11 SP4 with kdump enabled mkinitrd calls mkdumprd which calls
mkinitrd, but mkdumprd doesn't have any clue of the root device.
Call mkinitrd with rootdev environment variable to tell them all
what device to use as root.
Tested-By: Cédric Bosdonnat <cbosdonnat@suse.com>
mkinitrd in SLE guests < 12 tries to get the root device by scanning the
fstab: this will fail, since v2v already remapped the devices from
hd*/sd* to vd* in the guest (including in its fstab).
Since we know what is the root device in the appliance, pass it to
mkinitrd directly, so it does not have to do guesswork.
Thanks to: Cédric Bosdonnat, for reporting the issue, and testing
the fix.
Extend the guestcaps structure so it records whether a guest supports
(or drivers were added) for virtio-rng, the virtio memory balloon, and
the ISA pvpanic device.
Previously the kernel_info field 'ki_supports_virtio' really meant
that the kernel supports virtio-net. That was used as a proxy to mean
the kernel supports virtio in general.
This change splits the field so we explicitly test for both virtio-blk
and virtio-net drivers, and store the results as separate fields.
The patch is straightforward, except for the change to the
'rebuild_initrd' function. Instead of making the module list
conditional on whether virtio-net is available and using the old
(probably wrong) fallback if it happens to be unavailable, this now
tries to enable the common virtio kernel modules (just the ones needed
for virtio-blk and virtio-net to work). The fallback is only used if
none of the common virtio modules can be found.
This also adds a print_kernel_info function (as the
string_of_kernel_info function was unwieldy) and exports it from the
module.
We want to detect if a Linux kernel is Xen PV only ("PV-only"). Such
a kernel will not boot on KVM, and if a guest has only PV-only
kernels, it will not be able to boot at all on the target.
Our previous test was wrong. It tested whether the xennet.ko module
exists. This module was renamed in more recent kernels (to
xen-netfront.ko), so it happened to not detect modern kernels as
PV-only, but this was by chance.
Modern kernel images can be compiled with Xen PV guest support. The
same image can boot on baremetal, KVM, Xen PV or Xen HVM. Testing if
the xennet (or xen-netfront) module exists is irrelevant to this.
This test, which is based on ideas from Laszlo Ersek and
https://wiki.xen.org/wiki/Xen_Project_Software_Overview#Guest_Types
uses the kernel config test CONFIG_X86_XEN || CONFIG_X86_64_XEN to
determine PV-only kernels.
Note that these CONFIG flags were never upstream, and existed only in
Linux kernels forked by Xen ("XenLinux"). By the time Xen guest
support was added upstream, it was implemented using pvops support, so
a single image could boot on Xen and non-Xen as described above, and
these flags were no longer used.
Updates commit 7eb219d193.
Thanks: Laszlo Ersek.
Only in end-user messages and documentation. This change was done
mostly mechanically using the Perl script attached below.
I also changed don't -> don’t etc and made some other simple fixes.
See also: https://www.cl.cam.ac.uk/~mgk25/ucs/quotes.html
----------
#!/usr/bin/perl -w
use strict;
use Locale::PO;
my $re = qr{'([-\w%.,=?*/]+)'};
my %files = ();
foreach my $filename ("po/libguestfs.pot", "po-docs/libguestfs-docs.pot") {
my $poref = Locale::PO->load_file_asarray($filename);
foreach my $po (@$poref) {
if ($po->msgid =~ $re) {
my @refs = split /\s+/, $po->reference;
foreach my $ref (@refs) {
my ($file, $lineno) = split /:/, $ref, 2;
$file =~ s{^\.\./}{};
if (exists $files{$file}) {
push @{$files{$file}}, $lineno;
} else {
$files{$file} = [$lineno];
}
}
}
}
}
foreach my $file (sort keys %files) {
unless (-w $file) {
warn "warning: $file is probably generated\n"; # have to edit generator
next;
}
my @lines = sort { $a <=> $b } @{$files{$file}};
#print "editing $file at lines ", join (", ", @lines), " ...\n";
open FILE, "<$file" or die "$file: $!";
my @all = ();
push @all, $_ while <FILE>;
close FILE;
my $ext = $file;
$ext =~ s/^.*\.//;
foreach (@lines) {
# Don't mess with verbatim sections in POD files.
next if $ext eq "pod" && $all[$_-1] =~ m/^ /;
unless ($all[$_-1] =~ $re) {
# this can happen for multi-line strings, have to edit it
# by hand
warn "warning: $file:$_ does not contain expected content\n";
next;
}
$all[$_-1] =~ s/$re/‘$1’/g;
}
rename "$file", "$file.bak";
open FILE, ">$file" or die "$file: $!";
print FILE $_ for @all;
close FILE;
my $mode = (stat ("$file.bak"))[2];
chmod ($mode & 0777, "$file");
}
Make the documentation in Convert_linux more generic than just
Enterprise distros using RPM. Also, rename the name of the module from
"enterprise-linux" to simply "linux".
Previously the Convert_linux conversion module depended on one feature
of the output module (#keep_serial_console). This was extracted in an
ad-hoc way from the output module and passed as an extra parameter to
the conversion module.
Instead of doing it this way, just pass the output module into the
conversion module, so it can call output#keep_serial_console itself.
This is just a simplification of the existing code, but otherwise adds
no new features.
Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
Try to improve the way packages of VMware tools are removed from
YUM-based guests:
- when filtering the package itself from its providers, do a stricter
check so either the provide is the unversioned package, or it is
exactly its own name
- if the package has no other providers, then going further will cause
the invocation of 'yum install' with no packages, and thus the package
itself will not be added to the list of packages to be removed; to
overcome this issue, just mark the package as "to be removed" in that
case
Related to: RHBZ#1155150
Using update-initramfs is the native way of updating initrd on Debian
based systems.
To add some modules to the image we can list them in file
/etc/initramfs-tools/modules.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Create a new module [Linux_kernels] which does all kernel detection,
and also provides a place to define the kernel_info data structure.
This is essentially just code motion.
Methods in OCaml which don't take any parameters don't require the
dummy unit arg, ie writing:
method foo = ...
is fine. The reason you might need the unit arg is if you need to
create a closure from the method without calling it, for example if
you need to use the method in a callback.
In lablgtk2 the convention is to use unitless methods if either: the
method shouldn't be used as a callback; or: (conceptually) the method
doesn't change the object's internal state. Let's do that here.
The name of the initrd image on Debian-based systems is different from
what used on Fedora/RHEL/SUSE and derived; set a different regexp to
avoid making the current regexp even more complex.
Implement the 'remove', 'file_list_of_package', and 'file_owner' methods
of the Linux module for the "deb" package manager (dpkg basically, on
Debian and derived distributions).
Also allow it for the main conversion code.
Create an object hierarchy to represent different bootloaders for Linux
guests, moving the separate handling of grub1 and grub2 in different
classes: this isolates the code for each type of bootloader together,
instead of scattering it all around.
This is mostly code refactoring, with no actual behaviour change.
Add also /boot/efi/EFI/redhat/grub.conf as configuration of Grub 1;
since the "grub" lens of Augeas does not handle this path, add a
transformation so Augeas can parse it.
Do not print warning for 'sr' devices when converting fstab. Not all
systems create the /dev/cdrom symlink for SCSI CD-ROM devices. Moreover,
on systems with multiple CD-ROMs, having entries for /dev/sr* devices
may be inevitable.
RWMJ: Use String.is_prefix instead of String.find, to more accurately
match on the device name.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
The replacement string was wrong. There are only two match groups in the
regular expression, not three.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
This makes sure that in the internal configure_kernel_modules function,
for virtio or SCSI block types:
a) the warnings about leftover Xen modules are printed
b) the changes in Augeas are saved
Fixes commit ee02191483.
First check for the existence of the directory /etc/modprobe.d, in case
using a file under it; this also skips all the other checks, since they
are not needed at all. Also /etc/modprobe.d exists on recent Linux
versions, so let's give priority to the more common methods.
When /etc/modprobe.d does not exist, check for the file to edit using a
single list of possible files, now in order of priority, where the first
find is used without checking further for the rest.
Also, make sure all the returned paths are absolute: they are used in
Augeas paths later on, so relative paths will not do anything useful.
The debug() function is already sending these to stderr, but in a few
places we were using printf. Change those to eprintf, except for one
informational message which should have been using info().
Store the list of EFI system partitions on the inspect object in order to be
able to tune their contents later in the process.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
Guest tools for Linux in Parallels / Virtuozzo Server 6 come with a script
that can be used to uninstall them.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
To make sure we can also find the initrd on openSUSE and SLES, we need two improvements:
* the initrd filename may not end with '.img'
* don't use the version + release from the RPM data, rather from the
/lib/modules/<version>/ path as we need to find it out anyway.
This function is useful outside virt-v2v so move it into the core
utilities library.
The function has been rewritten from OCaml into C, but it should be
functionally identical.
Virtio-SCSI offers a number of advantages over virtio-blk, in
particular, it supports SCSI UNMAP (aka trim) which is crucial for
keeping the virtual drive from wasting host disk space.
This patch adds support for virtio-scsi as the virtual disk connection
type both on input and on output of v2v.
Virtio-blk remains the default, so for now virtio-scsi-based guests can
only be produced in --in-place mode.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
