The generator creates these files with 0444 mode, so they cannot be
overwritten by a simple ‘cp’ command. We could use ‘cp -f’ or ‘rm -f’.
Fixes commit 15394cb4dd.
After the proposed split of the libguestfs repo, we will end up with
the following layout:
libguestfs.git
common -> git submodule libguestfs-common.git
generator
virt-v2v.git
common -> git submodule libguestfs-common.git
guestfs-tools.git
common -> git submodule libguestfs-common.git
The generator will only be able to write to libguestfs directories and
the common directory/submodule. This is mostly the case already with
only 6 exceptions:
customize/customize-options.pod
customize/customize-synopsis.pod
customize/customize_cmdline.ml
customize/customize_cmdline.mli
v2v/uefi.ml
v2v/uefi.mli
This commit moves these files around so they appear under common/ml*
It is somewhat unsatisfactory because it involves copying files
around, but there are some mitigating factors:
(1) Any changes now give us more freedom to develop faster and thus
clean things up in future.
(2) The v2v/uefi files ought to go away in future anyway.
This is simple code motion and should have no effect on the built
programs or tests.
These two modules are a dependency of virt-v2v. Since we intend to
split virt-v2v from the other OCaml virt-* programs, we cannot have a
dependency between virt-v2v and virt-customize. Instead we must move
the modules to a common directory (common/mlcustomize) and have both
tools depending on the modules from there.
This is simple refactoring and should not affect how the programs work
or are tested.
This way no non-namespaced OCaml C symbols are used, reducing the risk
of clashes with other code.
The only exception is ocaml-augeas, which does not build with
CAML_NAME_SPACE; it will be fixed upstream, and it affects only
ocaml-augeas itself.
This option was removed from qemu for no apparent reason except to
break existing consumers. It does the same as -no-user-config, added
in May 2012, so use that instead.
After this commit, all annocheck errors are fixed except for:
Hardened: virt-get-kernel: MAYB: Gaps were detected in the annobin coverage. Run with -v to list.
After discussion with the annocheck maintainers this gap in coverage
(which corresponds to the OCaml runtime) seems to be caused either by
the runtime not being linked with the right flags, or might be a bug
in annocheck itself. In any case it's not something that can be
resolved within the scope of libguestfs.
The majority of the tools have already options (--echo-keys &
--keys-from-stdin) to deal with LUKS credentials, although there is no
way to automatically provide credentials. --keys-from-stdin is
suboptimal, because it is a usable solution only when there is just one
device to open, and no other input passed via stdin to the tool (like
the commands for guestfish).
To overcome this limitation, introduce a new --key option in tools:
* --key /dev/device:file:/filename/with/key
* --key /dev/device:string:the-actual-key
this way it is possible to pass all the credentials needed for the
specific devices to open, with no risk of conflict with stdin, and also
in a secure way (when using the "file" way).
On the technical side: this adds a new "key_store" API for the C tools,
making sure it is used only when needed. Partially mirror it also for
the OCaml tools, although there will be a conversion to the C API
because the decryption helpers used are in the common C parts.
Instead of returning directly a Getopt.t handle, now
Tools_utils.create_standard_options returns a struct, which at the
moment contains only the Getopt.t handle. This way, it will be easy to
add more data needed for handling standard command line options.
This is mostly refactoring, with no functional changes.
This requires that Windows guests have been created using the
procedure outlined here:
https://rwmj.wordpress.com/2018/09/13/creating-windows-templates-for-virt-builder/
For me:
PASS: test-firstboot-windows-6.2-server.sh
PASS: test-firstboot-windows-6.3-server.sh
PASS: test-firstboot-windows-10.0-server.sh
An incidental change is that we dump the firstboot log from the guest
(even on success). If the firstboot fails this is very useful for
determining the real cause.
This is just a mechanincal change, so that the public documentation[*]
refers to the latest release Fedora versions, instead of the EOLed
versions.
While at it, also update the `virt-cutomize` Makefile.am
[*] http://libguestfs.org/virt-builder.1.html
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
Kali Linux is a Debian derivative, so add basic support for it by using
most of the Debian code paths. The only exception is the crypto
algorithm for passwords in passwd, which is always assumed as SHA512
(as Kali Linux is relatively new).
libselinux defaults to "targeted" when no SELINUXTYPE is specified in
/etc/config/selinux. Hence do the same here, instead of failing because
of the missing key.
Add a slow test for checking SELinux relabeling on a Fedora 27 guest,
both with no changes, and with a modified configuration.
This is an unexpected error, so fail hard instead of leaking
End_of_file exception.
Nothing that calls into the Urandom module expects or handles
End_of_file.
However some existing functions had names which shadowed existing
functions in the List module, so I had to rename them:
assoc -> List.assoc_lbl
append -> List.push_back_list
prepend -> List.push_front_list
This is an extension of the previous commit.
Inspired by ocaml-extlib, introduce a module for handling option
types.
We already had the ‘may’ function (which becomes ‘Option.may’). This
adds also ‘Option.map’ (unused), and ‘Option.default’ functions.
Note this does *not* introduce the unsafe ‘Option.get’ function from
extlib.
This avoids warning 52 in OCaml code such as:
try URI.parse_uri arg
with Invalid_argument "URI.parse_uri" -> ...
which prints:
Warning 52: Code should not depend on the actual values of
this constructor's arguments. They are only for information
and may change in future versions. (See manual section 8.5)
In the long term we need to change fish/uri.c so that we can throw
proper errors.
If you have a struct containing ‘field’, eg:
type t = { field : int }
then previously to pattern-match on this type, eg. in function
parameters, you had to write:
let f { field = field } =
(* ... use field ... *)
In OCaml >= 3.12 it is possible to abbreviate cases where the field
being matched and the variable being bound have the same name, so now
you can just write:
let f { field } =
(* ... use field ... *)
(Similarly for a field prefixed by a Module name you can use
‘{ Module.field }’ instead of ‘{ Module.field = field }’).
This style is widely used inside the OCaml compiler sources, and is
briefer than the long form, so it makes sense to use it. Furthermore
there was one place in virt-dib where we are already using this new
style, so the old code did not compile on OCaml < 3.12.
See also:
https://forge.ocamlcore.org/docman/view.php/77/112/leroy-cug2010.pdf
This directory which previously contained random modules and functions
now has an official purpose: to be the place for any OCaml utility
needed by the OCaml virt tools.
This is just code movement, I didn't (yet) rename or move any of the
modules.
systemd defined an /etc/machine-id file which is supposed to contain a
unique, unchanging ID for the host. This file is initially zero-sized
and is meant to be set by systemd on the first boot of the system. In
virt-builder Fedora templates, the file is empty.
Unfortunately the Fedora kernel %post script requires the machine-id
to have been set, else the script exits with an error:
Running scriptlet: kernel-core-4.12.13-300.fc26.x86_64 209/209
Could not determine your machine ID from /etc/machine-id.
Please run 'systemd-machine-id-setup' as root. See man:machine-id(5)
warning: %posttrans(kernel-core-4.12.13-300.fc26.x86_64) scriptlet failed, exit status 1
This also leaves the kernel package half-installed. The files are
present in the filesystem, but important initialization is not done,
in particular the vmlinuz file is not copied into /boot.
A simple reproducer for this problem is:
$ virt-builder fedora-26 --update
which will leave the image with a half-installed kernel. (Add -v -x
to see the error above amongst the debug output).
This change makes virt-customize set /etc/machine-id to a random value
if the file exists and is zero sized. This is done unconditionally at
the same time as setting the random seed (a similar issue), and before
running any customize options such as installing or updating packages.
These are generated in many different ways in the various
subdirectories, and sometimes not generated correctly. Introduce a
script to do this in one place, and hopefully correctly.
This is mostly simple refactoring, but I got rid of a couple of
things:
(1) The ‘make depend’ rule doesn't appear to be needed. automake (or
make?) seems to rebuild the ‘.depend’ file automatically just because
it is included.
(2) I got rid of the hairy path rewriting sed expression. Possibly
that is needed for srcdir != builddir.
oraclelinux is the same as redhat/centos/scientificlinux, so add it
where it is missing. This fixes amongst other things, running sysprep
on an Oracle Linux image where it would previously fail operations like
setting the hostname, saying that it was not supported on that distro.
Currently we install the firstboot service under systemd target
‘default.target’. This change simply factors out this name.
Note that the name is not factored out in the code which deletes the
old ‘/etc/systemd/system/default.target.wants/firstboot.service’ file,
since that would have always been installed in the same location.
Create a module ‘C_utils’ containing functions like ‘drive_name’ and
‘shell_unquote’ which come from the C utilities.
The new directory ‘common/mlutils’ also contains the ‘Unix_utils’
wrappers around POSIX functions missing from the OCaml stdlib.
The new module ‘Std_utils’ contains only functions which are pure
OCaml and depend only on the OCaml stdlib. Therefore these functions
may be used by the generator.
The new module is moved to ‘common/mlstdutils’.
This also removes the "<stdlib>" hack, and the code which copied the
library around.
Also ‘Guestfs_config’, ‘Libdir’ and ‘StringMap’ modules are moved
since these are essentially the same.
The bulk of this change is just updating files which use
‘open Common_utils’ to add ‘open Std_utils’ where necessary.
Only in end-user messages and documentation. This change was done
mostly mechanically using the Perl script attached below.
I also changed don't -> don’t etc and made some other simple fixes.
See also: https://www.cl.cam.ac.uk/~mgk25/ucs/quotes.html
----------
#!/usr/bin/perl -w
use strict;
use Locale::PO;
my $re = qr{'([-\w%.,=?*/]+)'};
my %files = ();
foreach my $filename ("po/libguestfs.pot", "po-docs/libguestfs-docs.pot") {
my $poref = Locale::PO->load_file_asarray($filename);
foreach my $po (@$poref) {
if ($po->msgid =~ $re) {
my @refs = split /\s+/, $po->reference;
foreach my $ref (@refs) {
my ($file, $lineno) = split /:/, $ref, 2;
$file =~ s{^\.\./}{};
if (exists $files{$file}) {
push @{$files{$file}}, $lineno;
} else {
$files{$file} = [$lineno];
}
}
}
}
}
foreach my $file (sort keys %files) {
unless (-w $file) {
warn "warning: $file is probably generated\n"; # have to edit generator
next;
}
my @lines = sort { $a <=> $b } @{$files{$file}};
#print "editing $file at lines ", join (", ", @lines), " ...\n";
open FILE, "<$file" or die "$file: $!";
my @all = ();
push @all, $_ while <FILE>;
close FILE;
my $ext = $file;
$ext =~ s/^.*\.//;
foreach (@lines) {
# Don't mess with verbatim sections in POD files.
next if $ext eq "pod" && $all[$_-1] =~ m/^ /;
unless ($all[$_-1] =~ $re) {
# this can happen for multi-line strings, have to edit it
# by hand
warn "warning: $file:$_ does not contain expected content\n";
next;
}
$all[$_-1] =~ s/$re/‘$1’/g;
}
rename "$file", "$file.bak";
open FILE, ">$file" or die "$file: $!";
print FILE $_ for @all;
close FILE;
my $mode = (stat ("$file.bak"))[2];
chmod ($mode & 0777, "$file");
}
