The majority of the tools have already options (--echo-keys &
--keys-from-stdin) to deal with LUKS credentials, although there is no
way to automatically provide credentials. --keys-from-stdin is
suboptimal, because it is a usable solution only when there is just one
device to open, and no other input passed via stdin to the tool (like
the commands for guestfish).
To overcome this limitation, introduce a new --key option in tools:
* --key /dev/device:file:/filename/with/key
* --key /dev/device:string:the-actual-key
this way it is possible to pass all the credentials needed for the
specific devices to open, with no risk of conflict with stdin, and also
in a secure way (when using the "file" way).
On the technical side: this adds a new "key_store" API for the C tools,
making sure it is used only when needed. Partially mirror it also for
the OCaml tools, although there will be a conversion to the C API
because the decryption helpers used are in the common C parts.
The guestfish prepared disks (-N option) all defaulted to 100M. This
has always been too small for btrfs (so for example ‘-N fs:btrfs’ has
failed for a long time), and can be too small for LVs with new LVM2 /
kernel >= 4.16.
This changes the default from 100M to 1G. The beginning of the 1.39
branch seems like a good time to make this change.
virt-builder-repository allows users to easily create or update
a virt-builder source repository out of disk images. The tool can
be run in either interactive or automated mode.
Since glibc-2.26 removed the /usr/include/rpc/rpc.h, it caused problem even
though probing for libtirpc has taken place and worked properly. The problem is
that RPC_CFLAGS (usually `-I/usr/include/tirpc` on Linux) was not used in places
where it was needed. So the build failed with messages like the following:
In file included from guestfs_protocol.c:6:0:
guestfs_protocol.h:9:10: fatal error: rpc/rpc.h: No such file or directory
#include <rpc/rpc.h>
^~~~~~~~~~~
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
The device name is only used by guestfish (when using the -N option to
prepare drives). We constructed the device name very naively,
basically ‘sprintf ("/dev/sd%c", next_drive)’.
This stores the device index instead, and only constructs the device
name in guestfish. Also the device name is constructed properly using
guestfs_int_drive_name so it can cope with #drives > 26.
As a side effect of this change we can remove the extra parameter of
the add_drives macro.
Thanks: Pino Toscano
Only in end-user messages and documentation. This change was done
mostly mechanically using the Perl script attached below.
I also changed don't -> don’t etc and made some other simple fixes.
See also: https://www.cl.cam.ac.uk/~mgk25/ucs/quotes.html
----------
#!/usr/bin/perl -w
use strict;
use Locale::PO;
my $re = qr{'([-\w%.,=?*/]+)'};
my %files = ();
foreach my $filename ("po/libguestfs.pot", "po-docs/libguestfs-docs.pot") {
my $poref = Locale::PO->load_file_asarray($filename);
foreach my $po (@$poref) {
if ($po->msgid =~ $re) {
my @refs = split /\s+/, $po->reference;
foreach my $ref (@refs) {
my ($file, $lineno) = split /:/, $ref, 2;
$file =~ s{^\.\./}{};
if (exists $files{$file}) {
push @{$files{$file}}, $lineno;
} else {
$files{$file} = [$lineno];
}
}
}
}
}
foreach my $file (sort keys %files) {
unless (-w $file) {
warn "warning: $file is probably generated\n"; # have to edit generator
next;
}
my @lines = sort { $a <=> $b } @{$files{$file}};
#print "editing $file at lines ", join (", ", @lines), " ...\n";
open FILE, "<$file" or die "$file: $!";
my @all = ();
push @all, $_ while <FILE>;
close FILE;
my $ext = $file;
$ext =~ s/^.*\.//;
foreach (@lines) {
# Don't mess with verbatim sections in POD files.
next if $ext eq "pod" && $all[$_-1] =~ m/^ /;
unless ($all[$_-1] =~ $re) {
# this can happen for multi-line strings, have to edit it
# by hand
warn "warning: $file:$_ does not contain expected content\n";
next;
}
$all[$_-1] =~ s/$re/‘$1’/g;
}
rename "$file", "$file.bak";
open FILE, ">$file" or die "$file: $!";
print FILE $_ for @all;
close FILE;
my $mode = (stat ("$file.bak"))[2];
chmod ($mode & 0777, "$file");
}
glibc in Fedora is currently configured with `--enable-obsolete-rpc',
so I guess we can see which way the wind is blowing.
(1) This changes our configure script to prefer libtirpc if it is
available.
If libtirpc is _not_ available then:
(a) Headers must be located in <rpc/xdr.h>, or the user must supply
the right CFLAGS.
(b) XDR functions must be located in one of -lportablexdr, -lrpc,
-lxdr, -lnsl or no library at all (ie. -lc), and the user must set
LDFLAGS if needed.
(2) We no longer add these paths automatically to $(CFLAGS)/$(LIBS).
Any part of libguestfs which needs <rpc/*.h> or the xdr_* functions
must use $(RPC_CFLAGS)/$(RPC_LIBS) explicitly.
(3) Previously Mac OS X had a workaround for the broken 64 bit support
in the supplied rpcgen. This workaround "activates" all the time if
you use tirpc, so breaking Linux after the above changes. tirpc is
supported on OS X, so I think it's just better to use that rather than
the broken rpcgen. For that reason I removed the workaround
completely.
Thanks: Roy Keene
Apply this change across all the shell scripts containing tests.
Additionally this defines the environment variables $abs_srcdir,
$abs_builddir, $top_srcdir, $top_builddir, $abs_top_srcdir and
$abs_top_builddir which can now be used throughout test scripts.
Commit 004de6cf45 was not sufficient to
solve this problem. We are also using gperf in guestfish. Rewrite
the code in the same way to avoid having to prototype the hash
function.
Generate the random filename using our utility function
guestfs_int_random_string. This also means that we will not need to
call srandom() in guestfish or virt-edit.
Three more pieces of common code are moved under the common/
subdirectory. This is just code motion.
Note that windows.[ch] wasn't even being used by guestfish. That code
was only used in other virt tools.
This is mostly code motion but:
(1) I had to remove the compile-time COMPILING_GUESTFISH and
COMPILING_VIRT_RESCUE macros and replace them with runtime constants
and checks.
(2) I moved the fish/config.c file into this library.
Just code motion.
This commit makes it clearer what is a utility and what is part of the
library. It also makes it clear that we should rename:
guestfs-internal-frontend.h -> utils.h
guestfs-internal-frontend-cleanups.h -> cleanups.h (?)
but this commit does not make that change.
Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
This makes a number of small changes to how the internal documentation
is generated and what can be documented.
Header files are now permitted to contain internal documentation.
This works in the same wasy as .c files.
Also documentation can be added for structs as well as functions.
This commit also adds documentation to some header files and structs,
and fixes a few places which contained broken header documentation.
When TESTS_ENVIRONMENT already uses 'run', the VG variable
doesn't also need to use 'run'.
The specific problem is that if the command contains newlines
then double invocations of the 'run' script fails (in libtool).
ie the following command failed causing errors in check-valgrind:
$VG virt-builder phony-fedora \
-v --no-cache --no-check-signature $no_network \
...
--write '/etc/append4:line1
' \
...
Create a new libfishcommon convenience static library to build just once
(instead of 12 times!) the majority of the guestfish sources used in the
rest of the C tools (mostly for command line stuff, inspection, and
mount).
The notable exceptions not using libfishcommon are guestfish itself, and
virt-rescue: both need to build at least one of the common sources using
additional CPPFLAGS.
All the define was doing in options.h was masking the declaration of
inspect_mount_root, which was always built-in in inspect.c (because of
the unconditional #define there) anyway.
Since this is common code used by all the C tools, try to avoid extra
knobs which add different code paths for no benefit.
Make use of the recently added 'getprogname' module in gnulib: replace
our guestfs_int_program_name with the getprogname() provided by the
module, since it does the same thing, and in a portable way.
As consequence of the above, use gnulib in a couple of tests that use
getprogname().
Since guestfs_int_program_name is gone, drop the configure checks
associated with it.
By adding common CLEANFILES and DISTCLEANFILES variables to
common-rules.mk, we can remove these from most other Makefiles, and
also clean files more consistently.
Note that bin_PROGRAMS are already cleaned by 'make clean', so I
removed cases where these were unnecessarily added to CLEANFILES.
By moving these two functions out of the common options parsing code,
it means we don't need to depend on all the other machinery of options
parsing, such as the global variables ("verbose"), libconfig, etc.
One -Wnull-dereference warning is real: we deliberately cause a
segfault in one of the tests.
There is a -Wshift-overflow bug in a Gtk 2 header.
The others are the result of shortcomings in GCC.
In all cases we have to add GCC diagnostic overrides to ignore
the warnings when compiling with ./configure --enable-werror.
For OCaml tools this does essentially nothing useful because the
--help output is automatically generated from the options, and so
cannot be wrong. However for C tools this is a useful check.
It would be nice to generate C tools --help output, but there isn't
enough information in the getopt data to do that.
This commit also includes fixes to the --help output for a few tools.
podcheck.pl is run as part of the tests to perform various checks on
the documentation and the tool.
Currently we check only that the documented options matches the
options that the tool implements and vice versa. This commit would
also allow us (in future) to check --help, --long-options,
--short-options, --version output.
This commit includes scripts to run the tests and various fixes to the
manual pages to ensure that the tests pass.
For guestfish, guestmount, remove '?' from short options. Currently
those tools don't process -?, so I believe these are erroneous:
$ guestfish -\?
Try `guestfish --help' for more information.
For virt-format, the -c, -d and -q options are removed. These options
just give errors because they appear in the short options list but not
in the case statement.
On Linux 4.7.0 the old code was giving -EINVAL errors. This rewrites
the code based on online sources, and it works on at least RHEL 7,
Linux 4.4 and 4.7. Also checked with valgrind.
When possible, make the disk image format explicit when invoking tools
or using add-drive. This avoids warnings from qemu about the unspecified
format for the image, and also makes qemu slightly faster (skipping the
disk image probing).
Tests checking the image probing are not touched.
This changes also:
- old-style invocations of tools (`$tool $filename`) into new style
(`$tool -a $filename`)
- add-drive-ro/add-drive-with-if guestfish commands into add/add-drive
with explicit readonly/iface arguments
There should be no change in the tests results.
