Because of vagaries in how the debug messages get printed they can get
split over lines:
gulibguestfs: trace: truncate_size = 0
estfsd: => truncate_size (0xc8) took 0.00 secs
and the code which filtered out those messages before comparison was
failing. This makes the code more liberal on how it matches and
filters out these messages.
Every .ml file now needs a corresponding .mli file so that
dependencies are generated correctly. The check-mli.sh script picks
up the problem:
FAIL: check-mli.sh
==================
./builder/osinfo_config.ml: missing ./builder/osinfo_config.mli
FAIL check-mli.sh (exit status: 1)
Fixes commit 46abca0ec3
and commit 02184f55f9.
If running the external command fails in "argv mode" (ie. when
not using the shell), then exit with either 126 or 127 as defined
by POSIX.
This is mostly the same as what bash does, see
execute_cmd.c:shell_execve in the bash sources.
Note: saving errno around perror(3) if necessary, otherwise you will
see different behaviour between verbose and non-verbose mode. In
non-verbose mode, perror(3) tried to print to a closed file
descriptor, failing and overwriting errno. That took some time to
debug.
This safe wrapper around Unix.openfile ensures that exceptions
escaping cannot leave unclosed files.
There are only a few places in the code where this wrapper can be used
currently. There are other occurences of Unix.openfile but they are
not suitable for replacement.
In OCaml 4.06 we need to link to daemon to libasmrun_pic.a (the OCaml
runtime linked with -fPIC). Otherwise you will see many errors like
this:
/usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(startup_aux.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(startup.o): relocation R_X86_64_32S against symbol `caml_data_segments' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(fail.o): relocation R_X86_64_32 against symbol `caml_exn_Failure' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib64/ocaml/libasmrun.a(roots.o): relocation R_X86_64_32 against symbol `caml_frametable' can not be used when making a shared object; recompile with -fPIC
This commit use a configure-time test to find the best OCaml runtime.
The immediate issue is with Fedora/ppc64 and /ppc64le which currently
use extended partitions, breaking the virt-builder ‘--size’ parameter,
eg:
$ virt-builder --arch ppc64le fedora-26 --size 20G
...
[ 21.6] Resizing (using virt-resize) to expand the disk to 20.0G
virt-resize: error: /dev/sda5: partition not found in the source disk image
(this error came from '--expand' option on the command line). Try running
this command: virt-filesystems --partitions --long -a /var/tmp/vbf67b8c.img
However more generally MBR is broken and should die. GPT is supported
by all modern virtual bootloaders, so just default to it.
Notes:
* This is different from mandating a UEFI bootloader.
* I am not planning to rebuild any existing images except the
F26 ppc64 & ppc64le ones.
libfuse prints errors on stderr and there seems to be no way to change
that. It doesn't make any attempt to preserve errno either, so
printing an error based on errno is wrong.
Thanks: Assaf Gordon.
Add a function to properly write virt-builder source index entries.
Note that this function is very similar to Index.print_entry that is
meant for debugging purposes.
Add a simple OCaml-based implementation of reader of the osinfo-db:
the only interface is an iterator that invokes an user-supplied
function with each XML file found.
This implementation behaves like the current C implementation, and
still supports the old libosinfo db.
[RWMJ: Fixed trailing whitespace]
While 1.1.14 will technically still work, 1.1.16 adds the
‘selinux=yes|no’ flag to ‘--dump-config’ output. It's also the more
widely available and tested version at the time of writing.
Thanks: Ming Xie.
- move the space needed by mountpoint mapping to an own function,
outside of the checking loop
- remove the minimum limit of 100M on partitions, since existing
partitions that we check (e.g. /boot) may be smaller than that
- in case /boot is not on a separate mountpoint, enforce the space check
on the root mountpoint instead, since it is where /boot is in that
case
If visit_guest() fails, then it returns a null pointer; later on,
free_tree() is called unconditionally on the variables, thus
dereferencing null pointers.
As a solution, make free_tree() behave like free(), i.e. become no-op
with null pointers.
Remove two wrong error handling situations in btrfs_balance_status:
- if the calloc for 'ret' fails, the 'goto error' would try to
dereference the resulting null pointer (to free the items of the
return struct); hence, just return null directly, instead of jumping
to 'error'
- if the strdup() for 'btrfsbalance_status' fails, then the directly
return would leak 'ret'
When the LDM code was converted to the CLEANUP_* macros, a free()
invocation for a CLEANUP_FREE variable was left in the
ldmtool_diskgroup_volumes implementation, causing double-free on
success.
Updates commit 950951c67d.
Linux.file_owner is not used by any other function, so remove it.
Linux.is_file_owned is only used when removing kmod-xenpv on old RHEL
releases, and so is only required to work for RPM.
The old file_owner/is_file_owned functions were completely broken for
the RPM case. This replaces them with a simpler, working
implementation of is_file_owned only.
Thanks: Ming Xie for finding and reporting the original bug.
Some hypervisors including KVM and VMware expose SATA as a separate
device emulation from IDE. SATA devices are an evolution of IDE, but
in Linux they are handled like SCSI disks.
This change adds a new Source_SATA type to model this. In the Linux
conversion module we treat them like SCSI (ie. with "sd" prefix).
It's not very clear how to handle them for target bus assignment, so I
arbitrarily mapped them to the SCSI bus. It's possible we need a new
bus type to handle SATA properly. As the code only applies to
removable devices it's not very important to get this right.
Currently test820RHBZ912499.py fails with libvirt because libvirt
still doesn't work around qemu locking properly. Allow this test to
be skipped on a temporary basis using SKIP_TEST820RHBZ912499_PY=1
Latest libvirt (3.8, I think) made a very small adjustment to the
format of the generated -drive parameter when using iscsi, from:
file=iscsi://1.2.3.4:1234/iqn.2003-01.org.linux-iscsi.fedora,
to:
file=iscsi://1.2.3.4:1234/iqn.2003-01.org.linux-iscsi.fedora/0,
This makes a corresponding change to the test so that both patterns
can be recognized.
Commit 0b631d739b changed the debug
message format from the main loop in guestfsd. This test tried to
filter out these messages when comparing output, but that stopped
working. The problem was only evident when enabling debugging while
running the tests.
Previously we were abusing the ‘input#source ()’ and
‘output#prepare_targets’ methods to perform environmental checks.
Small refactoring, no functional change.
Make use of parse_key_value_strings to slightly simplify the parsing of
mdadm output, and the parsing of os-release, and lsb-release files.
In particular, the parsing of lsb-release is simplified a lot, bringing
it much like the parsing of os-release.
Add a simple helper to turn a list of strings into key/value pairs,
splitting by '=', with the possibility to apply a function to unquote
values.
Add also a simple unquote function.
This implements a missing feature from old virt-v2v, namely being able
to cope with a guest with snapshots. Note this only converts the top
(latest) snapshot. As in old virt-v2v it does NOT convert the whole
chain of snapshots.
This module had a selection of functions taking a different mix of
parameters and doing slightly different things. You could call one
function to return an https://... URL, or another function to return a
qemu URL, and there was a third function to get the session cookie but
you generally had to call that anyway (and it was implicitly called
when making the qemu URL!)
Integrate these into a single function which returns a struct
returning all possible values.
This is conceptually refactoring, except that the session cookie is no
longer memoized, but we didn't use this feature (of calling
get_session_cookie multiple times) anyway.
With modern libvirt, when fetching the XML of a VMware guest libvirt
passes us the datacenter path (dcpath). However with older libvirt we
had to guess this value, or else the user had to supply it on the
command line.
This commit removes all the guessing code and the --dcpath parameter
(which will now give an error).
This requires libvirt >= 1.2.20 for virt-v2v, released Oct 2015.
In case os-release does not contain VERSION_ID, and it is not an
exception (like Void Linux), then discard the results got from
os-release.
This matches what the old C code did -- e.g. see
commit 32d19e3289, and the changes in
commit 0251c0fcaa.
Instead of using ‘xpath_(string|int|int64)_default’ we can write the
equivalent code using ‘Option.default’. This is not quite so concise,
but may be easier to understand.
eg:
xpath_int_default xctx "xpath_expr" 10
->
Option.default 10 (xpath_int xctx "xpath_expr")
Inspired by ocaml-extlib, introduce a module for handling option
types.
We already had the ‘may’ function (which becomes ‘Option.may’). This
adds also ‘Option.map’ (unused), and ‘Option.default’ functions.
Note this does *not* introduce the unsafe ‘Option.get’ function from
extlib.
When testing if sudo -n requires a password, we tested for the prompt
earlier than testing for the magic sudo message ‘a password is
required’.
Since the shell will print the prompt just after the sudo message:
prompt$ sudo -n virt-v2v --version
sudo: a password is required
prompt$
the prompt nearly always matched and we missed the magic sudo message.
(The exception is in the case where we are running everything on
localhost where the sudo message could be read in a single call to
read(2) without seeing the prompt immediately afterwards. Even this
exception was non-deterministic.)
By swapping the priority of the sudo message and prompt we avoid this.
Looking at the debug output (enabled by editing common/miniexpect)
makes this clearer:
DEBUG: writing: sudo -n virt-v2v --version
DEBUG: buffer content: sudo: a password is required
ESC]0;rjones@hamr:~^G###bphcxtq5###
Fixes commit 5b6a8e0862.
Thanks: Ming Xie.
The upstream ocaml-augeas defines macros which may not be used in all
the cases, triggering a warning (which turns into error with -Werror)
because of -Wunused-macros.
Since the code is correct, disable -Wunused-macros with
-Wno-unused-macros before updating the embedded copy of ocaml-augeas.
The embedded copy of ocaml-augeas does Val_int(-1), which in turns
triggers warnings in newer GCC versions about "left shift of negative
value". The issue actually lies in the OCaml headers (mlvalues.h in
particular), and it was fixed in newer OCaml versions.
Since the code is actually correct, disable -Wshift-negative-value with
-Wno-shift-negative-value (checking whether the compiler has it).
Use also custom_compare_ext_default for the compare_ext field.
According to the git logs, this was introduced in OCaml 3.12.1, which is
earlier than out minimum required version.
mlaugeas is not touched by this change, since it is a copy of a 3rd
party library (and thus it will be fixed there first).
This function returns -1 if things like fork(2) fail, so it is right
to return an error here. If the PBMTEXT command fails then that's a
NOT_FOUND case.
We reimplemented some functions which can now be found in the OCaml
stdlib since 4.01 (or earlier). The functions I have dropped are:
- String.map
- |>
- iteri (replaced by List.iteri)
- mapi (replaced by List.mapi)
Note that our definition of iteri was slightly wrong: the type of the
function parameter was too wide, allowing (int -> 'a -> 'b) instead of
(int -> 'a -> unit).
I also added this new function to the Std_utils.String module as an
export from stdlib String:
- String.iteri
Thanks: Pino Toscano
Change the Planner.plan function so it returns an optional type. This
means it no longer raises Failure "plan" on error, so we can both
force the caller to deal with the error case and avoid Warning 52.
This avoids warning 52 in OCaml code such as:
try URI.parse_uri arg
with Invalid_argument "URI.parse_uri" -> ...
which prints:
Warning 52: Code should not depend on the actual values of
this constructor's arguments. They are only for information
and may change in future versions. (See manual section 8.5)
In the long term we need to change fish/uri.c so that we can throw
proper errors.
If you have a struct containing ‘field’, eg:
type t = { field : int }
then previously to pattern-match on this type, eg. in function
parameters, you had to write:
let f { field = field } =
(* ... use field ... *)
In OCaml >= 3.12 it is possible to abbreviate cases where the field
being matched and the variable being bound have the same name, so now
you can just write:
let f { field } =
(* ... use field ... *)
(Similarly for a field prefixed by a Module name you can use
‘{ Module.field }’ instead of ‘{ Module.field = field }’).
This style is widely used inside the OCaml compiler sources, and is
briefer than the long form, so it makes sense to use it. Furthermore
there was one place in virt-dib where we are already using this new
style, so the old code did not compile on OCaml < 3.12.
See also:
https://forge.ocamlcore.org/docman/view.php/77/112/leroy-cug2010.pdf
common/mlutils: Unix_utils.StatVFS.statvfs: This commit implements a
full-featured binding for the statvfs(3) function.
We then use this to reimplement the daemon statvfs API in OCaml.
Note that the Gnulib fallback is dropped in this commit. It
previously referenced non-existent field names in the fs_usage struct
so it didn't work. Also it's not necessary as POSIX has supported
statvfs(3) since 2001, it's supported in *BSD, macOS > 10.4, and there
is already a Windows fallback.
This directory which previously contained random modules and functions
now has an official purpose: to be the place for any OCaml utility
needed by the OCaml virt tools.
This is just code movement, I didn't (yet) rename or move any of the
modules.
Although it's not too likely that these libraries will contain
translatable strings, it's consistent to add them to po/POTFILES-ml
because mllib/*.ml are also in this file.
I also renamed the functions with the correct ‘guestfs_int_*’
namespace.
The corresponding change is made for OCaml C_utils and Visit.
No functional change, just code motion.
systemd defined an /etc/machine-id file which is supposed to contain a
unique, unchanging ID for the host. This file is initially zero-sized
and is meant to be set by systemd on the first boot of the system. In
virt-builder Fedora templates, the file is empty.
Unfortunately the Fedora kernel %post script requires the machine-id
to have been set, else the script exits with an error:
Running scriptlet: kernel-core-4.12.13-300.fc26.x86_64 209/209
Could not determine your machine ID from /etc/machine-id.
Please run 'systemd-machine-id-setup' as root. See man:machine-id(5)
warning: %posttrans(kernel-core-4.12.13-300.fc26.x86_64) scriptlet failed, exit status 1
This also leaves the kernel package half-installed. The files are
present in the filesystem, but important initialization is not done,
in particular the vmlinuz file is not copied into /boot.
A simple reproducer for this problem is:
$ virt-builder fedora-26 --update
which will leave the image with a half-installed kernel. (Add -v -x
to see the error above amongst the debug output).
This change makes virt-customize set /etc/machine-id to a random value
if the file exists and is zero sized. This is done unconditionally at
the same time as setting the random seed (a similar issue), and before
running any customize options such as installing or updating packages.
The old virt-v2v code ignored boot kernels with names like
"/boot/vmlinuz-*.rpmsave". The transscribed code did not because the
Str module requires ‘|’ to be escaped as ‘\|’.
This changes the code to use a simpler test.
Thanks: Pino Toscano
Commit dbe0b69f24 transscribed the Perl
regexp incorrectly so that it only matched the impossible case of
‘resume=/dev/X’ for a single non-whitespace character X.
This fixes the regular expression, referencing back to the original
Perl code in virt-v2v.
In the original conversion of virt-v2v from Perl
(commit 0131d6f666), the Perl regular
expression was incorrectly transscribed. ‘\b’ was not converted to
‘\\b’ so the new regexp was looking for an ASCII BEL character, not a
word boundary. Also ‘|’ was used, but Str requires ‘\|’ (ie. "\\|"
in the final source).
To fix these problems I converted the code to use PCRE, and went back
to the original virt-v2v code (virt-v2v.git:
lib/Sys/VirtConvert/Converter/Linux.pm) to find out what the Perl
regular expression should have been.
Note I have also removed ‘.*’ at the beginning and end of the regexp
because PCRE regexps are not anchored.
Only five simple flags are allowed so far, and not all of them are
actually used in any code. They are:
~anchored / PCRE_ANCHORED - implicit ^...$ around regexp
~caseless / PCRE_CASELESS - fold upper and lower case
~dotall / PCRE_DOTALL - ‘.’ matches anything
~extended / PCRE_EXTENDED - extended regular expressions
~multiline / PCRE_MULTILINE - ^ and $ match lines within the subject string
If configured with --without-ocaml, the build might fail because the
fix added in df5bd5741b was not active.
According to the automake documentation, it should be enough to set
BUILT_SOURCES.
This pure refactoring changes the convert () functions so that the
conversion operations are listed first, followed by the
implementations of those operations.
Add a --with-distro=ID argument for configure, so it is possible to
manually specify the distro to use for the packages (in case os-release
does not provide ID=.., or the ID is not recognized yet).
In the case when --with-distro is not set, keep doing the autodetection,
but using os-release only, i.e. dropping the checks for all the other
-release files -- since there is --with-distro, older distros with no
os-release can still be used.
RWMJ: Add documentation to guestfs-building(1).
Here is an example of a failure with the previous dependency
calculation:
$ make -j5
make: Entering directory '/home/rjones/d/libguestfs/common/mlstdutils'
OCAMLOPT guestfs_config.cmx
CC libmlstdutils_a-dummy.o
OCAMLOPT libdir.cmx
OCAMLCMI stringMap.cmi
OCAMLCMI stringSet.cmi
OCAMLCMI guestfs_config.cmi
OCAMLCMI std_utils.cmi
OCAMLC guestfs_config.cmo
OCAMLC libdir.cmo
OCAMLC stringMap.cmo
OCAMLC stringSet.cmo
OCAMLOPT stringMap.cmx
OCAMLOPT stringSet.cmx
OCAMLOPT std_utils.cmx
OCAMLC std_utils.cmo
ocamlfind ocamlc -package str,unix -I . -a guestfs_config.cmo libdir.cmo stringMap.cmo stringSet.cmo std_utils.cmo -o mlstdutils.cma
ocamlfind ocamlopt -package str,unix -I . -a guestfs_config.cmx libdir.cmx stringMap.cmx stringSet.cmx std_utils.cmx -o mlstdutils.cmxa
AR libmlstdutils.a
File "_none_", line 1:
Error: Files std_utils.cmx and guestfs_config.cmx
make inconsistent assumptions over interface Guestfs_config
make: *** [Makefile:2523: mlstdutils.cmxa] Error 2
make: Leaving directory '/home/rjones/d/libguestfs/common/mlstdutils'
What seems to be happening is that there is a rule:
std_utils.cmx : guestfs_config.cmi guestfs_config.cmx [...]
In this case, make chose to build guestfs_config.cmx and
guestfs_config.cmi in parallel (see the first 5 rules above). However
building guestfs_config.cmx also creates guestfs_config.cmi
(implicitly - this is not expressed in make dependencies, and make
doesn't "know" that guestfs_config.cmi has already been created).
Unfortunately the OCaml compiler doesn't create output files
atomically. Worse than that, it creates an intermediate version of
the output file, reads it back and then creates the final version. It
seems if the build of std_utils.cmi reads this intermediate version.
In any case, Std_utils sees the wrong hash of the Guestfs_config
module.
The above only happens where we have a *.ml file without a
corresponding *.mli file. That is because if there is a *.mli file,
ocamldep generates slightly different dependencies:
guestfs_config.cmx [...] : guestfs_config.cmi guestfs_config.ml
std_utils.cmx : guestfs_config.cmi guestfs_config.cmx [...]
std_utils.cmx still depends on both files, but there is an extra rule
which ensures that guestfs_config.cmx isn't built in parallel with
guestfs_config.cmi.
I tested this change by running this command:
$ while ( rm common/mlstdutils/.depend; make -C common/mlstdutils/ clean && make -C common/mlstdutils/ ) >& /tmp/log; do echo -n .; done
Before the change it would fail after about 100 iterations. After the
change it ran for 10000s iterations and did not fail ever.
Updates commit 6d0ad49d5e.
Since commit 65cfecb0f5,
‘guestfs_int_download_to_tmp’ was buggy because it did not deal with
the case where a guest had multiple roots. It cached the downloaded
file under a single name which was not distinguished by which root we
were looking at. As a result, if you inspected (eg.) the icon on such
a guest, then in some circumstances the same icon could be returned
for all roots (incorrectly).
This changes the function in a few ways:
- Don't cache downloaded files. It wasn't a useful feature of the
function in most scenarios. Instead a new name is generated for
every call.
- Use guestfs_int_make_temp_path.
- Allow an extension to be specified.
In the function ‘guestfs_int_make_temp_path’, allow an optional
extension to be specified. For example:
r = guestfs_int_make_temp_path (g, "ls", "txt");
will create paths like ‘<tmpdir>/ls123.txt’.
NULL can also be passed for the extension, which means no extension.
Move the last remaining function ‘guestfs_int_download_to_tmp’ to
lib/inspect-icon.c (the main, but not only user of this function).
Then remove lib/inspect.c entirely.
This is not quite code motion because I updated the comment for the
function to reflect what it does in reality.
Commit 4d3601eb4e made it clear that
ocaml-hivex is required, implying also that hivex is required.
However the configure test was still optional, and if you built
libguestfs without hivex you got a very long and confusing error in
the daemon/ subdirectory.
Thanks: Cédric Bosdonnat
We permit the following constructs in libguestfs code:
if (guestfs_some_call (g) == -1) {
fprintf (stderr, "failed: error is %s\n", guestfs_last_error (g));
}
and:
guestfs_push_error_handler (g, NULL, NULL);
guestfs_some_call (g);
guestfs_pop_error_handler (g);
Neither of these would be safe if we allowed the handle to be used
from threads concurrently, since the error string or error handler
could be changed by another thread.
Solve this in approximately the same way that libvirt does: by making
the error, current error handler, and stack of error handlers use
thread-local storage (TLS).
The implementation is not entirely straightforward, mainly because
POSIX doesn't give us useful destructor behaviour, so effectively we
end up creating our own destructor using a linked list.
Note that you have to set the error handler in each thread separately,
which is an API change (eg: if you set the error handler in one
thread, then pass the handle 'g' to another thread, the error handler
in the second thread appears to have reset itself back to the default
error handler). I haven't yet worked out a better way to solve this.
Acquire the per-handle lock on entering each public API function.
The lock is released by a cleanup handler, so we only need to use the
ACQUIRE_LOCK_FOR_CURRENT_SCOPE macro at the top of each function.
Note this means we require __attribute__((cleanup)). On platforms
where this is not supported, the code will probably hang whenever a
libguestfs function is called.
The only definitive list of public APIs is found indirectly in the
generator (in generator/c.ml : globals).
Define the types which will be used to communicate between the
different parts of the inspection code. The main types are:
fs corresponds to ‘struct inspect_fs’ in C code
root no direct correspondence with the C code, but in the C
code, ‘inspect_fs’ was overloaded to store roots
inspection_data
the inspection data which is incrementally collected about
each filesystem as we perform inspection steps
Other types have simple and obvious correspondences with the
equivalent C code.
Add some utility function which will be used by inspection.
Note that this commit has no effect on its own, it just links extra
dead code into the daemon.
After we move inspection code to the daemon, the library will no
longer have access to ‘struct inspect_fs’, and so we won't be able to
prefix downloads with the "root filesystem number".
Just remove this prefix (it's internal only). However it does mean
that this function can no longer cache downloaded files.
Unlike previous ‘daemon: Reimplement ...’ patches, this does not
reimplement the umount_all API completely (yet, but this
implementation could be completed in future and then replace the C
one). However it is necessary to have a version of umount_all which
we can call from the OCaml inspection code.
hivex has a function hivex_value_string. We were not calling it under
the mistaken belief that because hivex implements this using iconv,
the function wouldn't work inside the daemon. Instead we
reimplemented the functionality in the library.
This commit deprecates hivex_value_utf8 and removes the library side
code. It replaces it with a plain wrapper around hivex_value_string.
Thanks: Pino Toscano
This commit bundles the ocaml-augeas library (upstream here:
http://git.annexia.org/?p=ocaml-augeas.git;a=summary). It's identical
to the upstream version and should remain so.
We can work towards using system ocaml-augeas, when it's more widely
available.
In the case where we have a module.ml without a corresponding
module.mli file, ocamldep generates (correct) dependencies:
module.cmx module.cmi : module.ml
but we had no rule telling make how to generate the module.cmi
file from module.ml.
This didn't matter very much because when make came to build
module.cmx, the module.cmi file is generated as a side-effect.
However for highly parallel builds, the build ordering was still
incorrect. Any other module that depends on module.cmi could be built
in parallel. You would very occasionally see errors like this one:
File "_none_", line 1:
Error: Files index.cmx and utils.cmx
make inconsistent assumptions over interface Utils
Fixing this involves adding a ‘%.cmi: %.ml’ rule. However we have to
be careful that make doesn't run this rule instead of the ‘%.cmi: %.mli’
rule (if module.mli did exist). It turns out that GNU make says we
can depend on rule ordering in the Makefile for this.
I found that this only works correctly if we use "%"-style pattern
rules (not the ‘.ml.cmi:’ old-style rules).
This is *still* not a complete fix. Make still doesn't know that the
rules ‘%.cmo: %.ml’ and ‘%.cmx: %.ml’ also build the .cmi file as a
side-effect, so you can still occasionally see build failures.
However I could not work out how to add the extra information to the
dependencies without causing make itself to go into an infinite loop.
It may be that in the end we will have to get rid of pattern rules
completely and generate the complete OCaml rule set.
Because there are no *.mli in either of these directories, the .depend
file was not being built (silently). So dependencies were wrong.
Fixes commit 692195c6ba.
We ran some sed scripts over the output of ocamldep to get them into a
single line format. This actually broke the output of ocamldep,
combining multiple dependencies into single lines. I didn't debug
exactly why our sed rules were broken because modern ocamldep has an
option (-one-line) which make this hacking unnecessary.
Another useful option (-all) is added so that all dependencies are
made explicit.
Since old ocamldep in RHEL 6 doesn't support these options, we have to
detect them at configure time.
The historical reason we were trying to combine entries into a single
line was so we could sort them. That was only needed back when we
used to commit the dependency files (so the files were stable), but we
haven't committed dependency files in a long time.
QEMU >= 2.10 started to do mandatory locking. This checks the QMP
schema to see if we are using that version of qemu. (Note it is
sometimes disabled in downstream builds, and it was also enabled in
upstream prereleases with version 2.9.9x, so we cannot just check the
version number).
Rename the cache files like ‘qemu.stat’ etc so they include the qemu
binary "key" (ie. size and mtime) in the name. This allows a single
user to use multiple qemu binaries in parallel without conflicts.
This adds an extra test using QMP (the QEMU Monitor Protocol). This
allows us to get extra information about the qemu binary beyond what
is available from the version number or ‘qemu -help’.
The previous code duplicated a lot of common code for reading and
writing the cache file per data field. This change simply factors out
that common code. This makes it simpler to add new tests in future.
This is just refactoring, it should have no effect.
Rather unnecessarily this function returned the parsed qemu version.
This complicates further refactoring, so I have changed the function
not to return this, and instead there is a separate function you have
to call to get the version struct (‘guestfs_int_qemu_version’).
Apart from a tiny amount of extra copying this is simply refactoring
of the interface between the direct-mode backend and the qemu query
functions.
Unlike ordinary guestfs_int_cmd_run, the pipe version did not print
the command it was about to run when debugging was enabled.
Fixes commit e98bb86929.
These have been supported since forever, but it's good to have an
indication in the ‘virt-v2v --machine-readable’ output that they are
available:
$ virt-v2v --machine-readable
virt-v2v
libguestfs-rewrite
vcenter-https
xen-ssh
colours-option
vdsm-compat-option
[etc]
Thanks: Pino Toscano.
virt-v2v obviously cannot convert this kind of devices, since they are
specific to the host of the hypervisor. Thus, emit a warning about the
presence of passthrough host devices, so at least this can be noticed
when converting a guest.
Avoid passing an empty UUID string to mkswap, which mkswap does not
accept (correctly) as new UUID.
In addition, print a warning when the UUID of a swap partition changed,
since it may require manual fixups in the guest.
The previously selected 'atomic' recipe resulted in 2GB swap in a 6GB
volume. Also, we don't really need the boot partition, so just create a
partition using the whole disk space.
Commit 0a94cae15b added useful
headings to parts of the configure script. This refactors
the code by adding a common macro (‘HEADING’), and also changes
the output so it's clearer at a glance:
--- Checking for Haskell ---
checking for ghc... (cached) ghc
--- Checking for PHP ---
checking for php... (cached) php
checking for phpize... (cached) phpize
Modern ssh and RHEL 5 sshd are not always interoperable, requiring
enabling legacy crypto policies on the converion server (ssh client).
Thanks: Nikos Mavrogiannopoulos, Jakub Jelen.
Shuffle the order of the various CFLAGS used when building the Ruby
extension: first the flags from manywarnings (gnulib), then libguestfs
own CFLAGS, and then -DGUESTFS_PRIVATE=1 (so it is not overridden).
This matches also what happens in automake parts, and makes it possible
to override any flags from manywarnings also in the Ruby extension.
As the name suggests, caml_stat* functions are used for allocating
static blocks. The OCaml runtime doesn't bother freeing any of these
on exit. OCaml 4.05 allocates caml_executable_name this way, a newly
seen global variable. We might as well just ignore everything
allocated this way.
Further information here:
e7cf3b1846/byterun/caml/memory.h (L66)
These are generated in many different ways in the various
subdirectories, and sometimes not generated correctly. Introduce a
script to do this in one place, and hopefully correctly.
This is mostly simple refactoring, but I got rid of a couple of
things:
(1) The ‘make depend’ rule doesn't appear to be needed. automake (or
make?) seems to rebuild the ‘.depend’ file automatically just because
it is included.
(2) I got rid of the hairy path rewriting sed expression. Possibly
that is needed for srcdir != builddir.
Output each path where we attempt to find 'java', so it is easier to
spot whether the location for the current OS is missing. This also
removes the need to print the location next to the version, since the
location was already printed above.
All sorts of strings might be passed here hoping to make them
canonical LV names. We cannot be sure that the strings passed will be
devices which exist in the appliance.
For Device parameters we expect a block device name. However we were
only testing for "/dev/..." and so chardevs (from the appliance) could
be passed here, resulting in strange effects. This adds a function
is_device_parameter which tests for a valid block device name.
For Dev_or_Path parameters much the same, except we can also use the
is_device_parameter function elsewhere in the daemon to distinguish if
we were called with a device or path parameter. Previously we used a
simple test if the path begins with "/dev/...".
Reported by Mathieu Tarral.
The following functions were previously reimplemented in OCaml. This
commit replaces them with calls to the C functions:
- is_root_device
- prog_exists
- udev_settle
plus the internal get_verbose_flag function.
However note that we cannot do this for every utility function. In
particular the C function must not call any reply* functions.
When parts of the daemon were previously converted to OCaml, the
previous PCRE regexps were converted to Str regexps. Restore the
original PCRE regexps.
There was also one case where an original call to glob(3) was replaced
by a Str regexp, and this is replaced by a PCRE regexp (although it is
in fact identical in this instance).
This updates commit b48da89dd6
and commit eeda6edca1
and commit 2ca0fa778d.
oraclelinux is the same as redhat/centos/scientificlinux, so add it
where it is missing. This fixes amongst other things, running sysprep
on an Oracle Linux image where it would previously fail operations like
setting the hostname, saying that it was not supported on that distro.
Since the introduction of the TEST_FUNCTIONS framework in tests, with
commit 2cb0413049, using a non-bash shell
will not work anymore, since the framework currently relies on bash.
Hence, switch the interpreter of these tests to bash, so they can run
also on systems where /bin/sh is not bash (e.g. default Ubuntu
installations).
Since commit d5b6f1df5f, the daemon
requires ocaml-hivex; OTOH, nothing checks it is actually available, so
the build of the daemon will fail with a semi-cryptic error about the
lack of a directory in the OCaml install prefix.
As fix, check for the presence of the hivex module at build time,
failing earlier if not present. The check is performed only when the
daemon is enabled, as ocaml-hivex is not used for anything else than the
daemon.
This also reimplements the lv_canonical function in OCaml. We cannot
call the original C function because it calls reply_with_perror which
would break the OCaml bindings.
Move the list_filesystems API into the daemon, reimplementing it in
OCaml. Since this API makes many other API calls, it runs a lot
faster in the daemon.
The previously library-side ‘file_architecture’ API is reimplemented
in the daemon, in OCaml.
There are some significant differences compared to the C
implementation:
- The C code used libmagic. That is replaced by calling the ‘file’
command (because that is simpler than using the library).
- The C code had extra cases to deal with compressed files. This is
not necessary since the ‘file’ command supports the ‘-z’ option
which transparently looks inside compressed content (this is a
consequence of the change above).
This commit demonstrates a number of techniques which will be useful
for moving inspection code to the daemon:
- Moving an API from the C library to the OCaml daemon.
- Calling from one OCaml API inside the daemon to another (from
‘Filearch.file_architecture’ to ‘File.file’). This can be done and
is done with C daemon APIs but correct reply_with_error handling is
more difficult in C.
- Use of Str for regular expression matching within the appliance.
This change allows parts of the daemon to be written in the OCaml
programming language. I am using the ‘Main Program in C’ method along
with ‘-output-obj’ to create an object file from the OCaml code /
runtime, as described here:
https://caml.inria.fr/pub/docs/manual-ocaml/intfc.html
Furthermore, change the generator to allow individual APIs to be
implemented in OCaml. This is picked by setting:
impl = OCaml <ocaml_function>;
The generator creates ‘do_function’ (the same one you would have to
write by hand in C), with the function calling the named
‘ocaml_function’ and dealing with marshalling/unmarshalling the OCaml
parameters.
LVM is fine with a completely empty configuration file (meaning "all
defaults"), so start with one instead of copying the system
configuration file.
Also this means we can very easily implement lvm_set_filter
functionality without using Augeas, since we no longer have to worry
about existing filters being present.
Thanks: Alasdair Kergon, Zdenek Kabelac.
In SUSE guests, handle the case where
Bootloader::Tools::GetDefaultSection () returns undef.
Previously this would return an empty string and cause a bogus error
in subsequent code:
virt-v2v: error: libguestfs error: statns: statns_stub: path must start
with a / character
With OCaml < 4.02 when using the alternate Bytes module, this module
would be compiled twice during parallel builds, resulting in
occasional corruption. The reason for this is that the ocamldep file
mentions ‘bytes.cmo’ whereas the ‘$(OCAML_BYTES_COMPAT_ML)’ macro
expands to ‘../../common/mlstdutils/bytes.ml’. Make doesn't recognize
these as the same file.
Use an alternate way to specify this file to fix this.
GUESTFSD_EXT_CMD was used by OpenSUSE to track which external commands
are run by the daemon and package those commands into the appliance.
It is no longer used by recent SUSE builds, so remove it.
Thanks: Pino Toscano, Olaf Hering.
Even if ocamlopt is available, always build a bytecode version of
‘common/mlstdutils’.
Furthermore, because this library is pure OCaml, we should not be
using ‘ocamlmklib’. We should use ‘ocaml{c,opt} -a’ instead. This
doesn't make any difference for native code, but for bytecode it was
building a broken library.
The original reason for making this change is because the generator is
always built as bytecode, and it depended on
‘../common/mlstdutils/guestfs_config.cmo’ and
‘../common/mlstdutils/std_utils.cmo’. On native code platforms these
were not built before the generator and so the generator races to
build the .cmi and .cmo files. Since the generator doesn't have
correct dependencies covering the ‘common/mlstdutils’ directory you
can get a broken link on fast machines:
File "../common/mlstdutils/std_utils.ml", line 1:
Error: Corrupted compiled interface
../common/mlstdutils/guestfs_config.cmi
make[2]: *** [Makefile:1993: ../common/mlstdutils/std_utils.cmo] Error 2
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/builddir/build/BUILD/libguestfs-1.37.17/generator'
The C function mkdtemp(3) requires that the string ends with 6 'X'
characters, so appending a non-empty suffix causes the function to
raise EINVAL.
Luckily we only ever called this function with the last parameter "".
This is like the Perl chomp function, it removes a single \n from the
end of a string if present, else leaves the string alone.
I believe I found the only (two) places where such a function is used,
but there may be a few more lurking.
Require <caml/unixsupport.h> (an OCaml header file) and remove
alternate defintions of ‘Nothing’ and ‘unix_error’ which are defined
in this header file.
We require OCaml >= 3.11 which has this header file, so there is no
need to test for it or provide alternative definitions.
Thanks: Pino Toscano.
Previously the OCaml compiler was only required if building from git
but was at least theoretically optional if building from tarballs
(although this was never tested). Since we want to write parts of the
daemon in OCaml, this makes OCaml required for all builds.
Note that the ‘--disable-ocaml’ option remains, but it now only
disables OCaml bindings and OCaml virt tools. Using this option does
not disable the OCaml compiler requirement.
Also note that ‘HAVE_OCAML’ changes meaning slightly, so it now means
"build OCaml bindings and tools" (analogous to ‘HAVE_PERL’ and
others). The generator, daemon [in a future commit], and some utility
libraries needed by the generator or daemon do not test for this macro
because we can assume OCaml compiler availability.
The ‘vmware-uninstall-tools.pl’ script tries to rebuild the initrd.
On SUSE, if kdump initrd has been enabled, this would use the wrong
root device because ‘mkdumprd’ doesn't know what root device to use.
Fix that by setting the ‘rootdev’ environment variable.
See also https://bugzilla.redhat.com/1465849 (this is not a fix).
Currently we install the firstboot service under systemd target
‘default.target’. This change simply factors out this name.
Note that the name is not factored out in the code which deletes the
old ‘/etc/systemd/system/default.target.wants/firstboot.service’ file,
since that would have always been installed in the same location.
Or with LIBVIRT_DEFAULT_URI=qemu:///system which is the same
thing.
In either case the images are created as user qemu.qemu and then
aren't readable or modifiable by later parts of the script.
Place the Bytes fallback module in the right place (mlstdutils), with no
need to make it available directly also for generation, since it uses
mlstdutils now.
Fixes commit 61d4891ef4.
The gobject bindings are adequately covered in the usual manual pages:
guestfs(3). There is no need for separate generation of gtk-doc.
Also generating gtk documentation is the slowest part of the build,
and the tooling around gtk-doc is broken
(https://bugzilla.redhat.com/show_bug.cgi?id=1465665).
Note this removes the configure ‘--enable-gtk-doc’ option. Using this
option now gives a warning, but is otherwise ignored:
configure: WARNING: unrecognized options: --enable-gtk-doc
After the previous refactoring, we are able to link the daemon to
common/utils, and also remove some of the "duplicate" functions that
the daemon carried ("duplicate" in quotes because they were often not
exact duplicates).
Also this removes the duplicate reimplementation of (most) cleanup
functions in the daemon, since those are provided by libutils now.
It also allows us in future (but not in this commit) to move utility
functions from the daemon into libutils.
Create a module ‘C_utils’ containing functions like ‘drive_name’ and
‘shell_unquote’ which come from the C utilities.
The new directory ‘common/mlutils’ also contains the ‘Unix_utils’
wrappers around POSIX functions missing from the OCaml stdlib.
This refactoring change just moves the cleanup functions around in the
common/utils directory.
libxml2 cleanups are moved to a separate object file, so that we can
still link to libutils even if the main program is not using libxml2
anywhere. Similarly gnulib cleanups.
cleanup.c is renamed to cleanups.c.
A new header file cleanups.h is introduced which will replace
guestfs-internal-frontend-cleanups.h (fully replaced in a later commit).
The new module ‘Std_utils’ contains only functions which are pure
OCaml and depend only on the OCaml stdlib. Therefore these functions
may be used by the generator.
The new module is moved to ‘common/mlstdutils’.
This also removes the "<stdlib>" hack, and the code which copied the
library around.
Also ‘Guestfs_config’, ‘Libdir’ and ‘StringMap’ modules are moved
since these are essentially the same.
The bulk of this change is just updating files which use
‘open Common_utils’ to add ‘open Std_utils’ where necessary.
The ‘Xml’ module is a self-contained library of bindings for libxml2,
with no other dependencies.
Move it to a separate ‘common/mlxml’ directory.
This is not pure refactoring. For unclear reasons, the previous
version of ‘Xml.parse_file’ read the whole file into memory and then
called ‘xmlReadMemory’. This was quite inefficient, and unnecessary
because we could use ‘xmlReadFile’ to read and parse the file
efficiently. Changing the code to use ‘xmlReadFile’ also removes the
unnecessary dependency on ‘Common_utils.read_whole_file’.
The ‘Progress’ module is a self-contained library with the only
dependencies being:
- the C ‘progress’ implementation
Move it to a separate ‘common/mlprogress’ directory.
This change is pure code refactoring.
The ‘Visit’ module is a self-contained library with the only
dependencies being:
- the C ‘visit’ implementation
- the guestfs OCaml bindings
Move it to a separate ‘common/mlvisit’ directory.
This change is not entirely refactoring. Two other fixes are made:
- remove unsafe use of CLEANUP_FREE from a function which could
raise an OCaml exception (cleanup handlers would not be called
correctly if the exception is thrown)
- don't link directly to common/visit/visit.c, but instead use
the library (common/visit/libvisit.la)
When installing guests that need NVRAM variables, the cleanup of the
guest with `virsh undefine` will remove that file too, which is not what
we want. Instead, compress the NVRAM file right before the cleanup,
to ensure we have it.
Also, fix the filename for it, removing the double "-nvram" suffixes.
- set the mirror to deb.debian.org, which is the official redirector, so
generated images will use the closest mirror depending on their
location
- automake the task selection, using the choice we want
- do a full upgrade of the distro, so the template is already up-to-date
- install grub on the default device, which should also be the only one
in the automated installation
We were dropping the add_drive copyonread flag when using the libvirt
backend. This resulted in significant performance degradation (2x-3x
slower) when running virt-v2v against VMware servers.
Thanks: Kun Wei.
Currently -i libvirtxml mode only works for local files or NBD disks.
The purpose of NBD is to support virt-p2v.
This change adds support for network disks over http or https, ie:
<disk type='network' device='disk'>
<driver name='qemu' type='raw'/>
<source protocol="http" name="/scratch/disk.img">
<host name="server.example.com" port="80"/>
</source>
<target dev='hda' bus='ide'/>
</disk>
This is just for testing. It's especially useful for exercising curl
support in qemu without requiring VMware to be available.
This feature allows you to use different image formats for the fixed
appliance. The raw format is used by default.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
Caused the following error when running make-template:
syntax error at line 22: syntax error
virt-index-validate: ‘debian-9.index-fragment’ could not be validated, see errors above
Add a RELEASES file which contains the release date for each version
of libguestfs. When a new version is released, the configure script
checks that the date has not been omitted from the file.
This also fixes the release date for version 1.37.16 which I omitted
from commit 9455f21237.
New gnulib uses ‘-Wimplicit-fallthrough=5’ which warns (and hence
errors) on every switch statement with an implicit fallthrough.
We could use the new GCC attribute here:
#ifndef FALLTHROUGH
# if __GNUC__ < 7
# define FALLTHROUGH ((void) 0)
# else
# define FALLTHROUGH __attribute__ ((__fallthrough__))
# endif
#endif
but instead this changes the level to 4 which allows us to use
/*FALLTHROUGH*/ comments (which we are already using) to mark cases
where we expect fallthrough.
Gnulib installs two files: ‘GNUmakefile’ and ‘maint.mk’. The first
one runs the second one, and the second one sets LC_ALL=C universally.
However this breaks the ‘bugzilla’ script (or Python) because some
bugs contain non-ASCII characters in their Summary. The script prints
this error:
Traceback (most recent call last):
File "/usr/bin/bugzilla", line 1117, in <module>
main()
File "/usr/bin/bugzilla", line 1112, in main
_format_output(bz, opt, buglist)
File "/usr/bin/bugzilla", line 702, in _format_output
print(format_field_re.sub(bug_field, opt.outputformat))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 108-109: ordinal not in range(128)
The simplest fix for this is to unset LC_ALL before running the
external script, so that the normal locale settings are used.
The OCaml function ‘input_line’ throws an End_of_file exception if the
end of the file is read before any other input, ie. if the file here
is zero length. Return an empty string instead.
This can produce peculiar errors such as:
$ virt-customize ... --root-password file:/dev/null
libguestfs: uncaught OCaml exception in getopt callback: End_of_file
Remove (before opening round bracket) whitespaces in the documentation
of the Python binding, according to the PEP 8 specification.
This is just code reformatting, with no behaviour changes; no content
changed beside whitespaces, so "git diff -w" gives an empty diff.
Mostly modelled after a snippet implemented in dib, it is an helper
function to run multiple commands in parallel, waiting for all of them
at once, and returning all their exit codes. It is possible to pass
custom descriptors for collecting stdout and stderr of each command.
Common_utils.run_command is adapted to use few helper methods used by
run_commands, so all the existing code using it keeps working; in
addition, it gets labelled parameters for stdout and stderr FDs.
Add a simple unit tests for them.
LVM2 >= 2.02.171 requires the ‘--yes’ option to force pvresize to work
in various circumstances, eg. reducing the size of an existing PV.
Pass this flag unconditionally.
Note this does NOT break earlier versions which just ignore this flag.
If SIGTERM is blocked in the main program, then it ends up still being
blocked in the subprocess after we fork. This means that we cannot
kill qemu by sending SIGTERM to it. This commit fixes the problem by
unblocking SIGTERM unconditionally after fork.
Thanks: wtfuzz on IRC for reporting and analysis.
If you use the libguestfs tools which open disk images read-only
(eg. virt-df), with formats such as 'vdi', then you will see an error:
error: invalid value for backingformat parameter 'vdi'
This is because opening a disk image read-only will try to create a
qcow2 file with the original image as a backing file. However the
list of permitted backing formats was very restrictive and did not
include 'vdi' (nor many other uncommon formats).
Instead of using a whitelist for backing formats, just validate that
the string is alphanumeric and short.
Thanks: Mike Goodwin for reporting the bug.
Complementary fix of commit 2d25872df3.
On SLES 11 SP4 with kdump enabled mkinitrd calls mkdumprd which calls
mkinitrd, but mkdumprd doesn't have any clue of the root device.
Call mkinitrd with rootdev environment variable to tell them all
what device to use as root.
Tested-By: Cédric Bosdonnat <cbosdonnat@suse.com>
These were already included in the .ml implementation file, but just
weren't being exported in the interface, so there's no real change
here. It just allows these functions to be used.
Make use of the additional command line arguments, and API needed to
decrypt LUKS partitions. This extends to v2v the work done in other
OCaml tools with commit 6b26a0cce4,
since it seems to be working fine after a basic testing.
Related to: RHBZ#1362649
PCI devices don't exist/work. You would see errors such as:
qemu-system-s390x: -device virtio-rng-pci,rng=rng0: MSI-X support is mandatory in the S390 architecture
mkinitrd in SLE guests < 12 tries to get the root device by scanning the
fstab: this will fail, since v2v already remapped the devices from
hd*/sd* to vd* in the guest (including in its fstab).
Since we know what is the root device in the appliance, pass it to
mkinitrd directly, so it does not have to do guesswork.
Thanks to: Cédric Bosdonnat, for reporting the issue, and testing
the fix.
Verify the returned values of Python Object constructor functions
are not NULL before adding them to a collection.
This is particularly relevant when constructing Unicode strings in
Python 3 as they will return NULL if non UTF-8 characters are present.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
It looks like older versions of lvm2 (recent enough to have selectors)
do not recognize '1' (and '0') as boolean values. Switch to 'yes',
which seems to be supported.
Updates commit 7367945647.
Make use of functions and types that fit more, and that do the same job:
- use PyErr_NoMemory() on malloc failure
- use PyErr_SetFromErrno when setting an exception from an errno
- throw TypeError if not getting a list when required
The current need for #ifdef's based on the presence of
PyString_FromString makes both the OCaml code of the generator, and the
generated C code a mess to read.
Hence, add three simple wrappers to make both the OCaml, and C code more
readable, and easier to tweak in the future.
This should be just refactoring, with no actual behaviour changes.
Thanks to: Matteo Cafasso
If the parallel tools pigz or pxz are available, prefer them for
uncompressing gz- and xz-compressed OVA files respectively. If not
available then gzip or xz are used as normal.
In its current form this is very hard to implement because it requires
us to "unparse" the options, including removing any shell quoting.
It wasn't implemented at all for the libvirt backend.
Also contrary to the documentation, the configure script did not use
these options for testing, but constructed its own set of qemu test
options.
Previously the generator did not change any string returned from the
daemon. Thus guestfs_list_devices (for example) might return internal
device names like /dev/vda (if virtio-blk was in use).
This changes calls to the daemon so that returned strings are
annotated as plain strings, devices or mountables:
old ---> new
RString "uuid" RString (RPlainString "uuid")
RString "device" RString (RDevice "device")
RString "fs" RString (RMountable "fs")
For hash tables, keys and values must be annotated separately. For
example a hash table of mountables (keys) -> plain strings (values)
would be annotated like this:
old ---> new
RHashtable "fses" RHashtable (RMountable, RPlainString, "fses")
The daemon calls reverse_device_name_translation (currently a no-op)
for devices and mountables.
Note that this has no effect for calls which are handled on the
library side.
(cherry picked from commit 6b77cc196ecb8d7e1d73592ef65a189a7412c97c)
Remove much of the text detailing how device name translation
happened. Since we removed support for virtio-blk
(commit 9e0294f88f) and deprecated the
‘iface’ parameter, only /dev/sdX device names should be visible
through the public APIs, both in parameters and in return values from
calls like guestfs_list_devices and guestfs_list_partitions.
Note the above is in fact not true for the UML backend, but UML is
broken in the kernel and in any case this will be fixed later.
(cherry picked from commit 2727e589db216bf0731385966889a4f66dbfe225)
In particular the virt-rescue --scratch option makes it very easy to
add huge numbers of drives. Since the per-backend max_disks limit was
never checked anywhere you could get peculiar failures. Now you'll
get a clear error message:
$ virt-rescue --scratch=256
libguestfs: error: too many drives have been added, the current backend only supports 255 drives
This patch changes appliance search using paths with multiple directories. Now
all appliance checks will be done separately for each directory. For example
if the path LIBGUESTFS_PATH=/a:/b:/c, then all applainces are searched first in
/a, then in /b and then in /c. It allows to flexibly configure the libguestfs
to interact with different appliances.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
It is not something standard, and actually unused by libguestfs itself.
Possibly going to be used by the old virt-v2v (which was in Perl), but
never used for that either.
The device name is only used by guestfish (when using the -N option to
prepare drives). We constructed the device name very naively,
basically ‘sprintf ("/dev/sd%c", next_drive)’.
This stores the device index instead, and only constructs the device
name in guestfish. Also the device name is constructed properly using
guestfs_int_drive_name so it can cope with #drives > 26.
As a side effect of this change we can remove the extra parameter of
the add_drives macro.
Thanks: Pino Toscano
After the previous commit it become possible to construct various
"impossible" argument types, such as lists of FileIn strings. This
commit prevents these from happening.
Previously we had lots of types like String, Device, StringList,
DeviceList, etc. where Device was just a String with magical
properties (but only inside the daemon), and DeviceList was just a
list of Device strings.
Replace these with some simple top-level types:
String
StringList
and move the magic into a subtype.
The change is mechanical, for example:
old ---> new
FileIn "filename" String (FileIn, "filename")
DeviceList "devices" StringList (Device, "devices")
Handling BufferIn is sufficiently different from a plain String
throughout all the bindings that it still uses a top-level type.
(Compare with FileIn/FileOut where the only difference is in the
protocol, but the bindings can uniformly treat it as a plain String.)
There is no semantic change, and the generated files are identical
except for a minor change in the (deprecated) Perl
%guestfs_introspection table.
This patch improves the search of grub config on EFI partition. This
means that the config will be found not only for rhel but also for
many other distributions. Tests were performed on the following
distributions: centos, fedora, ubuntu, suse. In all cases, the config
path was /boot/efi/EFI/*distname*/grub.cfg
The main purpose of the patch is to improve support for converting of
vm with UEFI for most distributions. Unfortunately this patch does not
solve the problem for all distributions, for example Debian does not
store grub config on the EFI partition, therefore for such
distributions another solution is necessary.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
The yara_scan API parses the file generated by the daemon counterpart
function and returns the list of yara_detection structs to the user.
It writes the daemon's command output on a temporary file and parses it,
deserialising the XDR formatted yara_detection structs.
It returns to the caller the list of yara_detection structs generated by
the internal_yara_scan command.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The internal_yara_scan runs the Yara engine with the previously loaded
rules against the given file.
For each rule matching against the scanned file, a struct containing
the file name and the rule identifier is returned.
The gathered list of yara_detection structs is serialised into XDR format
and written to a file.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The yara_destroy API allows to claim resources back via the removal of
the previously loaded Yara rules.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The yara_load API allows to load a set of Yara rules contained within a
file on the host.
Rules can be in binary format, as when compiled with yarac command, or
in source code format. In the latter case, the rules will be first
compiled and then loaded.
Subsequent calls of the yara_load API will result in the discard of the
previously loaded rules.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
At least two control files (packages and base.tar.gz) are necessary for the
supermin appliance.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
Many of the properties for disks are the same for all of them, so
collect them only once, instead of doing that for every disk.
Should be just code motion, with no behaviour change.
When converting a guest with UEFI firmware, set the also
hw_firmware_type=uefi property for all the disks of the guest, so Nova
can properly boot the guest.
In every instance where we used the ‘cancel_stmt’ parameter of
these macros:
ABS_PATH
NEED_ROOT
the value was only ever ‘cancel_receive ()’ or empty. We only use
‘cancel_receive’ for FileIn functions, so replace it with a simple
flag for whether the current function is a FileIn function.
This also removes some incorrect comments about macros that cannot be
used with FileIn functions when in fact they can.
As a simple consequence of the previous commit, every instance of the
‘fail_stmt’ parameter to one of the following macros:
RESOLVE_DEVICE
RESOLVE_MOUNTABLE
REQUIRE_ROOT_OR_RESOLVE_DEVICE
REQUIRE_ROOT_OR_RESOLVE_MOUNTABLE
is now ‘return’ and therefore we can replace it in the macro and drop
the parameter completely.
The generated code had:
...
done_no_free:
return;
}
There was no possible code between the label and the return statement,
so both the label and the return statement are redundant. The
instances of ‘goto done_no_free’ can simply be replaced by ‘return’.
The only small complication is that there is a label just before this
which contains some optional code. So we might have ended up with
generated code looking like:
done:
// if there was no optional code, this would be empty
}
which provokes a parse error in C. Therefore I had to add a semicolon
after the ‘done:’ label. This will be removed in a subsequent commit,
so it's just to make the current commit bisectable.
Replace the monolithic 'test-max-disks.pl' script with a test program
written in C. The program is completely equivalent to the old script,
except for the enhancement that it is able to detect if disks are
added to the appliance in the wrong order.
The tests themselves are split out into some shell scripts:
- test-27-disks: Fully tests 27 disks.
This is the minimum supported configuration for all backends.
- test-255-disks: Fully tests 255 disks.
This is the most disks that libguestfs up to 1.36 supported.
- test-add-lots-of-disks: Add ‘a lot’ of disks and exit (without
any further testing). This is meant to try to exercise > 255
disk case but without being as slow as a test of the max number
of disks (which takes ages, even for as few as 4000 disks).
- test-max-disks: Test the maximum possible number of disks.
The maximum depends on several factors, notably which backend
is in use, and the limit on the number of open files.
‘test-max-disks’ is a slow test.
swap.c: In function 'do_mkswap_U':
swap.c:62:9: error: argument 1 null where non-null expected [-Werror=nonnull]
if (strlen (label) > SWAP_LABEL_MAX) {
^~~~~~~~~~~~~~
virtio-scsi has been supported in qemu since 2012, and it is superior
in every respect to virtio-blk. There's no reason to still be using
virtio-blk.
virtio-scsi support was initially added in 2012
(commit 0c0a7d0d86).
You can still use virtio-blk using the (deprecated) iface parameter,
but don't do that in new code.
Add udev_settle_file() to run 'udevadm settle' with --exit-if-exists option. It
will slightly reduce the waiting-time for pending events if we need to wait
for events related to a particular device/file.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
RWMJ:
- Use local variable for MAX_ARGS.
- Use commandv instead of commandrv, fix checking of return code.
When running under valgrind, the process takes a fraction of a second
to start up, changing the "Opening the guest" timestamp, which broke
the test.
Fixes commit ca40078cdd.
- convert the Fedora 20 examples to Fedora 25
- convert the yum examples to dnf
- convert the Debian 7 examples to Debian 8
- mention that --core updates all the installed packages, not "core"
ones
- convert i386 to i686 for the Fedora 32bit examples
- mention that autogenerated root passwords are printed on stdout
- mention --get-kernel is deprecated in favour of virt-get-kernel
- update URL to the Puppet installation documentation
- edit the repository index documentation to make it clearer
non-compressed templates are supported
- remove old wrong paragraph about customization specified in command
line not processed in the same order
- use --install instead of --run-command '$pkgmgr install ...'
- other minor changes
This is a mostly complete implementation of a VMX parser and input
class for virt-v2v. It parses the name, memory size, CPU topology,
firmware, video, sound, hard disks, removable disks and network
interfaces from the VMX file. It only omits support for floppies and
SCSI CD-ROMs.
The input class is split into two major parts: a generic VMX file
parser (Parse_vmx), and the Input_vmx module which translates the VMX
tree into the source device model.
This also contains tests. There are simple unit tests of the
Parse_vmx module, and also some more complete parsing tests taken from
real guests.
When outputting libvirt XML, create virtio-rng, a memory balloon
device, and a pvpanic device, if the guest supports it.
Thanks: Daniel Berrangé for advice on memballoon
Extend the guestcaps structure so it records whether a guest supports
(or drivers were added) for virtio-rng, the virtio memory balloon, and
the ISA pvpanic device.
Previously the kernel_info field 'ki_supports_virtio' really meant
that the kernel supports virtio-net. That was used as a proxy to mean
the kernel supports virtio in general.
This change splits the field so we explicitly test for both virtio-blk
and virtio-net drivers, and store the results as separate fields.
The patch is straightforward, except for the change to the
'rebuild_initrd' function. Instead of making the module list
conditional on whether virtio-net is available and using the old
(probably wrong) fallback if it happens to be unavailable, this now
tries to enable the common virtio kernel modules (just the ones needed
for virtio-blk and virtio-net to work). The fallback is only used if
none of the common virtio modules can be found.
This also adds a print_kernel_info function (as the
string_of_kernel_info function was unwieldy) and exports it from the
module.
We want to detect if a Linux kernel is Xen PV only ("PV-only"). Such
a kernel will not boot on KVM, and if a guest has only PV-only
kernels, it will not be able to boot at all on the target.
Our previous test was wrong. It tested whether the xennet.ko module
exists. This module was renamed in more recent kernels (to
xen-netfront.ko), so it happened to not detect modern kernels as
PV-only, but this was by chance.
Modern kernel images can be compiled with Xen PV guest support. The
same image can boot on baremetal, KVM, Xen PV or Xen HVM. Testing if
the xennet (or xen-netfront) module exists is irrelevant to this.
This test, which is based on ideas from Laszlo Ersek and
https://wiki.xen.org/wiki/Xen_Project_Software_Overview#Guest_Types
uses the kernel config test CONFIG_X86_XEN || CONFIG_X86_64_XEN to
determine PV-only kernels.
Note that these CONFIG flags were never upstream, and existed only in
Linux kernels forked by Xen ("XenLinux"). By the time Xen guest
support was added upstream, it was implemented using pvops support, so
a single image could boot on Xen and non-Xen as described above, and
these flags were no longer used.
Updates commit 7eb219d193.
Thanks: Laszlo Ersek.
Only in end-user messages and documentation. This change was done
mostly mechanically using the Perl script attached below.
I also changed don't -> don’t etc and made some other simple fixes.
See also: https://www.cl.cam.ac.uk/~mgk25/ucs/quotes.html
----------
#!/usr/bin/perl -w
use strict;
use Locale::PO;
my $re = qr{'([-\w%.,=?*/]+)'};
my %files = ();
foreach my $filename ("po/libguestfs.pot", "po-docs/libguestfs-docs.pot") {
my $poref = Locale::PO->load_file_asarray($filename);
foreach my $po (@$poref) {
if ($po->msgid =~ $re) {
my @refs = split /\s+/, $po->reference;
foreach my $ref (@refs) {
my ($file, $lineno) = split /:/, $ref, 2;
$file =~ s{^\.\./}{};
if (exists $files{$file}) {
push @{$files{$file}}, $lineno;
} else {
$files{$file} = [$lineno];
}
}
}
}
}
foreach my $file (sort keys %files) {
unless (-w $file) {
warn "warning: $file is probably generated\n"; # have to edit generator
next;
}
my @lines = sort { $a <=> $b } @{$files{$file}};
#print "editing $file at lines ", join (", ", @lines), " ...\n";
open FILE, "<$file" or die "$file: $!";
my @all = ();
push @all, $_ while <FILE>;
close FILE;
my $ext = $file;
$ext =~ s/^.*\.//;
foreach (@lines) {
# Don't mess with verbatim sections in POD files.
next if $ext eq "pod" && $all[$_-1] =~ m/^ /;
unless ($all[$_-1] =~ $re) {
# this can happen for multi-line strings, have to edit it
# by hand
warn "warning: $file:$_ does not contain expected content\n";
next;
}
$all[$_-1] =~ s/$re/‘$1’/g;
}
rename "$file", "$file.bak";
open FILE, ">$file" or die "$file: $!";
print FILE $_ for @all;
close FILE;
my $mode = (stat ("$file.bak"))[2];
chmod ($mode & 0777, "$file");
}
This patch fixes the problem of the first boot after installation virtio block
drivers on Windows 10 / Server 2016. The problem is related to the change in
the behavior of Windows-Kernel-Pnp. To fix we need to add a field "Version" to
HKLM/SYSTEM/DriverDatabase/DriverPackages/drv_inf_label then Windows-Kernel-Pnp
will be able to correctly install the virtio block driver.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
Otherwise the man page cannot contain any non-7-bit-ASCII characters.
The test error seen was:
Failed test 'POD test for blib/lib/Sys/Guestfs.pm'
at /usr/share/perl5/vendor_perl/Test/Pod.pm line 187.
Wide character in print at /usr/share/perl5/vendor_perl/Test2/Formatter/TAP.pm line 112.
blib/lib/Sys/Guestfs.pm (1811): Non-ASCII character seen before =encoding in 'caller’s'. Assuming UTF-8
Looks like you failed 1 test of 1.
Each scp command takes a considerable amount of time -- several
seconds -- because it must set up, authenticate and tear down a new
connection. Avoid this by combining several copies into a single
command.
We still have to use two scp commands because we want to check that
some files are copied and ignore failures in a second set of
informational files.
Ensure the full version of virt-v2v and virt-p2v is available in the
conversion log file, the only log file that we reliably get from users
and customers. "Full version" means the major, minor, release and
extra fields. The extra field is especially important as it contains
the downstream package release in Fedora, RHEL etc.
This change saves the virt-p2v version as a comment in the
physical.xml file, which is included in full in the conversion log (by
virt-v2v).
It also ensures that the initial virt-v2v debug message contains the
full version number including the 'extra' field.
$ cat virt-v2v-conversion-log.txt
virt-v2v: libguestfs 1.37.7local,libvirt (x86_64)
...
<!-- virt-p2v 1.37.7local,libvirt -->
It also adds 'p2v-version' and 'v2v-version' files in the virt-p2v
debug directory. These are strictly superfluous but could be useful
for end users.
$ cat p2v-version
virt-p2v 1.37.7local,libvirt
$ cat v2v-version
virt-v2v 1.37.7local,libvirt
GtkAboutDialog in GTK+ 3 can be configured with few standard licenses,
including GPL2+. Thus, just set that property, so the about dialog will
show its own license text, and there is no need for our custom one.
If you switch from ./configure --with-gtk=2 to =3 (or vice versa) the
dependencies files don't get rebuilt. As a result the wrong Gtk
libraries can be included in the ISO and virt-p2v won't run.
This make sure there is no mismatch between the size of the integer
value that Int64_val returns, and the size of the guestfs_h pointer.
This should fix the warning on 32bit environments (and thus build, when
--enable-werror is enabled).
Don't get the CPU information from libvirt, because including libvirt
and all dependencies in the virt-p2v ISO bloats everything.
Instead get most of the information we need from the util-linux
program 'lscpu'.
Unfortunately the CPU model cannot be retrieved.
Example output:
$ ./run virt-p2v --cmdline="p2v.dump_config_and_exit"
[...]
cpu vendor . . . Intel
cpu sockets . . 2
cpu cores . . . 8
cpu threads . . 1
flags . . . . . acpi apic pae
This updates commit 963d6c3be7.
Calculate the offset of the physical host's Real Time Clock (RTC) from
UTC and pass this to virt-v2v through the libvirt XML description of
the physical machine.
The libvirt XML is modified to add one of the following:
(no <clock/> element)
- if the RTC could not be read or there was some other time
calculation error.
<clock offset='utc' />
- if the RTC is the same as UTC.
<clock offset='localtime' />
- if the RTC is in local time.
It's not possible to distinguish between UTC and localtime in
timezones that lie along the Greenwich Meridian (obviously
including the UK), when daylight savings time is not in effect. In
that case, UTC is chosen.
Necessarily in timezones that use DST, this depends on when
virt-p2v is run, so in the unusual case where a physical machine is
switched off for a long time and then booted directly into virt-p2v
when DST has changed, the calculation can be wrong.
<clock offset='variable' basis='utc' adjustment='<seconds>' />
- if the RTC is some other offset to UTC, other than equal to UTC or
localtime.
Check for a "product.id" file in an architecture-specific subdirectory
of the main partition, and use its data to improve the data on the
media.
Only Mageia as distribution name is recognized there, since most
probably this file will not be available on other distros.
Export a new environment variable for extra-data.d scripts, that
contains a YAML representation of the elements in use.
This is a new addition in d-i-b 2.0.
d-i-b 2.0 is written in Python, and thus it passes the information about
the interpreter used for it (sys.executable) to the scripts that need
it; this mechanism replaces the old discovery of what is the default
Python interpreter in the system.
Since we are not Python-based, look for 'python' and use it as default
interpreter, with the --python command line option to set a different
one.
The only place where it is used so far are extra-data.d scripts; in case
it will be used also in other out-of-chroot phases, a different solution
will be needed.
Output the get_image_element_array function like d-i-b does, together
with the eval stuff needed to make it work.
get_image_element_array is basically a bash array, with the keys are the
used elements, and the values are the paths of each element; it is a new
addition in d-i-b 2.0.
This is used so far only for the extra-data.d phase.
Change mnemonics so 's' is not used twice:
Start Conversion: 's' -> 'c'
Output Connection: 'c' -> 'o'
Output Storage: continues to use 's'
Thanks: Ming Xie.
There were two related bugs in the virt-v2v error messages when source
inspection failed.
Firstly it would print the message about running 'virt-v2v -v -x' twice:
virt-v2v: error: inspection could not detect the source guest (or physical
machine).
Assuming that you are running virt-v2v/virt-p2v on a source which is
supported (and not, for example, a blank disk), then this should not
happen. You should run 'virt-v2v -v -x ... >& log' and attach the complete
log to a new bug report (see http://libguestfs.org).
No root device found in this operating system image.
If reporting bugs, run virt-v2v with debugging enabled and include the
complete output:
virt-v2v -v -x [...]
And secondly it would tell you to run virt-v2v -v -x even if you were
running with verbose messages.
Further fix for https://bugzilla.redhat.com/show_bug.cgi?id=1167623
which updates commit ab7197476d.
Thanks: Kun Wei.
Use a 64px icon which is in the menu-icons-default package, hopefully
part of the default installation of any flavour.
Updates commit 9928296c1c.
Thanks to: Xiang Hua Chen.
On Debian systems, usually daemons are run automatically after their
installations; since we are upgrading the template, and installing
packages such as openssh, network-manager, etc, this may cause few
daemon to stay running during the virt-builder execution, causing it to
fail when shutting down (since e.g. /sysroot/dev would be in use).
As workaround, use a temporary policy-rc.d policy layer [1]: give an
error code so no daemon is ever started at any point of package upgrades
and installation. The script is removed at the very end of the
building, so the final image will spawn daemons just fine when booted.
[1] https://people.debian.org/~hmh/invokerc.d-policyrc.d-specification.txt
Usually systemd is configured with prefix=/, so units are in
/lib/systemd. On a system with UsrMove, such as Fedora/RHEL7, /lib is
a symlink to /usr/lib, so both /lib/systemd and /usr/lib/systemd work;
OTOH, on non-UsrMove systems, such as Debian, the two paths are
different locations, and thus trying to edit system files as
/usr/lib/systemd/etc will fail.
As solution, just reference /lib/systemd for systemd files.
On Debian, network-manager-gnome contains nm-applet, which is what
launch-virt-p2v spawns. network-manager-gnome is already in the list of
packages for Debian, while network-manager-applet does not exist.
On Debian, the names of the packages containing the Gtk 2 and 3
libraries have different patterns, and thus just the name will not work
for Gtk 3.
Apply a bit of m4 to use a different package name on Gtk 2 and other
versions. It can be easily adapted to new models in future Gtk 4+,
should the naming pattern change from Gtk 3.
An old version of btrfs-progs (3.17 in this case) has a different
version string, so adapt our filtering to include this line as well.
Related to/updates commit 839ae5bcd5.
mdadm 4.0 no longer lets you create devices called literally "boot" or
"root", giving a very obscure error message:
https://bugzilla.redhat.com/show_bug.cgi?id=1433575
Work around this by calling them something else.
Move all the dependencies with the same name in all the distributions to
a single list at the end.
There should be no change to the package list used to build the p2v ISO.
Where supported, pass the source CPU vendor, model and topology to the
target hypervisor.
-i ova: We can get just cores per socket via a proprietary VMware
extension to OVF.
-i libvirt, virt-p2v: We can get all of these fields from the libvirt
XML.
-o libvirt, -o local: We can preserve all of the information in the
target XML.
-o rhv/vdsm: We can only pass through the topology, since neither
oVirt Engine nor the OVF format supports CPU vendor or model.
Topology uses an extension of OVF which is not part of the standard.
-o qemu: We preserve the topology only because versions of qemu vary
widely in their support for CPU models.
-o glance: Only the topology is preserved since that is all that
Glance currently supports.
Thanks: Pino Toscano.
In the fake <domain type='physical'> libvirt XML that we create to
describe the physical host, we did not accurately pass any information
about the host CPU except the number of cores (<vcpu/>).
This commit extracts detailed information about the vendor, model and
topology of the host CPU and adds that to the libvirt XML for
virt-v2v. Conveniently we can use libvirt capabilities to get this
information without needing to parse /proc/cpuinfo or similar
techniques.
The libvirt XML looks like this:
<domain type="physical">
...
<cpu match="minimum">
<vendor>Intel</vendor>
<model fallback="allow">Broadwell</model>
<topology sockets="1" cores="2" threads="2"/>
</cpu>
...
<features>
<acpi/>
<apic/>
<pae/>
</features>
It looks like Unix.O_CLOEXEC does not exist in OCaml < 4, so resort to
manually using Unix.set_close_on_exec on the filedesc after opening it.
Thanks to: John Sucaet and Richard W.M. Jones.
Unfortunately, there are very few osinfo types available for Debian and
Ubuntu, so make sure to use them only when the version is enough, and
use the higher version available also for any following version.
In cross-compilation, it is not possible to run tests, so the configure
aborts when checking for attribute cleanup.
The comment above the test hints at why we would need to do an actual
run test rather than a compile test: gcc only warns on unrecognised
attributes.
But warnings can be turned into error with -Werror.
Change the AC_RUN_IFELSE into an AC_COMPILE_IFELSE test that runs with
-Werror set. Save the CFLAGS before adding -Werror, and restore them
after the test.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
In the case where we are going to read the disk directly from the OVA
file (partial = true), we will create a qcow2 image backed by the OVA.
If running as root, libvirt will run qemu as a non-root user (because
of no qemu:///session support for root, which is a libvirt bug). qemu
will not be able to read the backing file and thus will fail.
To work around this problem, chmod the backing (OVA) file.
This isn't a complete fix since (eg) the OVA might be located in a
directory which isn't accessible by root, but there's only so much we
can do here until libvirt is fixed.
Thanks: Kun Wei.
Mixing the XML parsing with the other functions of this module made it
very hard to understand. Splitting the XML parsing into another
module simplifies the flow considerably.
This is just code refactoring and should not affect the semantics.
Support also a "socket" listen type with no explicitly specified socket
(which will be generated by libvirt).
Updates commit a4adf48915.
Related to: RHBZ#1378022
Commit a6330e9d3a enabled /run for
systemd-tmpfiles: while this works fine in most of the cases, there are
few tmpfiles configurations that still references /var/run instead of
/run. As result, include also /var/run in the systemd-tmpfiles
execution.
When converting and throwing away the output (-o null), the input
format should not really matter. However it does currently matter for
a couple of reasons:
(1) If the input is not raw or qcow2, then you'll get the error below
because virt-v2v wants to write the same output format as input format
but only allows raw or qcow2 as output formats:
virt-v2v: error: output format should be 'raw' or 'qcow2'.
(2) If you use -o null -oa preallocated then a non-sparse temporary
disk is created before being deleted, but that just wastes disk space
for no reason.
Therefore this change
(i) prevents '-o null -oa preallocated' from being specified on the
command line,
(ii) prevents '-o null -of <anything>',
(iii) forces the output (temporary file) to be raw sparse.
glibc in Fedora is currently configured with `--enable-obsolete-rpc',
so I guess we can see which way the wind is blowing.
(1) This changes our configure script to prefer libtirpc if it is
available.
If libtirpc is _not_ available then:
(a) Headers must be located in <rpc/xdr.h>, or the user must supply
the right CFLAGS.
(b) XDR functions must be located in one of -lportablexdr, -lrpc,
-lxdr, -lnsl or no library at all (ie. -lc), and the user must set
LDFLAGS if needed.
(2) We no longer add these paths automatically to $(CFLAGS)/$(LIBS).
Any part of libguestfs which needs <rpc/*.h> or the xdr_* functions
must use $(RPC_CFLAGS)/$(RPC_LIBS) explicitly.
(3) Previously Mac OS X had a workaround for the broken 64 bit support
in the supplied rpcgen. This workaround "activates" all the time if
you use tirpc, so breaking Linux after the above changes. tirpc is
supported on OS X, so I think it's just better to use that rather than
the broken rpcgen. For that reason I removed the workaround
completely.
Thanks: Roy Keene
`virt-rescue -a disk -i' does the right thing.
`-m' was previously an alternate form of `--memsize'. By sniffing the
parameter we can make `-m MB' continue to work, while also allowing
`-m' to be used as a short form for the `--mount' option.
This also removes most of the description of `--suggest' from the man
page, since it is no longer needed.
Instead of using "direct mode" (which was basically a quick hack),
virt-rescue now launches the appliance with a running daemon.
The daemon doesn't do much -- there is still a bash shell which the
user interacts with. The daemon is there simply to provide the
initial GUESTFS_LAUNCH_FLAG message and to handle shutdown a bit more
gracefully.
To interact with the shell, and replacing direct mode, virt-rescue now
prints out log messages (the output of the shell), and sends input
typed by the user directly to the console socket. This uses the
guestfs_internal_get_console_socket API added previously. Most of the
complexity behind this is hidden in virt-rescue.
This fully fixes the handling of ^C (RHBZ#1152819). Also there were
earlier reports that full screen commands like 'vim' didn't work well,
(RHBZ#1171654), but in this version vim appears to work fine, albeit
only using 80x24 of the screen because of the serial console.
Test:
- Bad SHA1 sum
- Bad SHA256 sum
- Invalid lines in manifest (x2)
- Good checksum and manifest
These tests were originally written by Tomáš Golembiovský. All I have
done is to integrate them into the virt-v2v test suite.
Avoids a lengthy conversion followed by failure if we discover at the
end that OVMF is not installed.
This also changes the order of the methods in -o libvirt and -o qemu
so that it matches the order in the class interface, and also
logically makes more sense.
Thanks: Christopher Brown
Create own blocks for all the parts dealing with FILE*: this way there
is no need to recycle the same FILE* variable for all the operations,
and have each block its own variable automatically cleaned up.
This also fixes a potential undefined behaviour on error: POSIX says
that after a call fclose(), a FILE* cannot be used anymore, not even
on fclose() failure. The previous behaviour for fclose == -1 was to jump
to the error label, which would then try to call fclose() again (since
the FILE* pointer was still non-null).
Filling some of their fields may cause the flow to skip to throw the
"out of memory" exception, and return immediately. To avoid leaking the
struct, or struct list, from the C implementation, use a cleanup handler
so there is no need to manually clean it up.
The previous code:
fcntl (fd, F_SETFL, O_NONBLOCK)
was technically incorrect, because it would have reset any
other flags on the file descriptor.
Thanks: Eric Blake
In case there are no event handlers registered with the handle,
get_all_event_callbacks will count 0 elements, trying to malloc a buffer
of that size. POSIX says that this can result in either a null pointer,
or an unusable pointer.
Short-circuit get_all_event_callbacks to allocate nothing when there are
no events, making sure to use its results only when there were events.
In case there are no event handlers registered with the handle,
get_all_event_callbacks will count 0 elements, trying to malloc a buffer
of that size. POSIX says that this can result in either a null pointer,
or an unusable pointer.
Short-circuit get_all_event_callbacks to allocate nothing when there are
no events, making sure to use its results only when there were events.
In case there are no event handlers registered with the handle,
get_all_event_callbacks will count 0 elements, trying to malloc a buffer
of that size. POSIX says that this can result in either a null pointer,
or an unusable pointer.
Short-circuit get_all_event_callbacks to allocate nothing when there are
no events, making sure to use its results only when there were events.
In case there are no event handlers registered with the handle,
get_all_event_callbacks will count 0 elements, trying to malloc a buffer
of that size. POSIX says that this can result in either a null pointer,
or an unusable pointer.
Short-circuit get_all_event_callbacks to allocate nothing when there are
no events, making sure to use its results only when there were events.
We aren't generating Java bindings for internal and other undocumented
methods, and therefore providing {@link #...} for deprecated methods
replaced by internal methods doesn't work. The easiest way is just to
check for this and turn such links into plain text.
Call windows_path with the actual path to resolve, instead of a null
pointer ('filename' is still null at that point), so Windows paths can
be properly resolved.
Use CLEANUP_FREE to properly dispose the temporary array used for
StringList, DeviceList, FilenameList, and OStringList parameters: this
way, an early return (jumping to the ret_error label) will not forget
cleaning them up.
When convering FBuffer fields of structs in each element of the return
list, make sure to allocate enough buffer to hold also the trailing null
character.
The JNI library uses CLEANUP_FREE macros, whose functions are built in
the internal libutils. Currently, trying to use functions that use
CLEANUP_FREE variables will cause the java execution to stop with a
symbol lookup error (for guestfs_int_cleanup_free).
It's not possible to define an action which takes a parameter called
'message' because the Erlang bindings use that as the name of an
internal variable. Solve this by renaming the Erlang internal
variable.
Fixes commit 84763d7fca.
Note this requires either the following fix in autoconf:
http://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=e17a30e98
OR gnulib sys_types module plus gnulib
commit a512e041120e9012e69afa2f5c3adc196ec4999a (any gnulib more
recent than Sep 2016) which corrects the AC_HEADER_MAJOR macro in a
similar way.
add udev_settle calls to print_partition_table and
sgdisk_info_extract_field because the inspect-os calls
guestfs_part_get_parttype and guestfs_part_get_gpt_guid for all
parition devices found and this causes intermittent with opening block
devices that are certainly present yet RESOLVE_DEVICE macro would fail
wiht ENOENT.
This test failed on ppc64le with:
Failure:
</wrong argument type Fixnum \(expected Array\)/> was expected to be =~
<"wrong argument type Integer (expected Array)">.
In addition the test generated a warning:
tc_800_rhbz507346.rb:29: warning: ambiguous first argument; put parentheses or a space even after `/' operator
This commit fixes both of these and also makes it simpler and faster
by not bothering to launch the appliance.
Fixes commit 227b1eea90.
If libmagic isn't installed then the guestfs_file_architecture API
doesn't work. This means that inspection will always return
<arch>unknown</arch> for every guest. This subtly breaks a few
features. In particular it was reported that the
virt-builder/virt-customize --install option did not work because the
"unknown" architecture of the guest was not compatible with the host.
libmagic is a small, widely available C library, so the easiest fix is
just to make it mandatory.
Reported by: Solarix on IRC
This change refactors how the qemu shell script is generated, making
it a bit less likely that we'll get the quoting wrong. Parameters are
now added to a list, unquoted, and all quoting is done when printing
out the list.
Checked by running virt-v2v -o qemu before and after the change and
manually comparing the output (which is not identical, but still
correct).
Make the documentation in Convert_linux more generic than just
Enterprise distros using RPM. Also, rename the name of the module from
"enterprise-linux" to simply "linux".
In particular, read the URL, the source name, and both the summary and
the description. For the long description, add a small system to read
continuation lines.
Adapt the expected result of virt-inspector for Debian and Ubuntu phony
guests, so test-virt-inspector.sh still passes.
A message was translated without the trailing \n. As it was in a
verbatim section of POD, I actually reverted the whole translation.
The error was:
./uk.po:53533: 'msgid' and 'msgstr' entries do not both end with '\n'
msgfmt: found 1 fatal error
In fact po4a (or whatever calls msgfmt) seems to ignore the error and
continues which is another problem.
Fixes commit 794b6a1acf.
Commit 794b6a1acf added cs.po, but
because this wasn't also added to lingas_not_translated the file was
not included in EXTRA_DIST or processed properly.
Fixes commit 794b6a1acf.
Compiling po-docs/ja/guestfish.pod failed with:
Pod input around line 277: L<> starts or ends with whitespace
Pod input around line 362: L<> starts or ends with whitespace
I reverted the part of the L</...> link which had been translated.
The fix isn't quite correct -- headings *can* be translated, and the
corresponding internal links should be translated too. However
pod2man has problems dealing with such links and in any case the
partial translation was not correct, so revert to the English version
of the link.
Updates commit 794b6a1acf.
Somewhere between OCaml 4.01 and OCaml 4.04, <caml/memory.h> changed
the way that CAMLlocalX() values on the stack are initialized. In
OCaml 4.01:
#define CAMLlocal1(x) \
value x = 0; \
CAMLxparam1 (x)
but in OCaml 4.04:
#define CAMLlocal1(x) \
value x = Val_unit; \
CAMLxparam1 (x)
The code in mllib/visit-c.c assumed that the value would be
initialized to Val_unit, and used a check (exn != Val_unit) to see if
the value had been updated. This failed badly (with a segfault) on
OCaml 4.01 in Debian 8.
Resolve this by always initializing CAMLlocalX() values before use.
We are redy to restore the disabled test. The relevant libvirt fixes
should be available in 3.1.0.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Libvirt < 3.1.0 lacks enough support for json: pseudo-URLs. Notably it
does not allow use of "raw" driver that we need.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The test tests/regressions/rhbz914931.c works by causing the daemon to
segfault while writing to it.
For reasons unknown, when configured --without-libvirt, this causes
the test to fail receiving SIGPIPE (exit code 141). We can prevent
this by installing a signal handler to ignore SIGPIPE, so the signal
is converted to EPIPE which the code handles properly.
The old version of qemu-img available in Debian 8 doesn't support the
preallocation option for raw format. However this doesn't matter
because the default is preallocation=off (ie. sparse) for old and new
versions of qemu-img so we can just omit this parameter.
Fixes commit 53317e3f07
and commit fa83b47876
and commit 74ded0dbc1.
Slightly change the logic for installing RHEV-APT and VMDP:
- Only warn on failure to install RHEV-APT if the output hypervisor
requested it. Otherwise the warning is pointless because RHEV-APT
can't be used anyway.
- Make the warning clearer and actionable.
- Try to install VMDP unconditionally (rather than only if RHEV-APT
failed). It is always useful for SuSE hosts. However don't warn if
it's not available.
Previously the Convert_linux conversion module depended on one feature
of the output module (#keep_serial_console). This was extracted in an
ad-hoc way from the output module and passed as an extra parameter to
the conversion module.
Instead of doing it this way, just pass the output module into the
conversion module, so it can call output#keep_serial_console itself.
This is just a simplification of the existing code, but otherwise adds
no new features.
Non-Linux Unix guests may have static devices for CDs, so make sure to
skip them when reading /etc/fstab. This is the same as done for
/dev/fdN devices, i.e. floppy devices.
Currently imports from vCenter sometimes obey proxy environment
variables (eg. $http_proxy) and sometimes don't. The initial libvirt
connection to fetch metadata never uses the proxy, but because the
subsequent conversion and copying uses libcurl, it will pick up on
proxy environment variables.
This makes no sense, and in any case vCenter is really slow as it is
without putting another device into the data path. Therefore ensure
that libcurl does not see any proxy environment variables by unsetting
them.
Concatenate six small modules containing Unix/POSIX library call
bindings into a single module called Unix_utils.
The previous modules and the library functions bound were:
- Dev_t: makedev(3), major(3), minor(3)
- Exit: _exit(2)
- Fnmatch: fnmatch(3)
- Fsync: sync(2)
- Mkdtemp: mkdtemp(3)
- StatVFS: statvfs(2)
Apply this change across all the shell scripts containing tests.
Additionally this defines the environment variables $abs_srcdir,
$abs_builddir, $top_srcdir, $top_builddir, $abs_top_srcdir and
$abs_top_builddir which can now be used throughout test scripts.
This macro(?) expands to some shell script to source the
tests/test-functions.sh file from its correct location. The intention
is to use this in all tests, but in this commit only the existing
tests which already include test-functions.sh are modified.
Daemon 'proc_nr's have to be assigned monotonically and uniquely to
each daemon function. However in practice it can be difficult to work
out which is the next free proc_nr. Placing all of them into a single
table in a new file (proc_nr.ml) should make this easier.
Group the APIs logically and move them into new modules:
Actions_core:
Core APIs and anything that doesn't fit into another group, eg. launch.
(With some more effort this could be split further.)
Actions_augeas:
Augeas APIs, eg. aug-init.
Actions_debug:
Debug APIs.
Actions_hivex:
Hivex APIs, eg. hivex-open.
Actions_inspection:
Inspection APIs, eg. inspect-get-type.
Actions_properties:
Handle properties, eg. set-hv, get-hv.
Actions_tsk:
SleuthKit APIs, eg. filesystem-walk.
*_deprecated:
All of the above modules have deprecated variants, where we
place the deprecated actions.
Sort the functions so the output is stable.
This changes the order in which the C API tests run. Previously we
ran the newest tests first, which was useful when we were frequently
adding new APIs. Now we run them in sorted order.
Also rename the function 'uuidgen ()' (conflicts with a same-named
function in Common_utils) to 'stable_uuid'. Notice that the UUID is
now only computed once per run of the generator, whereas previously
the same value was computed over and over again.
Sadly, there are elements (hello "gentoo"!) that, during the cleanup.d
phase, wipe out almost everything in the /tmp of the guest, including
the /tmp/aux where virt-dib mounts the auxiliary data. Since that
removal excludes things starting with "in_target" (mostly to avoid
wiping the "in_target.d" that disk-image-create itself sets up, then
rename our "aux" subdirectory to "in_target.aux" to work it around.
Hopefully, the "gentoo" element will be fixed upstream:
https://review.openstack.org/#/c/436101/
Commit 8ee51ee396 re-scans for the
available scripts when running certain phases; OTOH, some of them may be
missing, usually due to the lack of scripts for that phase in the
selected set of elements.
Indeed, if there is no directory for a phase, safely raise Not_found so
the case is handled as if the phase was missing in the final_hooks
Hashtbl.
The real sudo does it as well, and leaving them when preserving the
environment (-E) maybe breaks the applications, as e.g. chroot will have
a TMPDIR path pointing outside of it.
The inspection code already computed the paths of the software hive
(twice!) and the system hive, plus we also recompute the same paths
elsewhere, in virt-v2v for example. Therefore it makes sense to store
the paths from the inspection code and make them available through two
new APIs.
Add a convenient tuple Registry.t for the currently open hive. It
contains the guestfs handle and the root node of a registry.
The functions with_hive_readonly and with_hive_write are modified to
pass this tuple to their callbacks.
For reasons unknown virt-v2v recomputed the CurrentControlSet from
first principles, and passed %systemroot% around to all functions.
However that data is available from the libguestfs inspection APIs.
Move the functions decode_utf16le, encode_utf16le, get_node,
with_hive_readonly and with_hive_write to common code in a new module
called Registry.
This also defines types for nodes and values, instead of using int64
directly.
Just code motion.
Adding a new optional parameter to hivex_open in
commit 1f99251223 reveals a subtle bug
in the virt-p2v slow test.
Because we didn't pass LIBGUESTFS_PATH through to the instance of
virt-v2v, it was running with the new binary and library code, but
with the installed appliance (or would have failed if libguestfs
wasn't installed on the host when running the test). In particular
this bug was revealed when the new virt-v2v binary passed the
GUESTFS_HIVEX_OPEN_UNSAFE flag to the guestfs_hivex_open call, which
the (old, installed) daemon did not recognize and rejected with an
error.
For the same reason we also have to pass in LIBGUESTFS_CACHEDIR to
make sure that supermin doesn't reuse the cached appliance from
/var/tmp.
When extracting the content of the guest as tar, save also the extended
attributes and the SELinux attributes. This makes sure guests exported
as tar, tgz, and as docker image will work fine afterwards.
This is what disk-image-create does as well.
Pass the HIVEX_OPEN_UNSAFE flag when opening hives for reading.
Do NOT pass it when opening hives for writing.
This should make inspection, virt-win-reg and virt-v2v more tolerant
about handling Windows Registry corruption, without increasing the
risk of causing new corruption in hives.
In hivex >= 1.3.14, there is a new HIVEX_OPEN_UNSAFE flag allowing
heuristics to be used to deal with corrupted hives. Map this flag
into the libguestfs API.
If the flag is not supported (because libguestfs was compiled with
hivex < 1.3.14) then the flag is ignored. This is safe behaviour:
opening corrupted hives will give an error, as happened previously.
Because v2v/test-harness is a subdirectory of v2v, and because both
paths are listed in $(DIST_SUBDIRS), using find $(DIST_SUBDIRS) will
list files in v2v/test-harness twice. (This probably happens in other
directories too, but I noticed it here.) The easiest fix for this is
simply to use 'sort -u' to remove the duplicates.
Implement the "squash" output format, i.e. a squashfs compressed
filesystem.
This was implemented in diskimage-builder upstream as
commit 9d13084c4183b63587e1f5e4b03395a8df6538f6.
Introduce a new API to create a new squashfs filesystem out of a path
in the guest. It can be configured to exclude paths based on patterns,
and to select which compression use for the filesystem.
The advantage of running mksquashfs directly in the appliance is that
ownerships are properly saved, as opposed to tar_out + local untar.
It will be useful also for APIs different than tar-out, so move it to
guestfsd.c, and add it a parameter to specify the function name that
invoked it.
This is mostly code motion.
Recently, diskimage-builder moved the log cleanup from the 'base'
element to disk-image-create proper; the cleanup done is:
- truncate any file in /var/log
- remove *.log files in /root
This was implemented in diskimage-builder upstream as
commit 022d93ee822e71245af52c4cf8f8a8e82f599af3.
Scripts (especially root.d and extra-data.d) may add new scripts to the
existing set during their execution: in this case, re-read the list of
available scripts, so we do not miss any new addition.
It looks like scripts (in particular in the extra-data.d phase) may
in-flight add new scripts to the existing ones. As first step to
handle this situation, change the path of hooks passed for extra-data.d
scripts: instead of the local temporary directory, use the in-guest
location that is visible as result of the local fuse-based mount of the
guest.
Source the "die" script, part of the diskimage-builder library, when
running hooks that run outside the guest chroot. This is what
disk-image-create does, and scripts expect to use the "die" function
without sourcing the "die" script containing it.
Query awk for the list of environment variables, instead of trying to
extract the list from the output of `env`: the old approach breaks when
any of the environment variable contains more than one line.
GCC 7.0.1 can determine if there is likely to be sufficient space in
the output buffer when using sprintf/snprintf, based on the format
string.
The errors were all either of this form:
bindtests.c:717:29: error: '%zu' directive output may be truncated writing between 1 and 19 bytes into a region of size 16 [-Werror=format-truncation=]
snprintf (strs[i], 16, "%zu", i);
^~~
bindtests.c:717:28: note: directive argument in the range [0, 2305843009213693951]
snprintf (strs[i], 16, "%zu", i);
^~~~~
or this form:
sync.c: In function 'fsync_devices':
sync.c:108:50: error: '%s' directive output may be truncated writing up to 255 bytes into a region of size 251 [-Werror=format-truncation=]
snprintf (dev_path, sizeof dev_path, "/dev/%s", d->d_name);
^~
Fixed by converting these into dynamic allocation, or making the
output buffer larger, whichever was easier.
There is a gnulib macro we can use to make this simpler for integers.
It requires a new gnulib module (intprops), but it turns out that we
were already pulling that in through dependencies, so the change to
bootstrap is a no-op. (thanks: Dan Berrange)
This happens with GCC 7.0.1. The errors were all of the form:
qemu-speed-test.c: In function 'main':
qemu-speed-test.c:153:7: error: this statement may fall through [-Werror=implicit-fallthrough=]
usage (EXIT_SUCCESS);
^~~~~~~~~~~~~~~~~~~~
qemu-speed-test.c:155:5: note: here
default:
^~~~~~~
Commit 74ded0dbc1 changes existing calls
to truncate to run 'guestfisk disk-create' instead. However this
fails if guestfish has not been installed. By using the ./run script,
we can use the just-built guestfish instead.
Fixes commit 74ded0dbc1.
This adds a contrib script which can be used to build the virt-p2v ISO
on top of RHEL 5 or RHEL 6, i686 (32 bit) or x86-64 (64 bit) base.
There is also a script for testing the ISOs produced this way.
I don't think this will really make any difference. However there is
some puzzling disk corruption in the recently built RHEL 5 & 6 i686
images, and I want to make absolutely sure they are not caused by an
unsynchronized disk image.
There were still a few places where we were testing bash rather than
the intended program. Add `libtool --mode=execute' to the beginning
of the @VG@ macro to try to get around this.
If supported, use socket activation to pass a pre-opened listening
socket to the NBD server. This means we can guarantee to choose an
unused port even if there are multiple instances of virt-p2v running
(ie. when testing) or if there are unexpected services running on the
same machine.
This change applies to both NBD servers, since both now (or will
shortly) support this feature.
In preparation for using socket activation, move the nbd_local_port
global inside nbd.c and make it private.
This is largely code motion. However I rearranged the order in which
the ssh data connection and the NBD server are started up. Previously
the ssh connection was made first, and the NBD server was then
started. Now the NBD server starts first. This happens so that nbd.c
can choose the local port, and when we implement socket activation it
will allow nbd.c to open the socket and choose a free port too.
This is almost pure code motion, but I changed the name and prototype
of the function 'wait_nbd' to make its purpose clearer, and to remove
the unnecessary timeout setting (which is hard-coded).
Parse the output disk as URI, and use all its attributes just like
it is done for the input disk. The only change is that the fsync of the
output disk is limited now for local URIs only, since it will not work
with remote protocols.
Implement the "tgz" output format, i.e. "tar" compressed as gzip.
This was implemented in diskimage-builder upstream as
commit da41ee6012b064aa6901c871a1104a3a3933117e.
Implement a system similar to the operations in virt-sysprep, so each
output format has its own file and attributes (such as command line
arguments).
The result is that there is no more need to spread everywhere the job
required by each output format, such as checking for tools, or hooking
at the right point.
Make sure they are handled as options for the actual mkfs driver,
instead of mkfs itself. Also, improve the documentation of
--mkfs-options accordingly.
This was fixed in diskimage-builder upstream as
commit 5e2330d89c6c0e55b270464abea7e99a4d3246ad.
Add a new option to create files with the MD5 and SHA512 checksums of
all the disk image types produced.
This was implemented in diskimage-builder upstream as
commit 2ea5feca5c4b64867ac327736edfb20408f8840e and
commit 22952b7ea0543bb4f446752976d1d8ba232b021a.
Improve the error message that is shown when trying to selecting an
element, and another element provides it.
This was implemented in diskimage-builder upstream as
commit 452f7b8d5aaa6e85e3b01b970cd9cbc842b34b86.
We don't have to always extract all files from the OVA archive. The OVA,
as defined in the standard, is plain tar. We can work directly over the
tar archive if we use correct 'offset' and 'size' options when defining
the backing file for QEMU. This puts much lower requirement on available
disk space.
Since the virt-v2v behaviour for OVA input now depends on QEMU version
available this affects some of the tests. Expected result of the
affected also has to depend on the QEMU used thus such tests will have
two *.expected files.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The function looks up file in tar archive and returns a tuple containing
which at byte it starts and how long the file is.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The function returns version of qemu-img as a tuple (major, minor), or
the value (0,9) in case there was an error detecting the version.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
In commit 08f82f2e3d we increased the
memory size to 800MB (on x86) so that the semodule command would work.
However it has been discovered that another SELinux command takes
large amounts of RAM (setfiles during the SELinux relabel step).
Therefore increase the memory size again, this time to 2000MB.
The information whether the disk is gzip compressed or not is stored
in the OVF. There is no reason to do the detection.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
- Make module name explicit, so it's more obvious which module a
function is defined in.
- Group lines of code by feature.
- Capitalize some module names properly.
Just code motion, no functional change.
Commit 004de6cf45 was not sufficient to
solve this problem. We are also using gperf in guestfish. Rewrite
the code in the same way to avoid having to prototype the hash
function.
Add code in virt-p2v so that it can use nbdkit (with the file plugin)
as an alternative to qemu-nbd.
This is controlled through the virt-p2v --nbd command line option,
allowing you to select which server (out of qemu-nbd or nbdkit) you
prefer (with a fallback). The default is: --nbd=qemu-nbd,nbdkit so
qemu-nbd will be used preferentially.
When testing virt-p2v (eg. on the host machine) this prevents us from
testing two instances of virt-p2v at the same time because both will
try to use the same port.
Generate the random filename using our utility function
guestfs_int_random_string. This also means that we will not need to
call srandom() in guestfish or virt-edit.
The prototype of the hash function changed in gperf 3.1:
* The 'len' parameter of the hash function and of the lookup function is now
of type 'size_t' instead of 'unsigned int'. This makes it safe to call these
functions with strings of length > 4 GB, on 64-bit machines.
This commit moves the guestfs_int_string_to_errno function into the
third (functions) section of the gperf file so it is able to use the
lookup function without needing a prototype.
Thanks: Marius Cirsta
The prototype of the hash function changed in gperf 3.1:
* The 'len' parameter of the hash function and of the lookup function is now
of type 'size_t' instead of 'unsigned int'. This makes it safe to call these
functions with strings of length > 4 GB, on 64-bit machines.
This change tries to detect the required type in ./configure
Thanks: Marius Cirsta
Three more pieces of common code are moved under the common/
subdirectory. This is just code motion.
Note that windows.[ch] wasn't even being used by guestfish. That code
was only used in other virt tools.
This is mostly code motion but:
(1) I had to remove the compile-time COMPILING_GUESTFISH and
COMPILING_VIRT_RESCUE macros and replace them with runtime constants
and checks.
(2) I moved the fish/config.c file into this library.
Just code motion.
This commit makes it clearer what is a utility and what is part of the
library. It also makes it clear that we should rename:
guestfs-internal-frontend.h -> utils.h
guestfs-internal-frontend-cleanups.h -> cleanups.h (?)
but this commit does not make that change.
This commit, which is just code motion, moves the common XDR protocol
code (libprotocol) and the common errno handling (liberrnostring) into
libraries which are each built once and shared between the library and
daemon.
The error is:
gobject/docs/gtk-doc.make:52: warning: CLEANFILES multiply defined in condition TRUE ...
gobject/docs/Makefile.am:95: 'gobject/docs/gtk-doc.make' included from here
common-rules.mk:28: ... 'CLEANFILES' previously defined here
gobject/docs/Makefile.am:18: 'subdir-rules.mk' included from here
subdir-rules.mk:20: 'common-rules.mk' included from here
In the ancient version of PCRE in RHEL 5, partial match support does
not support /.*/, returning PCRE_ERROR_BADPARTIAL. However the
equivalent regular expression /(?:.)*/ works fine, so use that
instead.
RHEL 5 has ancient Gtk 2.10 which lacks several functions and whole
features we need (eg the spinner). Add various macros and function
definitions to work around this.
The guestfs_canonical_device_name API was rewriting /dev/mdX as
/dev/sdX. This is wrong since (eg) /dev/sd0 is not a device name, so
if you pass the canonicalized name back to the API it will fail.
virt-v2v was one tool doing this.
Bug found by valgrind on aarch64:
==21350== Syscall param faccessat(pathname) points to unaddressable byte(s)
==21350== at 0x4F05274: access (access.c:29)
==21350== by 0x4954DC3: guestfs_int_get_uefi (appliance-uefi.c:90)
==21350== by 0x49752D7: launch_libvirt (launch-libvirt.c:413)
==21350== by 0x496FD83: guestfs_impl_launch (launch.c:98)
==21350== by 0x4902003: guestfs_launch (actions-3.c:145)
[...]
==21350== Address 0x0 is not stack'd, malloc'd or (recently) free'd
Because we used 'return' directly, the effect of the CAMLparam0()
macro was not undone, resulting in local roots corruption when the
visit callback throws an exception.
The visit_tests unit test picked this up, but the corruption was only
seen on aarch64.
Fixes commit 4a62e52111.
appliance-uefi.c: In function 'guestfs_int_get_uefi':
appliance-uefi.c:67:9: error: implicit declaration of function 'access' [-Werror=implicit-function-declaration]
if (access (codefile, R_OK) == 0 && access (varsfile, R_OK) == 0) {
^~~~~~
appliance-uefi.c:67:5: error: nested extern declaration of 'access' [-Werror=nested-externs]
if (access (codefile, R_OK) == 0 && access (varsfile, R_OK) == 0) {
^~
appliance-uefi.c:67:27: error: 'R_OK' undeclared (first use in this function)
if (access (codefile, R_OK) == 0 && access (varsfile, R_OK) == 0) {
^~~~
appliance-uefi.c:67:27: note: each undeclared identifier is reported only once for each function it appears in
cc1: all warnings being treated as errors
Fixes commit dd519e8a8e.
Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
EDD (https://en.wikipedia.org/wiki/Enhanced_Disk_Drive) is a BIOS INT 13h
subcall for communicating the extended size of the boot disk over 8GB
to the OS. Since libguestfs doesn't use a boot disk, and the
appliance disk is limited to 4GB, and we use virtio-scsi, this is all
useless.
EDD is also broken currently on RHEL 7.3, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1404287
Also the EDD probing takes significant extra time (about 80ms on my
laptop), and using edd=off reduces this time although doesn't entirely
eliminate it.
One of the tests for btrfs_subvolume_show tries to invoke it on the
toplevel subvolume, expecting its failure (since btrfs-progs used to not
be able to list that). However, since v4.8.3 btrfs-progs can do that
(even if the data returned for that subvolume is limited/incomplete),
and thus this test fails because the API does not fail anymore.
Since we can consider that functionality working (for some kind of it),
then drop this test.
Very recent versions of tar (most probably as a consequence of
CVE-2016-6321) may refuse archive members with '..', like the relative
paths to upper level directories.
Since these are just tests, simply copy the files in the temporary
directories where tar (or zip as well) is run, so all the files are in
the same directory.
The return value struct was freed using a simple free() instead of the
own cleanup function of each struct: this meant dynamically allocated
values (such as strings) were leaked.
Use the proper cleanup functions instead.
The return value struct was freed using a simple free() instead of the
own cleanup function of each struct: this meant dynamically allocated
values (such as strings) were leaked.
Use the proper cleanup functions instead.
The definition of BUILDFILES was accidentally removed, resulting
in the error:
cp /home/rjones/d/websites/libguestfs/download/builder/
cp: missing destination file operand after '/home/rjones/d/websites/libguestfs/download/builder/'
Fixes commit 65a0570385.
It fails with:
File "../builder/templates/make-template.ml", line 1, characters 0-1:
Parse error: [a_LIDENT] expected after "#" (in [implem])
End_of_file
An error occurs while processing.
In any case this doesn't matter because this file should not be
translated.
Fixes commit a1ea9a2599.
See:
http://caml.inria.fr/pub/docs/manual-ocaml/comp.html#ss%3Awarn57
I believe the code as written previously was incorrect. However we
are lucky because if neither clause matches then it will fall through
to displaying an error message, allowing the user to correct the
problem.
For a very long time we have maintained two sets of utility functions,
in mllib/common_utils.ml and generator/utils.ml. This changes things
so that the same set of utility functions can be shared with both
directories.
It's not possible to use common_utils.ml directly in the generator
because it provides several functions that use modules outside the
OCaml stdlib. Therefore we add some lightweight post-processing which
extracts the functions using only the stdlib:
(*<stdlib>*)
...
(*</stdlib>*)
and creates generator/common_utils.ml and generator/common_utils.mli
from that. The effect is we only need to write utility functions
once.
As with other tools, we still have generator-specific utility
functions in generator/utils.ml.
Also in this change:
- Use String.uppercase_ascii and String.lowercase_ascii in place
of deprecated String.uppercase/String.lowercase.
- Implement String.capitalize_ascii to replace deprecated
String.capitalize.
- Move isspace, isdigit, isxdigit functions to Char module.
Support for RHEV with RHEL 6 nodes required us to output the old style
qcow2 compat=0.10 images. Since RHEV 3.6 GA, RHEL 6 has not been
supported as a RHEV node type. Since RHV 4.1, compat=1.1 is
supported. (Support for compat=1.1 is uncertain in RHV 4.0 even on
RHEL 7 nodes.)
There are significant downsides to using qcow2 compat=0.10 instead of
the modern default (compat=1.1).
Therefore this patch does two things:
For -o rhev, it drops support for compat=0.10 completely. You must
use RHV 4.1.
For -o vdsm, it adds an interim flag (--vdsm-compat=0.10 or
--vdsm-compat=1.1) which controls the compat level of the qcow2 output
file. VDSM should use --vdsm-compat=1.1 when it is known that modern
qemu is available. We can make this the default later when all RHV
instances have moved to 4.1.
It also adds:
vdsm-compat-option
to the `virt-v2v --machine-readable' output to indicate that this flag
can be used.
Thanks: Yaniv Kaul, Michal Skrivanek.
Flag the filesystems for Linux /usr properly as USR role, and detect
some data out of it, like the distro information from an os-release
(if present), and the architecture (since the binaries used for our
architecture check will be available there only).
Later on, collect the results in a way similar to what is done for
CoreOS: for each non-CoreOS root, try to find its /usr filesystem, and
if found then merge what is missing from root; in the last case, also
override the distro inspection data (version, product name) if available
in /usr.
When checking for the existance of /dev/mapper devices found in the
fstab of a filesystem, using guestfs_exists means they are checked as
files in the guest, while they really appear as devices on the
appliance. Instead, try the lvm name resolution anyway, and ignore them
when they are reported as missing.
Thanks to: Richard W.M. Jones.
Fixes commit 96b6504b09.
Move the checks for empty xmlXPathObjectPtr, and for extracting the
result string out of it, to a new helper functions.
This is just code motion, there should be no behaviour changes.
Add a new inspect role for "/usr" partitions, and use that to mark the
/usr partition in CoreOS: this additional role allows to ease its lookup
later on, when merging its results into those of the root.
Introduce a new enum to classify the role of a filesystem, if available.
This will help later on when doing operations on non-root filesystems,
like detecting particular mountpoints such as /usr.
The new enum has only "root" as known role, which replaces the is_root
flag.
Tools could require the use of pseudo-terminals, so make sure we have
/dev/pts available in the appliance. The "command" API already
bind-mounts it when running commands, so this is the only bit needed.
This refactors command line parsing into a parse_cmdline function.
The function uses the Arg module to parse the command line instead of
ad hoc parsing.
This also enforces use of the ./run script to run the program, since
otherwise you can end up with mixed versions of the OCaml bindings and
the C library.
This updates commit 65a0570385.
initviocons package provides tools to resize the terminal. Having it
in the appliance will allow SUSE users to have proper line wrapping
in their terminal when using virt-rescue.
Rebuild these images with the new make-template.ml script. This fixes
the lack of virtio-scsi driver and broken initramfs.
These images all use LVM because without that both Fedora and RHEL put
the root filesystem on an extended partition (/dev/sda5) which
virt-resize cannot handle.
(Note the RHEL images are not public.)
Create a new directory (builder/template). Integrate all of the
scripts into a single program, so that templates are generated more
consistently.
This also changes how the index file is generated. The script now
generates the index file fragment and saves it under version control,
and then generates the final index file by concatenating these.
(Previously the index was written by hand which was tedious and
error-prone.)
The new script also saves the generated kickstart under version
control so it can be referenced later.
It turns out that adding the virtio-scsi driver does not help
to fix the dracut problem.
This partially reverts commit f766c84c39,
keeping the change to index.
Unfortunately I was unable to build s390 binaries since multilib was
dropped in Fedora 24 on s390x. Going from the source of the 'file'
command it seems as if it prints "32-bit" (the architecture is really
31 bit).
As noted by Pino in another patch, the logic passes the first member of
the struct which happens to be the right address to the callback
function.
This will break the callback if order of the members of the struct will
change.
As the callback is using the entire struct, better to pass the pointer
to the struct itself.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
If an environment variable such as XDG_RUNTIME_DIR or one of the
tmpdirs or cachedir is set to a non-existent directory, improve the
error message that the user will see so that (where possible) it
includes the environment variable or API call.
This is still not bullet-proof because it's hard to display the
environment variable if it is LIBGUESTFS_TMPDIR or
LIBGUESTFS_CACHEDIR, but the main problem is with XDG_RUNTIME_DIR
(because of systemd bugs).
Thanks: Hilko Bengen for identifying the bug.
This makes a number of small changes to how the internal documentation
is generated and what can be documented.
Header files are now permitted to contain internal documentation.
This works in the same wasy as .c files.
Also documentation can be added for structs as well as functions.
This commit also adds documentation to some header files and structs,
and fixes a few places which contained broken header documentation.
We do this by sending an Inhibit() message to logind and receiving a
file descriptor back, which we hold open until the conversion
completes (or fails). This is described here:
https://www.freedesktop.org/wiki/Software/systemd/inhibit/
This adds an additional optional dependency on DBus since we use DBus
to call the Inhibit() method.
Reported-by: Chris Cowley.
Create a single temporary directory for all the files created during the
virt-builder run (except big disk images, which already use the
libguestfs cache directory). A single directory means there is no need
to manually schedule all the temporary files and directories for removal
when the application quits.
Add a new optional parameter for the Curl ADT, so temporary files can be
created in a specified directory (which is supposed to be temporary, and
disposed only when the application quits).
$0 is set to the name of the test script, eg. test-settings.sh, and in
bash there is no way to change that.
Create a new environment variable, $script, which is set to the name
of the wrapper script (eg. test-settings-fedora-24.sh).
This should give more accurate error messages.
When TESTS_ENVIRONMENT already uses 'run', the VG variable
doesn't also need to use 'run'.
The specific problem is that if the command contains newlines
then double invocations of the 'run' script fails (in libtool).
ie the following command failed causing errors in check-valgrind:
$VG virt-builder phony-fedora \
-v --no-cache --no-check-signature $no_network \
...
--write '/etc/append4:line1
' \
...
Currently 'make install' installs the virt-p2v binary in
/usr/libexec/virt-p2v on the host. It is never supposed to be run
from there, even by another program, so use of /usr/libexec is
incorrect. It is only supposed to be copied into USB keys / ISOs /
etc created by virt-p2v-make-* scripts.
The other problem with shipping a "naked" binary on the host is that
packages built from that get all the dependencies of virt-p2v, for
example Gtk. This is unnecessary just for running the command line
scripts mentioned above.
This changes the Makefile and scripts so that the binary is stored
compressed in $libdir/virt-p2v/virt-p2v.xz. It is compressed to avoid
exposing the dependencies. It is stored under $libdir since the
binary is still architecture-dependent.
A further change is that when we copy the binary into the virt-p2v
ISO, it is now installed in /usr/bin instead of /usr/libexec. (And
note that we always use /usr/bin, not $bindir, since this path should
not need to be affected by the configuration of libguestfs).
Added:
- cdrtools: added as alternative to cdrkit
- multipath-tools: contains kpartx (in AUR)
Removed:
- ntfsprogs: the package is no longer available, it has been completely
replaced by ntfs-3g (already in packagelist.in)
- zfs-fuse: no longer in AUR
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The OVF standard allows the use of SHA256 hashes in the manifest file.
Adding support for this.
One of the tests was updated to put SHA256 into manifest file.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The regular expression for parsing the manifest line was wrong. There is
a mandatory space between '=' and the hash.
Another problem was that only the first line of the manifest file was
actually processed.
Also added some debugging info and warning to catch problems with
parsing.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Try to improve the way packages of VMware tools are removed from
YUM-based guests:
- when filtering the package itself from its providers, do a stricter
check so either the provide is the unversioned package, or it is
exactly its own name
- if the package has no other providers, then going further will cause
the invocation of 'yum install' with no packages, and thus the package
itself will not be added to the list of packages to be removed; to
overcome this issue, just mark the package as "to be removed" in that
case
Related to: RHBZ#1155150
When checking whether a kernel supports virtio or it is Xen-based, it is
assumed that the feature has the kernel module for it; this will fail if
the feature is built-in in the kernel, misrepresenting it.
The solution is to check the kernel configuration whether the feature
is built-in, in case there is no module available.
This fixes the virtio detection on Ubuntu kernels, where virtio is
built in and not as module.
Detect only once which method must be used to get and set the default
Grub2 kernel in the guest: this avoids the same checks in list_kernels
and set_default_kernel.
Also, add a "no method" option as well: Debian/Ubuntu guests do not have
neither grubby nor Perl's Bootloader::Tools, so there is no way to know
what is the default kernel, nor to change it. In this case, add a
warning about this situation.
Move the Checksums module from virt-builder mostly as it is; the only
change is that on checksum mismatch an exception is raised rather than
invoking "error" directly: this way users of verify_checksum &
verify_checksums can do their own handling of the situation.
Debian is also doing a UsrMove/UsrMerge:
https://wiki.debian.org/UsrMerge
However it is not finalized that Debian will actually make this
change.
Since some Debian systems have /sbin as a symlink and other have /sbin
as a real directory, and we should avoid choosing a symlinked
directory for the daemon, the easiest fix is simply to probe /usr/sbin
before /sbin since under all scenarios (and Fedora too) /usr/sbin is a
real directory.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=838995
Create a new libfishcommon convenience static library to build just once
(instead of 12 times!) the majority of the guestfish sources used in the
rest of the C tools (mostly for command line stuff, inspection, and
mount).
The notable exceptions not using libfishcommon are guestfish itself, and
virt-rescue: both need to build at least one of the common sources using
additional CPPFLAGS.
All the define was doing in options.h was masking the declaration of
inspect_mount_root, which was always built-in in inspect.c (because of
the unconditional #define there) anyway.
Since this is common code used by all the C tools, try to avoid extra
knobs which add different code paths for no benefit.
Some of the C tools were building also config.c as part of the shared
sources from guestfish, and thus bringing a dependency on libconfig.
Since none of them actually read the libguestfs configuration at all,
then exclude fish/config.c from their build, and stop linking to
libconfig.
Using update-initramfs is the native way of updating initrd on Debian
based systems.
To add some modules to the image we can list them in file
/etc/initramfs-tools/modules.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
A disk of type 'volume' is stored as
<source pool='pool_name' volume='volume_name'/>
and its real location is inside the 'volume_name', as 'pool_name': in
this case, query libvirt for the actual path of the specified volume in
the specified pool.
Adjust the code so that:
- for_each_disk gets the virConnectPtr, needed to do operations with
libvirt
- when extracting the disk filename depending on the type, the code
snippet doing it can directly set 'filename', without setting an XPath
result variable
Only file-based volumes are supported for now; more types can be added
(with proper testing) later on.
Since libvirt 2.0.0, these two new <listen/> types have been
supported: https://libvirt.org/formatdomain.html#elementsGraphics
This change just copies that configuration over from the source to the
destination if the destination is also libvirt.
Since we previously used 'LNone' to mean "no parseable <listen/>
element" I also had to change previous uses of 'LNone' to 'LNoListen',
so we can use 'LNone' to mean "<listen type='none'>".
Thanks: Ming Xie.
NTFS file system always has the MFT file at inode 0. This reliable
information helps testing the API.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Library's counterpart of the daemon's internal_find_inode command.
It writes the daemon's command output on a temporary file and parses it,
deserialising the XDR formatted tsk_dirent structs.
It returns to the caller the list of tsk_dirent structs generated by the
internal_find_inode command.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The internal_find_inode command searches all entries referring to the
given inode and returns a tsk_dirent structure for each of them.
The command is able to retrieve information regarding deleted
or unaccessible files where other commands such as stat or find
would fail.
The gathered list of tsk_dirent structs is serialised into XDR format
and written to a file by the appliance.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Make use of the additional command line arguments, and API needed to
decrypt LUKS partitions.
This affects only virt-customize, virt-get-kernel, virt-sparsify, and
virt-sysprep, as they are the main OCaml tools interacting with
user-provided images.
Expose via Common_utils the C functions & variables (part of guestfish)
that handle decryption of LUKS partitions, and the additional command
line arguments to tune the way they work. This way it will be easy to
provide (basic) crypto support also in OCaml-based tools.
Related to: RHBZ#1362649
With the current implementation, the root inode of the given partition
is ignored.
The root inode is now reported. Its name will be a single dot '.'
reproducing the TSK API.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
On Debian family of OSes Grub2 tools are prefixed with 'grub-', not with
'grub2-'. We have to detect the correct name of the tool to use it.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
When creating the XML for the new guest, always put the name of the pool
containing the disks, even when -os specified a pool UUID: libvirt does
not handle pool UUIDs for storage, but only names.
When running in verbose mode for debugging, run `glance --version` to
print the version of the glance client. This helps when dealing with
failures, or making its output actionable.
The glance client v1.0.0 defaults to the Image v2 API: this means that
an image can be referenced only by its UUID, and not anymore by the name
as well (which does not need to be unique). This caused our glance
invocation to set the properties for the newly created image to fail,
since we are using the name as identifier.
Instead of first creating the image and then setting all the properties
for it, set all the properties when creating the image: other than being
simpler, it also avoid parsing the output of the 'glance image-create'
command for the UUID ('id') of the image.
Create a new module [Linux_kernels] which does all kernel detection,
and also provides a place to define the kernel_info data structure.
This is essentially just code motion.
If the guest has no <Name> element in the OVF, previously we chose
"default" as the name. This changes that so it uses a name derived
from the basename of the OVA file instead.
For example:
virt-v2v -i ova /path/to/myguest.ova [...]
would use "myguest" as the name (assuming no <Name> was present).
Modifies the behaviour of
commit 1ae4252c93.
Upstream Perl is going to remove '.' from @INC (the include path for
modules) by default for the next major release (= 5.26) [1], as measure
to fix security issues. Debian already started backporting the fixes
for this [2], thus behaving this way in current Sid installations.
Since the affected Perl sources are only the local daemon testing
scripts, a simple fix is to force the 'requires' for the local
captive-daemon.pm module to start from the current directory: this way
there is no need to manually augment @INC, and only our local module is
loaded automatically.
[1] https://rt.perl.org/Public/Bug/Display.html?id=127810
[2] https://lists.debian.org/debian-devel-announce/2016/08/msg00013.html
Make use of the recently added 'getprogname' module in gnulib: replace
our guestfs_int_program_name with the getprogname() provided by the
module, since it does the same thing, and in a portable way.
As consequence of the above, use gnulib in a couple of tests that use
getprogname().
Since guestfs_int_program_name is gone, drop the configure checks
associated with it.
When running 'make maintainer-check-extra-dist', check also that all
generated files are included in the tarball. This is done so that end
users will not need OCaml to compile from tarball releases.
Previously we used an awkward hack to split up the large src/actions.c
into smaller files (which can benefit from being compiled in
parallel). This commit generalizes that code, so that we pass a
subsetted actions list to certain generator functions.
The output of the generator is identical after this commit and the
previous commit, except for the UUID encoded into tests/c-api/tests.c
since that is derived from the MD5 hash of generator/actions.ml.
This mostly mechanical change moves all of the libguestfs API
lists of functions into a struct in the Actions module.
It also adds filter functions to be used elsewhere to get
subsets of these functions.
Original code Replacement
all_functions actions
daemon_functions actions |> daemon_functions
non_daemon_functions actions |> non_daemon_functions
external_functions actions |> external_functions
internal_functions actions |> internal_functions
documented_functions actions |> documented_functions
fish_functions actions |> fish_functions
*_functions_sorted ... replacement as above ... |> sort
Commit ab2df2e659 attempted to rewrite
/etc/hosts if it contained the old hostname.
However this wasn't done reliably for a couple of reasons:
(1) Certain Debian/Ubuntu /etc/hostname contains
"localhost.localdomain" but still has "unassigned-hostname" or
"unassigned-hostname.unassigned-domain" in /etc/hosts.
(2) Even if (1) doesn't apply, you still need to split out the
hostname and domainname parts and deal with them separately.
Use /etc/os-release as first option, translating the distro name to the
current identifier used. The other options (the release files) are left
as following checks, avoiding them if any matches.
Don't remove the Processor and Intelppm nodes since that just breaks
the device driver.
The only remaining node being removed by the original code was
"rhelscsi" (the xenpv-win driver). I changed this so that instead of
deleting the whole node, it simply disables that driver. If you look
at RHBZ#737600, it's not even clear that deleting nodes was the right
fix for anything. I also renamed the function.
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
When put on new virtual hardware Windows will start driver installation
for newly discovered devices.
The problem is that it happens asynchronously and concurrently with
other activities, in particular, the firstboot scripts. This may result
in conflicts (because a firstboot script may want to install or
uninstall a driver, etc.) and eventually in the system left in
inconsistent state.
In order to prevent it, add another firstboot script which calls a
special utility, pnp_wait, whose sole purpose is to wait until all
PnP-related activities are finished. (It does so via a WinAPI call
which isn't available from a script so it has to be a compiled .exe.
The source code for it can be found in the corresponding directory in
https://github.com/rwmjones/rhsrvany). This firstboot script is put
first, so that other firstboot scripts do not have to worry about
interactions with PnP manager any more.
One caveat is that on Windows XP and Windows 2003 the PnP manager always
fires the "Found New Hardware" wizard, which doesn't allow PnP to make
any progress until the user closes it. As a result, this firstboot
script would never finish.
To work it around, follow the Microsoft KB
(https://support.microsoft.com/en-us/kb/938596) and set up a registry
key to make the the PnP manager not fire the wizard; the key is restored
to its initial state upon PnP completion. Unfortunately this only works
on systems having the mentioned update installed; otherwise the user
will have to interact with the UI.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
It will be used in new code in a followup patch.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
If multiple tests (eg. from different branches) are running in
parallel, they will attempt to use and delete the same temporary pool.
Use a random pool name to avoid this.
Other functions, such as ``-i ova'', also call Xml.parse_memory.
Therefore the error is not always from XML generated by libvirt, the
XML might be generated from other sources too.
Thanks: Ming Xie.
Following the same style used for the firstboot tests in the
customize/ subdirectory, use a separate test script for each real
guest that we test. This has two advantages: (a) the tests run in
parallel (and are thus faster) and (b) automake "knows" about each
test and reports the results separately.
Methods in OCaml which don't take any parameters don't require the
dummy unit arg, ie writing:
method foo = ...
is fine. The reason you might need the unit arg is if you need to
create a closure from the method without calling it, for example if
you need to use the method in a callback.
In lablgtk2 the convention is to use unitless methods if either: the
method shouldn't be used as a callback; or: (conceptually) the method
doesn't change the object's internal state. Let's do that here.
The name of the initrd image on Debian-based systems is different from
what used on Fedora/RHEL/SUSE and derived; set a different regexp to
avoid making the current regexp even more complex.
Implement the 'remove', 'file_list_of_package', and 'file_owner' methods
of the Linux module for the "deb" package manager (dpkg basically, on
Debian and derived distributions).
Also allow it for the main conversion code.
Move the actual job in an helper function, so the common bits (like the
check of the size of 'packages' and the reload of Augeas) can be done
for all the package manager implementations.
This should be code motion with no behaviour change.
Old perl/POD from Debian Wheezy has a bug where <name>NAME</name>
gives the error:
Unknown E content in E</name>
even though this is in a verbatim section. Fudge the documentation to
avoid this.
In OCaml 3.12, type inference was not as smart and could not use local
information to disambiguate same-named fields in different structures.
The easiest fix for this is just to rename the field so it doesn't
have the same name as a field in the Customize_cmdline.ops struct.
In Debian Wheezy with libvirt 0.9.12.3, HAVE_LIBVIRT was defined, but
HAVE_LIBVIRT_BACKEND was undefined, so src/libvirt-auth.c failed to
compile.
It's easy, and better, to check the minimum version in the configure
script.
Create an object hierarchy to represent different bootloaders for Linux
guests, moving the separate handling of grub1 and grub2 in different
classes: this isolates the code for each type of bootloader together,
instead of scattering it all around.
This is mostly code refactoring, with no actual behaviour change.
String.map was added in OCaml 4.00.0. However we use this function
to implement String.lowercase_ascii etc.
Therefore include a definition of the function for older versions of
OCaml. (Debian Wheezy has OCaml 3.12.1.)
By adding common CLEANFILES and DISTCLEANFILES variables to
common-rules.mk, we can remove these from most other Makefiles, and
also clean files more consistently.
Note that bin_PROGRAMS are already cleaned by 'make clean', so I
removed cases where these were unnecessarily added to CLEANFILES.
There was some code in configure.ac dating back to 2009
(commit b9014d6a0d) which attempted to
add the javac option `-source 1.5'. I don't think this code ever
worked.
However, if -source 1.5 is added, then you get this warning:
warning: [options] bootstrap class path not set in conjunction with -source 1.5
warning: [options] source value 1.5 is obsolete and will be removed in a future release
warning: [options] To suppress warnings about obsolete options, use -Xlint:-options.
and that's very hard to reliably fix because you have to know somehow
a magic path. For more details, see:
https://blogs.oracle.com/darcy/entry/bootclasspath_older_sourcehttps://blogs.oracle.com/darcy/entry/how_to_cross_compile_for
Anyway, remove the voodoo from configure.ac.
As a side effect, this allows you to set the EXTRA_JAVAC_FLAGS
environment variable before ./configure with any extra javac flags
that you want.
Add also /boot/efi/EFI/redhat/grub.conf as configuration of Grub 1;
since the "grub" lens of Augeas does not handle this path, add a
transformation so Augeas can parse it.
Expose the aug_transform API through the library, so it's possible to
add/remove Augeas transformations to handle files in custom places using
existing lenses.
Add new scripts for:
- guestunmount
- virt-copy-in
- virt-copy-out
- virt-customize
- virt-dib
- virt-diff
- virt-get-kernel
- virt-p2v-make-disk
- virt-p2v-make-kickstart
- virt-p2v-make-kiwi
- virt-tar-in
- virt-tar-out
Also combine the separate virt-resize script into the general script
virt-alignment-scan. There wasn't really any reason to have separate
scripts.
By moving these two functions out of the common options parsing code,
it means we don't need to depend on all the other machinery of options
parsing, such as the global variables ("verbose"), libconfig, etc.
The code to parse btrfs subvol entries in /etc/fstab failed if the
entry had more than one comma-separated option, for example:
/dev/sda4 /home btrfs rw,user,subvol=foo 0 0
This commit fixes that code to use Augeas correctly.
Fixes commit 7ba0e10501.
Reported by: Zhongfu Li
https://bugs.launchpad.net/ubuntu/+source/libguestfs/+bug/1615337
Libvirt >= 2.1.0 now allows you to open files which have a "json:"
QEMU pseudo-URL as backingfile, whereas previously it would fail hard
in this case (RHBZ#1134878).
When virt-v2v performs conversions from Xen (over SSH) or vCenter
(over HTTPS) it uses these pseudo-URLs as backingfiles. We had to
tell people to use LIBGUESTFS_BACKEND=direct to avoid libvirt in this
situation.
This commit narrows the check so it will now only print the error if
libvirt < 2.1.0 and LIBGUESTFS_BACKEND=direct is not set. Also the
error is modified to tell users they can either upgrade libvirt or set
LIBGUESTFS_BACKEND=direct to work around the problem.
Note there is not an easy way apart from checking the version number
of libvirt to tell if the json pseudo-URL is supported.
As a side-effect, this commit also prints the libvirt version number
in debugging output when virt-v2v starts up, which is sometimes useful
information for narrowing down bugs (it is in fact already printed by
libguestfs, so this is duplicate information, but it's a bit easier to
find when it's at the top of the debug).
Thanks: Peter Krempa.
Make 'which' gracefully handle the case where $PATH is not set
(it will raise Executable_not_found, but that is the expected thing to
do).
Related to RHBZ#1367839.
From RHEL 7.3, Red Hat have decided to only compile the secure boot
version of OVMF on x86-64, with flags -D SECURE_BOOT_ENABLE -D SMM_REQUIRE.
The filename has also changed to reflect this - it is now
/usr/share/OVMF/OVMF_CODE.secboot.fd. The old file
/usr/share/OVMF/OVMF_CODE.fd is no longer shipped.
However switching to using this variant of OVMF for UEFI guests is not
just a matter of changing the filename. The new OVMF variant won't
run unless we also change:
- The qemu machine model, from the default ("pc" ==
"pc-i440fx-rhel7.3.0" or later) to Q35 ("q35" == "pc-q35-rhel7.3.0"
or later).
- Add <smm> under <features>.
- Set <loader ... secure="yes">.
NB: On RHEL the changes requires qemu-kvm-rhev. It is no longer
possible to convert UEFI guests using the basic qemu-kvm.
Thanks: Laszlo Ersek, Ming Xie.
Previously we had a list of UEFI paths in src/uefi.c, which was
accessed in virt-v2v using a private (guestfs_int_*) API and some C
binding code. This was clumsy and required the paths to be replicated
in the virt-v2v unit tests.
Instead just generate the list of paths from the generator, creating
src/uefi.c and v2v/uefi.ml with the same content.
Remove the C bindings and the virt-v2v unit tests associated with UEFI
paths.
This *may* be required by some filesystems in order for fstrim to
work. I'm not actually sure if this is true, but it's what
virt-sparsify --in-place does, and that utility has been tested a lot
more in regards to trimming.
This reverts the change made for RHBZ#1168144. The warning is now
always displayed.
It would be nice to make the warning actionable, but there is not a
lot that end users can do since fstrim is such a complex topic
interacting with all filesystem and storage layers.
Do not print warning for 'sr' devices when converting fstab. Not all
systems create the /dev/cdrom symlink for SCSI CD-ROM devices. Moreover,
on systems with multiple CD-ROMs, having entries for /dev/sr* devices
may be inevitable.
RWMJ: Use String.is_prefix instead of String.find, to more accurately
match on the device name.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Conversion from Citrix Xen has now been tested and should work. Remove
the comment about it from the documentation.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The replacement string was wrong. There are only two match groups in the
regular expression, not three.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
The Invalid_argument exception is there to catch unexpected situation
when rpm returns no output. Such situation should be reported rather
then hidden.
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
Because we run the external fstrim command we don't have access to the
kernel errno when it fails. However in the case where it prints this
specific error message, turn that into errno ENOTSUP.
Add makefile variables to enable silent rules for simple command
invocations, such as ocamlc, ocamlopt, javac, and erlc.
This reduces the log output when building with silent rules, still
showing the full command lines otherwise.
Allow to specify the format of the helper drive, to avoid relying on the
format autodetection. This is the same approach used also in all the
other tools.
Turn the rc.d symlinks for the guestfs-firstboot service as relative,
instead of absolute paths: the result is the same (the service works the
same), and this way is more coherent with symlinks created by
update-rc.d.
Currently we install a systemd service named firstboot.service and a
SysV service named virt-sysprep-firstboot. On systems where systemd is
the init system and runs with the SysV compatibility, the different
names make systemd handle them as different services, and thus trying to
run the firstboot script runner twice.
Rename both the systemd service and the SysV one to guestfs-firstboot:
the new name is less generic, and allows the systemd service to be
shadowed by the SysV service (and thus running just once).
Also cleanup the old services: the old SysV service can be removed
directly, since its former name had "virt-sysprep" in it, and so there
could not be much room for confusion and conflict. Regarding the old
systemd service: to avoid leaving it behind, a simple cleanup strategy
is in place, checking the content of the old firstboot.service to really
ensure we are removing one of our versions of this file.
Export FS_TYPE for the elements, so they can know what is the filesystem
created.
This was implemented in diskimage-builder upstream as
commit be521bdec6233530cc952be246a66f1957b0dd58.
Implement the "docker" output format, to import an image into docker.
This was implemented in diskimage-builder upstream as
commit 1187f9d5df6d6eddba478ac75d3834391480755f and
commit edc06a20e57bdf0a9a03949b54395008d73520dc.
Instead of running them before lanching the appliance with the disks and
then uploading the result after root.d hooks run, mount the root in the
local temporary directory using FUSE and then run the hooks on the
guest. Other than being closer to what diskimage-builder does, it also
avoids issues with the extra-data.d scripts assuming to find things
already, and we don't error out while trying to unpack files on the
guest.
Since virt-dib requires FUSE now, build it only if FUSE is supported.
run_command uses Unix.create_process which forks a child process, and
executes execve: the latter fails when the executable does not exist,
triggering the exit which, in older OCaml versions [1], also runs the
at_exit handlers.
Since there is not much that can be done to avoid this on the OCaml
side, to keep run_command working also in older OCaml version then
manually search for the existance of the given executable, exiting with
code 127 (as a shell does) in this case.
[1] http://caml.inria.fr/mantis/view.php?id=7209
If the user doesn't specify a format (ie. they want autodetection),
and the drive is set to read-only, previously we created a qcow2
overlay to protect the drive but didn't set the backing_fmt field.
However libvirt disabled this so now libvirt doesn't work at all on
qcow2 files that have no backing_fmt field set.
We must therefore do autodetection in libguestfs and use that to set
the backing_fmt field.
We were already doing autodetection in the non-readonly case, so this
commit refactors that code into a new function
("get_source_format_or_autodetect") and uses it on the readonly path
too.
Because we didn't give the type of 'g' there wasn't sufficient
information for the type checker to pick up this problem. By
specifying the real type of 'g' in a type annotation, that reveals the
problem.
Thanks: Pino Toscano, Nyoxi on IRC.
What was happening in file_owner function did not match the description
in the comment. When a path is owned by multiple packages the returned
string was in fact a concatenation of the names of all packages that own
it. E.g. for `Linux.is_file_owned g inspect "/etc"` the returned value
was "filesystemyum" (i.e. "filesystem" + "yum").
Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
$OSINFO_SYSTEM_DIR (or its /usr/share/osinfo fallback) represents the
top-level directory of the osinfo data, while we need to read the "os"
subdirectory of it.
Fixes commit c2ae46a9eb.
Directly query the attributes in the media node, instead of using an
XPath expression to read it.
Saves ~10ms when parsing the libosinfo DB on a Fedora 24 installation
(other than making the code easier).
Check the presence of Gtk properly depending on the value of --with-gtk:
if a specific version is chosen, then let PKG_CHECK_MODULES fail if that
version if not found, otherwise fallback from gtk3 to gtk2 to no gtk.
Also move few common AC_SUBST in a single place.
It looks like flex 2.6.1 changed [1] the return code for EOF in
yyinput. Therefore, use the right value depending on the version of
flex which generates the lexer.
[1] f863c9490e
Setup the volatile /run in the appliance also with the tmpfiles
configurations available. In particular, setting up correctly the lvm
bits allow lvmetad to run.
Commit 2e16e3e993 added lv_active=active
as additional condition when listing LVs, to ignore those with the
activationskip flag set. OTOH, this check is too broad, and matches
also other kind of LVs.
Change the condition to lv_skip_activation!=1, so matching precisely
what was meant, and only that.
Related to: RHBZ#1306666
Currently lvmetad is started in init, and thus using the system
(= appliance) configuration of lvm. Later on, in the daemon, a local
copy of the lvm configuration is setup, and set it for use using the
LVM_SYSTEM_DIR environment variable: this means only the programmes
executed by the daemon will use the local lvm configuration, and not
lvmetad.
Thus manually start lvmetad from the daemon, right after having setup
the local lvm configuration, and still without failing if it cannot be
executed.
Additionally, since lvmetad now respects the right configuration, make
sure to update its cache when rescanning the VGs by passing --cache to
vgscan.
When editing the lvm configuration to set the LVM filter, edit the
'global_filter' key in addition to 'filter': the latter is not used when
lvmetad is running, when only the former works.
Commit b91b39e06a changed the separator to
':', although this creates parsing issues when there are fields with
colons (for example lv_tags=imgbased:pool).
Change once more the separator, but this time using a non-printable
character such as '\r': while it will produce uglier debug logs, this
should greatly reduce the possibilities of conflicts with texts of
metadata.
Switch to guestfs_int_version_from_x_y_or_x to parse version numbers --
although, restrict the parsing to what could look like a valid version
number, to discard quickly version strings like "unknown".
This makes sure that Debian, Fedora, Mageia, and SLED ISOs have the
right version number.
More recent versions of libosinfo switched the internal directory with
the XML files of OSes to a different layout (still with the same XML
format), causing libguestfs to not read them anymore. Furthermore, the
internal directory is going to disappear soon, replaced by a public
osinfo database [1].
Revamp the way libguestfs reads the data: first try the upcoming osinfo
layout, falling back to the current libosinfo layout (which is the same
as osinfo), and then to the old flat layout.
[1] https://gitlab.com/libosinfo/libosinfo/blob/master/docs/database-layout.txt
-Wnull-dereference and -Wshift-overflow are new warnings in GCC 6, so do
not try to disable them with pragmas on older GCC versions.
Fixes commit a8e15ea924.
One -Wnull-dereference warning is real: we deliberately cause a
segfault in one of the tests.
There is a -Wshift-overflow bug in a Gtk 2 header.
The others are the result of shortcomings in GCC.
In all cases we have to add GCC diagnostic overrides to ignore
the warnings when compiling with ./configure --enable-werror.
The commit claimed that SELinux was disabled in the
virt-p2v-make-kickstart script, and so this was just making things
consistent. However this is not true. SELinux *is* enabled in the
kickstart version. There is some code to disable it, but it is
commented out (admittedly it's hard to tell because the code is buried
in a multi-line sed expression).
The commit message also claimed (this time correctly) that the
'--selinux-relabel' step caused an extra reboot each time virt-p2v is
run from the ISO. However we have recently fixed this so it can
usually do the relabelling during the build, not at boot time, so this
is no longer a problem.
This reverts commit 25ffcc7d43.
For OCaml tools this does essentially nothing useful because the
--help output is automatically generated from the options, and so
cannot be wrong. However for C tools this is a useful check.
It would be nice to generate C tools --help output, but there isn't
enough information in the getopt data to do that.
This commit also includes fixes to the --help output for a few tools.
This function allows to download file system data units (blocks) from
the given partition.
The API can be used to detect data hidden within filesystem bad blocks
or slack space.
Moreover for filesystems such as Ext3 and Ext4, this function is the
only way to retrieve deleted files. An example is given in the function
tests.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Pass -q to virsh if virt-v2v was run with -q, to reduce its output in
quiet mode.
Unfortunately this does not currently work in virsh as it should, see
RHBZ#1358179, so only after that bug is fixed this change can actually
take effect.
ocamlmklib -g does not work in OCaml 4.01.0. It does work (and is
desirable) in OCaml 4.02.2. However since we would like to support
back to OCaml 3.11, we cannot use it.
In libguestfs 1.33.43 we managed to release a completely
non-functional virt-customize binary (it ran without error but did
nothing at all, whatever command line arguments you gave it).
Improve the test so this cannot happen again.
podcheck.pl is run as part of the tests to perform various checks on
the documentation and the tool.
Currently we check only that the documented options matches the
options that the tool implements and vice versa. This commit would
also allow us (in future) to check --help, --long-options,
--short-options, --version output.
This commit includes scripts to run the tests and various fixes to the
manual pages to ensure that the tests pass.
For guestfish, guestmount, remove '?' from short options. Currently
those tools don't process -?, so I believe these are erroneous:
$ guestfish -\?
Try `guestfish --help' for more information.
For virt-format, the -c, -d and -q options are removed. These options
just give errors because they appear in the short options list but not
in the case statement.
Store them directly in List_entries, an adding helper function to
convert from string.
Also use Getopt.Symbol for them, so there is no need to manually reject
invalid formats.
Instead of linking with individual objects, which is very tedious,
build a proper library and link virt-builder, virt-customize and
virt-sysprep to it.
This makes the binaries a tiny bit smaller because .cmxa/.a files
allow unused code to be removed by the linker, whereas explicitly
linking .cmx/.o files does not.
Instead of linking with individual objects, which is very tedious,
build a proper library and link the other tools with it.
This doesn't make the resulting binaries any larger.
This implements the --selinux-relabel option for virt-customize,
virt-builder and virt-sysprep. There is no need to autorelabel
functionality now.
Thanks: Stephen Smalley
Sadly, the dhclient-script shipped as part of isc-dhcp-client in Ubuntu
unconditionally reads from /etc/fstab without checking for its
existence. Since no package holds /etc/fstab, this file will not exist
in the appliance, cause dhclient to fail (actually keep looping calling
the failing dhclient-script) when the network is requested.
As a workaround, touch /etc/fstab just before enabling the network: if
that file exists nothing changes, while an empty file will be available
in the other case, making at least dhclient-script in Ubuntu working.
Since the function uses the StringMap module, I also had to
move this to mllib.
The function is renamed and the 'if verbose ()' part is moved into the
function, but apart from that, it's all just code motion.
Add a dummy description value to mark an option as "hidden", so it will
not be shown in the help text.
Mark few options as hidden:
- common: --short-options, --long-options, --debug-gc
- virt-sysprep: --dump-pod, --dump-pod-options
Since --debug-gc is now really considered internal, it is no more
documented.
Add a new Getopt module to mllib, to parse command line arguments with
handlers close to the ones used with Arg, but using getopt(3) (actually
getopt_long_only) to do the real parsing. This allow us to provide
options for OCaml tools with a syntax similar to the C tools, and use
the additional features getopt offers and Arg does not.
Getopt now handles every part of the command line handling, including
the output of short & long options.
Do a single-step conversion of Common_utils and all the OCaml tools to
the syntax of Getopt.
Move a couple of utility functions from Common_utils to Getopt, since
they fit better there (and Common_utils cannot be used in Getopt, as
the former already uses the latter).
As side-change due to the conversion, extra arguments for sysprep
operation can have more keys for the same argument.
In previous commit c5f12e47e4 we changed
the PATH environment variable to run the locally built virt-v2v, not
the system-installed virt-v2v.
However this meant that we were running the locally built virt-v2v
against the system-installed libguestfs.so.0. We also need to set
LD_LIBRARY_PATH.
Fixes commit c5f12e47e4.
In the case where we are building a Python dist (which can be
distributed separately), we generated calls to non-existent optargs
structures if an API was once no-optargs but had optargs added. A
typical error was this (library: 1.32.5, Python sdist: 1.33.42):
guestfs-py.c: In function ‘guestfs_int_py_glob_expand’:
guestfs-py.c:9648:40: error: storage size of ‘optargs_s’ isn’t known
struct guestfs_glob_expand_opts_argv optargs_s;
This is a simple solution to this, although it has the possible
disadvantage that such functions will not be bound at all.
Since commit 83e92b4a97, utils.c
includes "ignore-value.h". We copy utils.c into the python sdist
tarball, but it didn't not compile because of the missing header file.
Therefore we need to copy the header in too.
Fixes commit 83e92b4a97.
guestfs_set_uuid wants device as argument.
Moreover, btrfstune -U is unable to set uuid for subvolumes,
only unmounted partitions are supported.
Here we skip btrfs subvolumes.
Calling guestfs_is_lv on btrfs subvolume throws an error.
Here we workaround it by taking Mountable instead of Device
and returning 'false' for non-device mountables.
Access, modification, last status change and creation time in
Unix format as for statns.
Number of links pointing to a given entry.
If the entry is a symbolic link, report the its target path.
A new flag (DIRENT_COMPRESSED 0x04) indicating whether the file is
compressed using native filesystem compression support.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
This adds imperative list manipulation functions inspired by Perl.
The functions are passed list refs which get updated in place.
This allows us to replace some awkward pure functional code like:
let xs = ys in
let xs = if foo then xs @ zs else xs in
with:
let xs = ref ys in
if foo then append xs zs;
These are now passed using a curl configuration file, which is a
little bit safer than using command lines. virt-builder doesn't need
to pass usernames and passwords to curl, but if it ever does in future
this will be a lot safer.
Change the Curl module to use an ADT to store the name of the curl
binary and the arguments.
The callers in virt-v2v are changed accordingly.
This also adds a (currently unused) ?proxy argument to allow callers
to override the proxy. It also adds some safety arguments implicitly.
Declare most of the stringsbuf as CLEANUP_FREE_STRINGSBUF, so they are
freed completely on stack unwind: use take_stringsbuf() in return
places to take away from the stringsbuf its content, and remove all the
manual calls to free_stringslen (no more needed now).
This requires to not use free_stringslen anymore on failure in the
helper functions of stringsbuf, which now leave the content as-is (might
be still useful even on error).
This allows us to simplify the memory management of stringsbuf's, which
are not properly fully freed, fixing memory leaks in some error paths
(which were not calling free_stringslen).
If add_string_nodup fails free the passed string instead of leaking it,
as that string would have been owned by the stringbuf.
Adapt few places to this behaviour.
We have to download the old 32 bit explorer.exe in order to find the
icons as the 64 bit version doesn't contain any icons (where are they?).
Thus prefer the 32 bit (WoW64 subsystem) directory if found.
Fixes commit 7f16c346bb.
Thanks: Xiaoyun Hu
The current location doesn't exist unless you've installed GNOME,
which is not so common on Ubuntu. Unfortunately I couldn't find any
other location containing a clean, high quality logo.
This adds another low quality icon source, and also prevents any icon
being returned if the highquality flag was set (note this prevents
virt-manager from displaying an icon, but there's nothing we can do
about that, and it's no worse than the current situation).
Updates commit 1d0683964f.
Thanks: Xiaoyun Hu
This is now enabled by default and cannot be changed/disabled by the
user. The output is saved into /tmp/.../virt-v2v-conversion-log.txt
on the conversion server.
This makes sure that in the internal configure_kernel_modules function,
for virtio or SCSI block types:
a) the warnings about leftover Xen modules are printed
b) the changes in Augeas are saved
Fixes commit ee02191483.
First check for the existence of the directory /etc/modprobe.d, in case
using a file under it; this also skips all the other checks, since they
are not needed at all. Also /etc/modprobe.d exists on recent Linux
versions, so let's give priority to the more common methods.
When /etc/modprobe.d does not exist, check for the file to edit using a
single list of possible files, now in order of priority, where the first
find is used without checking further for the rest.
Also, make sure all the returned paths are absolute: they are used in
Augeas paths later on, so relative paths will not do anything useful.
We are now able to cancel the conversion instantly by sending ^C to
the remote virt-v2v process.
Also, this reverts:
"p2v: Poll to make Cancel Conversion button more responsive."
(commit 6da4941db7)
Previously we copied / creates files in the remote dir by running
complex shell commands like:
cat > file <<'EOF'
## the file was copied in here
EOF
This was a little hairy, but in particular it doesn't allow us to set
the ssh session to cooked mode (so we can send ^C to cancel a
conversion).
A cleaner way to do it is to use 'scp' to copy the files over.
The messages that come "through" from virt-v2v are already colourized.
However the other messages are not. This colourizes the ones
generated during kernel-mode conversions.
Note that because of the way journal/syslog works we have to write the
ansi_restore before the end of line, thus code like this:
ansi_magenta (stdout);
printf ("%s: %s", guestfs_int_program_name, data);
ansi_restore (stdout);
putchar ('\n');
This also adds a "Conversion finished successfully" message (in green).
Thanks: Ming Xie
Add powerpc64 (ppc64) and powerpc64le (ppc64le) cases to the
normalize_arch function. Also remove a few duplicate cases which are
already caught in the catch-all case at the end of the match.
In kernel conversion mode, we must ensure messages are flushed to the
journal as soon as they are received from virt-v2v, otherwise (now
that debugging is not the default) we won't see any messages at all
until the last moment.
Thanks: Ming Xie
This allows you to install extra packages in the disk/ISO. The
implementation of this option in virt-p2v-make-disk was particularly
simple and followed naturally from the previous commit.
- sanitize the logs, removing terminal color codes and carriage returns
- add an empty "properties" node
- put the log for skipped tests as "message" attribute, hoping it is
read from there
- do not blacklist the log of test-virt-rescue.pl, which should not
cause issues now
If an ssh server is completely unresponsive (ie. it doesn't even send
back ICMP errors), then eventually ssh times out after ConnectTimeout
seconds [ConnectTimeout is an ssh option].
Unfortunately the miniexpect timeout (default 60 seconds) was less
than ConnectTimeout so we never saw the ssh error. Instead we would
see the internal error:
remote server closed the connection unexpectedly,
waiting for: password prompt
This commit sets the ssh ConnectTimeout to an explicit value, and sets
the miniexpect timeout to be 20 seconds larger than ConnectTimeout, so
that in most cases we will see the ssh error message before miniexpect
times out.
Thanks: Ming Xie for finding and diagnosing the problem
In the GUI disks block, instead of using separate "Size" and "Model"
columns, combine that information into a markup column under "Device".
To make the GUI look consistent I also had to change the removables
"Device" column to be a markup column and embolden the device name
there too.
This is just refactoring, but it reveals and fixes a bug too. The
size_gb field was left as NULL when the --test-disk option was used.
Apparently passing NULL to gtk_list_store_set is fine, but just in
case I replaced it with "".
The error you would see is:
virt-v2v: error: insufficient free space in the conversion server temporary
directory /var/tmp (853.8M).
Either free up space in that directory, or set the LIBGUESTFS_CACHEDIR
environment variable to point to another directory with more than 1GB of
free space.
See also the virt-v2v(1) manual, section "Minimum free space check in the
host".
Also adds some documentation.
Thanks: Ming Xie and Xiaodai Wang
There is a tiny functional change in this patch, since overlay_dir is
now always evaluated once (eg. even in --inplace mode), whereas
previously it was evaluated twice but only in copying mode.
As described in the comment, this solves a number of problems with
non-standard remote configurations.
I tested this with:
- tcsh
- zsh
- ksh
and all behaved correctly.
Print a better error message when the non-root user on the conversion
server requires a password to use sudo, and p2v is told to use sudo.
See also RHZ#1340809.
Library's counterpart of the daemon's internal_filesystem_walk command.
It writes the daemon's command output on a temporary file and parses it,
deserialising the XDR formatted tsk_dirent structs.
It returns to the caller the list of tsk_dirent structs generated by the
internal_filesystem_walk command.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The tests check whether the filesystem_walk command is able to retrieve
information regarding both existing and deleted files.
A NTFS image is used as Ext3+ filesystems deletion is more aggressive
in terms of metadata removal.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
- generator: Added tsk_dirent struct
The tsk_dirent struct contains the information gathered via TSK APIs.
The struct contains the following fields:
* tsk_inode: inode of a file
* tsk_type: type of file such as for dirwalk command
* tsk_size: file size in bytes
* tsk_name: path relative to its disk partition
* tsk_flags: bitfield containing extra information
* tsk_spare[1-5]: extra space for future usage
- configure: Added libtsk compile-time check
Ensure libtsk is available at compile time.
If not, daemon routines depending on it won't be available.
- API: internal_filesystem_walk
The internal_filesystem_walk command walks through the FS structures
of a disk partition and returns all the files or directories
which could be found.
The command is able to retrieve information regarding deleted
or unaccessible files where other commands such as stat or find
would fail.
The gathered list of tsk_dirent structs is serialised into XDR format
and written to a file by the appliance.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
The previous code treated floppy disks and CD-ROMs as the same kind of
thing, resulting in malformed libvirt XML. You would see the
following error when importing a guest into libvirt:
error: Failed to define domain from /tmp/v2vlibvirt063486.xml
error: internal error: Invalid floppy device name: hdb
because we incorrectly generated this bogus libvirt XML fragment:
<disk device='floppy' type='file'>
<driver name='qemu' type='raw'/>
<target dev='hdb' bus='ide'/>
</disk>
This commit models floppy devices as a distinct type, occupying their
own bus ("/dev/fdX"). When writing to libvirt, we generate correct
XML fragments, looking like this:
<disk device='floppy' type='file'>
<driver name='qemu' type='raw'/>
<target dev='fda'/>
</disk>
Miscellaneous other changes were required in the code. There is also
a regression test (see following commit).
Note this ignores floppy disks in '-o qemu' mode.
aarch64 image isn't buildable at the moment because of
Anaconda bug RHBZ#1348980.
ppc64 & ppc64le images are not buildable because of delays updating
the Fedora mirrors.
This adds <BootOrder>1</BootOrder> to the first drive, which is
necessary to make the drive bootable (or would be necessary, if oVirt
hadn't implemented their own workaround for the OVF we were previously
generating).
https://bugzilla.redhat.com/show_bug.cgi?id=1308535#c11
This is not a full solution to the problem. Really we should be
copying the boot order over from the source hypervisor, but that
requires extensions to libvirt as discussed here:
https://bugzilla.redhat.com/show_bug.cgi?id=1308535#c7
Thanks: Shmuel Melamud
e2fsck returns 1 in case of "file system errors corrected".
We treat it as success in normal e2fsck, but fail if e2fsck
is run by resize2fs.
Change 'manual' execution of e2fsck to dedicated function call.
On distros not running NetworkManager, use the nm-online -x parameter
to exit quickly. The network connection will be checked just after
anyway when attempting to connect to the conversion server using ssh.
Kiwi is the tool used by openSUSE / SLES to generate many sort of
disk images. Add a virt-p2v-make-kiwi tool and his documentation to
geneate the p2v appliance kiwi configuration.
Use the common denominator for SLES and openSUSE in the dependencies.
For example most of NetworkManager pieces and metacity aren't provided
on SLES.
ifconfig is in the net-tools-deprecated package in openSUSE Factory,
which means after openSUSE Leap 42.2. Older versions have it in the
net-tools package. Adding this complexity only to add ifconfig because
sysadmins prefer it isn't too good: iproute2 is doing the job.
The password prompt is actually generated by the openssh client which
is under our control, so we can regexp match on the specific prompt
printed by openssh. This also avoids mistaking any server-side
issue.net message for a password prompt (eg. if the server prints a
message like "all users must change their passwords today!") which
would have prevented virt-p2v from logging in.
This can fail because the username is wrong. Also don't unnecessarily
reveal irrelevant implementation details in the error message, just
say the login failed.
Thanks: Juquin Zhou
Previously cancelling the conversion only set a flag, which was
checked when the run dialog displayed new output from virt-v2v. When
virt-v2v was showing hundreds of debugging messages, this wasn't a
problem, but now that we are hiding those messages, cancelling the
conversion might mean a wait of seconds or minutes.
By polling (albeit infrequently) we can make the cancel button more
responsive.
For some failures, 30 lines was not sufficient and only part of the
error was shown. Increase the number of lines shown to 50.
This also colourizes the failure message, and prominently displays the
location of the full log. The following message is now shown:
*** virt-v2v command failed ***
The full log is available on the conversion server in:
/tmp/virt-p2v-20160620-sga9rhk7/virt-v2v-conversion-log.txt
Only the last 50 lines are shown below.
[followed by up to 50 lines of log]
Updates and fixes commit 7447fe2478.
PLD Linux got /etc/os-release only in the recent 3.0 release; since
older versions have only /etc/pld-release, check for it to identify
the guest and get its version.
Previously we displayed the complete output of virt-v2v in the run
dialog. This output included all the debugging messages, and was very
long and confusing for users (especially we had false bug reports
about "errors" appearing in the debug output).
Only display stdout in the run dialog. However make sure everything
(stdout and stderr) is still logged to the conversion log.
Instead of constructing and directly executing a long virt-v2v command
line, build a wrapper script with the same command line and send it to
the remote server (stored in /<remote_dir>/virt-v2v-wrapper.sh).
This will make it a bit easier to construct more complex virt-v2v
wrappers.
Note this commit on its own is a simple refactoring and does not
change any functionality.
This option (alternately spelled: --color, --colour, --colors, or
--colours) enables ANSI colour sequences output even if that would be
disabled becaues the output is not a TTY.
The debug() function is already sending these to stderr, but in a few
places we were using printf. Change those to eprintf, except for one
informational message which should have been using info().
The virt-p2v slow test tested the local virt-p2v. However it ran
against the installed virt-v2v (and in fact would have failed if
virt-v2v was not installed).
This commit sets the environment up to run the locally built virt-v2v.
This is unfortunately quite a lot more complex than it should be.
There is no simple "set this environment variable" option in
sshd_config.
It makes no sense. By setting one field to non-sensitive when the
other field is populated, we can avoid this happening, and also make
tabbing between the fields simpler.
As a consequence of making this change, I also got rid of the now
unnecessary explanatory text telling you to leave one field blank.
In particular if you hit the 'Test Connection' button when the server
name field was empty, you would still see a stopped spinner.
This updates commit 036a11f379.
By changing the logging mode of the p2v.service unit, we can make sure
that the progress of virt-p2v is visible in non-GUI mode at the
console.
Since the output is now always shown on the console, there's no need
to print the journalctl command for following that output when the
user logs in.
This simulates doing a non-GUI conversion with virt-p2v being
downloaded over PXE. Conveniently we already had a test that was
doing this, and this rule just runs the same test.
Instead of throwing away the ssh error and printing the generic
message "unexpected end of file waiting for password prompt", we
capture the ssh error and print it. The user will see the ssh
diagnostic, eg. "No route to host".
For some reason I don't quite understand, in
commit fd82bb12fd if it got the password
prompt unexpectedly a second time then it would send the password
again, continually until ssh closed the connection because of too many
retries.
This obviously makes no sense. If we get the password prompt back,
then we must have sent an incorrect password so abort the connection
and report the error at once.
OCaml 4.02 introduced the 'bytes' type, a mutable string intended to
replace the existing 'string' type for those cases where the byte
array can be mutated. In future the 'string' type will become
immutable. This is not the default now, but it can be forced using
the '-safe-string' compile option.
This commit changes the code so that it could be compiled using
'-safe-string' (but does not actually make that change).
If we detect OCaml < 4.02, we create a dummy 'Bytes' compatibility
module ((nearly) an alias for the 'String' module). The only
significant difference from upstream OCaml is that you must write the
'bytes' type as 'Bytes.t' in interfaces, apart from that everything
else should work.
Removes this warning:
File "common_utils.ml", line 826, characters 24-28:
Warning 40: F_OK was selected from type Unix.access_permission.
It is not visible in the current scope, and will not
be selected if the type becomes unknown.
In commit ae6f726ecc we started to use
the virt-customize code to replace various virt-sysprep operations.
This had the effect of adding many more possible operations to
virt-sysprep, but some of them (specifically --install) did not work
unless the appliance network is enabled. It was not enabled in
virt-sysprep, so these operations never worked.
This change does NOT enable the network by default. However it adds a
--network flag which may be used in conjunction with --install etc to
make those commands work.
In addition we now emit a warning for certain customize operations if
they fail and if the network is not enabled. It will print:
[ 4.5] Installing packages: tcpdump
Error: Cannot retrieve repository metadata (repomd.xml) for repository: base. Please verify its path and try again
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os&infra=stock error was
virt-sysprep: warning: the command may have failed because the network is
disabled. Try either removing '--no-network' or adding '--network' on the
command line.
virt-sysprep: error: yum -y install 'tcpdump': command exited with an error
(If the network is already enabled, or if the command is successful,
then the warning is not printed.)
Thanks: Xianghua Chen
You will now see an error such as:
$ virt-customize -a centos-6.img --truncate-recursive /home/foo
[ 0.0] Examining the guest ...
[ 16.5] Setting a random seed
[ 16.5] Recursively truncating: /home/foo
virt-customize: error: libguestfs error: find0: /home/foo: No such file or
directory
Thanks: Xianghua Chen
Commit 82df768514 breaks the regression
test for RHBZ#1232192 because g#part_list can return an error when
called on a completely blank disk.
By rewriting and simplifying the code using higher-order functions we
can avoid the need for the double-nested imperative loop and calling
g#part_list at all.
Fixes commit 82df768514.
The 'semodule' command, called by the guest tools uninstaller, will
run out of memory with the default of 500MB
(https://bugzilla.redhat.com/426775).
Allocate a large amount of RAM, that will be at least 800MB, but scale
correspondingly on other architectures like ppc64 that have larger
memory requirements.
Thanks: Pavel Butsykin
This is a mostly mechanical renaming of the internal pointer types
used in the libxml2 bindings. Also I have added a longer comment to
make the system clearer.
Guest tools for Windows in Parallels / Virtuozzo Server 6 contain a
paravirtualized video driver, which wants to take full control of the video
mode. To avoid excessive video mode switches in UEFI VMs it executes
"bcdedit /set {current} graphicsmodedisabled true".
When such a VM is imported into a QEMU/KVM-based hypervisor, the
standard video drivers in Windows do not set the video mode as they
expect it to have already been configured by the bootloader. As a result,
the VM runs with black screen.
So patch the BCD store (which is a Windows registry hive with a special
structure, located on the EFI system partition) to clear this setting.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
Store the list of EFI system partitions on the inspect object in order to be
able to tune their contents later in the process.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
Guest tools for Linux in Parallels / Virtuozzo Server 6 come with a script
that can be used to uninstall them.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
'tc_410_close_event.rb' was not being run. You could prove this by
simply inserting "exit 1" into that test.
The reason is unclear, but by renaming every class and method in the
tests to be unique, this ensures the tests are run.
Parallels proprietary hypervisor uses RDPMC as the hypercall
instruction. As this instruction is supported since early P6 family,
the drivers didn't even bother to check for the presence of the
corresponding feature in CPUID.
In QEMU/KVM, however, this instruction triggers #GP unless the VM is run
with PMU (performance monitoring unit) enabled, which is often not the
case (due to its impact on perfromance and migratability).
So, to prevent crashes upon conversion, stop respective drivers from
loading by disabling the corresponding services. Note that the services
are being disabled ("Start" value set to 4) rather than their node
removed, to avoid confusing the uninstaller which is scheduled to run
from a firstboot script.
In addition, prl_strg (storage filter driver) is unlinked from the
storage subsystem following the DelReg directive from its .inf file,
otherwise Windows crashes with BSOD 0x7b due to missing dependency of
the storage subsystem.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Some virtio-win drivers contain more files than just *.{cat,inf,sys}.
They are filtered out currently, which prevents the drivers from being
installed by PnP.
Stop filtering driver files by extension, and copy all of them instead.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Use this command as a starting point:
git shortlog -s -- p2v v2v | awk -F'\t' '{print $2}' | sort -f
However note the above will include people who made only incidental
code changes, and also misses out QA team members, so the actual list
still has to be curated by hand.
It's better to have this overview of how the code is arranged before
any details about modifying the code, since most people will be
reading code before writing.
Recent updates of Virtuozzo Server 6 (fka Parallels Cloud Server 6) have
"Virtuozzo tools" instead of "Parallels tools" as their display name in
the corresponding Uninstall registry key.
Recognize those, too, and schedule their uninstallation on first boot.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
On Linux 4.7.0 the old code was giving -EINVAL errors. This rewrites
the code based on online sources, and it works on at least RHEL 7,
Linux 4.4 and 4.7. Also checked with valgrind.
Because EXTRA_DIST included $(TESTS) this ended up distributing the
binary file v2v_unit_tests.
Unfortunately to fix this we need to list all the test scripts and
data files explicitly a second time, but there we are.
If this flag is not set (ie. we are doing local testing) then we
disable some dangerous features. Currently this is just the "Reboot"
button, but may include other features in future.
Currently virt-p2v requires Gtk 2. This commit changes virt-p2v so it
can be built with either Gtk 2 or 3.
By careful use of macros, this code should compile on both recent
Gtk 2 and Gtk 3.
With no other options, ./configure will now prefer Gtk 3 if it finds
it, or fall back to Gtk 2. But you can control this by setting
'./configure --with-gtk=2|3|check|no' where the options mean:
* --with-gtk=2 - Only test for Gtk 2
* --with-gtk=3 - Only test for Gtk 3
* --with-gtk=check - Check for Gtk 3 then Gtk 2 (default)
* --with-gtk=no - Don't build virt-p2v
In the ./configure output you will see something like this:
checking for --with-gtk option... 2
checking for GTK... yes
checking if we can build virt-p2v... yes, with Gtk 2
Gtk 3 deprecates the GDK thread synchronization locks (functions such
as gdk_threads_enter/gdk_threads_leave), and requires that you issue
all Gtk calls in the main thread. To do this you have to restructure
any code that calls Gtk from other threads so it is wrapped in an idle
task, and so is run from the main thread.
This commit transforms the code in this way.
I found through experimentation that libxcb crashes noisily if you
issue any X11 calls (and hence any significant Gtk calls) when the GDK
thread sychronization initializer has been removed, so if there are
any remaining calls that are missed by this patch we should find out
about them quickly.
Use, for example:
./run virt-p2v --test-disk=$(pwd)/test-data/phony-guests/windows.img
to test conversions using a file of test data instead of the real host
/dev/sda.
But allow the warning to be turned off using --no-warn-if-partition.
Ming Xie tried to create a bootable USB key using
'virt-p2v-make-disk -o /dev/sdX1'. That works, but doesn't create a
bootable key because it puts everything into the first partition.
Emit a warning if someone tries to do this to try to prevent user
error:
virt-builder: warning: output device (/dev/sdb1) is a partition. If you
are writing to a USB key or external drive then you probably need to write
to the whole device, not to a partition. If this warning is wrong then you
can disable it with --no-warn-if-partition
Returns true if the device is a host partition.
The only tedious bit of this patch is that now Common_utils depends on
Dev_t so we have to add extra objects every time something links with
Common_utils.
For virt-p2v-make-kickstart this doesn't do very much. But for
virt-p2v-make-disk it enables verbose mode in virt-builder which is
extremely useful for debugging problems.
Thanks: Ming Xie.
Run sudo with -n (non-interactive), so it will fail right away when not
configured to not require a password. This will avoid the connection to
time out.
You can run `xterm -display :0' at the virtual console in order to get
a shell in the graphical environment.
This is useful for debugging and doesn't have much overhead unlike
other terminals.
Fetch the full version from the remote virt-v2v, including the
--with-extra field, since that tells us which distro virt-v2v was
compiled for which is useful information.
Display this information in all debug output and dialogs.
Multiple cell alignment issues in the treeviews on the right of the
conversion dialog. This particularly showed up when you had multiple
heterogeneous disks or network cards because the description cells did
not align nicely.
Thanks: Ming Xie.
Use KVM, fallback to TCG.
Add more network devices so we can test the layout of that
dialog box.
Move the documentation into guestfs-hacking(1).
Updates commit f8cfdf274f.
The kickstart version of virt-p2v has SELinux disabled, so this is at
least consistent. Mainly it avoids the very long relabel when the
disk boots first time.
Since Void Linux provides only an /etc/os-release with no VERSION_ID
field, then special-case it to avoid that the os-release parsing ignore
it.
This provides basic distro identification, and icon.
When parsing lsb-release, allow a version number in the format "X" other
than "X.Y" for the DISTRIB_RELEASE key.
This fixes version parsing in Mageia 4 guests.
Add a simple helper to run a command from a sequence of arguments,
without using a shell: this should help reducing the amount of quoting
ineeded, since arguments are passed straight as such.
Make use of it in the places currently using shell_command, and which
don't assume they can run anything (so no shell redirections, `env`,
etc).
Move the make_dib_args helper function to Common_utils as
stringify_args, so it can be used also within Common_utils itself.
This is mostly code motion.
Some ova files like those produced by SUSE Studio have their ovf, mf
and other files inside a folder rather than at the root of the
tarball. Consider the paths relative to the ovf and mf files to cover
this case too.
To add this support, two things are needed:
* make the existing code searches for either the viostor
or the SUSE VMDP (Virtual Machine Driver Pack) files.
* add a firstboot script setting up VMDP.
Note that 2 firstboot scripts are intentionally added for the VMDP
setup. This is due to windows potentially rebooting after loading the
virtio block driver. It may happen that this reboot interrupts the VMDP
setup in the firstboot script, we thus make sure the setup is run a
second time in case it needs to finish the previous run.
Sync the windows firstboot script with its linux brother. Also change
the main redirection to append to the log rather than overwriting it.
With this change, the firstboot script will resist reboots in the
executed scripts.
Add a common debug function for printing debugging messages. It only
emits the debug message when the verbose (-v) flag is used on the
command line.
It sends the output to stderr, which is flushed immediately after the
message is printed (to help with debugging unexpected crashes). There
are good arguments for sending the debug to either stdout or stderr,
and almost all existing debug messages replaced by this change went to
stdout. However using stderr is consistent with libguestfs's own
debug messages which also go to stderr.
I only made simple changes to code of the form 'if verbose () then
printf ...'. There are more places which could be changed in future.
In a few places I removed gettext calls since we probably should not
translate debug messages.
It is still only emitted if we are debugging in order not to cause
unnecessary alarm. Note this code needs a better long term fix, this
is still a hack.
Now that UEFI is fully open source the UEFI firmware can be included
in Fedora. However the location is slightly different from the
location that was used by kraxel's out of distro firmware. This
commit searches for UEFI files in the new location, falling back to
the old location (until the end of this year when we will drop it
entirely).
Thanks: Cole Robinson, Gerd Hoffmann (kraxel) & Microsoft for
relicensing the FAT code.
Since Linux 4.7, the process umask is available in /proc/<pid>/status.
See:
3e42979e65
Use this value if available, else fall back to the existing codepath
for Linux <= 4.6 and other Unix.
Newer versions of e2fsprogs refuse modifying journal devices, causing
the test #1 of get_e2uuid to fail with:
libguestfs: error: set_e2uuid: Cannot modify a journal device.
Since the test case just needs to verify get_e2uuid runs without
failing, run it without expecting a certain UUID set earlier.
Followup of commit da4812ab6b.
Only in very recent OCaml versions (or in Fedora) the Arg module
supports the --foo=bar syntax for options with arguments. Do not
require it yet.
Fixes commit 64bb9edd52.
To make sure we can also find the initrd on openSUSE and SLES, we need two improvements:
* the initrd filename may not end with '.img'
* don't use the version + release from the RPM data, rather from the
/lib/modules/<version>/ path as we need to find it out anyway.
When possible, make the disk image format explicit when invoking tools
or using add-drive. This avoids warnings from qemu about the unspecified
format for the image, and also makes qemu slightly faster (skipping the
disk image probing).
Tests checking the image probing are not touched.
This changes also:
- old-style invocations of tools (`$tool $filename`) into new style
(`$tool -a $filename`)
- add-drive-ro/add-drive-with-if guestfish commands into add/add-drive
with explicit readonly/iface arguments
There should be no change in the tests results.
Turn the snippet reading user information from /etc/passwd in a slightly
more generic function, so there is no need to copy&paste for other
details.
Mostly code motion.
At least on openSUSE and SLES, the /etc/mtab file is already existing.
Skipping the symlink creation in init removes one error message during
the appliance boot.
SUSE Linux Entreprise Server doesn't have dhcpcd and the hivex package
is not in the default repositories. Better use dhcp-client and
libhivex0.
openSUSE needs ntfs-3g and ntfsprogs packages.
To allow saving space in the package shipping the windows virtio
drivers, we can use symlinks between the drivers folders. For example
SUSE VMDP drivers are the same for Win8.1 and Win2012r2, one folder
is a symlink to the other.
To take advantages of this, find must use the -L flag.
qemu feature detection takes about 95ms on my laptop. The overhead is
almost all due to the time taken by the glibc link loader opening the
170+ libraries that qemu is linked to (×2 because we need to run qemu
twice).
Fixing that is seriously hard work.
Therefore memoize the results of guestfs_int_test_qemu.
This is keyed on the size and mtime of the qemu binary, so if the user
changes the qemu binary (eg. setting LIBGUESTFS_HV) we discard the
memoized result and rerun the qemu commands. There is also a
generation number so we can bump the generation in future versions of
libguestfs to invalidate all previously cached data.
The memo is stored in the supermin cache directory (eg. /var/tmp/.guestfs-*)
in the files:
qemu.stat Result of 'stat(2)' of the qemu binary
qemu.help qemu -help output
qemu.devices qemu -devices ? output
Note these files are only stored when using the 'direct' backend. For
the libvirt backend, libvirt itself memoizes this data in its own
place.
This is code motion, but I have cleaned up and formalized the
interface between this module and other parts of the library.
Also this adds documentation to the interface.
The maxrss is described in getrusage(2) as "This is the maximum
resident set size used (in kilobytes).". This lets us find out how
much memory qemu really used, and is useful for determining if memory
minimization techniques such as DAX are having any effect.
When creating the dummy scratch disk for a /dev/null drive, force its
format as raw if it was not specified already. After all, raw was
already the only format allowed for this kind of drives, so it makes
sense to specify the format explicitly and avoid the autodetection in
qemu.
Use the version struct in inspect_fs to hold the version of a
filesystem, adapting the inspection code to that.
Also, move the parts of the version parsing to helper functions of the
version struct, so common bits like parsing "X.Y" or "X" version strings
is done only once.
Introduce a new struct version with few helper functions for it: this
allows to "atomically" represent a version number, without different
variables to be used and checked together.
Add a initialization function from a libvirt-style version number, and
apply it for the qemu and libvirt versions in the direct and libvirt
backends.
Previously I disabled ACPI because it was not necessary and was slow.
However measuring it again now, I can see no significant difference in
performance, and it is necessary if we are going to use DAX.
It's also useful to keep the appliance kernel state as similar to the
ordinary state, and setting acpi=off is a massive change to the way
that the kernel boots.
Therefore, reenable it.
Versions older than 4.3.1 output also the version string after the
output. Ignore such line, since it isn't relevant to what
btrfs_filesystem_show needs.
In /etc/fstab the UUID= or LABEL= field may be quoted. Augeas returns
the field including the quotes, and we passed this directly to
guestfs_findfs_uuid or guestfs_findfs_label. It happens that this
works on upstream findfs, although it doesn't work in RHEL 7.2. The
correct thing to do is to remove the quotes before passing the UUID or
label to these functions.
Thanks: Thom Carlin for reporting the bug, Karel Zak for identifying
the change in behaviour in util-linux.
This function is useful outside virt-v2v so move it into the core
utilities library.
The function has been rewritten from OCaml into C, but it should be
functionally identical.
I suspected data corruption (but didn't prove it). This commit just
adds some magic numbers to the structs so we will see data corruption
quickly if it happens again.
Previously we were running ldconfig to create /etc/ld.so.cache.
This is required, at least on Fedora, if we need to run any binary
that uses a library with a weird path. libiscsi (a dependency of
qemu-img, used by virt-dib) is an example of such a weird library,
since it puts its single library into /usr/lib64/iscsi for no readily
understandable reason, and drops a configuration file into
/etc/ld.so.conf.d/ so that this new directory gets picked up.
By copying the /etc/ld.so.cache from the host we get an already
configured cache which should contain every library on the host, so
there is no need to run ldconfig.
Running ldconfig adds about 100ms to the boot time. I would prefer
that we understood which libraries need ldconfig to be run, and fix
that. We could also consider running ldconfig in parallel, but since
it might be required by just about any binary that the init script
runs it's not clear what benefit that gives.
This reverts commit 66aa98265d.
If the guest uses SELinux, then make sure to run a relabel (or at least
schedule one) after the image build: this way the template is
relabelled, or at least it will do that at the next boot, without the
need for the user to ask for a relabel.
This just covers the case of building a new image with no additional
operations on it though.
Add unit tests for the following internal functions:
- guestfs_int_getumask
- guestfs_int_new_command and other src/command.c functions
- guestfs_int_qemu_escape_param
- guestfs_int_timeval_diff
- guestfs_int_match, match1, match2
- guestfs_int_add_string and other src/stringsbuf.c functions
Because we now need to use a libguestfs handle, we have to link the
unit tests to the library. But because we also need to access the
internal functions (to test them) we need to link the test statically
to the objects making up the library. So this requires a small change
to the linking of this test too.
Use /etc/hosts as alternative of /etc/fstab to detect whether a partition
represents the root of a Linux installation; the latter might not exist
in smaller/special installations like Docker images.
Improve the error messages produced by C-based tools in case of issues
with the command line options:
- explicitly mention to use -a/-d (and -A/-D in virt-diff)
- when extra arguments are found, mention the correct way to pass
options to certain command line switches (like --format)
- in virt-inspector, give a cleaner error message when neither -i nor
any -m is specified
In all the cases, keep the extra notice to use 'TOOL --help' to get more
help with it.
python/guestfs.py:136:37: E712 comparison to True should be 'if cond is True:' or 'if cond:'
python/t/tests_helper.py:42:8: E713 test for membership should be 'not in'
No functional changes, as the new versions follow the suggested Python
coding style to do the same things.
Reindent Python scripts to make sure lines are not longer than 80
columns.
Regarding autogenerated code (guestfs.py and bindtests.py): add an
helper function to make sure comma-separated lists are wrapped at the
wanted length.
This produces only differences in the indentation of long Python lines,
with no behaviour changes.
Add (after comma) or remove (before opening round bracket, and around
'=' in arguments) whitespaces according to the PEP 8 specification.
This is just code reformatting, with no behaviour changes; no content
changed beside whitespaces, so "git diff -w" gives an empty diff.
For Windows, we now print:
$ virt-customize -a ./test-data/phony-guests/windows.img --install MSSQL
[ 0.0] Examining the guest ...
[ 14.2] Setting a random seed
virt-customize: warning: random seed could not be set for this type of
guest
[ 14.2] Installing packages: MSSQL
virt-customize: error: cannot use '--install' because no package manager
has been detected for this guest OS.
If this guest OS is a common one with ordinary package management then this
may have been caused by a failure of libguestfs inspection.
For OSes such as Windows that lack package management, this is not
possible. Try using one of the '--firstboot*' flags instead (described in
the manual).
In cases where parsing the release file failed and so we have an
obviously incorrect major version number, don't try to infer the
package manager from the major version number.
In the bug report, parsing the /etc/redhat-release file of a CentOS
7.1 guest failed, so major version was set to 0, and the package
manager was inferred as "up2date". virt-customize then failed with a
peculiar error:
virt-customize: sorry, don't know how to use --install with the 'up2date' package manager
Instead this sets it to "unknown" which will cause virt-customize to
fail with:
virt-customize: --install is not supported for this guest operating system
which is (only very slightly) better.
Problem reported by novegin on IRC.
All developer documentation should go in guestfs-hacking(1) and the
"internal documentation" (ie. documentation about internal functions
and interfaces) belongs here, not in the coincidentally synonymous
guestfs-internals(1).
Since moving boot-benchmark-range.pl out from under a tests/
subdirectory, it is now being added to po/POTFILES. This causes the
following error:
/usr/bin/xgettext: Non-ASCII string at utils/boot-benchmark/boot-benchmark-range.pl:205.
Please specify the source encoding through --from-code.
Add --from-code parameter as instructed.
The Arg module of OCaml does not support hiding options from the --help
output: hence, mark --short-options and --long-options as internal
options, since we need them only for the bash completion scripts.
Create a new top level directory called 'utils' and move the
following programs there:
tests/qemu/boot-analysis -> utils/boot-analysis/
tests/qemu/boot-benchmark -> utils/boot-benchmark/
tests/qemu/qemu-boot -> utils/qemu-boot/
tests/qemu/qemu-speed-test -> utils/qemu-speed-test/
Also we only build the boot-analysis program on x86-64 and aarch64,
since it requires custom porting to each architecture.
Recently Debian switched to btrfs-progs, so almost all the distributions
(except openSUSE) have btrfs-progs which is then moved as common
package.
The old btrfs-tools name is left there, to support Debian Jessie and
older, and Ubuntu Xenial and older.
When (in --verbose mode) we dump the pass data, dump the times in
milliseconds (instead of nanoseconds) so they are consistent with
other output. Also dump the time difference from the previous event.
Useful for detailed debugging/analysis of problems.
When we are debugging use the verbose (debug messages enabled) version
of the AAVMF (UEFI) firmware. This is useful for diagnosing what is
causing failures to boot and what inside UEFI is causing slow booting.
Use an array of structs for the UEFI firmware instead of an array of
strings.
This change is hopefully just refactoring and there is no functional
difference.
It is safe to use /dev/urandom to seed the guest. In libvirt 1.3.4
this silly restriction has been lifted.
This restores commit b2c845333f.
See also commit 9423c16607.
Thanks: Cole Robinson for doing the libvirt implementation.
The old mapping code was directly copied from old virt-v2v, translated
from Perl to OCaml.
The new mapping code does a few things more accurately:
- Use the i_product_variant field (Windows InstallationType) if available.
- Simplify rules, so there is only one special case needed for RHEL 3/4.
- Don't assume Fedora == Desktop.
- Don't assume all later Windows variants are server.
- Works for Windows > 7.
Since commit b23e149774, we have been
strict about POD errors. An error in the Japanese translation caused
"L</guestfish" (ie. an unclosed L<>) to appear in the POD.
This commit fixes the problem, but when building you may need to do:
rm po-docs/ja/guestfish.pod
rm po-docs/podfiles; make -C po-docs update-po
in order to update the broken po-docs/ja/guestfish.pod file.
Unfortunately I have been unable to upload this change back to the
Zanata servers because of a variety of client problems.
PHP (5?) renamed the PHP_EXECUTABLE variable to TEST_PHP_EXECUTABLE.
As a result of that if you enabled debugging, the tests broke because
we no longer used our custom PHP wrapper to filter out debugging
environment variables before running the tests, so debug output was
mixed with the expected output.
This commit also updates an old comment telling you how to debug PHP
tests.
Prevents us from accidentally "losing" a disk during conversion. I
believe from code inspection that this assertion is always true for
the current code, so this should have no effect.
It's a large feature / function with its own set of nested functions,
so move the inspect_source function into its own module. It also lets
us specify and document the interface explicitly.
Also: Define a root_choice type in Types module. I turned it into a
non-polymorphic variant type for extra type safety.
This is just code motion.
- Add headings, further documentation to the Types module.
- Move two type definitions in Types module. Not a functional change.
- Add more comments and clean up comments throughout v2v/v2v.ml.
- Refactor guestcaps / conversion in v2v/v2v.ml. Not a functional
change, just hides local variables from the rest of the code.
- In --in-place mode, change inspection message to 'Inspecting the
source VM'. This matches the later message 'Closing the source VM'.
Enable a few POD options:
- Don't generate an errata section in the output.
- Send errors and warnings to stderr.
- Die if any errors or warnings are seen while generating the outputs.
The drive used for the appliance is a raw (sparse) disk: specify that
explicitly in its -drive qemu command line options, so qemu can skip the
autodetection of its format and save a tiny bit of time.
Allow kernel options such as p2v.o=libvirt to override internal
defaults, even for GUI configuration.
The main change is to split up the kernel conversion into two steps:
reading the kernel command line configuration, and performing the
conversion. The kernel command line can then be read by the main
program and used to initialize the config structure for either kernel
conversion or GUI conversion.
A small adjustment is required in the test because p2v.pre no longer
runs before kernel command line parsing. (The aim is to have
p2v.pre/post/fail still only run when doing a kernel conversion, not
in the GUI case.)
Since we started to use the parallel tests framework in automake,
'make check-slow' has been broken. This is because parallel tests
doesn't allow you to run 'make check TESTS=...' with a set of test
scripts which do not also appear in the static list of tests in the
Makefile.am. We would like to list and run only "fast" tests in the
Makefile.am, and have other scripts for slow tests.
The solution is to add the slow tests to Makefile.am, but condition
those tests on an environment variable SLOW=1 being set.
This commit fixes all the existing slow tests in this way, and updates
the documentation (guestfs-hacking(1)) to document how slow tests
should be written in future.
Valgrind has a weird hack where it invokes a glibc function called
__libc_freeres on exit. See:
http://valgrind.org/docs/manual/faq.html#faq.exit_errors
This is intended to free up memory that glibc won't normally free
(since glibc doesn't free everything on exit for efficiency reasons).
More importantly, valgrind runs __libc_freeres even if the process
calls _exit, resulting in this bug:
https://bugs.kde.org/show_bug.cgi?id=361810
(either a bug in valgrind, or in glibc, or in both, depending on your
point of view).
In any case we don't want this behaviour, so disable it.
Also we have to add suppressions for new "leaks" in glibc found by
valgrind because __libc_freeres no longer runs. In fact there is only
one such suppression needed, for TLS allocation in multithreaded
tests.
When we are valgrinding we don't really care about the child
processes, which might be qemu, libvirtd, etc. So disable tracing
into children (at least, as far as is possible with valgrind, which is
not entirely disabling it, but suppressing it).
This is convenient because it allows you to do:
rm -f docs/C_SOURCE_FILES
make docs/C_SOURCE_FILES
if you ever wanted to recreate that file (without doing 'make dist').
Note that the rules depend on configure.ac so that the files get
updated on each new version (otherwise the files would never get
updated).
On Linux this will load the whole file into the page cache. However
the output file is empty and zero sized just after it is opened, so
this has no effect. Note that the advice is not persistent, so this
really does nothing.
I considered adding the call back after the file has been written,
just before the close, but:
- If we do a virt-resize next then we will open and read the file mostly
sequentially, so readahead will deal with any missing pages.
- If we do a virt-customize next then we will only access a small
part of the disk image, so loading it all into the page cache adds
extra work.
- In any case, since we have just written the file it's likely to
still be in the page cache.
This commit adds guestfs_int_fadvise_normal, but it's not enabled
since nothing calls it.
It also documents what Linux actually does, which is a bit different
from what I thought these settings did. Note this is for Linux 4.6.0rc3
and may change in future.
This updates commit 83e92b4a97.
Setting POSIX_FADV_RANDOM makes no measurable difference, but at least
it's the right thing to do.
POSIX_FADV_WILLNEED makes no measurable difference either.
Moving the calls to posix_fadvise to just after the open() makes no
measurable difference, but does make the code a bit clearer.
Changing POSIX_FADV_NOREUSE on the input file descriptor to
POSIX_FADV_DONTNEED slows things down by about 10%.
Add wrappers around posix_fadvise and use them in places we were
calling posix_fadvise directly before.
Also in virt-builder we were doing this (and ignoring the result):
posix_fadvise (fd, 0, 0, POSIX_FADV_RANDOM|POSIX_FADV_DONTNEED);
However the POSIX_FADV_* flags are _not_ bitmasks! In fact
POSIX_FADV_RANDOM|POSIX_FADV_DONTNEED == POSIX_FADV_NOREUSE so we were
giving a completely different hint from what we thought we were
giving.
The current implementation of getumask involves writing a file with
mode 0777 and then testing what mode was created by the kernel. This
doesn't work properly if the user set a per-mount umask (or fmask/
dmask).
This alternative method was suggested by Josh Stone. By forking, we
can use the thread-unsafe method (calling umask) and pass the result
back over a pipe.
This change also fixes another problem: mode_t is unsigned, so cannot
be used to return an error indication (ie. -1). Return a plain int
instead.
Thanks: Josh Stone, Jiri Jaburek, Eric Blake.
Exclude more files from the documented sources in C_SOURCE_FILES:
- sources generated by XDR definitions
- sources generated by gperf
- errnostring.c (generated by the generator, and copied around from src
to daemon during build)
Also update C_SOURCE_FILES as well, so libguestfs builds again from
scratch.
Virtio-SCSI offers a number of advantages over virtio-blk, in
particular, it supports SCSI UNMAP (aka trim) which is crucial for
keeping the virtual drive from wasting host disk space.
This patch adds support for virtio-scsi as the virtual disk connection
type both on input and on output of v2v.
Virtio-blk remains the default, so for now virtio-scsi-based guests can
only be produced in --in-place mode.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
There's no particular reason that we have to update these files in
every 'make'. It's slow, and it doesn't really matter if a single
file is accidentally omitted from translations on a developer's build
machine.
On the other hand generating it from 'make dist' has advantages:
- We only do it once per release.
- The maintainer can check the changes carefully.
- If a developer adds some random files in their local build tree,
these won't be picked up accidentally.
- Changes to POTFILES etc won't leak into general commits.
Add the real list of C source files to the Makefile, instead of trying
to calculate it. This (will, in the next commit) fix a problem with
the build on Koji where we created a subdirectory (for python3/) but
the find command picked up files from that directory for the
documentation (and then failed).
This fixes commit 04229c68d6.
As in libvirt, allow internal documentation comments in C files.
These are marked using '/**' comments. Unlike libvirt which has an
ill-defined and inconsistent markup, the markup we use is Perl POD.
These comments are added to guestfs-internals(1), but are most likely
to be read in-place.
This commit changes some existing comments in src/launch.c to
demonstrate how this can be used.
Refactor registry patching to share common parts between Win <= 7 and
Win >= 8. Also branch between the two based on the presence of the
DriverDatabase subtree rather than on the OS version, which allows not
to pass the inspect handle into the functions.
OTOH make the patching functions accept the driver name and device PCI
id as arguments; this will allow to easily expand the code to support
other drivers (e.g. vioscsi) later on.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
It turned out (by trial and error) that for Windows >= 8 / 2012, even
though the PnP system uses names related to the original driver's .inf
file for certain entries under DriverDatabase registry subtree, this
doesn't have to hold for the storage driver to make it boot.
So use our own artificial names there and simplify the surrounding code.
This will also make sure the added entries don't confilct with those
generated by PnP proper once the driver is installed, and that the
entries are easy to look up in the eventual registry.
The name chosen to refer to the driver -- "guestor" -- is intentionally
different from the original "viostor" both to avoid collisions and to
naturally expand to other relevant drivers (e.g. "vioscsi") later on.
Tested on
Win 8 x32
Win 8 x64
Win 8.1 x32
Win 8.1 x64
Win 2012R2 x64
Win 10 x64
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
The patch to the Windows registry allowing it to boot off a virtio-blk
drive was initially conceived by comparing the state with virtio-blk
driver properly installed, to that without.
However, we don't want to replicate the Windows PnP system; rather we
need to apply just enough edits to make the system boot, and then let
the Windows PnP manager figure out the rest.
It turned out (by trial and error) that for Windows >= 8 / 2012 the
"just enough" set includes only the service entry for the driver, and a
few entries under DriverDatabase.
Tested on
Win 8 x32
Win 8 x64
Win 8.1 x32
Win 8.1 x64
Win 2012R2 x64
Win 10 x64
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
The patch to the Windows registry allowing it to boot off a virtio-blk
drive was initially conceived by comparing the state with virtio-blk
driver properly installed, to that without.
However, we don't want to replicate the Windows PnP system; rather we
need to apply just enough edits to make the system boot, and then let
the Windows PnP manager figure out the rest.
It turned out that for Windows <= 7 / 2008R2 the "just enough" set
includes only the service entry for the driver, and a
CriticalDeviceDatabase entry matching the PCI id of the device and
pointing at that service entry.
While at this, also drop CriticalDeviceDatabase entries for older and
no longer relevant PCI ids of the virtio-blk device.
Tested on
Win XP SP3 x32
Win XP SP2 x64
Win 2003R2 SP2 x32
Win 2003R2 SP2 x63
Win Vista SP2 x32
Win Vista SP2 x64
Win 2008 SP2 x32
Win 2008R2 SP1 x64
Win 7 SP1 x32
Win 7 SP1 x64
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Reviewed-by: "Richard W.M. Jones" <rjones@redhat.com>
Adapt to the API changes in the newer versions of the Zend framework, in
particular regarding:
- strings handling
- resources handling (used for the guestfs_h pointer)
- iterating in hash maps
- data types for function arguments
Introduce helper macros to reduce greatly the amount of #if's all around
the generated C code.
In the generated description of the guestfish commands, wrap and indent
the help text, so it is a multiline string instead of a very long single
one.
This has no behaviour changes, only makes cmds.c more readable (and
easier to diff when the description of actions change).
Like with the previous commit, this replaces instances of:
if (something_bad) {
fprintf (stderr, "%s: error message\n", guestfs_int_program_name);
exit (EXIT_FAILURE);
}
with:
if (something_bad)
error (EXIT_FAILURE, 0, "error message");
(except in a few cases were errno was incorrectly being ignored, in
which case I have fixed that).
It's slightly more complex than the previous commit because we must be
careful to:
- Remove the program name (since error(3) prints it).
- Remove any trailing \n character from the message.
Candidates for replacement were found using:
pcregrep --buffer-size 10M -M '\bfprintf\b.*\n.*\bexit\b' `git ls-files`
Wherever we had code which did:
if (something_bad) {
perror (...);
exit (EXIT_FAILURE);
}
replace this with use of the error(3) function:
if (something_bad)
error (EXIT_FAILURE, errno, ...);
The error(3) function is supplied by glibc, or by gnulib on platforms
which don't have it, and is much more flexible than perror(3). Since
we already use error(3), there seems to be no downside to mandating it
everywhere.
Note there is one nasty catch with error(3): error (EXIT_SUCCESS, ...)
does *not* exit! This is also the reason why error(3) cannot be
marked as __attribute__((noreturn)).
Because the examples can't use gnulib, I did not change them.
To search for multiline patterns of the above form, pcregrep -M turns
out to be very useful:
pcregrep --buffer-size 10M -M '\bperror\b.*\n.*\bexit\b' `git ls-files`
It was confusing to have a mix of seconds and milliseconds. For all
upstream testing (eg. with SeaBIOS) we are discussing everything in
milliseconds, so use those exclusively.
Add a new test program called 'boot-benchmark'. This is similar to
'boot-analysis' but it simply boots and shuts down the appliance
several times in a row and measures how long it takes, calculating
mean and standard deviation.
When a struct containing FChar fields is defined
in generator/structs.ml, the Python code generator will generate
code referencing a dirent struct instead of the correct one
thus breaking at compile time.
This bug was left unnoticed because the dirent struct is the only
one using FChar fields. Yet it prevents any other struct
from using such fields.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Look for /lib/os-release in the /usr partition and try to use it, if
present, before using lsb-release later on. This should not change the
final result of the inspection, while using the os-release detection
method also for CoreOS.
Update the phony CoreOS image to use os-release instead, uploading the
version found in the current stable version.
The "icat" name comes from the employed command line tool which might be
replaced at any time with a different implementation.
The command name is a bit confusing because it's similar to "cat" but
act as "download".
download_inode is more clear and descriptive.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Use the proper length modifier for asprintf, instead of a padding
modifier. This fixes the parsing of versions in VERSION_ID which are a
single number (i.e. "X" and not "X.Y", etc).
Bug introduced with commit 32d19e3289, but
uncovered recently with the switch away from VLAs, i.e.
commit 07c496c53c.
In order to support the new features I am renaming the file with a better name.
The file sleuthkit.c will contain the code depending on the sleuthkit package.
The original tsk.c file will contain the logic built using libtsk
which is the sleuthkit core library.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Some of the systemd-tmpfiles snippets need the machine ID of the running
system; the current lack of this file produces warning messages during
the appliance boot like:
[/usr/lib/tmpfiles.d/systemd.conf:26] Failed to replace specifiers: /run/log/journal/%m
[/usr/lib/tmpfiles.d/systemd.conf:28] Failed to replace specifiers: /run/log/journal/%m
[/usr/lib/tmpfiles.d/systemd.conf:29] Failed to replace specifiers: /run/log/journal/%m
Thus create a new randomly-generated /etc/machine-id on boot.
These options allow you to control the appliance memory size, number
of vCPUs, and extra kernel options respectively.
Note that using --smp is not usually a good idea. Not only does it
slow down the appliance, but it tends to break the boot analysis
program because it makes runs (more) non-deterministic.
the description of the --selinux-relabel option suggests that it
perform an immediate relabel, when in fact it may (and probably will)
instead simply touch /.autorelabel on the image, which schedules a
relabel operation for the next time the image boots. This can be
surprising because it results both in an extended initial boot time
*and* results in an automatic reboot (on some distributions).
It seems that the documentation for --in-place mode of v2v wasn't clear
enough, so try to explain it better.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
In in-place mode, the decisions on which interfaces to use are made and
the source VM configuration is created by the outside entity. So in
that case v2v needs to look it up in the source configuraion, and try to
follow.
For that, the source VM configuration is used to populate requested caps
object which is then passed to the convert routine.
It's assumed that the configuration has exactly one type of storage
devices and no more than one type of network and video adapters;
anything else is rejected.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
Give the caller certain control over what kind of interface to use for
virtual disks, network and video cards upon conversion.
For that, make convert functions accept additional rcaps parameter
containing an object with optional capabilities similar to the ones
produced on output, with None indicating that the decision is left to
the convert function itself.
To facilicate review, this patch unconditionally passes rcaps with no
preferences; populating it with more sensible values is done in a
followup patch.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
Introduce a type to contain the guestcaps that are to be put in effect
in the converted VM: options of the block type, net type, and video.
It'll be populated by the caller and passed into convert function to
affect its choice of devices and drivers.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
Those will be useful when making decisions about what configuration to
set on output.
The data is also included in --print-source so the tests are adjusted
accordingly.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: "Richard W.M. Jones" <rjones@redhat.com>
This tedious workaround avoids a 0.26 second pause when using sgabios
(the Serial Graphics Adapter). It's basically a workaround for buggy
code in sgabios, but much easier than fixing the assembler.
Since commit bb5d30ab2a, we don't
require any external programs like grep to parse the command line. We
only use bash intrinsics.
Therefore we can do it early (but after /proc is mounted).
This allows verbose mode to enable set -x early on, so we can trace
most things that the init script does.
This function was kind of like debug(), except that it didn't check
the g->verbose flag and it only worked in the library (it would crash
if used after fork).
It also wasn't very useful. The sort of boot analysis done by
[forthcoming program] tests/qemu/boot-analysis.c means that
timestamping (some) messages is even less interesting than before.
Remove it and replace calls with debug() instead.
Dr. David Gilbert pointed out to me that the first line of the
qemu -help output includes the qemu version, so we don't need
to run qemu -version at all.
This saves about 0.04s on the launch time when using the direct
backend.
Read the content of /proc/cmdline using bash features, and use its
[[ ... ]] expression to find texts in a variable.
This shaves off 5 grep invocations.
Instead of hardcoding "make" in run-php-tests.sh, pass the actual name
of make from the Makefile; the default is still "make", mostly to use
the script without having to set $MAKE.
sha1sum(1) does not exist everywhere, so wrap it in an own function so
the right implementation can be chosen on each OS. Also, wrapping it
avoid using awk everytime.
Move it to the test-data directory, giving it a more generic name so it
can be used also for more than just md5; adjust qemu tests accordingly.
This is just code motion, no behaviour change.
The quiet option suppresses kernel messages. On my laptop it improves
appliance boot times by about 40% (3.5s -> 2.5s).
The emulated UART is slow and has a fixed, small FIFO (16 bytes). But
it has the advantage of being a simple ISA device which is available
very early in boot, thus enabling us to diagnose early boot problems.
So the aim is to reduce our usage of this UART on fast paths.
Of course when we are in verbose mode, we should not add this flag
because we want to see all the messages.
This change is not entirely invisible:
(1) Progress messages use the "Linux version ..." string from kernel
output in order to determine part of where we are in the boot process.
This string will no longer be detected. We should probably use a BIOS
message or maybe drop this altogether. I have added a comment to the
code.
(2) It is possible for programs to be listening for
GUESTFS_EVENT_APPLIANCE events, and they will see fewer messages now
(although what kernel messages programs see is never defined).
In case /usr is a symlink to /usr/bin, then we cannot rely on /usr/bin
to exist, since /usr might be in a different partition. Thus, in case
/bin is a symlink, check it points to "usr/bin".
Provide new <define> for the following elements: "distro",
"package_format", "package_management", and "format", adding all the
values so far possible for each.
Update the system at the end of the installation, so the generated image
is up-to-date. This also removes the need to manually update dnf on
i686 and x86_64.
The list of filesystems that is printed when there was an error prints
the internal mountable string even for the btrfs subvolumes. Let's
printing a valid -m option value instead.
Add command line options to optionally disable the output of the
installed application, and the guest icon.
This makes the inspection slightly faster, producing a smaller XML to
parse/process in case there is no interest in these details.
GCC has two warnings related to large stack frames. We were already
using the -Wframe-larger-than warning, but this reduces the threshold
from 10000 to 5000 bytes.
However that warning only covers the static part of frames (not
alloca). So this change also enables -Wstack-usage=10000 which covers
both the static and dynamic usage (alloca and variable length arrays).
Multiple changes are made throughout the code to reduce frames to fit
within these new limits.
Note that stack allocation of large strings can be a security issue.
For example, we had code like:
size_t len = strlen (fs->windows_systemroot) + 64;
char software[len];
snprintf (software, len, "%s/system32/config/software",
fs->windows_systemroot);
where fs->windows_systemroot is guest controlled. It's not clear what
the effects might be of allowing the guest to allocate potentially
very large stack frames, but at best it allows the guest to cause
libguestfs to segfault. It turns out we are very lucky that
fs->windows_systemroot cannot be set arbitrarily large (see checks in
is_systemroot).
This commit changes those to large heap allocations instead.
Implement the helper function for guestfs_inspect_list_applications2 to
be able to parse the list of installed applications with the APK package
manager (used on Alpine Linux).
Update the check for the first line (the one containing the version) to
match also the message with OpenJDK; switch to awk so it is easier to
extract the version inside the double quotes.
Introduce and use a new inspect_mount_root function to mount all the
mountpoints of a root in the guest, replacing the same code doing that
in different tools.
inspect_mount_root_ro is inspect_mount_root with readonly mount option.
If there are no labels set for the disks, the directory with the
symlinks will not even exists, causing list-disk-labels to fail with
ENOENT. In this situation, act as if the directory was there, but
empty.
This makes a bit easier to diagnose failures on mkfs, without the need
to restart the filesystem creation with verbose output (which will
produce a lot more output).
Use PRIi64 as format string for int64_t, so it builds and works fine
also on 32bit. Also switch from asprintf_nowarn to asprintf, since no
custom formats (eg %Q, %R) are used.
Adding ntfscat_i command for downloading files based on their inode number.
This allows the dowload of files unaccessible otherwise from a NTFS guest disk image.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Add a comment to appliance/Makefile.am describing the purpose of this
directory.
Add a 'make clean-supermin-appliance' rule which forces the supermin
appliance to be reprepared from scratch. See the discussion here:
https://www.redhat.com/archives/libguestfs/2016-February/msg00264.html
Document which feature, if any, is needed for a function; this should
help users in properly checking feature availability when using certain
functions.
Old virt-v2v changed the <graphics> to VNC if the guest was using the
Cirrus hardware and Spice if the guest used QXL.
In commit 3d1cb74b3e I got the logic
backwards, using Spice if the guest used Cirrus and VNC if the guest
used QXL, which obviously makes no sense.
In this commit, I preserve the original <graphics> type from the
source guest. This has the advantage that the user can use the same
method to access the guest after conversion. If the source guest had
no <graphics> element, then we force VNC (a safe choice), and if the
source guest is a local disk that we use SDL, but this should only be
used for testing.
Thanks: Xiaodai Wang for the original bug report here:
https://bugzilla.redhat.com/show_bug.cgi?id=1225789#c10
Set the (libguestfs handle) program name correctly when running
programs like virt-copy-in, so the program name is not "guestfish" but
"virt-copy-in".
Note this feature cannot be tested using the ./run script, since
libtool (buggily) resets the program name to "lt-guestfish". However
I tested it on the installed copy and it worked there.
When listing the subvolumes of a btrfs filesystem, ignore the default
subvolume: we get the content of it when mounting the whole device
(without specifying any particular subvolume), so avoid listing it
twice.
Extend the generator to generate a source (and the header for it) with
functions that print the content of a guestfs struct. The code is
modelled after the code for it in fish.ml, although made a bit more
generic (destination FILE*, line separator) so it can be used also in
the library, when tracing.
This just introduces the new code and builds it as part of the helper
libutils.la.
Since the daemon links to pcre and use regular expressions, and since
the COMPILE_REGEXP macro doesn't depend on any aspects of the
library-side code (eg. the guestfs_h handle etc), we can allow the
daemon to use the COMPILE_REGEXP macro. Move the macro to
"guestfs-internal-all.h" to permit this.
It seems the default behaviour of rake is to run tests sequentially, and
not in parallel (there are separate gems to achieve that behaviour).
Hence just invoke "rake test" to run all the available tests at once.
This reverts commit 8f30c3c3f8.
Since we enabled parallel tests, you can no longer run tests
under valgrind merely by doing:
TESTS_ENVIRONMENT = $(top_builddir)/run --test $(VG)
check-valgrind:
$(MAKE) check VG="@VG@"
The reason is that the parallel tests framework doesn't run
``$(TESTS_ENVIRONMENT) <test>''. It inserts some other processes in
between the environment and the test, so you end up valgrinding some
unrelated process (currently the 'env' program).
In run.in, remove out of date documentation for using $(VG).
In guestfs-hacking(1), document how to do it properly.
Instead of running all the tests manually, the unittest module has a
'discovery' mode to run tests by importing them from a directory: this
requires the tests to have different filenames, since they need to be
imported as modules now (hence an empty __init__.py is added), and the
current naming does not match the convention.
Using unittest as loader/runner brings another change: tests skipped as
whole cannot be done anymore with exit(77), since they are not run but
imported: thus introduce an helper module with decorators applied to the
test classes to skip them according to the current checks. This also
gets us nicer recordings in the unittest log.
Due to the relative imports (needed for the helper code), it is no more
possible to execute tests anymore by invoking them manually; although
it is possible to run single tests, still using unittest's runner:
$ cd python
python$ ../run python -m unittest discover -v t test010Load.py
This does not change anything in what the tests do/check.
If you ran 'make installcheck' when libguestfs-devel was not
installed, then the installcheck would fail in many places with:
gcc: error: ../../src/.libs/libguestfs.so: No such file or directory
We should stop the installcheck script immediately if the .so file was
not found.
Also this commit ensures that src/.libs/libguestfs.so* not existing
before installcheck is not fatal.
Make this error message useful and actionable by suggesting commands
that the end user can run to find available pools, etc.
I also had to extend the fixed size buffer we use for messages since
the message got truncated.
Introduce a new kind of bindings tests, 090-retvalues, to check all the
possible return values in bindings; start implementing them for
scripting languages such as GObject introspection, Perl, PHP, Python,
and Ruby, reusing existing implementations where existing.
Thanks Joshua Pincus for finding the bug.
I checked back with the original W2K8R2 guest which I had used for
getting these registry entries, and there was a transcription error
with two of the CompatibleIDs.
- use assertIsInstance, assertNotEqual, and assertIsNotNone as more
specific checks (will produce better logging)
- use assertRaises when expecting exceptions being thrown
- when testing internal_test_rhashtable, instead of checking type and
elements of the return values just check the return value as a whole
(easier and already getting all the work needed by unittest)
When listing logical volumes, ignore the ones which don't get
activated automatically. No /dev/VG/LV device node is created for
these ones which confuses APIs that attempt to do 'guestfs_lvs'
followed by opening the device node. Note that 'guestfs_lvs_full' is
unaffected by this change.
Rename the existing tests according to the naming/numbering described in
guestfs-hacking(1), and improve the current ones:
- guestfs_php_001.phpt: rename to guestfs_020_create.phpt
- guestfs_php_003.phpt: rename to guestfs_070_optargs.phpt
- guestfs_php_bindtests.phpt: rename to guestfs_090_bindtests.phpt
- guestfs_090_version.phpt: new, checks taken from the former
guestfs_php_002.phpt
- guestfs_100_launch.phpt: new, modelled after the equivalent in e.g.
OCaml/Perl/Python
- guestfs_php_002.phpt: remove, as what it did is now covered by
090_version and 100_launch
And that READ_ONLY access is insufficient. I believe this is a bug in
libvirt, but it requires further investigation. If it is resolved in
libvirt then we can update this documentation again.
Thanks: Ulhas Surse
Now that all the stuff related to Windows virtio drivers has been moved
into a dedicated module, it makes sense to move the definition of
virtio_win there, too, and stop passing it around as a parameter.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Move to lazy_make_tmpdir the logic for making world-readable (but only
for root) newly-created temporary directories, removing the non-fatal
code doing that in guestfs_impl_launch.
Followup of commit 772f649e59.
When running as root libvirt will launch qemu as qemu.qemu, which will
not be able to read inside the socket directory in case it is set as
XDG_RUNTIME_DIR under /run/user/0.
Since normal users don't need particular extra access permissions for
their sockdirs, restrict /tmp as only possible sockdir for root,
changing the permissions only for this user (and making this operation
fatal).
Fixes commit 55202a4d49 and
commit 7453952d24.
1 GB should be enough to create a btrfs filesystem, even with 64K page
size; hence, make the /dev/sda and /dev/sdb test devices smaller so
there is less space taken during the test run.
Followup of commit 8ffad75e5b and
commit 9e9b648770.
This patch updates the guestfs_inspect_get_drive_mappings API call to
also return drive letters for GPT paritions. Previously this worked
only for MBR partitions. This is achieved by matching the GPT partition
GUID with the info stored in the blob from
HKLM\SYSTEM\MountedDevices\DosDevices keys. For GPT partions this blob
contains a "DMIO:ID:" prefix followed by a 16 byte binary GUID.
Support globbing in paths passed to --delete, telling glob to not
return directories with leading slash.
This re-adds back globbing for --delete in virt-sysprep, which was
available before the integration with common code from virt-customize.
Add a new optional bool "directoryslash" to indicate whether the caller
wants trailing slashes in names of directories, defaulting to true (the
current behaviour); this helps with interoperability with other tools
(such as rm).
Related to RHBZ#1293271.
The current VDSM .meta file parser is incorrect:
https://code.engineering.redhat.com/gerrit/gitweb?p=vdsm.git;a=blob;f=vdsm/storage/fileVolume.py;h=20573bd2a3fe6909a8474bd7c53c8d0fea79091c;hb=c1289fd70edcde0a0b3bc8e37ecad1e2a5babe1a#l285
When parsing the key=value fields in the .meta file, it uses the
Python expression:
key, value = l.split("=")
which throws an exception "ValueError: too many values to unpack" if
the value part also contains an "=" character (because split returns
an n-tuple with n > 2). The correct way to split the string in this
situation is to use l.split("=", 1).
In commit 399e031c25 I changed the
DESCRIPTION field in .meta so that in RHEL it would look like this:
DESCRIPTION=generated by libguestfs 1.32.2rhel=7,release=1.el7,libvirt
thus containing an "=" character in the value and throwing an
exception in VDSM.
Although this is a bug in VDSM which needs to be fixed separately,
change any occurrence of "=" in the value to "_" to avoid this
problem.
Thanks: Derrick Ornelas for helping to diagnose this bug and finding
the offending code in VDSM.
One of the mkfs-btrfs tests used two 200 MB partitions. That isn't
enough to create a btrfs filesystem on aarch64 (with 64K page size).
Since we now have a 10 GB disk, we can make the test partition 10
times larger.
In btrfs-progs 4.4, the --leafsize parameter is deprecated. It's now
just an alias for --nodesize.
On aarch64, --nodesize 4096 does not work because it's smaller than
the page size (64K). Luckily we can test a 64K nodesize on any
platform, so use that instead.
Previously these were rather small - just 500 MB. This is too small
to create a btrfs device on aarch64 (where page size may be 64K), and
barely enough even on x86-64. This change makes both these devices
10 GB, and adjusts a few tests so they continue to pass.
Introduce a new read-only API to get a path where to store temporary
sockets: this is different from tmpdir, as we need short paths for
sockets (due to sockaddr_un::sun_path), and it is either
XDG_RUNTIME_DIR if set, or /tmp; adapt guestfs_int_create_socketname
to create sockets in that location.
Furthermore, print sockdir and XDG_RUNTIME_DIR in test-tool for
debugging.
Introduce an internal helper to create paths for sockets -- will be
useful for changing later the logic for placing sockets.
Futhermore, check that the length of sockets won't overflow the buffer
for their filenames.
Extract the bit of code for lazy creation of a temporary directory, so
it can be used for more directories as well.
This is just code motion, with no behaviour changes.
Move the paths of the sockets used, from the libvirt_xml_params struct
to backend_libvirt_data; this way, they will be usable also outside the
launch callback.
Simply code motion.
I find the code slightly easier to read if we get rid of the ~write
flag, and instead split the with_hive function into two functions, one
for readonly access, and one for write access.
This is just code refactoring.
Add a common module `Windows_virtio' and move all the code related to
finding and installing Windows virtio drivers to this module.
This is just code motion and code refactoring, there is no change in
functionality.
Instead of pass 0x3fffffff as mask value, pass a simplier 0xfff, which
is the value of the IN_ALL_EVENTS define. This will still catch all
the inotify events, and avoid a EINVAL error with Linux 4.4.
Actually these are all static functions, so they don't really need the
'guestfs_' prefix at all, but using a prefix avoids any possibility of
a collision with a standard C function.
This is a mechanical refactoring.
For functions linked to virt tools, make sure the name of the function
actually matches the virt tool. Because of the history of moving
functions across tools, some names no longer matched.
For mllib, use `guestfs_int_mllib_' prefix.
For OCaml bindings, use `guestfs_int_ocaml_' prefix.
This commit is by no means complete. There are many other C functions
in other language bindings which could do with being more consistently
named.
There is no functional change, this is just refactoring.
lvm2 2.02.107 adds the -S/--select option used in lvs to filter out only
public LVs (see RHBZ#1278878). To make this work again with versions
of lvm2 older than that, only on old versions filter out thin layouts
and compose the resulting device strings ourselves.
The filtering done is much simplier than what "-S lv_role=public" will
do, but should be good enough for our need.
In Fedora 23, dracut moved to /usr/bin. However we only looked for
/sbin/dracut, so we didn't find it. You would have seen a failure
like this:
virt-v2v: error: unable to rebuild initrd
(/boot/initramfs-4.2.3-300.fc23.x86_64.img) because mkinitrd or dracut was
not found in the guest
Look for dracut in /usr/bin as well.
Thanks: Arnaud Bougeard.
print "" is no more available in Python 3, and print(...) is available
since Python 2.7; as one of the tests was using print(), use this form
everywhere so the tests can run fine with Python 3.
The warning (see below) is fairly useless. This modification to the
code avoids it.
inotify.c: In function 'do_inotify_read':
inotify.c:219:32: error: logical 'or' of equal expressions [-Werror=logical-op]
if (errno == EWOULDBLOCK || errno == EAGAIN) /* End of list. */
^~
The btrfs implementation of vfs_minimum_size requires
btrfs-progs >= 4.2, erroring with ENOTSUP otherwise; thus mark the btrfs
test case of the vfs_minimum_size tests as TestRunOrUnsupported, so it
will not cause failures if the available btrfs-progs is not enough.
Create a new TestRunOrUnsupported test type, which represents a test
sequence where a failure with ENOTSUP in the last command only marks the
test as skipped. To be used mainly when testing features available only
with some versions of helper tools used in the appliance, for example.
Having to choose the os-version of the virt-p2v disk was confusing.
Users thought it had something to do with the physical machine being
converted.
In some cases, virt-p2v-make-disk can now choose a suitable os-version
for the user. Usually it's the latest numeric version of Fedora or
Debian, depending on the host distro. If we don't recognize the host
distro, we bail and ask the user to choose, but this is still an
improvement.
Actually the OCaml compiler rewrites all printf functions at compile
time to a series of primitive string operations, so the use of sprintf
here doesn't really matter. Still, it's a bit tidier not to use
sprintf where it's not needed.
Also this commit contains some code reformatting.
This is just refactoring.
When the user clicks on the second column of the list of network
interfaces, run 'ethtool --identify <if_name> 10', which (on supported
cards) flashes a light on the physical interface for 10 seconds,
allowing it to be identified by the operator.
Resolve the paths of the binaries we are looking in the guest to
identify the architecture. This way we can identify also busybox-based
guests such as Alpine Linux, where all the "base" binaries in /bin are
symlinks to busybox.
Apparent newer versions of that report everything on stdout, including
error messages; since we only print something on failure, fold stdout to
stderr so we can see everything on failure.
This flag cannot be disabled (yet) in V5 xfs filesystems; since 2 out
of the current 3 tests of xfs_admin check other results than that flag,
avoid setting it when not needed.
Apparently with newer btrfs-progs (seen with 4.4) 100M are not enough
for a btrfs filesystem; hence double the size of the partitions
created in the test of btrfs_image, so now 200M are enough for btrfs.
In commit af762e4342 (2015-03-05)
I stopped libguestfs from using the supermin -dtb parameter when
building appliances.
It has not been necessary for quite a long time to specify a DTB,
since qemu itself creates a device tree structure in the guest
(appliance) memory from the devices presented to the guest. Qemu does
this on the fly each time the guest starts.
As I promised in the commit mentioned above I would remove this code,
and this commit does that (for libguestfs only).
This allows you to build a 32 bit virt-p2v USB key, even on a 64 bit
host.
Because of problems with the Fedora build system it's unfortunately
not easy to build a 32 bit virt-p2v binary, so this patch punts on
this issue and assumes that the user can supply the binary themselves.
This action moves second(backup) GPT header to the end of the disk.
It is usable in in-place image expanding, since free space after
second GPT header is unusable. To use additional space, we have
to move second header. This is what sgdisk -e does.
However, sgdisk -e may perform additional actions if the partition
table has unexpected params (e.g. if we call sgdisk -e /dev/sda1,
it may fix partition table thus destroying the filesystem).
To prevent such cases, we do a dry-run at first and fail if
additional actions are scheduled.
If present, Parallels Tools may stand in the way of proper running the
windows guests in non-Parallels hypervisors, so we're better off
uninstalling them on the first boot into the new environment.
With this patch, the uninstall records for Parallels Tools are looked up
in the registry and, if found, corresponding firstboot actions are
registered, taking special care that those actions are run fully
unattended.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
This test would fail on machines with around 4 GB of free memory.
Adjust the test so it would now work on such machines (it will however
still fail on a machine with around 2 GB of free memory).
Some OSes (e.g. Windows Server 2012 R2) fail to boot if the disk
GPT GUID has changed. To preserve disk guid e.g. during virt-resize,
we need a way to get/set disk GUIDs.
Libvirt has a fixed 15 second timeout for qemu to exit. If qemu is
writing to a slow USB key, it can hang (in D state) for much longer
than this - many minutes usually.
The solution is to check specifically for the libvirt EBUSY error when
this happens, and retry the virDomainDestroyFlags operation
(indefinitely).
See also the description here:
https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html
Similar to the following OpenStack Nova commit:
http://git.openstack.org/cgit/openstack/nova/commit/?id=3907867
Thanks: Kashyap Chamarthy and Daniel Berrange.
The err->domain field is essentially random from the point of view
of libguestfs.
The err->int1 field has some useful information in some circumstances:
it has the errno value from the failed system call.
Resolve in the guest the given path, so absolute symlinks can be listed
using appliance tools without resolution errors.
Also remove the note about the possibility to escape the sysroot using
ll and llz, since realpath won't return paths outside sysroot.
Fixes part of RHBZ#1293276.
Fedora/i686 does not have a package called 'kernel', so the command
'rpm -q kernel' fails. In any case the method used wasn't exactly
very robust.
The new method is based on how febootstrap used to do this.
This updates commit d723b352f8.
Sometimes the user wants to know minimum size
for dirty (e.g. mounted) filesystems. In this case,
resize2fs -P will require calling e2fsck -f, while
"in general, it is not safe to run e2fsck on mounted filesystems".
Since resize2fs -P does not modify filesystem, we force it
to display (probably approximate) minimum size.
Allow the user to specify a template in --list mode, which will be the
only result in the resulting output (instead of all the available
templates).
This makes it easier to find out the details of a specific template.
For every user command line tool (eg. guestfish or virt-cat) require
that the manual page has a DESCRIPTION section. This doesn't apply to
non-CLI man pages (eg. guestfs-perl, guestfs-faq).
Require qemu >= 1.3.0, the first version that supported
`qemu-img --output=json'.
This means we require yajl (for parsing the JSON output of qemu-img),
and that in turn has consequences elsewhere.
Move the creation of iscsi URIs away from make_uri, composing them
manually: this is needed because libxml assumes colons (':') to separate
user and password for the authority part, while with iscsi URIs the
separator is percentage ('%'), which would be percent-encoded by libxml.
It's a bitmask, so unsigned is the right choice. eg. We can more
easily print it using %x.
This patch changes the size of the fd mask and the values of the
COMMAND_FLAG_* constants, but since these are internal definitions
that doesn't matter.
Commit e6f7d23782 added new po files for
pt_BR to po/ and po-docs/ (they are only very minimal). However they
were not added to the list of linguas, so amongst other things the
files were not included in EXTRA_DIST automatically.
./uk.po:1049: 'msgid' and 'msgstr' entries do not both end with '\n'
./uk.po:9638: format specifications in 'msgid' and 'msgstr' for argument 2 are not the same
This updates commit e6f7d23782.
Two tests opened the phony Fedora image in parallel without the
readonly flag. This occasionally resulted in strange errors
(especially observed in test-inspect.sh). Add the --readonly flag to
avoid this.
- add a flag to request chroot for the process, which is done only as
very last (before chdir) operation before exec'ing the process in the
child: this avoids using CHROOT_IN & CHROOT_OUT around command*
invocations, and reduces the code spent in chroot mode
- add failure checks for dup2, open, and chdir done in child, not
proceeding to executing the process if they fail
- open /dev/null without O_CLOEXEC, so it stays available for the
exec'ed process, and thus we don't need to provide an own fd for stdin
Followup of commit fd2f175ee7, thanks also
to the notes and hints provided by Mateusz Guzik.
Prior to this commit:
$ virt-builder --help
[...]
--sm-attach SELECTOR Attach to a subscription-manager pool
--sm-credentials SELECTOR credentials for subscription-manager
--sm-register Register using subscription-manager
Really the generator ought to check this.
'sfdisk --part-type' used to merely set the MBR ID byte in the
partition. However since sfdisk was rewritten, it now "helpfully"
corrupts the first sector of the partition if you change the type byte
from an ordinary partition to an extended partition.
So we need to change the order in which we sets the partition type
byte, to do it before copying the partition content.
This fixes a major regression in the way that virt-resize handles any
guest with logical partitions (specifically Ubuntu guests).
If uncompressing a template to a filesystem which is nearly full,
virt-builder displays an error which points to the wrong (source) file:
$ virt-builder fedora-22
[ 1.2] Downloading: http://libguestfs.org/download/builder/fedora-22.xz
[ 1.7] Planning how to build this image
[ 1.7] Uncompressing
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
(The error message is usually printed once by each thread, so it is
printed several times.)
Change the error message to point to the output file, which might be a
temporary or the final file depending on the build plan:
$ virt-builder fedora-22
[ 1.1] Downloading: http://libguestfs.org/download/builder/fedora-22.xz
[ 1.6] Planning how to build this image
[ 1.6] Uncompressing
fedora-22.img: No space left on device
fedora-22.img: No space left on device
fedora-22.img: No space left on device
fedora-22.img: No space left on device
On ARM machines with vGICv3, we need to use the extra -M
parameter gic-version=3 (or gic-version=2 for the old vGIC).
qemu can detect this feature for us if we pass gic-version=host,
so add that to the -M option. Unfortunately we cannot detect if
qemu itself supports this parameter, but this change only affects
aarch64.
Thanks: Wei Huang
Exit with 3 as return value when fusermount fails, because the specified
mount point is not considered mounted for the user. This is in line
with what the guestunmount documentation says.
Adapt the test-guestunmount-fd test to the updated return value.
Thanks to: Maxim Perevedentsev.
When running commands in the mounted guest (using the "command" API, and
APIs based on it), provide the /dev/null from the appliance as open fd
for stdin. Commands usually assume stdin is open if they didn't close
it explicitly, so this should avoid crashes or misbehavings due to that.
Older GCC versions (e.g. < 4.9) may complain about inline functions that
cannot be inlined (growing the generated code). It is not really a
problem for us, so just suppress again the warning.
Followup of commit c37c554fc5.
This allows certain guests which virt-v2v cannot access to be copied
off the remote hypervisor and converted. Essentially this just
automates the process of copying the guest's disks and adjusting the
libvirt XML.
This also adds a test of the new tool.
Split the map_source_to_uri function up so that we can get at just the
https://... URI.
This is almost refactoring. It doesn't quite handle the case where we
are passed a source path from the libvirt VMware driver that doesn't
match the expected "[datastore] path" format, but probably if we hit
that case it's a bug anyway.
If you try to convert a guest that has a zero-sized disk, it
will currently fail in a rather strange way. Usually you will
see errors in the debug output:
[ 0.562714] sd 2:0:0:0: [sda] Read Capacity(16) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 0.563884] sd 2:0:0:0: [sda] Sense Key : Not Ready [current]
[ 0.564587] sd 2:0:0:0: [sda] Add. Sense: Logical unit not ready, manual intervention required
followed by virt-v2v failing with:
libguestfs: trace: v2v: inspect_os = []
virt-v2v: error: inspection could not detect the source guest (or physical
machine).
Additionally, because of a problem with the ssh driver in qemu (or
perhaps, with sftp-server on the remote side) it is not possible to
use the ssh driver to open a block device path on the remote server.
The drive will appear as zero-sized, triggering the above error.
Therefore detect this situation, and emit an error (see below). Also
add a section to the manual describing the workaround required for
converting RHEL 5 Xen guests which are located on block devices.
virt-v2v: error: guest disk sda appears to be zero bytes in size.
There could be several reasons for this:
Check that the guest doesn't really have a zero-sized disk. virt-v2v
cannot convert such a guest.
If you are converting a guest from an ssh source and the guest has a disk
on a block device (eg. on a host partition or host LVM LV), then
conversions of this type are not supported. See "XEN OR SSH CONVERSIONS
FROM BLOCK DEVICES" in the virt-v2v(1) manual for a workaround.
If the user tries to close the running dialog during a running
conversion, act as if the cancel button was pressed (cancelling the
conversion).
If the conversion has finished/been cancelled, then this closes the
dialog and exits virt-p2v, as now.
Cancelling the conversion in this way ensures that qemu-nbd and ssh
subprocesses are cleaned up.
If the virt-p2v ISO doesn't contain the correct drivers for the
hardware on the physical server, that can cause peculiar conversion
problems. This was hard to diagnose because we could not see the
'dmesg' output from the physical server.
Upload 'dmesg' output from the physical server (more precisely: from
the virt-p2v ISO's kernel running on the physical server) to the
virt-v2v conversion server debugging directory, so that this
information is available for debugging later.
Unfortunately this involves yet another copy of the 'read_whole_file'
function.
Instead of creating a special 'Exported by ...' string just for this
module, use the Common_utils.generated_by string. Since the string is
just descriptive, this should make no difference.
Adding this causes 'make dist' to fail with:
make[1]: *** No rule to make target 'ChangeLog', needed by 'distdir'. Stop.
This updates commit a2edda266e.
See the description in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1281578
Thanks: Richard Tollerton for alerting me to this change in the way
that Windows version numbers are stored in the Registry starting with
Windows ≥ 10.
The separator character (currently comma) was picked arbitrarily to
allow us to parse the output of commands like 'lvs' for APIs such as
'lvs-full'. Unfortunately the choice of comma conflicts with the 'lvs
-o modules' column, since the list of modules is separated by commas,
breaking our parser.
Change the separator to another arbitrary character (colon) which
hopefully is not used by any column.
See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1278878#c11
When a disk image uses LVM thinp (thin provisioning), the guestfs_lvs
API would return the thinp pools. This confused other APIs because
thinp pools don't have corresponding /dev/VG/LV device nodes.
Filter the LVs that are returned using "lv_role=public".
Thanks: Fabian Deutsch
I enabled the -Wshadow warning temporarily in order to do these fixes,
but had to disable it again afterwards. The reason is that this warns
about shadowing globals, which is sort of a good thing, but because we
have a global called "verbose" just about everywhere, and at the same
time we baked a function argument called "verbose" into several
unchangable APIs, well, we're stuck without being able to use this
warning.
This seemingly redundant change avoids a gcc warning/error:
error: cannot optimize possibly infinite loops
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=34114#c3 and
following for an explanation.
Of course gcc is completely wrong and stupid here, because there's no
possibility on current or future hardware that an array of size
SSIZE_MAX could be allocated since it would by definition occupy at
least all addressible memory (if it was an array of bytes, which this
isn't), leaving no room for the code that is being compiled.
'virt-inspector --xpath' can segfault.
When run under valgrind, it shows this error:
==2254== Invalid free() / delete / delete[] / realloc()
==2254== at 0x4C29D6A: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==2254== by 0x53BA198: xmlFreeNodeList (tree.c:3690)
==2254== by 0x53B9F65: xmlFreeDoc (tree.c:1247)
==2254== by 0x405BFA: do_xpath (inspector.c:808)
==2254== by 0x405BFA: main (inspector.c:250)
==2254== Address 0x1030a037 is 311 bytes inside a block of size 1,048 alloc'd
==2254== at 0x4C28C50: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==2254== by 0x545DE86: xmlDictAddString.isra.0 (dict.c:270)
==2254== by 0x545E961: xmlDictLookup (dict.c:923)
==2254== by 0x539C6DC: xmlDetectSAX2 (parser.c:1067)
==2254== by 0x53B0B92: xmlParseDocument (parser.c:10725)
==2254== by 0x53B1276: xmlDoRead (parser.c:15295)
==2254== by 0x40587D: do_xpath (inspector.c:772)
==2254== by 0x40587D: main (inspector.c:250)
The cause appears to be that when copying the matching node(s) found
by the xpath expression, we have to copy them into the new document
(using xmlDocCopyNode instead of xmlCopyNode).
This bug has existed since this functionality was originally added in
commit d1ee71782a.
Extend open_guestfs to take an optional ?identifier parameter.
Use this parameter in virt-v2v which is currently the only place where
we use the handle identifier.
This higher order function encapsulates the following pattern:
match x with
| None -> ()
| Some x -> f x
(replaced with: `may f x`)
This is taken from OCaml lablgtk (Gtk bindings) where this pattern is
used frequently.
See also: https://ocaml.org/learn/tutorials/labels.html
Instead of creating Guestfs handles and manually apply common options
(e.g. debug and trace), use the open_guestfs in Common_utils.
This also applies the common options to handles which didn't set them
before, so we can inspect also their messages if needed.
Resolve also symlinks when checking whether a path passed to --edit is a
file: the file editing code supports symlinks anyway (properly changes
the target file, instead of overwrite the symlink with a real file), so
it is safe to allow this operation in customize/builder.
Check that a path provided to --edit exists already; while the is_file
call later will fail for non-existing files, with an explicit check a
better error message can be provided.
With commit bc2b417784
HAVE_LIBVIRT_BACKEND is defined based on the libvirt version (using its
version macro), although libvirt.h is included only after that check:
because of this, variables in the guestfs_h struct after the
HAVE_LIBVIRT_BACKEND block would be used wrongly if libvirt.h was not
included before guestfs-internal.h, like in the recently added
available.c (all the other places using libvirt features in the handle
already happened to do so).
Considering guestfs-internal.h already includes libvirt.h, move its
inclusion up, right before the libvirt version check.
Replace the custom XSLT + script to turn output from $RUN_OUTPUT_FILE
(from previous run.in versions) with a OCaml script to analyze .trs and
.log files of the automake test framework.
The final result should be the same, just with classname= for each test
set to the "path" (subdirectories) to that test.
Use (implicitly) parallel tests on new enough automake. The tests run
a bit quicker with this change. On my laptop, I measured 27 mins down
to 18 mins.
Change the ./run function so that ./run --test no longer spools the
test output to a file. That is not necessary when using parallel
tests, since the test harness does the same thing.
Note: This commit removes the $RUN_OUTPUT_FILE functionality. We will
change the CI integration in a future commit so it uses the .trs and
.log files.
The user did not have the 'file' command installed, and when they
attempted to do a v2v conversion of a Windows guest, inspection
succeeded, but the Types.i_arch field was found to contain "unknown".
virt-v2v conversion failed to locate the correct virtio drivers as a
result.
The 'file' command is required, and there's no other reason we should
not be able to detect the architecture of any Windows guest, so fail
hard in this situation.
We want to test if virt-v2v can find drivers using the *.inf files.
For this to work, one field is important:
[Version]
DriverVer = 07/17/2013,6.1.0.10018
Copy the [Version] section header and DriverVer field from the real
*.inf files to our fake *.inf files, which is enough for testing.
See also:
https://www.redhat.com/archives/libguestfs/2015-October/msg00352.html
Note that the fields are case insensitive and may or may not contain
spaces.
The virtio-win ISO has a different layout and superset of drivers
compared to the virtio-win RPM directory. Previously we simply put
all the RPM files into the fake ISO. Instead create a fake ISO which
has the same layout and (fake) drivers as the real ISO.
Refactor the internal_feature_available to return the result for just
one group, so it is easier to know on the library side what was the
actual error, and which group refers to; drop internal_available, as no
more needed after this.
On the library side, implement in available and feature_available the
real logic to iterate through the requested group, and error out or
return whether the groups are available. This also introduces caching
for the features, so each needs to be queried just once in each
appliance run.
The result of this is that there should be much less communication with
the daemon to know about available features; the downside is that
queries for more than one group at once, not already cached, will be
cause more queries to the daemon.
Rename the current available and feature_available into internal daemon
functions, and provide non-daemon functions wrapping them at library
side. This will make it possible to e.g. add caching for them.
Should be only refactoring, no actual behaviour change.
The error messages changes from:
$ virt-v2v -i disk /tmp/blank.img -o null
[...]
[ 14.6] Inspecting the overlay
virt-v2v: error: no root device found in this operating system image.
to:
$ virt-v2v -i disk /tmp/blank.img -o null
[...]
[ 15.2] Inspecting the overlay
virt-v2v: error: inspection could not detect the source guest (or physical
machine).
Assuming that you are running virt-v2v/virt-p2v on a source which is
supported (and not, for example, a blank disk), then this should not
happen. You should run 'virt-v2v -v -x ... >& log' and attach the complete
log to a new bug report (see http://libguestfs.org).
No root device found in this operating system image.
Various tests cannot be run in parallel just because they happen to
use conflicting names for temporary output files (eg. "test.out").
Change these tests to use unique temporary names, so the tests could
be run in parallel.
When virt-sparsify receives a user quit signal (eg from ^C) it
currently kills virt-sparsify and qemu instantly, meaning any mount +
fstrim in progress is uncleanly stopped. The (minor) side effect of
this is that the guest filesystem may require a journal replay or fsck
on boot.
Let virt-sparsify capture the user quit signal and shut down
gracefully.
It is not thought that the previous behaviour could cause guest
corruption; see
https://lists.nongnu.org/archive/html/qemu-devel/2015-11/threads.html#00402
for discussion.
- include <sys/un.h> for sockaddr_un, on any OS (removing the Mac OS X
blocks)
- include <fcntl.h> for O_* flags
This commits reverts a couple of changes in these files done in
commit 3b17152ec3.
Remove man pages and other pages which 'make clean' did not remove
before.
To evaluate which pages could be removed, I did a full build and
check, and then ran 'make clean' followed by 'git clean -xdf'. By
examining the output of the git clean command I could see which files
were being missed.
Files that are _not_ removed by make clean or make distclean:
- generator-built files
- Makefile, Makefile.in, .deps, .depend
- any ./configure output files (maybe they should be?)
It's a nice idea, but has never actually found a problem. After using
afl to find bugs in qemu-img, it's pretty clear to me that no
successful approach could be run in the short time demanded by the
libguestfs test suite. Afl tests run for days, weeks or even months.
A better approach may be to use a proven fuzz tester (such as afl) on
the inspection code; or to run this code for days on end on a separate
testing machine.
This reverts commit c2dcd10964.
Also move guestfs-performance to the right place on the website.
This updates commit 2eb01004bf
and commit d07515b947
and commit 74fba6e692
and commit 15b8b6453d.
Move the random set of HTML files we build from html/ into
the website/ directory.
Also in the website/ directory, put the index.html file from
http://libguestfs.org, which was previously not under version control.
It is generated from index.html.in so we can automatically add the
current version and release date.
Also in the website/ directory, put various CSS file, images, etc.
which are required by the website and were also previously not under
version control.
Change the 'make website' rule to 'make maintainer-upload-website'.
As the name suggests, it is only useful for the maintainer, and will
fail with an error for anyone else.
When I conceived commit dc1d0880b0, I
thought it would be a good idea to build test-data files, including
the phony guest images, during the 'make' phase. Previously these
were built early in the 'make check' phase.
However this has two disadvantages: (a) We run the appliance
unnecessarily during builds. For example, it is not necessary to
build the phony guests if we are going to do an 'installcheck'.
(b) Even if we are testing, the critical first time we run the
appliance is not in 'quickcheck' or 'tests/qemu', but some random
guest build.
Reorder the build sequence to fix this, and also a few other things.
This updates commit dc1d0880b0.
This test failed during release testing. It turned out to be because
the tests are run with LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 which
causes other messages to be printed.
Updates commit b5a6f899cd.
Create a new top-level directory called test-data, which will carry
all the test data which is large and/or shared between multiple tests.
There are actually several new subdirectories created:
test-data/binaries: The pre-built binary and library files for random
architectures that we use to test various architecture detection
features (was part of tests/data).
test-data/blank-disks: The blank disks which are used for disk format
detection (was part of tests/data).
test-data/files: Other miscellaneous test files from tests/data that
are not included in the above.
test-data/phony-guests: The phony guests (was tests/guests).
test-data: The top-level directory builds the 'test.iso' image file
that is used for testing the C API and in miscellaneous other tests.
Create a Windows module, analogous to the Linux module we already
have. Move utility functions related to Windows into this module.
In particular:
- detect_antivirus function moves here, and the Detect_antivirus
module goes away entirely
- copy_virtio_drivers / virtio_iso_path_matches_guest_os moves here,
and the virtio_iso_path_matches_guest_os can now be hidden
This is just refactoring, but allows us to make deeper changes to
copy_virtio_drivers.
The three utility functions xml_quote_attr, xml_quote_pcdata and
uri_escape were only used in a single module. Move them close to
where they are used.
This is just code refactoring.
Unfortunately Coccinelle places a Config module in the ocaml libdir,
and this confuses ocamlfind (only when Coccinelle is installed).
Since this is a private module that only libguestfs tools use
internally, just rename it from Config -> Guestfs_config.
Do this early since these tests generally don't depend on anything
else (the external program is either there or its not), and because
other things depend on some of these tests.
The error was:
libguestfs: trace: pread_device = "NTFS "
libguestfs: trace: part_to_dev "/dev/sda"
guestfsd: main_loop: new request, len 0x34
guestfsd: error: device name is not a partition
guestfsd: main_loop: proc 272 (part_to_dev) took 0.00 seconds
libguestfs: trace: part_to_dev = NULL (error)
virt-v2v: error: libguestfs error: part_to_dev: device name is not a
partition
This way, we can make sure that if a template needs to be expanded, the
result will have all the filesystems correctly expanded, fitting into
the new sizes of their partitions.
Introduce a new option to control how virt-resize behaves when asking to
expand a filesystem, either unknown to libguestfs or that virt-resize
cannot expand. The default keeps the current behaviour, i.e. just warn.
Replace ExtUtils::MakeMaker with Module::Build.
'perllocal.pod' and 'bindtests.pl' are no longer incorrectly
installed.
This change also removes the following phony deps: appliance src_deps
test_images. No other language binding needs explicit dependencies
for their tests, they just rely on the top level build order being
correct (ie. SUBDIRS in /Makefile.am).
'ocamlfind install' doesn't like to install the same package if it is
already installed. It will give errors like:
ocamlfind: Package guestfs is already installed
or:
ocamlfind: Conflict with file: /usr/lib64/ocaml/guestfs/mlguestfs.cmxa
Fix this by removing the old installed files before running
'ocamlfind install'. An alternative would be to use
'ocamlfind uninstall', but this seems more direct.
The error is:
ln -s xdg/virt-builder /tmp/p/etc/virt-builder
ln: failed to create symbolic link '/etc/virt-builder/virt-builder': File exists
There is no 'LN_SF' macro, but according to the documentation at
least, it should be safe to add a '-f' flag since it will work for
both the 'ln' and 'cp' variants of the macro.
When running in verbose mode, dump the XML we created for libvirt:
this way it is easier to debug mismatches between what we want to setup
in libvirt, and what actually gets defined.
Create the correct tags for <video> in the libvirtxml, so all the
attributes are in a <model> child of <video> instead of <video> itself.
Adapt the XML of test-v2v-i-ova to the different expected XML.
Move most of the content of either po-docs/ja/Makefile.am or
po-docs/uk/Makefile.am to po-docs/language.mk, and use it exclusively
instead of the former contents of the languange-specific Makefile.am.
This way, either adding a new documentation or enabling a new language
will not require copying over the same make code.
Move most of open_guestfs to Types, so a common function to open a
Guestfs handle is available. Since it does not do all the things the
old open_guestfs did, the two operations have been moved to the only
place requiring them.
This function has been placed in Types and not Utils, as will be needed
by other functions in Types; Utils uses Types, so this avoids a circular
dependency.
Use FilenameList as type for lists of file names, as used in some
listing-alike APIs. This way we can ensure anything different than just
file names in those lists is rejected outright.
As a consequence, test-big-dirs.pl does not need to prepend the
directory name anymore before calling listing-alike APIs: previously
they didn't fail, but the returned lists contained only invalid
elements (and only their size was checked).
Furthermore, add a new regression test for it.
We have phony tools for v2v tests, introduced with
commit c1af1ba32f126b7be275bcc8cd4ea4e5af27a88e; make use of it also in
tests related to v2v outside the v2v directory.
I have also removed the text equivalent of this file. Originally I
added this because it was thought good to have the release notes
available in a format that doesn't require any special tools to read.
But:
- POD files are basically text.
- Debian tooling generates the text file in a slightly different way
from Fedora tooling, resulting in git marking the file as being
updated when it isn't really.
- github can format and display POD files.
In this mode, converting of the VM configuration, setting up the
rollback path for error cases, transforming the VM storage and so on is
taken care of by a third-party toolset, and virt-v2v is only supposed to
tune up the guest OS directly inside the source VM, to enable it to boot
and run under the input hypervisor.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Factor out perfoming the actual convertion of the guest, which includes
determinig the appropriate guest os specific conversion module and
running its conversion routine.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Iterating over source disks and creating temporary overlays for easy
rollback fits nicely into a separate function. In addition, determining
their size doesn't need to wait until the guestfs is launched: the size
can be obtained via disk_virtual_size() method.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Opening the source VM and amending the properties in its internal
representation in accordance with command-line options fit nicely into
two isolated functions.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
If Augeas fails to parse a file, more information is available in at
least these fields:
><fs> aug-ls /augeas/files/etc/fstab/error
/augeas/files/etc/fstab/error/char
/augeas/files/etc/fstab/error/lens
/augeas/files/etc/fstab/error/line
/augeas/files/etc/fstab/error/message
/augeas/files/etc/fstab/error/pos
Pull out some of these fields and add them to the error message.
The new error message looks like this:
virt-v2v: error: libguestfs error: /etc/fstab:17:4: augeas parse failure:
Iterated lens matched less than it should
This updates commit 279addeb7d.
In the case where libvirt was installed, but is too old to run the
backend, the libvirt backend would be compiled out but still
referenced by the library.
Fix this by splitting the definition of HAVE_LIBVIRT (libvirt is
installed at all) and HAVE_LIBVIRT_BACKEND (libvirt is new enough to
run the libvirt backend).
Reported by: Efstratios Skleparis.
During the inspection phase, check for errors after aug_load: if any of
the errors happened in any of the requested files, then report the error
straight away; ignoring the error means that information would be
silently lost. For example, a malformed /etc/fstab would have caused
the inspection to not handle any of the additional mount points, giving
cryptic errors later on when trying to access files in any of the mount
points.
Now guests with invalid files such as /etc/fstab, /etc/mdadm.conf, and
/etc/sysconfig/network will cause the inspection to fail, instead of
being reported with a single mount point ('/').
Now that virtio driver copying is localized in a single function and the
auxiliary guestfs handle doesn't leak outside of it (and thus an extra
qemu instance doesn't survive beyond that function's runtime) there's no
need in the forced GC which used to trigger closing of that handle and
termination of that qemu instance.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Refactor copying of virtio windows drivers into the guest so that the
matching of the drivers to the guest os flavor and copying the files
happens one next to the other in a single function, and no guestfs
handle (nor any other irrelevant info) is leaked outside.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Copy the appropriate driver files from the virtio-win storage into
%SYSTEMROOT%\Drivers\VirtIO once they are discovered, and stick to using
those copies later on.
This way the knowledge of where the drivers come from originally is
consolidated in one place, so the lifetime of the associated entities
becomes easier to control (to be implemented in followup patches).
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Move this function to the VCenter module. This is easier since it
doesn't have to do dcPath calculation (see previous commit).
The readhead parameter is no longer labelled.
In libvirt >= 1.2.20, the VMware libvirt driver supplies the correct
dcPath to use via <vmware:datacenterpath> in the libvirt XML. If
libvirt passes us this element, use it.
This code still allows the user to override dcPath using the --dcPath
option on the command line, but that's mainly for safety so we can fix
any problems in virt-v2v or libvirt in the field. As we get more
confident in libvirt and as libvirt 1.2.20 is more widely adopted, we
will be able to deprecate this parameter entirely.
Thanks: Matthias Bolte for adding the <vmware:datacenterpath> element
to libvirt in
https://libvirt.org/git/?p=libvirt.git;a=commit;h=636a99058758a0447482f3baad94de8de3ab1151
libguestfs: trace: v2v: read_lines "/etc/os-release"
[...]
libguestfs: trace: v2v: read_lines = ["NAME=Fedora", "VERSION="20 (Heisenbug)"", "ID=fedora", "VERSION_ID=20", "PRETTY_NAME="Fedora 20 (Heisenbug)"", "ANSI_COLOR="0;34"", "CPE_NAME="cpe:/o:fedoraproject:fedora:20"", "HOME_URL="https://fedoraproject.org/"", "BUG_REPORT_URL="https://bugzilla.redhat.com/"", "REDHAT_BUGZILLA_PRODUCT="Fedora"", "REDHAT_BUGZILLA_PRODUCT_VERSION=20", "REDHAT_SUPPORT_PRODUCT="Fedora"", "REDHAT_SUPPORT_PRODUCT_VERSION=20"]
At this point libguestfs segfaulted.
It seems as if 'major' is not used here, so don't try to free it.
This fixes commit 32d19e3289.
On 32 bit platforms, reading the memory values can cause some numbers
to be read as negative numbers. Fix this by treating memory values as
64 bit integers throughout the parsing and calculation.
The log file was not being closed at the right time, resulting in the
process running out of open files. Also the log filename buffer was
not freed.
This fixes commit 7bbf5cab28.
Set the handle identifier of the main handle to "v2v".
Whenever we open a second handle for any substantial operation, set
the handle identifier for that handle too.
This should make traces clearer.
For that, add autofoo machinery to generate the iso out of the fake
virtio-win drivers introduced in the previous patches, and pass the
resulting image in $VIRTIO_WIN to v2v in a new test otherwise similar to
test-v2v-windows-conversion.sh.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
In order to test the copying of virtio-win drivers into the guest during
v2v, create a set of fake virtio-win drivers and make use of them in the
corresponding v2v tests.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
This option can be used to direct output from each individual
boot test to a different file. Usual usage is:
qemu-boot --log /tmp/output.% -v -x -n 1000
Add a per-handle identifier. The main effect of this is to change
trace output from:
libguestfs: trace: foo
to:
libguestfs: trace: ID: foo
which makes it simpler to follow traces from multi-threaded programs.
v2v/test-v2v-windows-conversion.sh used to query if the expected
directories and files were present in the VM upon conversion; however it
would ignore the results of that query.
That lead to the test passing even though the checks failed.
To fix it, parse the output of guestfish and verify that all is-file and
is-dir commands report the expected "true".
The approach taken is slightly more elaborate than what is being used in
other components, but it allows to generate comprehensible reports in
case of test failure, and facilitates future expansion of the set of
checks including negative ones.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Without rhev-apt.exe the code path where firstboot scripts are created
isn't taken, so it isn't tested and v2v/test-v2v-windows-conversion.sh
is supposed to fail (it actually doesn't but for another reason,
addressed in a followup patch).
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
The tests that depend on rhsrvany.exe being available, actually don't
need it to be the real one: all they want is the file being present.
So, instead of skipping the tests if the real rhsrvany.exe is not found,
store a fake one in the repository, and point $VIRT_TOOLS_DATA_DIR at
its location.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Make sure to close the open files even when test_fuse() returns earlier
because of failures, otherwise the mountpoint is still in use by open
files when later guestunmount is called on it.
** NB: This is an API break for OCaml programs using Guestfs.event_callback. **
Because of the way I implemented Guestfs.event_callback which had the
Guestfs.t handle as the first parameter, we had to store the (OCaml)
Guestfs.t handle in the C handle's private data area. To do that, we
had to create a global root pointing to the handle.
This of course meant that the handle could not be garbage collected
(thanks Roman Kagan for spotting this).
This changes the API of Guestfs.event_callback so that a handle is no
longer passed. The OCaml handle can now be garbage collected again.
For programs that need the Guestfs.t handle in the callback function
(which turns out to be *none* of the OCaml programs we have written),
you can do:
g#set_event_callback (callback_fn g) [Guestfs.EVENT_FOO];
In this case, since the closure passed to Guestfs.set_event_callback
is still registered as a global root, that will capture a reference to
the handle, so the handle won't be able to be garbage collected until
you delete the callback.
These are considerably more efficient than ordinary global roots, but
with the caveat that the program is not allowed to modify them without
calling a special function. We don't modify them, so this change is
safe.
This requires OCaml >= 3.11, but we have that on RHEL 6
(since we dropped support for RHEL 5).
See also:
http://caml.inria.fr/pub/ml-archives/caml-list/2008/03/c3bf86990088236ceeb9a0f0f4c35390.en.html
This is just a straight refactoring. Various ad hoc string_*
functions that appeared in Common_utils have been renamed and placed
in the String.* namespace. The old vs "new" functions are:
string_prefix -> String.is_prefix
string_suffix -> String.is_suffix
string_find -> String.find
replace_str -> String.replace
string_nsplit -> String.nsplit
string_split -> String.split
string_lines_split -> String.lines_split
string_random8 -> String.random8
In Common_utils, override the Char and String modules from stdlib.
This hides the original (stdlib) modules, and means that whenever you
use Char.foo or String.foo you are in fact calling the
Common_utils.Char.foo or Common_utils.String.foo function.
As it stands, this change does nothing, but it will allow us to drop
unsafe functions (like String.lowercase) and add extra functions.
Calling _exit(2) in the child process has the side effect that
tmp/libguestfsXXXXXX is not cleaned up. Clean it up by ensuring the
handle is properly closed before _exit.
This test has been broken for a while. It is meant to test that when
the appliance cachedir is empty, two simultaneous runs of libguestfs
(both rebuilding the full appliance) will not cause conflicts, because
(eg) the locking in either supermin or libguestfs is not working.
However the test only set $TMPDIR, but the ./run script sets
$LIBGUESTFS_CACHEDIR which overrides $TMPDIR, so it was simply reusing
the existing appliance, and hence not testing anything.
Fix this by clearing $LIBGUESTFS_CACHEDIR.
Note the test now takes a lot longer to run since it does a full
appliance rebuild.
Accoding to its documentation [1], it takes as second parameter a
PangoWrapMode and not a GtkWrapMode. While the change should reflect
the actual value intended, it should slightly change the label wrap
mode: GTK_WRAP_WORD is 2, which corresponds (as int value) to
PANGO_WRAP_WORD_CHAR.
[1] https://developer.gnome.org/gtk2/stable/GtkLabel.html#gtk-label-set-line-wrap-mode
There is GUESTFSD_EXT_CMD defining a string for udevadm (so it is marked
as "used tool" in the appliance), but it is not actually used when
starting udevadm.
There should be no behaviour change.
Instead of hardcoding the location of perl (assuming it is installed in
/usr), use /usr/bin/env to run it, and thus picking it from $PATH.
This makes it possible to run these scripts also on installations with
perl in a different prefix than /usr.
Also, given that we want enable warnings on scripts, turn the -w
previously in shebang to explicit "use warnings;" in scripts which
didn't have it before.
Apparently implicitly pulled by something else on Linux, it is not when
building on e.g. FreeBSD.
This reverts the change to this file of
commit 3b17152ec3.
We don't support RHEL 5 upstream (see the 'oldlinux' branch for a
version that works with RHEL 5). Therefore remove a bunch of hacks
that were only needed on RHEL 5.
Because of previous automated commits, such as changing 'guestfs___'
-> 'guestfs_int_', several function calls no longer lined up with
their parameters, and some lines were too long.
The bulk of this commit was done using emacs batch mode and the
technique described here:
http://www.cslab.pepperdine.edu/warford/BatchIndentationEmacs.html
The changes suggested by emacs were then reviewed by hand.
When searching for virtio drivers, it makes no sense to fail if we
encounter a file without extension.
Ignore such files instead, just like we do for the files whose extension
doesn't match that of a driver-related file.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Our documentation of set-label says that labels for btrfs are limited
to 256 characters, while btrfs-filesystem(1) says:
Note
the maximum allowable length shall be less than 256 chars
which is indeed true, as trying to set a label with 256 characters
fails. Hence, fix our documentation to say that the limit is actually
255 characters.
Related to RHBZ#1164708.
Previously we used (on arm): -M virt -machine accel=...
After this change, we use: -machine virt,accel=...
On x86 there is no change.
The -M option is an abbreviation for -machine, and we can combine the
two options together. (Qemu combines the two options internally, but
it's more predictable if we do it ourselves).
Upstream qemu has supported the -machine option for over 5 years, and
in fact we implicitly relied on it (despite what the comment in the
code said).
- fix the syntax of hyperlinks
- replace the deprecation text with a simplier named list item, so it's
more visible
- use a named list item for the pointer to the C documentation of each
API
- add a named list item for the version of each API
If you've ever tried to use this option, you'll know that it didn't
work well. It broke random things (probably RHBZ#1020216, definitely
RHBZ#1023630), and caused random failures generally, while often not
actually failing when valgrind itself found problems.
Commit f6a790b67d changed the Makefile
so it only builds the tests when you run 'make check'.
If you didn't run 'make check' then (in particular) bindtests won't be
built, and so when you install it cannot be removed.
The simple fix for this is to make the removal of bindtests.*
conditional.
This fixes commit f6a790b67d.
Use the generator_built variable to list any files which are built by
the generator. This ensures they get automatically rebuilt.
Three files were missing from EXTRA_DIST, so they were not included in
the tarball, meaning you would have needed OCaml to build from the
tarball. You would have seen this when building:
cd .. && /builddir/build/BUILD/libguestfs-1.31.9/generator/generator
written gobject/docs/guestfs-title.sgml
written customize/customize-synopsis.pod
written customize/customize-options.pod
generated 476921 lines of code
Miscellaneous improvements to the ocamldoc:
- Generate more sub-headings.
- Document the object-oriented API.
- Use a common function to generate the doc for module and OO APIs.
Using 'ocamlfind ocamldoc' is much faster than running 'ocamldoc'
directly, because ocamlfind will run the native code program
'ocamldoc.opt' if it is available.
This change approximately halves the time taken to compile the ocaml
bindings.
Modify the copy-in/copy-out APIs to use the new guestfs_int_cmd_pipe
internal API.
This new API allows us to capture the stderr from the tar subprocess
if it fails, fixing RHBZ#1267032. The user will now see errors like
this:
$ guestfish -N fs -m /dev/sda1 copy-in '/tmp/d/*' / : ll /
libguestfs: error: tar subprocess failed: tar: *: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
Anything that doesn't involve pipes [parent-child communication] is
moved into the run_child function, so we can reuse the same code when
writing the new pipe APIs (see following commit). Anything involving
setting up pipes stays in the existing run_command function.
This is just refactoring.
Allow this useful function to be called from elsewhere.
This commit is almost, but not completely refactoring. I made a minor
change to the function so that it \0-terminates the returned data,
which is convenient for some callers.
When building from git, you need OCaml to compile the generator to
build the generated files. Previously configure didn't detect that
situation, resulting in failed builds. This commit causes the
configure script to error out if the generated files are not present
and OCaml is not installed.
Note that you can still build without OCaml, if you get the generated
files from somewhere else. The source tarballs distributed on the
website contain the generated files.
As in commit e102bcf3cf (for Perl), any
Python program has the handle program name field set to 'python'. Set
it to the true name (derived from sys.argv[0]).
When compiling the libguestfs bindings as a pip module, it's helpful
if you can compile the bindings with a different version of libguestfs
(eg. installed 1.28, pip module 1.30).
That broke the previous assumption that we were always compiling
against precisely the same version of libguestfs (which made sense
because the python bindings are part of the libguestfs source tree).
Python distutils has no way to check a C library to find out if
functions are defined. Therefore we have to rely on GUESTFS_HAVE_*
macros from <guestfs.h>.
Previously we only defined GUESTFS_HAVE_* macros for functions that
were not deprecated, test or debug functions. The reasoning for that
is given in this commit message [note that 'LIBGUESTFS_HAVE_' is the
old/deprecated macro]:
commit 2d8fd7dacd
Author: Richard Jones <rjones@redhat.com>
Date: Thu Sep 2 22:45:54 2010 +0100
Define LIBGUESTFS_HAVE_<shortname> for C API functions.
The actions each have a corresponding define, eg:
#define LIBGUESTFS_HAVE_VGUUID 1
extern char *guestfs_vguuid (guestfs_h *g, const char *vgname);
However functions which are for testing, debugging or deprecated do
not have the corresponding define. Also a few functions are so
basic (eg. guestfs_create) that there is no point defining a symbol
for them.
This wasn't in fact carried through very consistently, since when we
replaced s/LIBGUESTFS_HAVE_/GUESTFS_HAVE_/, we kept the old
LIBGUESTFS_HAVE_* macros, but defined them for every function. Oops.
This commit defines GUESTFS_HAVE_* for every function in <guestfs.h>,
making it easier to write the Python language bindings (see following
commits).
Allow the tests to be run on the installed libguestfs.so library and
on installed programs (but not installed language bindings, yet).
To run the tests, you have to have a copy of the libguestfs source
tree with exactly the same version as the installed libguestfs. You
have to build the libguestfs source tree. Then you do:
make installcheck
I investigated making automake 'make installcheck' work, but it's
basically broken (no surprise there).
This is derived from the old 'make-check-on-installed.pl'
script (which is also removed in this patch).
If either zcat or cpio fails when spawned in initrd-list, pclose will
return the actual return value of it, but reply_with_perror still uses
errno regardless; thus, the reported error is:
libguestfs: error: initrd_list: pclose: Success
which is not much helpful.
Instead, when pclose returns > 0, extract the actual return value of the
subprocess, and print that. Thus now we get for example:
libguestfs: error: initrd_list: pclose: command failed with return code 1
It's currently impossible to correctly predict the dcPath parameter
from the data that libvirt gives us. So allow the user to override
--dcpath themselves.
Eventually we will get better support in libvirt for this, and this
option will no longer be needed:
https://www.redhat.com/archives/libvir-list/2015-September/thread.html#00201
This enhances commit 51bc573d0c
and commit 20f1eb400b.
If a Linux guest has /etc/os-release (mandatory for systemd, so all the
recent Linux distributions have it), try to parse it for the distro, its
version, and its product name. If we cannot get all of them, the
detection as done so far continues.
Other than changes in the product_name string (which is free-form
anyway, and already documented to not be parseable), there should be no
regressions in the detected distro and version.
Filter out from the git ls-files output the files we don't care they
appeared (gnulib and intltools stuff), and use comm to only show the
remaining added files (i.e. files not being distributed).
The output of comm is redirected to file and then printed, so we can
check it is empty (failing the target if not).
Add a new "simplestreams" repository type, and a simple parser for
fetching and reading the JSON indexes of the Simple Streams v1.0
format.
Read only datatype=image-downloads contents, and only the latest
versions of each content available as disk image (disk.img or
disk1.img).
Add a simple test, using the "released" images from the CirrOS project.
Check if the Windows guest has Group Policy Objects installed, or one
of several popular pieces of anti-virus software. If we are
installing a virtio block driver, then experience has shown this may
cause a 7B boot failure.
Print a warning when this combination happens.
The warnings look like this:
[ 19.9] Converting Windows Server 2008 R2 Enterprise to run on KVM
virt-v2v: warning: this guest has Windows Group Policy Objects (GPO) and a
new virtio block device driver was installed. In some circumstances, Group
Policy may prevent new drivers from working (resulting in a 7B boot error).
If this happens, try disabling Group Policy before doing the conversion.
virt-v2v: warning: this guest has Anti-Virus (AV) software and a new virtio
block device driver was installed. In some circumstances, AV may prevent
new drivers from working (resulting in a 7B boot error). If this happens,
try disabling AV before doing the conversion.
virt-v2v: This guest has virtio drivers installed.
Change the handling of -ic, -if, -oc, -of, -on, -os, --password-file,
--vdsm-vm-uuid, --vdsm-ovf-output, --vmtype options, so that if any
appear multiple times on the command line we error out:
$ virt-v2v -i disk /tmp/centos-6.img -o null -of qcow2 -of raw
virt-v2v: error: -of option used more than once on the command line
Thanks: Juquin Zhou for finding the bug.
Turns out that some of the Windows guests in the test suite try to
reactivate if the date is more than some time delta (maybe 1 year?)
after the previous activation. Just disabling the network wasn't
enough to prevent this.
qemu allows us to boot guests with the clock set to a particular date
in the past, and libvirt exposes this (albeit very clumsily):
https://docs.fedoraproject.org/en-US/Fedora/18/html/Virtualization_Administration_Guide/sect-Virtualization-Tips_and_tricks-Libvirt_Managed_Timers.html
Allow guests to be booted with the date set to a particular past date,
by setting the test_plan.guest_clock field.
This required adding some more libxml2 bindings.
When calling test_connection to test the connection to a conversion
server, we also query for the remote server's version and driver
information (using 'virt-v2v --version' and
'virt-v2v --machine-readable').
However we didn't clear the existing information from previous runs.
One consequence of this was that if you used the back button in the
GUI, the list of output drivers would be repopulated two or more
times. https://bugzilla.redhat.com/show_bug.cgi?id=1227609
Another is that connecting to a different server could confuse
test_connection into thinking the connection was successful (because
of data from the previous run) when in fact it was not.
Thanks: Tingting Zheng for finding the original bug.
Split in an own function the code dong the mounting, the inspection of
the kernel files, and the downloading, including a per-OS function for
the actual kernel files. This gives few advantages:
- the download phease is not repeated for all the files to fetch
- it is easier to eventually support multi-root disk images
- it is possible to support OSes different than Linux; virt-get-kernel
now will just report the unsupported OS, instead of a generic
"no kernel found" message
This is mostly code refactoring, with (on Linux) no actual behaviour
change.
Make print_version_and_exit, long_options, display_short_options, and
display_long_options private, as set_standard_options now takes care of
handling the job for the common command line options.
Move --debug-gc as common option for all the OCaml-based tools, even a
couple of them which didn't have it previously.
As implementation note, make set_debug_gc private to
set_standard_options, as it needed to be moved otherwise, and it is no
more required as public function.
Introduce a new common helper to add the common options for libguestfs
tools (short/long options, version, verbose, trace), and sort them.
All the OCaml-based tools had these options already, so there are no
functional changes in the interface they provide.
The only difference is that now the options are always sorted, while
some tools didn't had them like that previously: because of this, a
couple of ditto markers (as descriptions) don't match what's above them
anymore, and thus their full description is put instead.
When running (eg) dnf on a 32 bit i686 guest when the host is 64 bit
x86_64, dnf believes it is running on a 64 bit machine and so tries to
install x86_64 packages. We can 'trick' dnf into believing it's a 32
bit machine using the setarch program.
$ virt-builder fedora-22 --arch i686 --install 'gperf'
...
[ 27.4] Installing packages: gperf
...
Running transaction test
Error: Transaction check error:
package libgcc-5.1.1-4.fc22.x86_64 is intended for a different architecture
...
The use of a heredoc to solve quoting issues comes from:
http://stackoverflow.com/a/3435460
Thanks: Jan Sedlák for finding the solution.
In -i libvirtxml, -i ova and -o libvirt drivers, replace the ad hoc
xpath_to_* functions with use of the new xpath convenience functions
introduced in the previous commit.
This is not entirely refactoring because I fixed a few bugs found by
type safety.
Previously given a path such as:
vpx://vcenter/Folder/Datacenter/esxi
we calculated dcPath=Folder. However this is obviously wrong. We
should chop the path at the final (esxi) element to give
dcPath=Folder/Datacenter.
In vCenter, the datacenter is identified by a path, and the parameter
used for this is called 'dcPath'. Rename the function to avoid any
confusion about what we're getting here.
This is just renaming function/variable names and has no other effect.
Several tools handle --debug-gc command-line option, by explicitly
forcing GC on every exit path. This is tedious and prone to forgetting
some of the exit paths.
Instead, add a generic handler for --debug-gc, which installs an at_exit
hook to do the GC consistency check, and which can be called right in
the command-line parser. Also adjust all users of --debug-gc to use
that handler.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Build the P2V disk image and boot it. We don't actually use PXE
specifically, but we do test the whole PXE / kernel command line path
much more thoroughly.
This is a 'check-slow' test because it takes ages to run.
When using the virt-p2v ISO in command line mode, we did not wait for
the network to come online before starting virt-p2v. Therefore
virt-p2v could exit with an error when testing the ssh connection (or
on the other hand, it might work randomly). If the user logs in and
runs 'launch-virt-p2v' by hand, then it would usually work because the
network had been brought online in the meantime.
Fix this by waiting for NetworkManager to bring the connection online
before calling test_connection(). Note that the obvious way to fix
this (changing the systemd service to wait for network-online.target)
does *not* work - I added a comment to the service about this.
Thanks: Tingting Zheng
Raise a warning for each kernel specified in grub which does not
actually exist, keep going on with the conversion using the remaining
(existing) kernels.
Put in a list the errnos to expose, filling the content of the
Guestfs.Errno submodule from that.
Also, generate a separate guestfs-c-errnos.c with the implementations of
the functions returning the errno codes.
Only code motion and refactoring, no actual changes on the content of
the ocaml Guestfs module.
pcre < 8 has limitations on the types of partial matching
possible. As noted in the pcrepartial(3) man page, these can
be worked around by adding non-capturing parentheses around
various subexpressions, as I have done here.
Apparently gtk_init_check doesn't (always?) print an error message
when it returns false, so virt-p2v just exits without any message.
This was observed on RHEL 6 when $DISPLAY was not set.
File input_libvirt_other.ml, line 1, characters 0-1:
Error: The implementation input_libvirt_other.ml
does not match the interface input_libvirt_other.cmi:
Class declarations do not match:
class virtual input_libvirt :
'a ->
string option ->
string ->
object
method adjust_overlay_parameters : Types.overlay -> unit
method as_options : string
method virtual source : unit -> Types.source
end
does not match
class virtual input_libvirt :
string option ->
string option ->
string ->
object
method adjust_overlay_parameters : Types.overlay -> unit
method as_options : string
method virtual source : unit -> Types.source
end
A parameter has type 'a but is expected to have type string option
When the output format is qcow2 and -oa preallocated is used,
previously it would only allocate the metadata. That was a regression
over previous behaviour of virt-v2v 0.9. Change it so it allocates
the full file size in this case.
For raw, this allows "off" as a synonym for "sparse" (to make it
consistent with qcow2).
For qcow2, this allows "sparse" as a synonym for "off".
It also adds qcow2 "full" preallocation, which is actually mapped to
the qemu option "falloc" (see arguments about this on the qemu-devel
mailing list, which we lost).
This also updates the test.
Normalize the target architecture, and also each architecture when
checking for a compatible image.
This sort of reverts the effects of
commit 8864c47b94, but at least it is
possible to build e.g. Debian-based amd64 images on any x86_64 system
without being considered as foreign architecture.
Small helper to normalize an architecture string, so it is easier to
compare them and check for compatibilities.
Make use of it in guest_arch_compatible, simplifying it.
If a function name, with its trailing parentheses, is in the
environment , trying to unset it will error out with a message of "not
a valid identifier". Handle this as a special case, and unset it with
the -f option which can handle the parentheses in the supplied
identifier.
It turns out that -- some of the time -- systemd ignores unit files
which are not placed in one of the official unit directories. This
caused the service to be timed out and killed when the systemd daemon
was reloaded, as could happen for long-running package installs at
first boot.
Thanks: Marius Vollmer for diagnosing the problem and finding the
solution.
If the format of the disk image of the template is not specified in the
index, virt-builder would print
Converting auto to qcow2
when asking to generate a qcow2 image. Instead, in this case print only
the destination format, like
Converting to qcow2
I discovered that 'dnf upgrade' doesn't actually upgrade to the latest
versions of packages unless you also supply the '--best' flag.
This also changes update -> upgrade, since apparently 'dnf update'
is deprecated.
Because this 'feature' is broken (since 2013):
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=13928
we have to replace all instances of $(top_srcdir) in *_SOURCES lines
with a relative path. According to what I read, this shouldn't break
split builds, but I didn't test it.
The only things automake moans about now are:
* Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/\${ <-- HERE ([^ \t=:+{}]+)}/ at /usr/bin/automake line 3936.
- This is another bug in automake
* autoreconf: configure.ac: AM_GNU_GETTEXT is used, but not AM_GNU_GETTEXT_VERSION
- This is a feature, not a bug.
dhcpcd requires an interface name as parameter to work. We are now
getting it from /proc/sys/net/ipv4/conf/ folder children. dhclient
on Debian also has the problem, thus use the guessed interface name
for it too.
Instead of rm + touch, just truncate the file if existing.
In case there was no file previously, now nothing is created, as most
probably the system was fine even without it.
Also, turn the single path into a list, so more machine-id files can be
handled at once.
When importing a key, read the list of the valid subkeys of it, and use
it to check whether a signature was done by one of them.
This allows index provides to sign them using a subkey instead of the
main key.
Extend Index_parser.entry to hold a list of checksums to validate, and
validate all of them.
This does change nothing currently, as only sha512 is read, while still
allowing us to fetch more checksums if needed.
Introduce a new Checksums module to handle the check of checksums,
moving part of the Sigchecker code to it.
Adapt the rest of virt-builder to this new module.
When warning that there is no known default password encryption for the
current guest, print also its minor version (help debugging unsupported
guests).
Also, fix wrapping (and indentation) of the message.
I also sorted the list.
Between libguestfs 1.28 and 1.30, the appliance grew from 95MB to 213MB.
Using guestmount and filelight (see link below) I could see that the
main contributor was these two directories, which should not be
necessary.
With this change, the size goes down to 119MB.
See also:
https://rwmj.wordpress.com/2015/07/23/why-has-the-libguestfs-appliance-grown-by-118-mb/
Run `qemu-nbd --help` to know whether it supports --format, and if so
pass it explicitly. This avoids warnings with newer qemu versions,
where they really recommends to explicit the format of disk images being
specified on command line.
virt-p2v looks in /sys/block to find disks, and ignores the hard disk
containing the root device. This is correct when virt-p2v runs off
the ISO, but when running the tests on a machine that has a single
hard disk, all_disks would be NULL, resulting in a test failure.
Fix this by allowing all_disks to be NULL, but adding an extra check
later so that we don't perform the conversion with no hard disks (the
gui.c path already has this check, so it is only needed in the
kernel.c path).
Move the read_password_from_file helper in Password to mllib with a more
generic name, and use it in place of the former.
Also, use it in v2v instead of reading the whole file contaning a
password: given that the documentation says that the whole content is
used, there will not be newlines in the password file, so the behaviour
will be preserved. The oly difference is that newline is no more an
acceptable character for passwords, but that is a really unlikely
(if not impossible at all) situation.
We are no longer planning to ship virt-v2v as an appliance. In RHEL 7.2
it will be included in the base RHEL Server product.
This reverts commit 4e4ece5364.
If UEFI is required by the guest, but not supported by the host, then
you wouldn't see an error message until after copying.
Add an additional method to the output object so we can check this
before copying, to avoid a long wait.
Thanks: Junqin Zhou
https://bugzilla.redhat.com/show_bug.cgi?id=1184690#c22
When converting the old Perl virt-v2v code, I made a silly mistake
with the exception that aug_get throws when it doesn't find any node.
It throws a 'Guestfs.Error' exception, not 'Not_found'.
As a result of this, the exception was escaping and the proper error
message was not displayed. With a malformed grub configuration you
would see this error:
$ virt-v2v -i disk centos-6.img -o null
[...]
virt-v2v: error: libguestfs error: aug_get: no matching node
After applying this patch:
$ virt-v2v -i disk centos-6.img -o null
[...]
virt-v2v: error: no kernels were found in the grub configuration.
This probably indicates that virt-v2v was unable to parse the grub
configuration of this guest.
which is the correct error message.
Updating gnulib has caused -Wformat-signedness to be enabled. This
has revealed many problems in C format strings. The fixes here fall
into the following main categories:
- Using %d with an unsigned parameter.
- %x and %o expect an unsigned argument.
- uid_t and gid_t are unsigned on Linux. The safe way to print these
is to cast them to uintmax_t and then print them using the %ju
modifier (see http://stackoverflow.com/a/1401581).
- Using %d to print an enum. Since enums may be either char or int,
I fixed this by casting the enum to int.
- strtol_error & lzma_ret are both unsigned types.
Pass the target_buses assignment to the output#create_metadata method.
Now output modes have a choice: they can either ignore the new
parameter and continue to use the flat list of targets. This is
suitable for output modes that cannot model multiple buses (eg.
-o glance) or can model it but don't bother (currently -o rhev).
Or they can ignore the flat targets parameter and use the new
target_buses parameter, translating that into the appropriate list of
devices. This is implemented in this commit for these modes:
-o libvirt
-o local
-o qemu
The target VM will have several buses to which disks can be attached.
Commonly it will have an IDE bus, and possibly a virtio-blk "bus" (not
really a bus) and/or a SCSI bus.
Virt-v2v does not model this at the moment. Disks are just added to
the output XML in the order that we write them, and so they can move
around with respect to the target VM.
This commit introduces the idea that we should model the target disk
buses, and we should try to assign fixed and removable disks to slots
on each of those buses. In this commit, the modelling is not used by
any output mode, but that will be fixed in the next commit.
When we see libvirt source XML for a removable drive like this:
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<target dev='hdc' bus='ide'/>
</disk>
as well as recording the bus (s_removable_controller) as we do at the
moment, also record the slot number (eg. hdc == 2 in the example
above).
Do the same for OVA input files.
They create an own partition layout, so don't do that over the
testing-created layout.
At least for me, this fixes parted issues when creating the new
partition layout and the old one is still reported as in use.
Followup/fix for commit 52d1b0c593.
When cleaning the directories with cron/at jobs, remove only files
there, as subdirectories might be used by other systems; for example
in Debian under /var/spool/cron/ there is the atjobs subdirectory with
the actual at queue.
Make sure to not remove .SEQ files anymore, as they represent the at job
counter which is needed by the at daemon. Instead, reset these files to
0.
Furthermore, add also the path to the .SEQ location in Debian-based
systems.
Collect this small snippet to get the part of a string after the last
occurrency of a character; replace with it the current snippets doing
the same.
Should be just code motion.
When checking if a Windows variant is a server variant, assume it is a
server variant unless the i_product_variant field is "Client".
Windows 2008 and Windows 2008R2 does not have an InstallationType key
in the registry at all, so for these the product variant is returned
as "unknown".
Windows "Core" would have, according to the reference below,
"Server Core".
https://msdn.microsoft.com/en-us/library/ee391629%28v=vs.85%29.aspx
This fixes commit 47b5f245be.
"my" variable $output masks earlier declaration in same scope at /home/libguestfs/tests/daemon/test-btrfs.pl line 66.
"my" variable @r masks earlier declaration in same scope at /home/libguestfs/tests/daemon/test-btrfs.pl line 72.
Signed-off-by: Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
This refines the previous commit by shutting down the daemon cleanly
at the end of the test (assuming the test was successful). It
repurposes the 'internal_exit' API for this, which was previously used
by the now defunct --enable-valgrind-daemon functionality.
This allows us to test the daemon running as a host process, allowing
us to meaningfully test it using valgrind.
This commit only adds a single test that check that the daemon starts
up, can be pinged, and exits.
Instead of parsing /proc/cmdline from the daemon, move all of that
parsing into the init script, and pass the argument via the daemon
command line.
For example, previously the daemon and init script both looked for
guestfs_network=1 in /proc/cmdline. Now the init script still looks
for it, and if found it runs `guestfsd --network'.
This option, used for testing, causes the daemon to create the Unix
domain socket (from guestfs_channel), listen on it, and accept a
single connection.
We'll now just require that valgrind is new enough.
As best as I can tell from the valgrind subversion(!) repository, this
option was added to valgrind in around 2011.
If you free an xmlDocPtr before any xmlXPathObjectPtrs that reference
the doc, you'll get valgrind errors like this:
==7390== Invalid read of size 4
==7390== at 0x4EB8BC6: xmlXPathFreeNodeSet (xpath.c:4185)
==7390== by 0x4EB8CC5: xmlXPathFreeObject (xpath.c:5492)
==7390== by 0x400A56: main (in /tmp/test)
==7390== Address 0x60c0928 is 8 bytes inside a block of size 120 free'd
==7390== at 0x4C29D2A: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7390== by 0x4E8784F: xmlFreeNodeList (tree.c:3683)
==7390== by 0x4E87605: xmlFreeDoc (tree.c:1242)
==7390== by 0x400A4A: main (in /tmp/test)
The following simple test program demonstrates the problem:
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#include <libxml/xpath.h>
int
main (int argc, char *argv[])
{
xmlDocPtr doc;
xmlXPathContextPtr xpathctx;
xmlXPathObjectPtr xpathobj;
doc = xmlReadMemory ("<test/>", 7, NULL, NULL, XML_PARSE_NONET);
assert (doc);
xpathctx = xmlXPathNewContext (doc);
assert (xpathctx);
xpathobj = xmlXPathEvalExpression (BAD_CAST "/test", xpathctx);
assert (xpathobj);
xmlFreeDoc (doc);
xmlXPathFreeObject (xpathobj);
xmlXPathFreeContext (xpathctx);
exit (EXIT_SUCCESS);
}
In virt-v2v we were not freeing up objects in the correct order,
because we didn't express the dependency between objects at the C
level into the OCaml, where the OCaml garbage collector could see
those dependencies. For example code like:
let doc = ... in
let xpathctx = xpath_new_context doc in
let xpathobj = xpath_eval_expression xpathctx "/foo" in
might end up freeing the 'doc' (xmlDocPtr) if, say, there were no
further references to it in the code, even though the 'xpathobj'
(xmlXPathObjectPtr) remains live.
To avoid this, we change the OCaml-level representation of objects
like xpathobj so they contain a reference back to the higher-level
objects (xpathctx & doc). Therefore holding an xpathobj means that
the doc cannot be freed.
However that alone is not quite sufficient. There is a further
problem when the program calls Gc.full_major, Gc.compact etc., or even
just when xpathctx & doc happen to be freed at the same time. The GC
won't necessarily free them in the right order as it knows both need
to be freed but doesn't know that one must be freed before the other.
To solve this we have to move the finalisers into OCaml code, since
the OCaml Gc.finalise function comes with an explicit ordering
guarantee (that finalisers are always called in reverse order that
they were created), which the C-level finaliser does not.
The 'note:' comment is added, as below:
<Section xsi:type='ovf:DiskSection_Type'>
<Info>List of Virtual Disks</Info>
<!-- note: actual_size field is estimated -->
<Disk ovf:actual_size='7' [...]/>
</Section>
This allows you to append one file to another:
copy-file-to-file /input.txt /output.txt append:true
will append the contents of /input.txt to /output.txt.
This makes several changes to the handling of virtio-win drivers:
The VIRTIO_WIN_DIR environment variable has been renamed
VIRTIO_WIN (but you can still use the old name).
You can point the VIRTIO_WIN either at a RHEL virtio-win directory
(ie. /usr/share/virtio-win), OR at a loopback-mounted virtio-win ISO,
OR at the virtio-win.iso file itself. In the latter case, libguestfs
is used to open the ISO file and read drivers from it.
The code is more flexible about the pathnames of drivers, because the
paths in the ISO are completely different from the paths in RHEL
/usr/share/virtio-win.
Modify the function from a fixed delimiter to a variable. So,
it can be used in more APIs later. Also modified the existing
callers
Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Gerd is making EDK2 builds available here:
https://www.kraxel.org/repos/
These are convenient for Fedora/aarch64 users, so use this firmware if
available.
This commit also refactors the UEFI searching code.
In commit 4c73d1d4f1, I changed the
behaviour of virt-v2v so it ignores all errors from the
g#part_get_parttype method. However that would ignore I/O errors and
similar.
Only ignore the "unrecognised disk label" error from parted, by
testing if the return code is EINVAL.
This fixes commit 4c73d1d4f1.
See: https://bugzilla.redhat.com/show_bug.cgi?id=1232241#c3
However adding the -s parameter changes the error code returned by
parted when it processes a blank disk (or other unrecognized partition
table). Without -s it prints an error but returns a non-error exit
code (0). With -s it prints an error and returns an error exit code.
Because it's useful to be able to catch this case in user code, turn
"unrecognised disk label" into EINVAL.
Change virt-alignment-scan to catch this error and ignore it.
Mold split_lines_sb from split_lines, so it returns the strings buffer
with the result of the split. This way, we can have the number of lines
in the array, with no need to count them again later.
split_lines is rewritten to take the ownership of the result of
split_lines_sb.
If we are not interested in output of a command being run, then there's
no need to still capture it in a (automatically freed, but still)
variable.
If any of those outputs is needed, a variable for it can be easily added
back.
Move the "FIXED APPLIANCE" section from
libguestfs-make-fixed-appliance(1) to the "INTERNALS" section of
guestfs(3), so it's more visible and less hidden in the documentation
of a rarely-used tool. Also slightly improve it.
Expand the FAQ question about libguestfs without supermin, mentioning
the build options needed and pointing to the aforementioned new section.
When hitting a filesystem which we cannot or don't know how to expand,
instead of print a warning in verbose mode only, print it always when
showing the summary of the changes.
In the virt-resize documentation, add a paragraph about this warning.
Thanks to: Luigi Toscano.
Extract the guest kernel/ramdisk extraction from virt-builder into a
separate utility, so it can be used and improved without cluttering
virt-builder.
Currently it does what virt-builder --get-kernel did, adding also the
options for remote disks and libvirt access, much like other libguestfs
tools.
virt-builder --get-kernel now just spawns virt-get-kernel.
Done using a sequence of regular expressions like this:
perl -pi.bak -e 's{C</}{F</}g' `git ls-files \*.pod` generator/actions.ml
perl -pi.bak -e 's{C<C:\\}{F<C:\\}g' `git ls-files \*.pod` generator/actions.ml
[etc]
and then tediously checking every change by hand.
This also adds a couple of deliberately undocumented (and unsupported)
command line parameters to make testing simpler:
p2v.skip_test_connection - don't try to test the connection
p2v.dump_config_and_exit - after parsing command line, print it and exit
This updates commit 716244c337.
The sscanf call always failed because we checked for the wrong return
code.
We don't need to allocate two characters for the mem_code field since
sscanf doesn't append a nul byte.
This commit also allows you to use lowercase 'm' or 'g' as a memory
unit.
Also clarify the documentation: the memory unit must be given.
Fix the parsing of /proc/cmdline, including allowing double quoting of
parameters.
This allows you to pass parameters to p2v on the command line which
include spaces.
When convering a POD documentation, possibily combining various
sub-documents together, simplify the L<> links that explicitly point to
the very same documentation being generated.
For example, when generating the virt-builder documentation, links like
L<virt-builder(1)/SECTION>
will be turned into
L</SECTION>
thus not making Pod::Simple parse them as external reference.
While it is a slightly crude hack, it seems there's no easy way to
process the link parsing with Pod::Simple subclasses.
When using the kernel command line to configure virt-p2v, if any p2v.*
parameter appeared right at the end of the kernel command line, then a
bogus \n would be appended. Remove this.
This is essentially the same fix as:
4819655b3c
I'm trying to think if there was any reason why we would have set
g->last_errnum directly instead of using the proper function
(guestfs_int_error_errno), but I cannot think of one.
Since error (g, ...) is just a macro that calls
guestfs_int_error_errno (g, 0, ...), this code is equivalent.
Currently if /tmp (on the host) is a symlink, then the symlink is
copied into the appliance, probably pointing to a non-existent
directory, and everything goes downhill from there.
Avoid this by making sure that /tmp and /var/tmp are real directories.
In newer CentOS 7 versions /etc/redhat-release says that the distro is
derived from RHEL, so we need to look at /etc/centos-release for
actually identifying it as CentOS.
The old code is needed as sub-case of /etc/redhat-release, as on
CentOS < 7 that file is a symlink to /etc/centos-release.
In RHEL-based distros, the provided icons have (obviously) different
sizes than the RHEL ones, used in icon_rhel as reference.
Since 100K should be a reasonable threshold for avoid keeping a
per-distro list of limits, just use it as only size limit.
Add a new guestfs_int_merge_fs_inspections() function that merges the OS
inspection information of two inspect_fs instances into one. This
function is useful if the inspection information for an OS are gathered
by inspecting multiple filesystems.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Instead of having virt-p2v-make-disk and virt-p2v-make-kickstart
separately contain a list of the virt-p2v ISO dependencies, combine
the dependencies into a single file ('p2v/dependencies.m4') and use
that to generate the respective dependencies in each script.
After fuzzing 'qemu-img info' I found that certain files can cause the
command to use lots of memory and time. Modify the command
mini-library to allow us to place resource limits on subprocesses, and
use these to limit the amount of space and time used by 'qemu-img info'.
This change was generated automatically using the following Perl
script, with a few hand corrections afterwards.
use strict;
my %map;
open ADDED, "src/api-support/added" or die "added: $!";
while (<ADDED>) {
chomp;
if (/^guestfs_(.*)_(argv|va|opts) /) {
# ignore _argv, _va, _opts entries.
}
elsif (/^guestfs_(compare|copy)_/) {
# ignore guestfs_compare_*, guestfs_copy_* struct functions.
}
elsif (/^guestfs_(.*) [01]\.(\d+)\.(\d+)$/) {
$map{$1} = [1, $2, $3];
}
elsif (/^guestfs_(.*) 0\.(\d+)$/) {
$map{$1} = [0, 0, $2];
}
else {
die "cannot parse line: `$_'\n"
}
}
close ADDED;
open ACTIONS, "generator/actions.ml" or die "actions.ml: $!";
open ACTIONS_OUT, ">generator/actions.ml.out" or die "actions.ml.out: $!";
while (<ACTIONS>) {
chomp;
if (/name = "(.*)";$/ && exists $map{$1}) {
my @v = @{$map{$1}};
$_ .= sprintf (" added = (%d, %d, %d);", $v[0], $v[1], $v[2]) ;
delete $map{$1};
}
print ACTIONS_OUT "$_\n";
}
close ACTIONS_OUT;
close ACTIONS;
In APIs such as guestfs_download, when the FileOut parameter exactly
matches "/dev/stdout" or "/dev/stderr", don't reopen the possibly
redirected output file with O_TRUNC (truncate). Instead dup the file
descriptor.
This magic behaviour doesn't happen for /dev/fd/* (or any other output
file) allowing callers the choice of using /dev/stderr or /dev/fd/2
depending on whether or not they want truncation.
This works around an annoying virt-builder bug. If you do:
$ virt-builder fedora-21 --install no_such_package -v -x >& /tmp/log
then when the `--install' command fails, virt-builder will download
the log file using `guestfs_download (g, log, "/dev/stderr")'. Since
this truncates the redirected /dev/stderr, the final log file is
truncated and corrupted.
With this patch the log file is no longer corrupted.
Previously it was only being pulled in implicitly. However virt-p2v
runs this binary from the "Configure Network" button.
This fixes commit e464774a79.
This patch will add fields of sector size for:
- partition sector data size
- target partition sector data size
Signed-off-by: Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
In cases where we are asked to run commands in the guest (eg. options
such as --run-command or --install), give a clear error in the cases
where the guest arch is not compatible with the host arch.
Similar code existed in virt-builder, but I have removed that. Users
will still get an error message, it will just happen a bit later on.
There is a slight change in semantics here, in that architectures are
no longer normalized when matching, but that's probably fine since
`virt-builder -l' prints the exact arch string that people should use.
When you get an error in tools, it will print a message like:
If reporting bugs, run virt-v2v with debugging enabled and include the
complete output:
virt-v2v -v -x [...]
Only print this message if -v or -x were not already specified on the
command line.
This large commit is just code refactoring. Instead of having
every OCaml tool define 'prog' the same way, always as:
let prog = Filename.basename Sys.executable_name
move that into a single place, Common_utils.prog. Then we can use
that global value everywhere else, instead of having to pass it as a
parameter into a dozen different functions.
The common error function already prints the program name, so
we don't need to print it twice.
Before:
$ virt-resize --expand ""
virt-resize: error: virt-resize: empty --expand option
After:
$ virt-resize --expand ""
virt-resize: error: empty --expand option
This displays the MAC address and vendor next to each network
interface, aiding users in determining which network cards they want
to transfer to the virtualized machine.
The man page for getline says:
ssize_t getline(char **lineptr, size_t *n, FILE *stream);
[...]
If *lineptr is set to NULL and *n is set 0 before the call, then get‐
line() will allocate a buffer for storing the line. This buffer should
be freed by the user program even if getline() failed.
which seems to indicate that we must initialize both line and len to 0
before the first call to getline.
In several places we were not initializing len. The program still
worked fine, but it seems better to initialize the length anyway.
Show partition sector data and target partition sector data
in debug_partition.
Also, if --verbose, call debug_partition again before
parted partitions.
Signed-off-by: Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
RWMJ: Changed ~sectsize:sectsize => ~sectsize
Create a single ocaml-link.sh script, which supports a -cclib parameter
so it can be used instead of the per-project link.sh scripts.
As result, the libraries for each OCaml application can be moved back to
each Makefile.am.
The new behaviour is as follows:
Skip all of the fuse tests:
SKIP_TEST_FUSE_SH=1
Individual tests can be skipped by setting:
SKIP_TEST_FUSE_UMOUNT_RACE_SH=1
SKIP_TEST_GUESTMOUNT_FD=1
SKIP_TEST_GUESTUNMOUNT_FD=1
SKIP_TEST_GUESTUNMOUNT_NOT_MOUNTED_SH=1
When running gpg to validate a signature, use also --batch in
non-verbose mode to avoid the extra "Good signature" messages that are
printed by gpg2 even in quiet mode.
We are parsing the status output anyway to check the validation result,
so this should be harmless (and the full gpg output is printed in
verbose mode anyway).
In every SigChecker, trust by default the keys imported.
This should make gpg happier when using the keys later, used only when
validating the signatures of index files.
Since RHEV 3.3, OS variant names like "RHEL6x64" were deprecated and
replaced by names like "rhel_6x64". "RHEL7x64" was never valid, and
we should have used "rhel_7x64". "Windows2012R2x64" was also never
valid and should have been "windows_2012R2x64".
For backwards compatibility, RHEV still understands the old names, and
we still pass them for compatiblity with old RHEV.
However for the new OS variants supported by virt-v2v in RHEL 7.2, use
the new names.
This commit also adds support for SLES and ppc64.
This updates commit 3c51302d69
and also RHBZ#1213324.
This is for consistency with the rest of the documentation.
This commit also adds the files to EXTRA_DIST so they are shipped in
the tarball.
This updates commit 4e4ece5364.
The old version of virt-p2v had a whole custom-written dialog which
interacted with NetworkManager over dbus. After trying that approach,
it's really complex to get right.
Instead this button simply opens NetworkManager's connection editor.
This also adds nm-applet to the disk and starts it. However nm-applet
does not display any visible indication -- probably because we are
lacking a system tray.
Matchbox only supports a single window on screen. This worked fine
until we added the NetworkManager connection dialog -- which tries to
open in a new top level window, and of course fails.
Metacity supports multiple top-level windows so doesn't suffer this
problem.
Apart from that it's hard to tell the difference between the two.
Display a warning if the number of vCPUs or memory selected in the GUI
would be larger than the limits currently supported on RHEL 6 and RHEL 7.
This is just a warning -- the user is free to ignore it and continue
anyway.
This warning does not apply if the selections were made through the
kernel command line, because there is no place to display the warning
for automated conversions.
Add the Types.target_firmware type, which stores our final decision
for whether the guest requires BIOS or UEFI on the target.
Not all output modes support UEFI. In order to fail as early as
possible if conversion isn't going to be possible, there is an
output#supported_firmware method which returns the list of supported
target_firmware.
Add the target_firmware parameter to output#create_metadata so it can
select the correct firmware in the final metadata.
Use a heuristic to detect if the guest could boot with UEFI.
This is only used where we have missing metadata (in the
source.s_firmware == UnknownFirmware case). Currently that only
applies for `-i disk' and `-i libvirt/libvirtxml'.
Eventually we'll be able to get this information from the libvirt
metadata (RHBZ#1217444), so it'll only be used for `-i disk'.
Also modify the input_* drivers so they pull out the firmware from the
metadata. Currently this is only possible for `-i ova', since libvirt
does not expose it (RHBZ#1217444).
Old virt-v2v had a bunch of code for removing EFI configuration from a
Linux guest, and replacing it with grub. I copied this code to new
virt-v2v but it was never tested.
Now that OVMF exists, we can simply boot EFI guests directly on KVM.
Thus these hacks can finally be removed.
This test fails on recent kernels, sometimes.
Apparently calling 'fusermount -u mp' can exit with an EBUSY error,
but still unmount the filesystem. Or possibly guestmount crashes
coincidentally. It's impossible to debug because debugging tools like
strace prevent fuse from working at all.
guestunmount had a -v / --verbose option, but it didn't change the
behaviour of the program in any way.
Make it print the invocations of the underlying fusermount program.
Collect sound card information from the source, and where possible,
create a compatible sound card on the target.
Notes:
* VMware's libvirt driver, and also OVF files, do not appear to
contain any sound card information, so it cannot be collected from
VMware sources.
* Xen does emulate sound cards and makes that information available
through libvirt XML.
* There are no paravirt drivers for sound that I'm aware of.
Therefore we can just copy the same sound model to the target (so
the sound device does not appear to change). If the target, KVM,
does not support the device, it is dropped. But ...
* ... Unfortunately we cannot easily tell which sound cards are
supported by KVM on the target. This is especially a problem for
RHEL, where many sound drivers have been removed. There is a
convenience function, `Utils.qemu_supports_sound_card', which can be
modified by packagers to hard code the list of supported sound
cards.
* If a sound card is dubious / not supported by the target / has any
other problem, then we drop it, since it is more important that the
guest boots on the target than that sound works.
virt-v2v understands relative paths in libvirt XML (even though they
are not legal and libvirt itself wouldn't generate it). We don't need
to generate this file.
Create a version string that includes the "extra" string as well, and
make use of it in print_version_and_exit (which can avoid creating a new
Guestfs handle).
Folloup of commit 478a552ab4.
fstrim is an optimization. If it fails, it's not fatal (although
virt-v2v will run a lot more slowly).
We also expect that it will fail for non-aligned NTFS partitions found
on old versions of Windows, and in that case there's nothing that can
be done about it.
Therefore only emit the warning when we are debugging.
When generating OVF (eg. for -o rhev, -o vdsm) we have to put the
operating system type into a particular field of the output, in a
format that oVirt/RHEV will understand. Add further Windows OS
variants to this list. The variants are derived by examining
ovirt-engine sources.
Thanks: Tingting Zheng for testing Windows conversions to RHEV and
finding this bug.
Rename test-v2v-i-ova-zip.sh (and associated data) to
test-v2v-i-ova-formats, and generalize it a bit so it can test different
compression types for the ova "envelope".
Since I created these with virt-builder without specifying the format,
they have format=qcow2 (not raw). Instead of tediously recreating or
reuploading these images, just change the metadata.
Do not rely on .ovf and .mf files being in the top-level of the ova
archive, but search them anywhere within the content of the ova.
This also changes the result of the search of the .ovf file: previously,
one (random) file was picked in case there were more than one, while now
this situation triggers an error.
Related to RHBZ#1186800.
When dealing with a ova detected as gzip of xz, uncompress few bytes of
it to check whether it is a compressed tarball, and if so untar it.
Related to RHBZ#1186800.
Factor out the connection and pool loading out of v2v_pool_dumpxml, so
it can be used in later implementations requiring a pool.
Should be just code motion.
Read the port configuration from the XML of libvirt domains, restoring
it when writing new libvirt XMLs instead of always setting the
"autoport" option.
Extract them to find out the architecture of the data they hold.
Useful to detect the architecture of e.g. compressed Linux modules.
Provide in the test.iso two samples (compressing existing test data) of
binaries compressed with gzip and xz.
vCenter 5.0 was released in 2011. We have not tested against any
earlier versions, but it was reported that it does not work with
vCenter 4.
Thanks: Sokratis
I don't think it's worth building the test harness as a separate
Fedora package. It just means the harness gets out of date
quickly and is hard to install. Instead the test cases can now
be configured to use the libguestfs source directory to get the
test harness.
Grub2 in RHEL 6 responds to any key including shift keys by entering
the grub menu (and thus waiting for the 'user' to press Enter). This
means our previous strategy of always hitting the shift key before
taking a screenshot didn't work well.
Instead, take a screenshot, if it's all (or mostly) black, hit a shift
key and take another screenshot.
Previously we fuzzy-matched on screenshots of the entire display.
There were several problems:
- dates/times in images required fuzzy matching, but it's not clear
how much fuzz to use (Windows 8.1/2012R2 timestamp is huge,
requiring an enormous amount of fuzz)
- cannot cope with Windows 7 irregular placement of windows on
the desktop
It's better to match on subimages.
This requires some changes to the test data, using gimp to clip the
images and remove any changing elements (dates/times/etc) but in the
long run it's more flexible than fuzzy matching on the whole display.
Note that because of peculiarities in 'compare' we still need to have
a little bit of fuzz in the matching. Seemingly bit-for-bit identical
images still have a similarity > 0.
Take screenshots each time around the loop (every 10 seconds). These
are very useful for debugging.
Check for the final screenshot on every iteration. If the final
screenshot is encountered then the guest is terminated immediately.
Reverse a few loop tests to try to make the code more readable.
unxz --keep fails on the symlinks used by git-annex with this
rather unnecessary error:
unxz --keep 'rhel_511_x86_64_pv.img.xz'
unxz: rhel_511_x86_64_pv.img.xz: Is a symbolic link, skipping
Use xzcat instead.
Make the location of the Windows virtio drivers overridable with the
environment variable VIRTIO_WIN_DIR, in the same vein as is done for
virt-tools.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
Add p_mbr_p_type as member of type partition
to describe mbr partition type.
Currently we use:
List.filter (fun p -> parttype <> MBR || p.G.part_num <= 4_l)
to filter out logical partitions.
Commit 0c396a4bce
introduce API part_get_mbr_part_type,
we could use this to know the part_type.
Furthermore, we could also use p_mbr_p_type for resizing
logical partitions.
Signed-off-by: Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
Rebuilding the ruby documentation takes 51 seconds on my laptop, and
that's a significant fraction of the entire build. Make sure we only
rebuild the documentation when necessary.
RHEL 5 was unusual in having separate sub-repositories (Server, VT,
Cluster etc) within the main RHEL-5-Server directory. Thankfully no
other version of RHEL does this. Previously we set up the repository
baseurl incorrectly so it didn't include .../Server in the path.
In commit 1a1cb1ec3c, I tried to fix the
RHEL 5 repo location to include .../Server. However that broke the
virt-install --location parameter.
Use a separate baseurl path, which should fix both problems.
This fixes commit 1a1cb1ec3c and
commit 3bc9ba6c84.
The error you would have seen before was:
File "v2v_test_harness.ml", line 1:
Error: The files ../../ocaml/guestfs.cmi and v2v_test_harness.cmi
make inconsistent assumptions over interface Guestfs
btrfs-prog commit:
58a39524619f38d193b8adc3d57888ec07b612aa
change the default output to binary prefix,
and introduced a new option '--raw'
to keep the traditional behaviour.
This patch will add a check function to determine
whether to add '--raw' option to 'btrfs show qgroup'.
Signed-off-by: Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
Properly skip empty lines before the first section, otherwise they will
trigger a syntax error in parsing.
Add a simple test for it, so it doesn't regress.
Instead of save every time there's a new element in the cache, batch the
saving to disk every 100 changes, saving the unsaved remainder at the
exit.
While not a big optimization, this reduces a bit the disk usage during
generator run.
The --lcontext option was removed from coreutils ls (in 2007, cough).
Fedora carried a patch re-adding these options [I believe
unintentionally], but that patch was dropped in Fedora 23, so the
--lcontext option is gone for good.
This reverts commit f9c5b4aa2a.
Since wallclock time differs between boots, we cannot just compare two
screenshots by checking if they are identical -- if the screenshot
contains any time information, then that will be different, and it
turns out that VMs print the current time at boot.
Therefore parse out and use the "similarity" metric printed by the
ImageMagick "compare" command when it compares the screenshots, and
allow a small epsilon for the case where a few oixels are different.
block.header_size is uint32_t so cannot be compared directly with n (ssize_t).
pxzcat-c.c: In function 'worker_thread':
pxzcat-c.c:602:21: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
if (n >= 0 && n != block.header_size-1) {
^
This API converts the C guestfs_h pointer to a 64 bit integer,
which should always be possible on reasonable architectures.
Therefore silence a GCC warning (which only occurs on 32 bit ARM).
Fix the 'make check-valgrind' rule in the ocaml subdirectory so it
doesn't run '$VG run-bindtests', effectively running valgrind on bash.
Bash in Fedora Rawhide has added a new memory leak, resulting in a
failure, but we don't want to test bash anyway.
Instead of doing all firstboot actions in a single script, take the
advantage of the firstboot infrastructure and store and run unrelated
actions as individual steps.
This facilitates troubleshooting and fault recovery; besides it makes
adding more actions easier.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
The firstboot infrastructure used to give the firstboot scripts some
cryptic names which were impossible to relate to the actions they were
supposed to take.
This patch reworks the scheme such that the caller registering a
firstboot script has to provide a descriptive name for the action. That
name, with non-alphanumeric characters replaced with dashes, prefixed by
the serial number, is then used as the name of the script, e.g.
0004-install-gcc
or
0002-msiexec--i-foo-msi.bat
OTOH the numbering becomes internal to the API, i.e. the scripts are
numbered and executed in the order they are registered.
This greatly facilitates debugging and troubleshooting in case when
there are multiple firstboot scripts.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
This patch is an attempt to enhance the firstboot driver script for
Windows, and make it somewhat closer in functionality to what is done
for Linux guests.
Specifically, for every firstboot script it now will log its exit
status, and, if the script reported success, move it to -done directory.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
This patch moves line ending conversion for windows scripts into a
separate helper function.
This simplifies code a bit, and fixes the problem that actual firstboot
scripts used to remain with unix-style line endings.
Signed-off-by: Roman Kagan <rkagan@parallels.com>
Use commandrvf() instead of commandvf() to execute e2fsck. A non-zero
exit status does not always indicate a failure.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
When the user chooses to not verify the signatures on the indexes
(using --no-check-signature), there is no point in requiring as many
--fingerprint as --source (or even just one), as they will not be used
anyway.
In this case just ignore all the values of the specified --fingerprint
arguments.
Process the sources specified via the --source command line arguments
before sources read from repositories, so the former have a chance to
"override" the latter.
Related to RHBZ#1193238.
Ensure the arch string is sane before using it in the following XPath
expression. Since the arch string can be derived from untrusted guest
data [see src/filearch.c], this prevents a possible XPath injection
vulnerability.
We also need to be more careful about PyString_FromString and similar
functions returning NULL on failure. Currently we don't check this
every time. This commit adds more checks, but is still not complete.
libguestfs has used double and triple underscores in identifiers.
These aren't valid for global names in C++.
The second step is to replace all guestfs__* (2 underscores) with
guestfs_impl_*.
These functions are used where a libguestfs API call is implemented on
the library side. The generator creates a wrapper function which
calls guestfs_impl_* to do the work. (Libguestfs APIs which are
passed directly by the daemon work differently and don't require a
guestfs_impl_* function).
This is an entirely mechanical change done using:
git ls-files | xargs perl -pi.bak -e 's/guestfs___/guestfs_impl_/g'
Reference: http://stackoverflow.com/a/228797
libguestfs has used double and triple underscores in identifiers.
These aren't valid for global names in C++.
The first step is to replace all guestfs___* (3 underscores) with
guestfs_int_*. We've used guestfs_int_* elsewhere already as a prefix
for internal identifiers.
This is an entirely mechanical change done using:
git ls-files | xargs perl -pi.bak -e 's/guestfs___/guestfs_int_/g'
Reference: http://stackoverflow.com/a/228797
* fusermount is Linux-only; on BSD and OS X umount should be used
* fuser has no -v flag on BSD/OSX, and -c is the
POSIX-compatible equivalent of -m
* Does not solve the lack of pretty output of fuser -v, but does make it
work on Mac OS X.
Add a bash function 'prepend' for intelligently prepending elements to
paths. eg:
prepend PYTHONPATH "/foo"
would set PYTHONPATH to "/foo" or "/foo:<previous-contents-of-PYTHONPATH>"
Tested by:
(1) Building and testing libguestfs twice: first without libguestfs
installed as a system library, and then with it installed.
(2) Examining the output of './run printenv' by hand and comparing
environment variables to the expected values.
As advised by Daniel P. Berrange, the third parameter can be passed on all
platforms rather than specifically Mac.
Quoting a libvirt commit rationale after Daniel:
commit 9fa3a8ab6fd82ad2f5a14b490696085061418718
Author: Doug Goldstein <cardoe@cardoe.com>
Date: Wed Oct 30 11:22:58 2013 -0500
MacOS: Handle changes to xdrproc_t definition
With Mac OS X 10.9, xdrproc_t is no longer defined as:
typedef bool_t (*xdrproc_t)(XDR *, ...);
but instead as:
typedef bool_t (*xdrproc_t)(XDR *, void *, unsigned int);
For reference, Linux systems typically define it as:
typedef bool_t (*xdrproc_t)(XDR *, void *, ...);
The rationale explained in the header is that using a vararg is
incorrect and has a potential to change the ABI slightly do to compiler
optimizations taken and the undefined behavior. They decided
to specify the exact number of parameters and for compatibility with old
code decided to make the signature require 3 arguments. The third
argument is ignored for cases that its not used and its recommended to
supply a 0.
Generate a simple bindtests test for the PHP binding, so it is possible
to easily test all the argument types.
Unlike the bindtests for other languages, optional arguments are not
tested, due to the limitations of optional arguments in PHP (or maybe
they way we implement them).
Make sure to copy the strings we add to the char** array, otherwise they
are stale pointers which we'll try to free later.
Also, properly destruct the temporary zval.
OptString maps to a "s!" argument, which makes zend_parse_parameters not
touch the variables (char* and length) when NULL is passed as parameter.
Hence, set both to NULL/0, and check for non-NULL char* variable before
checking its length.
- use <p>..</p> for text paragraphs, instead of just using <p> to
separate them
- slightly improve the metadata in eventToString and set_event_callback
- fix the textual @see in set_event_callback, so it is accepted also in
JDK 8
- escape the doc text, so &, <, and > will not be considered as HTML
tags but actual text
- use the @deprecated tag instead of adding the customary deprecation
note to the doc text
The gnulib 'error' module uses 'program_name'. On some platforms --
but not Linux / glibc -- it references it as:
extern char *program_name;
This means when you compile libguestfs on non-glibc (eg. Mac OS X)
gnulib requires 'program_name' as an external string reference, which
we don't provide.
This change doesn't define this string reference for gnulib, but it
does change the name of the macro we use to avoid conflicts if we
eventually need to export 'program_name' as a string.
Thanks: Margaret Lewicka
When copying disks that use EFI, we created a new partition table,
randomizing the GPT GUID of the first partition. Since EFI may store
the GUID in its NVRAM variables, this could make the guest unbootable.
In GPT, each partition has a GUID assigned randomly. Allow this GUID
to be written and read.
Note this is different from the GUID type code which is used to
identify the type of the partition.
Commit 9135129b0f changed
two stack buffers to pointers:
- uint8_t buf[BUFFER_SIZE];
- unsigned char outbuf[BUFFER_SIZE];
+ CLEANUP_FREE uint8_t *buf = NULL;
+ CLEANUP_FREE uint8_t *outbuf = NULL;
but we were still using sizeof buf to calculate the size of the
buffer. sizeof buf == 8 so the original code which used large buffers
for reading/writing the file changed to using 8 byte buffers.
Currently implemented as guestfish commands, provide them instead as
single source -> destination functions for the library, so they can be
used also in other places.
These functions are not added to guestfish, since guestfish has its own
implementation (which will soon switch to call copy-in and copy-out for
multiple paths).
test-dlopen needs to be added only when libdl is available, and it is
like that already (within if HAVE_LIBDL); hence don't unconditionally
add it as well.
Check whether --part-type is supported (in the rewritten sfdisk in
util-linux >= 2.26), and use it instead of --print-id & --change-id.
The actual result should be the same, just not using a (recently)
deprecated command line API.
Unlike -o rhev which have only one data domin, -o vdsm can and usually
does have multiple data domain.
The path to vdsm is pre mounted so no need to assume nfs path with -os
Example:
-o vdsm -os /rhev/data-center/<data-center-uuid>/<data-domain-uuid>
Bug-Url: https://bugzilla.redhat.com/1176591
Signed-off-by: Shahar Havivi <shaharh@redhat.com>
Kernel 3.19 has problems uncompressing the RAM disk with <= 500 MB.
(This is likely to be a kernel bug)
64 KB pages are common on aarch64, so treat this case the same as ppc,
and use a larger default appliance memory size.
Thanks: Laszlo Ersek for help and reproducing the bug.
Since libguestfs >= 1.26 / supermin 5, the appliance has lived in a
subdirectory of $TMPDIR/.guestfs-$UID. As a result using 'ls -l'
didn't display any information about the files in the appliance.
Using '-R' causes this information to be displayed again.
ls -Z stops ls from displaying the size field. --lcontext uses longer
lines but has the advantage of still displaying the size field as well
as the SELinux context.
It is documented as 'p2v', but in fact this was not implemented
so there is no way to log in as root on the console.
On RHEL we also have to add the 'passwd' package, else you get:
Error creating Live CD : Unable to set unencrypted password due to lack of /usr/bin/passwd
Covert common_utils_tests to use oUnit as testing framework, replacing
the hand-made assert in favour of structured unit tests and better
error reporting.
common_utils_tests is now built only when the oUnit module has been
found.
Pass to --commands-from-file the name of a file containing customization
commands in each line, as if they were specified as command line
arguments.
This eases the reuse of commands among different
builder/customize/sysprep invocations.
When libguestfs is running using TCG on aarch64, we need to pass the
-cpu cortex-a57 parameter to qemu. Libvirt doesn't let us do this,
complaining "Unable to find CPU definition".
As a temporary workaround only, use <qemu:commandline> to pass this
argument directly to qemu. When libvirt is fixed we can remove this
hack.
This is a workaround for libvirt bug RHBZ#1184411.
See:
https://www.redhat.com/archives/libvirt-users/2014-August/msg00043.htmlhttps://bugzilla.redhat.com/show_bug.cgi?id=1184411
AAVMF is an open source UEFI implementation for aarch64 based on OVMF.
As aarch64 is heading for requiring UEFI even inside guests, if the
AAVMF firmware is installed on the host, use it as a hint that we
should boot the guest using AAVMF instead of the default "empty
machine".
Note this requires very recent AAVMF, libvirt, qemu. However that's
OK since it's only applicable to aarch64. On non-aarch64, this patch
does nothing.
Thanks: Laszlo Ersek for a lot of help getting this right.
If "strdup (d->d_name)" fails with "i" > 0, then both "p" and
"ret->guestfs_int_dirent_list_val" are non-null pointers, but the latter
is no more valid (since "p" is the new realloc'ed buffer). Hence, trying
to free both will access to invalid memory.
Make sure to free only one of them, "p" if not null or
"ret->guestfs_int_dirent_list_val" otherwise.
wait_qemu_nbd would fail in Rawhide where /etc/hosts contains:
127.0.0.1 localhost [...]
::1 localhost [...]
This peculiar mapping caused qemu-nbd to always select an IPv6
address. However since we used the obsolete inet_pton function and
forced AF_INET, we were always trying to connect to the IPv4
localhost.
As this was never going to work anyway, fix the code to use
getaddrinfo to resolve "localhost", and use a loop to connect to all
possible addresses.
Also: Add a short sleep in the reconnection loop so that we don't
hammer the qemu-nbd server while it is starting up.
This test launches a dummy guest using IDE. Aarch64 doesn't support
IDE, and because it also doesn't support PCI (virtio-mmio instead)
it's hard to formulate libvirt XML that will work on both x86 and
virtio-mmio.
Since this bug is not architecture specific (RHBZ#912499), skip it on
non-x86. The test will print:
skipping test: arch is not x86 and does not support IDE
Since systemd 209, all the functionalities of the former libsystemd-*
(including libsystemd-journal) have been merged into a single
libsystemd, with the former libraries left as compatibility ones.
Thus, first look for libsystemd, and if not found try again with the
libsystemd-journal as used before.
With commit a594b7f90a the checks for
input and output files with ':' were removed; while the input file is
handled correctly by the disk_create API, an output filename with ':'
could still be handled as specifying a transport, failing the last
"qemu-img convert" call.
Use the new qemu_input_filename helper to handle the output filename
correctly.
It mimics a bit in OCaml the logic used in
commit a95214b198 (for the main library)
and commit 588af1953e (in disk-create),
so in OCaml tools direct calls to qemu/qemu-img with filenames provided
by the user can properly handle filenames with e.g. ':'.
Since mkfifo, mknod_b, and mknod_c add the correct file type to the
modes of the resulting file, make sure the specified mode contains only
permissions bits.
blkid(1) (or actually, libblkid) seems to handle filesystem labels up
to 127 characters. Considering that btrfs labels can be up to 128
characters, this means long labels are not read correctly (i.e. get
truncated) by blkid. Furthermore, ntfs labels are actually unicode,
and libblkid seems to not decode them correctly.
Hence, if ntfsprogs is available invoke `ntfslabel` to get the label
of ntfs filesystems.
Related to RHBZ#1164708.
blkid(1) (or actually, libblkid) seems to handle filesystem labels up
to 127 characters. Considering that btrfs labels can be up to 255
characters, this means long labels are not read correctly (i.e. get
truncated) by blkid.
Get the filesystem type, and if btrfs is available invoke
`btrfs filesystem` to get the label of btrfs filesystems.
Related to RHBZ#1164708.
Add the 'label' optional argument to the mkfs action, so it is possible
to set a filesystem label direct when creating it. There may be
filesystems not supporting changing the label of existing filesystems
but only setting it at creation time, so this new optarg will help.
Implement it for the most common filesystems (ext*, fat, ntfs, btrfs,
xfs), giving an error for all the others, just like set_label does.
Obviously no one reads the FAQ, so make the error message
even clearer.
The new error messages are variation on this:
libguestfs: error: /bin/false exited with error status 1.
To see full error messages you may need to enable debugging.
Do:
export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again. For further information, read:
http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.
The test assumed (for no particular reason) that it could always run a
KVM guest. When run on a machine without KVM, or in a VM without
nested KVM, you get this error:
libvirt: Domain Config error : internal error: no supported architecture for os type 'hvm'
Traceback (most recent call last):
File "./t/820-rhbz912499.py", line 77, in <module>
dom = conn.createXML (xml, libvirt.VIR_DOMAIN_START_AUTODESTROY)
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3523, in createXML
if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
libvirt.libvirtError: internal error: no supported architecture for os type 'hvm'
FAIL: ./t/820-rhbz912499.py
Use a qemu guest instead, which should have a better chance of working
on all hosts.
Builds of bison-generated files currently fail with:
index-parse.y: In function 'yyparse':
index-parse.c:1256:6: error: assuming pointer wraparound does not occur when comparing P +- C1 with P +- C2 [-Werror=strict-overflow]
if (yyss + yystacksize - 1 <= yyssp)
^
This error only occurred on one machine, and it was unclear why. It
turned out to be because the other build machines I use had:
CFLAGS="-Wall" ./configure ...
whereas on the failing machine I was just using ./configure w/o CFLAGS.
This caused the compile flags of index-parse.c to be changed from:
gcc -Wstrict-overflow ... index-parse.c
to:
gcc -Wstrict-overflow -Wall ... index-parse.c
but since -Wstrict-overflow is short for -Wstrict-overflow=2, and
-Wall is short for -Wstrict-overflow=1, and since the error only
occurs at level >= 2, using CFLAGS="-Wall" was effectively turned off
the warning.
Anyway, I believe the optimizations that GCC does when
-fstrict-overflow is enabled to be dubious, frequently dangerous, and
useless. They are only done because we want C99 to work on
architectures that don't exist in reality (and which even if they did
exist, libguestfs would probably not support).
The Linux kernel uses -fno-strict-overflow, and we should too, for the
same reasons.
Normally this requires the program to have at least one C source file.
virt-sysprep doesn't have this, but it uses C sources from other
directories, and therefore we can make it work.
This returns the C pointer to the guestfs_h handle (ie. guestfs_h *).
The purpose of this API is to interwork with other non-C libraries.
See RHBZ#1075164 for the rationale when applied to another library.
This allows the Perl binding of guestfs_add_libvirt_dom to work.
This relies on the internal Perl representation of virDomainPtr, but
Dan assures me this isn't going to change.
Commit 96158d42f5 enabled the previously
private guestfs_add_libvirt_dom API. It also tried to enable the
existing test for this API, but failed to do that correctly. Also the
test was broken. Fix all of this.
This fixes commit 96158d42f5.
YACC index-parse.c
index-parse.y:97.34-41: warning: type <fields> is used, but is not associated to any symbol [-Wother]
%destructor { field_free ($$); } <fields>
^^^^^^^^
index-parse.y:95.36-45: warning: type <sections> is used, but is not associated to any symbol [-Wother]
%destructor { section_free ($$); } <sections>
^^^^^^^^^^
We had code (added for RHBZ#912499) which labels overlay files
correctly so that libvirt can read them. Unfortunately this code was
broken by subsequent commits: the new backend setting for the
imagelabel is only copied to the 'data' struct during launch, but the
create_cow_overlay callback is called before launch (when adding
drives).
The fix is easy: ensure create_cow_overlay_libvirt checks for the
backend setting and initializes the 'data' struct.
This change also means we need to free (data->selinux_imagelabel)
before setting it in launch (and we do the same for
data->selinux_label, although that's not strictly necessary).
For background on this, see:
https://bugzilla.redhat.com/show_bug.cgi?id=912499#c10
This allows the Python binding of guestfs_add_libvirt_dom to work.
There is a regression test to ensure this keeps working.
Note this requires a patched libvirt-python, supporting the
c_pointer() method.
This API already existed (as guestfs___add_libvirt_dom), and was used
by a few tools.
This commit changes it to a public API.
Note that for reasons outlined in the previous commit message, it is
impossible to call this from guestfish or from non-C language
bindings.
This implements Pointer arguments properly, at least for certain
limited definitions of "implements" and "properly".
'Pointer' as an argument type is meant to indicate a pointer passed to
an API. The canonical example is the following proposed API:
int guestfs_add_libvirt_dom (guestfs_h *g, virDomainPtr dom, ...);
where 'dom' is described in the generator as:
Pointer ("virDomainPtr", "dom")
Pointer existed already in the generator, but the implementation was
broken. It is not used by any existing API.
There are two basic difficulties of implementing Pointer:
(1) In language bindings there is no portable way to turn (eg.) a Perl
Sys::Virt 'dom' object into a C virDomainPtr.
(2) We can't rely on <libvirt/libvirt.h> being included (since it's an
optional dependency).
In this commit, we solve (2) by using a 'void *'.
We don't solve (1), really. Instead we have a macro
POINTER_NOT_IMPLEMENTED which is used by currently all the non-C
language bindings. It complains loudly and passes a NULL to the
underlying function. The underlying function detects the NULL and
safely returns an error. It is to be hoped that people will
contribute patches to make each language binding work, although in
some bindings it will always remain impossible to implement.
This didn't work at all because the regular expression did not match
the returned Augeas path.
In future if the regular expression doesn't match, this will give an
internal error instead of continuing with a bogus value.
Thanks: Junqin Zhou for providing the test case and debug information.
Regular EFI disks have /EFI on the first (VFAT) partition, but they
are not installers.
Fix this by only considering something to be an installer if it has a
single partition (or whole disks like ISOs).
Implementing this is quite complex since when checking a filesystem,
we don't have information about whether we are even looking at a
partition, nor about whether it's the first partition out of how many.
The majority of the commit is changing the code to collect that
information.
Because of the size of an extended partition reported by Linux is always 1024
bytes, so it will always fail to copy to an extended partition.
This patch fixes this problem by copying to the offset of an extended
partition in the destination disk.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Wait up to 10 seconds for qemu-nbd to start up and respond to clients.
Otherwise the conversion server may attempt to connect before qemu-nbd
is ready to serve it.
After a very long and trying episode with a Windows guest that refused
to boot after conversion, we managed to successfully boot it by
disabling Windows Group Policy. It appears that Group Policy
prevented the virtio driver from being used.
Document this in the manual.
Prior to this commit the distro for a FreeBSD or a NetBSD system, in
conjuction to what happened for OpenBSD, was shown as 'unknown'.
*BSDs are complete OSes, not a kernel like Linux, but theoritically you
could have FreeBSD as ostype and PC-BSD as distro.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Take into granted that partition 'c' has a special purpose. It has
always the same size as the enclosing slice and is not mapped under
Linux.
This is a best effort try. The mapping will be incorrect if there is a
gap in the disklabel partitions sequence, e.g. 'b' (swap) partition is
missing but 'd' partition is defined.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Make a best effort try to map NetBSD disklabel partitions to Linux
partitions. The mapping will be incorrect if there is a gap in the
disklabel partitions sequence, e.g. 'b' (swap) partition is missing but
'e' partition is defined.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
This patch lets guestfish show command synopsis if the syntax of command issued
by user is wrong, rather than telling user that the number of parameters is wrong.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
The assumption that Linux will map the MBR partition to /dev/sda1
and the BSD 'a' partition to /dev/sda5 is not always correct.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Because get-uuid looks more like a counterpart to set-uuid, and is
more likely come to mind if one knows set-uuid.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Just like --long-options, it makes it possible to know which short
options are supported by each tool; this can help improving the bash
completion, for example.
As contrast to shortdesc which is forced to begin with lowercase, this
patch forces longdesc to begin with uppercase.
Signed-off-by: Hu Tao <hutao@cn.fujitsu.com>
Commit 588af1953e started with a very
conservative list of permitted backing formats (just "raw" or
"qcow2"). We can allow almost any format permitted by qemu, but this
commit just adds "vmdk" to this whitelist.
Include the version of virt-p2v and its MD5 checksum in the source
kickstart. This is useful for working out which version of virt-p2v
the kickstart corresponds to.
This relies on the remote to keep sending us data. If it hangs, then
the cancel button won't work. This could also be fixed by introducing
a timeout to the read syscall.
I was fetching cancel_button from the wrong dialog (conv_dlg, not
run_dlg), and therefore getting the wrong button.
This explains why the back button on the conversion dialog was always
insensitive. It's because cancel_button was assigned to this button,
and then I was setting cancel_button to insensitive.
It did nothing and was left over from an earlier attempt to add
fullscreen mode. That is not needed any longer since we now run
virt-p2v under the matchbox window manager.
The <target dev> attribute in libvirt isn't very informative. What we
really want to know is whether the source disk used IDE or SCSI, as
that allows us to remap block devices accurately during conversion.
For example, if the source was IDE and RHEL 5, and after conversion
virtio will be supported, then we know that we need to remap
"/dev/hda" to "/dev/vda".
Drop the s_target_dev and s_removable_target_dev fields and replace
them with s_controller and s_removable_controller.
For -i libvirt, use the <target bus> attribute to get this information.
For -i ova, use the OVF disk controller's ResourceType.
(http://blogs.vmware.com/vapp/2009/11/virtual-hardware-in-ovf-part-1.html)
If /etc/favicon.png is a symbolic link, follow it.
Unfortunately RHEL 7 and Fedora have crappy 16x16 /etc/favicon.png
symlinks in the base distro. It would be nice to ignore this symlink,
but it's almost impossible to determine if the symlink is part of the
base distro or was added by the user. (This is a bug in those
distros.) virt-inspector and virt-mananger both ignore favicons.
Commit fd70cdc3c5 passes the -annot flag
to the compiler, generating *.annot files (containing type information
used by emacs and IDEs). Remove these files when doing 'make clean'.
Use AS_CASE to properly map from the host cpu to the qemu cpu suffix;
this avoids doing multiple string replacements using sed, and it is
possible to use case-like syntax.
Hardcoding -ltinfo breaks on distros that do not build the sep library
(which is the default ncurses behavior). Use pkg-config to look up the
right libraries regardless of how the distro built things.
The labels 'hw_disk_bus' and 'hw_vif_model' were swapped. This
happened to work for virtio guests. For non-virtio guests you ended
up with absurdities like requiring an rtl8139 driver for the disk.
Thanks: Junquin Zhou
Previously we used to try to find the 'vmlinuz' file by running 'rpm
-ql kernel' and looking for any file called 'vmlinuz-*'.
If there were multiple 'kernel' packages installed, the rpm command
would list files from all of them, resulting in a random 'vmlinuz-*'
being chosen, not necessarily the one corresponding to the kernel
package we were looking at.
Use 'rpm -ql kernel-<VERSION>-<RELEASE>' instead so that we only look
for files in the right kernel package.
Thanks: James Mighion
In tools except guestfish, we don't use ncurses, or even curses, just
2 termcap functions and a global variable. These are provided by
libtinfo, so just link to that.
In guestfish we use readline which needs ncurses. Leave that alone.
md5sum(1) does not exist everywhere, so wrap it in an own function so
the right implementation can be chosen on each OS. Also, wrapping it
avoid using awk everytime.
Query RbConfig::CONFIG for "libdir", and use that when trying to link to
the ruby library. This fixes the libruby detection when it is installed
in a non-standard library directory.
Create an autoconf macro to deal with the possible combinations of names
and versions of the libdb tools. This also eases the addition of a libdb
version, or a new pattern for the names of tools.
There is no functional change.
Use a %top block for including <config.h>, so it is included prior to
any other include, even for system ones. This makes sure its definitions
are properly used in gnulib replacement headers.
Do a configure check to see whether UNIX_PATH_MAX is defined; if it is
not defined, provide it with the size of sun_path in the struct
sockaddr_un, thus with the proper size for any libc.
Use CLEANFILES to remove the generated symlinks, and just ignore the
errors when copying the scripts in srcdir!=builddir situations.
Followup of commit 40ac54829d.
If a script does not finish, hangs, etc, it would be executed again at
the next boot, since the injected firstboot.sh helper removes it only
after it finished.
Before executing a script, move it to another internal subdirectory
(scripts-done) and execute it from there, so it is not run again by
firstboot.sh. The downside is that now scripts are executed only once
at all, so if a script fails it will not be attempted at the next boot.
Also, remove all the files found in scripts-done, as they have been run
(or at least attempted) in a previous boot.
This fixes RHBZ#1159651.
This adds a customize option:
virt-customize --ssh-inject USER
virt-customize --ssh-inject USER:string:KEY_STRING
virt-customize --ssh-inject USER:file:FILENAME
(ditto for virt-builder and virt-sysprep)
In each case this injects into the guest user USER
a) the current (host) user's ssh pubkey
b) the key specified as KEY_STRING
c) the key in FILENAME
adding it to ~USER/.ssh/authorized_keys in the guest.
For example:
virt-builder fedora-20 --ssh-inject root
will add the local user's ssh pubkey into the root account of the
newly created guest. Or:
virt-customize -a disk.img \
--ssh-inject 'mary:string:ssh-rsa AAAA.... mary@localhost'
adds the given ssh pubkey to mary's account in the guest.
This doesn't set the SELinux labels correctly on newly created files
and directories, so you have to use --selinux-relabel (probably we
should fix this as part of the general effort to fix SELinux
relabelling). However it should preserve the labels if the
~/.ssh/authorized_keys file already exists.
Most of this work is based on a patch sent to the mailing list by
Richard W.M. Jones <rjones@redhat.com>:
https://www.redhat.com/archives/libguestfs/2014-November/msg00000.html
Since the daemon has long since used the same build system as the rest
of libguestfs, it no longer needs its own gnulib.
This arcane bit of code was left over from
commit e05ddc70f1 (added 2009-08-03 !)
2014-10-31 18:30:36 +00:00
3065 changed files with 1800621 additions and 651538 deletions
error(f_"no GNU Privacy Guard (GnuPG, gpg) binary was found.\n\nEither gpg v1 or v2 can be installed to check signatures. Virt-builder looks for a binary called either ‘gpg2’ or ‘gpg‘ on the $PATH. You can also specify a binary using the ‘--gpg’ option. If you don't want to check signatures, use ’--no-check-signature’ but note that this may make you vulnerable to Man-In-The-Middle attacks.")
(* --- If we get here, we want to create a guest. --- *)
(* Warn if the user might be writing to a partition on a USB key. *)
(matchcmdline.outputwith
|Somedevicewhenis_partitiondevice->
ifcmdline.warn_if_partitionthen
warning(f_"output device (%s) is a partition. If you are writing to a USB key or external drive then you probably need to write to the whole device, not to a partition. If this warning is wrong then you can disable it with --no-warn-if-partition")
device;
|Some_|None->()
);
(* Download the template, or it may be in the cache. *)
error(f_"%s checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the ‘-v’ option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!")
error(f_"no plan could be found for making a disk image with\nthe required size, format etc. This is a bug in libguestfs!\nPlease file a bug, giving the command line arguments you used.");
error(f_"no plan could be found for making a disk image with\nthe required size, format etc. This is a bug in libguestfs!\nPlease file a bug, giving the command line arguments you used.")in
error(f_"no guest operating systems or multiboot OS found in this disk image\nThis is a failure of the source repository. Use -v for more information.")
in
Customize_run.run~verbose~quietgrootops;
Customize_run.rungrootcmdline.customize_ops;
(* Collect some stats about the final output file.
* Notes:
@@ -659,7 +723,7 @@ let main () =
* - Never fail here.
*)
letstats=
ifnotquietthen(
ifnot(quiet())then(
try
(* Calculate the free space (in bytes) across all mounted
* filesystems in the guest.
@@ -668,7 +732,7 @@ let main () =
letfilesystems=List.mapsnd(g#mountpoints())in
letstats=List.mapg#statvfsfilesystemsin
letstats=List.map(
fun{G.bfree=bfree;bsize=bsize;blocks=blocks}->
fun{G.bfree;bsize;blocks}->
bfree*^bsize,blocks*^bsize
)statsin
List.fold_left(
@@ -693,7 +757,7 @@ let main () =
elseNonein
(* Unmount everything and we're done! *)
msg(f_"Finishing off");
message(f_"Finishing off");
g#umount_all();
g#shutdown();
@@ -705,7 +769,7 @@ let main () =
* and therefore bypasses the host cache). In general you should not
* use cache=none.
*)
ifsyncthen
ifcmdline.syncthen
Fsync.fileoutput_filename;
(* Now that we've finished the build, don't delete the output file on
error(f_"could not export public key\nUse the '-v' option and look for earlier error messages.");
ignore(import_keyfilegpgtmpdirverbosefilename);
fp
error(f_"could not export public key\nUse the ‘-v’ option and look for earlier error messages.");
import_keyfilegpggpgtmpdirtmpdirfilename
)else
""in
"",[]in
{
verbose=verbose;
gpg=gpg;
fingerprint=fingerprint;
subkeys_fingerprints=subkeys;
check_signature=check_signature;
gpghome=tmpdir;
gpghome=gpgtmpdir;
tmpdir=tmpdir;
}
(* Compare two strings of hex digits ignoring whitespace and case. *)
@@ -127,6 +157,9 @@ and getxdigit = function
|'A'..'F'asc->Some(Char.codec-Char.code'A')
|_->None
letverifying_signaturest=
t.check_signature
letrecverifytfilename=
ift.check_signaturethen(
letargs=quotefilenamein
@@ -137,63 +170,52 @@ and verify_detached t filename sigfile =
ift.check_signaturethen(
matchsigfilewith
|None->
error(f_"there is no detached signature file\nThis probably means the index file is missing a sig=... line.\nYou can use --no-check-signature to ignore this error, but that means\nyou are susceptible to man-in-the-middle attacks.\n")
error(f_"there is no detached signature file\nThis probably means the index file is missing a sig=... line.\nYou can use --no-check-signature to ignore this error, but that meansyou are susceptible to man-in-the-middle attacks.")
error(f_"GPG failure: could not verify digital signature of file\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!");
error(f_"GPG failure: could not verify digital signature of file\nTry:\n - Use the ‘-v’ option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!");
error(f_"could not run sha512sum command to verify checksum");
letcsum_actual=read_whole_filecsum_filein
letcsum_actual=
letlen=String.lengthcsum_actualin
iflen>0&&csum_actual.[len-1]='\n'then
String.subcsum_actual0(len-1)
else
csum_actualin
ifcsum<>csum_actualthen
error(f_"checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!")
This template was generated by a script in the libguestfs source tree:
builder/templates/make-template.ml
Associated files used to prepare this template can be found in the
same directory.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
