The old virt-v2v code would either edit the grub config file (grub
legacy) or use grubby --set-default (grub2).
When translating this code, I figured we could use grubby in both
cases, since grubby exists for grub legacy guests.
However it doesn't appear to work for grub legacy guests. Use the
configuration file editing method for these, same as old virt-v2v.
Thanks: Tingting Zheng
The existing APIs guestfs_stat, guestfs_lstat and guestfs_lstatlist
return a stat structure that contains atime, mtime and ctime fields
that store only the timestamp in seconds.
Modern filesystems can store timestamps down to nanosecond
granularity, and the ordinary glibc stat(2) wrapper will return these
in "hidden" stat fields:
struct timespec st_atim; /* Time of last access. */
struct timespec st_mtim; /* Time of last modification. */
struct timespec st_ctim; /* Time of last status change. */
with the following macros defined for backwards compatibility:
#define st_atime st_atim.tv_sec
#define st_mtime st_mtim.tv_sec
#define st_ctime st_ctim.tv_sec
It is not possible to redefine guestfs_stat to return a longer struct
guestfs_stat with room for the extra nanosecond fields, because that
would break the ABI of guestfs_lstatlist as it returns an array
containing consecutive stat structs (not pointers). Changing the
return type of guestfs_stat would break API. Changing the generator
to support symbol versioning is judged to be too intrusive.
Therefore this adds a new struct (guestfs_statns) and new APIs:
guestfs_statns
guestfs_lstatns
guestfs_lstatnslist
which return the new struct (or array of structs in the last case).
The old APIs may of course still be used, forever, but are deprecated
and shouldn't be used in new programs.
Because virt tools are compiled with -DGUESTFS_WARN_DEPRECATED=1, I
have updated all the places calling the deprecated functions. This
has revealed some areas for improvement: in particular virt-diff and
virt-ls could be changed to print the nanosecond fields.
FUSE now returns nanoseconds in stat calls where available, fixing
https://bugzilla.redhat.com/show_bug.cgi?id=1144891
Notes about the implementation:
- guestfs_internal_lstatlist has been removed and replaced by
guestfs_internal_lstatnslist. As the former was an internal API no
one should have been calling it, or indeed can call it unless they
start defining their own header files.
- guestfs_stat and guestfs_lstat have been changed into library-side
functions. They, along with guestfs_lstatlist, are now implemented
as wrappers around the new functions which just throw away the
nanosecond fields.
This gives us finer control over how system calls are done,
and also potentially lets us test more.
Currently two tests are disabled:
- utimens because of https://bugzilla.redhat.com/show_bug.cgi?id=1144766
- utimes because our stat call does not return the nanosecond fields
This file causes the test ISO to be rebuilt every time.
Instead of including it as a Makefile dependency, create it just
before making the ISO, which has the same effect.
A test in fish/ also used this file. Change it so it doesn't need it.
Add support for aarch64 to the file_architecture API, so we can
detect aarch64 guests correctly.
Add a test binary and library. These were generated on an aarch64
machine by doing:
echo 'main(){}' > bin.c
gcc bin.c -o bin-aarch64-dynamic
strip --strip-all bin-aarch64-dynamic
echo '' > lib.c
gcc -shared lib.c -o lib-aarch64.so
strip --strip-all lib-aarch64.so
Previously we required that the source hypervisor knew the
architecture of the guest, and passed that through to KVM.
However this was error-prone for several reasons:
- OVF (-i ova) doesn't define the architecture
- Disk images (-i disk) don't have an associated architecture
- Libvirt XML sometimes lacks the <type arch=...> field, especially
for RHEL 5-era libvirt.
- It might not be set correctly for the guest.
We know the real architecture from inspection of the guest, so use
that instead.
The escape sequence \e[0;30m sets the colour to black. This is fine
if you use a black-on-white terminal, but fails rather badly if you
use a white-on-black terminal.
Instead use the escape sequence \e[0m which restores the terminal to
the defaults, and works in both cases.
If the RPM database in the source guest is corrupt, then subsequent
operations will fail randomly.
Old virt-v2v appears to have avoided this by cleaning the RPM database
very early on. When translating the code I inadvertantly moved this
cleanup to a later stage -- in particular, after inspection of kernels
was done which requires running rpm commands. Move this back to the
start.
When chowning the final directories, instead of chowning them to 36.36
(ie. hard-coding those UID/GIDs), use a --reference chown so it should
take the ownership from the parent directory.
Unfortunately chowning is very complex because of:
- NFSv3 vs NFSv4
- root squash
- idmap
which creates approximately 8 different ways for this to fail.
When presented with a RHEL 7 guest, virt-v2v would print the following
warning message:
virt-v2v: warning: could not update grub2 console: aug_get: no matching
node (ignored)
This happened because on RHEL 7, /etc/sysconfig/grub is a symlink to
/etc/default/grub, and the Augeas grub2 lens returns entries under
/files/etc/default/grub/*, but the code was checking for
/files/etc/sysconfig/grub/*.
The fix is to check all the possible Augeas paths until one matches.
(Only the first match gets updated however).
Also, don't rebuild the grub2 config unless we've actually changed it.
This avoids doing a potentially failure-prone operation when it's not
necessary.
I suspect that the old virt-v2v code did not work correctly for RHEL 7
guests, but old virt-v2v didn't print a warning here, it just failed
to update silently.
All tests run under the ./run binary. For a long time the ./run
binary has set the $PATH environment variable to contain all of the
directories with binaries in them.
Therefore there is no reason to use ../fish/guestfish instead of just
plain guestfish (and the same applies to other built binaries).
If do_part_get_parttype returns NULL, then return from the function
instead of dereferencing the NULL pointer and crashing. Note that
do_part_get_parttype has already called reply_with_error in this case.
This fixes commit 0316d24770.
Found by: Lingfei Kong
Old virt-v2v rebuilt the initrd late. Currently we rebuild the initrd
right after choosing a kernel, but it's best to update the initrd
at the end since mkinitrd may depend on other configuration files
that we are changing (especially see next commit).
The old virt-v2v code had:
my $prefix;
if ($virtio) {
$prefix = 'vd';
} elsif ($libata) {
$prefix = 'sd';
} else {
$prefix = 'hd'
}
The translated code dropped the test for virtio by accident, which
meant that device names would be mapped to sd* or hd* even if the
virtio-blk driver was enabled for the guest.
Thanks: Tingting Zheng
For Xen guests, a device can be presented twice as (eg) xvda and sda,
and therefore we should map any otherwise unmapped xvdX.
For non-Xen guests, this should have no effect.
I made a mistake while translating the Perl conversion code and
transcribed --set-default as --set-kernel. grubby has no --set-kernel
option so it failed.
Thanks: Tingting Zheng
Previously only "running" domains were blocked from conversion.
However "blocked" is essentially the same as running (on Xen it just
means the domain is running but waiting on an I/O resource). And
"paused" domains probably shouldn't be converted either since their
disks are only crash-consistent.
Don't rely on libvirt's implicit print-to-stderr. Instead make sure
that libvirt error messages are raised in the exception message.
Also allow error strings to be translated.
When you call close on any block device, udev kicks off a rule which
runs blkid to reexamine the device. We need to wait for this rule to
finish running since it holds the device open and can cause other
operations to fail, notably mkfs.
(cherry picked from commit a9c8123c72)
Estimate the amount of space required on the target by each source
disk after conversion. The estimate is approximately a ceiling on
this, and actual use should be considerably lower.
Provide an extra output method (#check_target_free_space) which output
modules may use to check that there is sufficient free space to
proceed, before conversion starts. None of the output modules
actually implement this at the moment.
In most C tools, virt-sysprep and virt-customize, you have to put the
--format parameter before the corresponding -a parameter. ie. The
following is correct:
guestfish --format qcow2 -a disk1 -a disk2
But the following is incorrect. The --format parameter is dangling
and prior to this commit would have been silently ignored:
guestfish -a disk1 -a disk2 --format qcow2
After this change, dangling --format parameters now lead to an error:
guestfish: --format parameter must appear before -a parameter
In virt-customize, also check that --attach-format parameter appears
before --attach parameter.
Thanks: Lingfei Kong
If there is an authentication error when getting the session cookie,
emphasize in the error message that the username can be specified in
the URI.
Unfortunately libvirt asks for the username previously, but we cannot
access that.
In theory this discards the output. Unfortunately in practice we have
to write the output to a temporary file and delete it (because of
limitations in qemu-img convert).
You will see an error like this:
$ virt-sysprep --remove-user-accounts foo,bar -a /dev/null
virt-sysprep: error: user-accounts: --remove-user-accounts parameter was
used, but the "user-account" operation is not enabled
The --user-accounts option, with its double-negative '-' prefix on
user names, is confusing. Replace it with '--remove-user-accounts'
and '--keep-user-accounts' options.
This updates commit 128d474095.
-o rhev: This is "traditional" output to a RHEV-M Export Storage Domain.
It requires no cooperation from RHEV itself, and in all other respects
works the same way as old virt-v2v did.
-o vdsm: This is the new mode which requires VDSM cooperation. VDSM
manages the conversion, creating the directories, suggesting UUIDs,
and cleaning up on failure. This allows direct import into a Data
Domain.
The --rhev* command arguments now only apply to -o vdsm, and therefore
have been renamed to --vdsm*
The --vmtype command line argument may be used with -o rhev or -o vdsm.
Also give an error if users use the incorrect one:
$ virt-builder --list --format json
virt-builder: virt-builder --list: use '--list-format', not '--format'.
The default charset for these filesystems depends on how the kernel was
configured with, so explicitly set the default one (as specified in
Linux sources), eventually setting later a new one for the tests.
Previously the core virt-v2v code had to know which Convert_* module
to call for each guest type (based on inspection). After this change
the core code doesn't need to know that, but instead Convert_* modules
register their interest in particular guest types.
Fedora patches OCaml's Arg module to support --option=arg. Since this
won't work on other distros or upstream OCaml, don't include such
examples in the documentation.
The two adjacent sections can be read separately, and the second
section doesn't mention not using make install / using ./run, so
repeat it a second time.
Before trying to perform the conversion, estimate if there is
sufficient free space on the guest filesystem(s) for the conversion to
be successful.
If not, you will see an error similar to this one:
[ 6.0] Checking for sufficient free disk space in the guest
virt-v2v: error: not enough free space for conversion on filesystem '/'.
4354048 bytes free < 20000000 bytes needed
virt-p2v can run without a window manager. However it then sits in
the top left corner of the screen. Add a lightweight window manager
to the ISO so that position and geometry hints work.
Two scripts are added, replacing the make targets from the
previous commit:
virt-p2v-make-disk: Builds a virt-p2v disk or USB key using
virt-builder.
virt-p2v-make-kickstart: Builds a kickstart file (only) which can be
fed to livecd-creator or similar tools on Red Hat-derived distros.
Kickstart generation is slightly different: Instead of requiring the
downstream 'libguestfs-pvhelper' package, the kickstart now embeds the
virt-p2v binary(!) making it self-contained and downstream packaging
simpler.
In addition there are documentation changes.
As part of reworking the scripts that are used to build the P2V ISO,
first remove the separate p2v-iso directory. Similar scripts and
documentation will be introduced back into the p2v/ directory in
following commits.
Import the upstream lens for the shadow file, just with a different
identifier and not matching /etc/shadow by default. Instead, apply a
transformation to have it match /etc/shadow only if the version of
augeas is at least 1.2.1 [1].
[1] While the last upstream version is 1.2.0, all the development seems
to happen in master, so whatever the next version is going to be
numbered (e.g. 1.2.1 or 1.3.0), the check will be fine anyway.
It will print an error like this:
virt-v2v: error: internal error: invalid argument: libvirt domain 'windows'
is running, it must be shut down in order to perform virt-v2v conversion
This only works for libvirt domains, and is explicitly disabled for
test:/// URIs since these domains are always "running", but we use
these URIs for testing.
Unhelpfully, Filename.chop_extension throws an exception if the
filename has no extension (instead of just returning the filename).
Therefore the -i disk option would fail if you passed any filename
that didn't have an extension.
When setting the password for a locked account, make sure to still write
the password after the "!!" marker, otherwise the account will have no
password.
Previously we only looked for <driver name=qemu type=...>. However
for Xen imports, the name field could be name=tap (or other values in
fact). Since there can only be one <driver/> element under <disk/>,
look for any <driver type=...> in order to get the disk format.
A further complication for Xen is that type=aio means "raw" (for
obscure historical reasons), so map that.
Thanks: Dan Berrangé.
The empty_element macro was not safe when used in a "naked" if
statement such as:
if (config->flags & FLAG_ACPI) empty_element ("acpi");
Fix the macro so it uses do { ... } while (0) to make it safe here.
The new output of virt-v2v --machine-readable now lists the input and
output drivers available in the binary:
$ virt-v2v --machine-readable
virt-v2v
libguestfs-rewrite
input:disk
input:libvirt
input:libvirtxml
output:glance
output:libvirt
output:local
output:rhev
With this information, we can now pre-populate the virt-p2v GUI
combo box.
Instead of having all interfaces connect to the "default" network on
the hypervisor target, you can now create a map of interface to target
network, either on the kernel command line, eg:
p2v.network=em1:rhevm
p2v.network=em1:rhevm,em2:management,other
or through the GUI.
Enhance the user-account sysprep operation, so it is possible to select
which user accounts should be removed only (instead of all), or which to
eventually keep.
Name these the same as the virt-v2v options, so for example
'-oa' -> 'p2v.oa'.
Also add the missing p2v.oc (-oc) option.
This updates commit c516d2d07d.
On Ubuntu, exclude the Ubuntu-only 85-lvm2.rules since it causes the
automatic activation of volume groups on addition or change. This
interferes with libguestfs, as vgs which are disables suddently get
enabled.
ocamldep -all (introduced in commit a3881445ef) creates a dependency
rule
optgroups.cmi ... : utils.cmx ...
Because we never build a native code version of the generator,
utils.cmx can never be remade, and so this results in optgroups.cmi
being always rebuilt and hence rebuilding the whole directory.
It's unclear how to fix this, but reverting the ocamldep -all change
in this directory works around it.
Partially reverts commit a3881445ef.
In v2v, 'stringMap.ml' has no corresponding *.mli file. ocamldep does
not generate a *.cmi dependency for such modules, and this can result
in parallel builds failing rarely because:
(1) stringMap.ml is built (which has the side-effect of generating
stringMap.cmi)
(2) at the same time, types.mli is compiled. types.mli depends on
stringMap.cmi, but it is half way through being created by (1).
There is no dependency to serialize (1) and (2).
The resulting error is:
File "types.mli", line 1:
Error: Corrupted compiled interface
stringMap.cmi
Using 'ocamldep -all' generates extra deps for the *.cmi files, and
otherwise appears to be safe, so use it.
There are currently a few upstream OCaml bugs with dependencies and
parallel builds:
http://caml.inria.fr/mantis/view.php?id=3190http://caml.inria.fr/mantis/view.php?id=4991http://caml.inria.fr/mantis/view.php?id=5000
It's not clear which of these apply here.
The parsing of p2v.disks, p2v.removable and p2v.interfaces did not
correctly end at the first space, and therefore any other parameters
that happened to be on the command line afterwards would be consumed
as a list of disks or interfaces.
In commit 53e0d3ea5f I fixed a cause of
disk corruption in the overlay file during conversion.
To ensure this doesn't occur again unexpectedly, add a simple test to
the virt-v2v code and to the tests.
When saving, resolve the path of the file being edited and use that as
real target to write to. Otherwise, if the file name is a symlink then
it will be replaced by a regular file with the new content, leaving the
old file untouched.
Extend test-edit.sh to check for this situation.
Allow null as value for the editor parameter of edit_file_editor, which
will then get it from the EDITOR envvar (falling back on vi).
This is basically code motion from the two edit_file_editor users to it.
To speed up conversions by storing everything we read during
conversion in the overlay instead of having to reread it from the
remote server during copying.
Thanks: Matthew Booth.
When drives are added with this flag, reads from the backing file are
copied into the overlay, improving performance of reads to the same
area of disk (at the expense of local storage). This is excellent for
reading remote / slow sources like HTTP.
Add the perl file editing, mostly based in the virt-edit implementation.
This introduces a mild code duplication with edit_file_editor; will deal
with it in a later commit.
Do not unconditionally exit if guestfs_case_sensitive_path, but let
windows_path still return null. Make virt-edit then check for that, and
eventually exit on its own.
Move the code handling Windows paths from virt-edit to a common file,
so that can be shared by various tools.
Mostly code motion, with a minimum touch (the additional guestfs_h*
parameter in mount_drive_letter) to make it build and work.
This seems* to cause disk corruption in the overlay files. A
manifestation of this is that the qcow2 header is not written back
correctly to disk, resulting in the backing file of the overlay
"disappearing" and hence hardly any data being copied over to the
target.
* Note this could be for a variety of reasons and doesn't necessarily
indicate a fault in qemu or qcow2.
Previously if you used `-i libvirt -ic [a remote URI]' then the remote
URI would be opened during object creation, which occurs during
command line parsing. The practical effect was that virt-v2v could
appear to hang or give an error (eg if the remote URI is inaccessible)
before printing any log messages. This could be very confusing for
the user.
Split out -i libvirtxml into a separate module, containing the libvirt
XML to source parsing code as a function called
Input_libvirtxml.parse_libvirt_xml.
Modify -i libvirt so it does all initialization and calls the helper
function during the #source method, and not at object creation time.
This was based an incorrect translation of what the old virt-v2v was
doing. %systemroot%\Drivers is not expected to exist already, and
indeed does not exist in a real Windows 7 guest.
Add a mini-binding to libvirt virStoragePoolGetXMLDesc to get the pool
XML directly using the libvirt API. Change existing external calls to
`virsh pool-dumpxml' to use this API instead.
The `virsh dumpxml' command doesn't work properly when the libvirt
source requires authentication. This is because the authentication
prompts are sent to stdout, but stdout is also the place where we are
reading the output XML from.
Add a mini-binding to libvirt virDomainGetXMLDesc which avoids this,
getting the XML directly using the libvirt API. Change existing
external calls to `virsh dumpxml' to use this API instead.
The way to set the filter for lvm devices was to open lvm.conf, look
for uncommented "filter =" lines and replace the configuration there.
This had the issue that if there is no uncommented filter line, then the
filter cannot be changed at all; considering newer lvm2 releases ship a
sample configuration with no uncommented filter lines, then the old way
became wrong and not sufficient.
Instead, take a copy of the upstream lvm.aug lens, with a simple change
to allow parsing also negative values in configuration, and install it
in the daemon. When asking to change the lvm filter, use augeas making
sure to use this custom lens for the lvm.conf copy used within the
appliance.
The C code didn't have -Werror enabled (when configured) and didn't
enable the full set of warnings we would want.
However OCaml-C bindings commonly declare functions which are
called directly from OCaml, eg:
external parse_memory : string -> doc = "v2v_xml_parse_memory"
value v2v_xml_parse_memory (value xmlv)
These do not require prototypes as no other C code will call them, so
we have to switch off the GCC -Werror=missing-prototypes flag for
these files.
Also fixes some confusion between _CPPFLAGS and _CFLAGS.
Move the fields from Types.overlay which were related to the target
file out. Now Types.overlay is only concerned with the actual qcow2
overlay files, not with a mixture of both files.
For every guest disk, there is one chain which looks like this:
source_disk overlay target
+------------+ +----------------+ +---------------+
| s_qemu_uri |<------ ov_source |<------ target_overlay|
| etc | | ov_overlay_file| | target_file |
+------------+ | ov_sd | | target_format |
| ov_virtual_size| +---------------+
+----------------+
describes source describes temp. describes target
disk file overlay (qcow2) file
This is just refactoring.
This function had got smaller and smaller until now we might as well
just inline it. Having it after the main function just obscured the
order that the code runs.
No functional change, just refactoring.
The ov_preallocation field was essentially an implementation detail of
the V2v module. There was no need to carry it around in the overlay
struct, and nothing else used it. This also allows us to simplify
other code.
If the <memory> or <vcpu> elements was missing from the source
metadata then we would choose useless defaults, resulting in (for
example) <vcpu>0</vcpu> being written to the output.
Choose sensible defaults for these integer values.
When we map the network or bridge names (using --network or --bridge
command line options), this commit keeps the original name and adds it
as a comment to the output metadata.
The comment will look something like this:
<interface type='bridge'>
<!-- bridge mapped from "VM Network" to "bridge1" -->
<source bridge='bridge1'/>
<model type='rtl8139'/>
</interface>
Previously Types.input and Types.output were variants, resulting in
the main v2v.ml code having to have knowledge of all the input and
output types. So you would end up with code in v2v.ml like:
match output with
| OutputLibvirt -> (* call some libvirt-specific code *)
| OutputRHEV -> (* call some RHEV-specific code *)
This changes these types to be abstract superclasses, with specific
implementations provided by (and hidden inside) each input/output
module. The above code in v2v.ml now becomes:
output#call ... (* no backend-specific knowledge needed *)
Although this commit is very large, it is just refactoring. There is
no functional change in the code.
This allows `-ic esx://...' to work because we map the strange source
URI used by ESX to a remote HTTPS connection.
Example disk section that this handles:
<disk type='file' device='disk'>
<source file='[datastore1] Windows/Windows.vmdk'/>
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
Instead of the ?dir parameter which was essentially a hack, allow
callers of create_xml to provide functions that map
<source file="..."> or <source dev="..."> to qemu URIs.
The default for these mappings is the identity function.
For -i libvirtxml we replace the old absolute_path_of_disk function
with a source file mapping function which does the same thing.
There is no functional change in this patch.
<source dev=...> in the libvirt XML should always refer to a block
device, eg. /dev/sda1. Don't attempt to make these paths absolute
(they should already be absolute paths, and if they are not it's wrong
to try to make them).
There is already a partial binding for this in fish/uri.c &
mllib/uri-c.c. However it is specialized to parsing the -a parameter
on the command line and we want access to the full underlying
functionality of xmlParseURI.
These are now collected from the source metadata and (where
possible) passed through to the destination.
For reference, see the metadata structure of the old virt-v2v (roughly
equivalent to Types.source in the new code):
https://git.fedorahosted.org/cgit/virt-v2v.git/tree/metadata-format.txt
As you can see the new code now covers all the metadata features of
the old code.
Mostly complete and making most of the features work.
The notable lack is ntfsprogs and ntfs-3g, which due to the way
mount.ntfs is handled (i.e. using alternatives) do not work properly.
This is simpler to use and more convenient than -i libvirtxml, because
users don't need to bother with writing libvirt XML. However it is
less expressive because many source capabilities such as memory and
number of vCPUs cannot be specified this way.
As initializing the target (-o ...) is a distinct operation, which can
fail if the user supplied the wrong command line arguments, make sure
we display an explicit message before we perform this step.
It prints something like this:
[ 5.0] Initializing the target -o libvirt -os default
Since `-o local' and `-o libvirt' both need to generate libvirt XML
they should share the same code to generate it.
Also generate it using the DOM module instead of printing the XML
directly. This is cleaner and avoids quoting issues.
No functional change here.
Notes:
- Drop the acpi feature for RHEL 3. Old virt-v2v says there is
a problem presenting ACPI to RHEL 3 guests although it is not
specific about what this is.
- Old virt-v2v does not pass features through to OVF at all.
This seems to be the new way [1] in Archlinux to specify, as symlink,
the default version of Java.
Thanks to Peter Wu for the notice about it.
[1] https://wiki.archlinux.org/index.php/Java
This warning was meant for the case where /dev/kvm is inaccessible and
the user must add themselves to a special group.
However we didn't take into account whether /dev/kvm was openable in
this test. If it's openable, no point issuing the warning.
Move the is_openable ("/dev/kvm") test earlier, and don't issue the
warning if that was successful.
The virt-v2v tests here are not meant to be thorough tests of guest
conversion. There is a test suite used to test conversion which is
kept outside libguestfs because it contains lots of proprietary
unredistributable guests like Windows.
The tests here instead are testing:
- virt-v2v command line arguments
- virt-v2v reads and writes files to the correct places
- valgrind & memory leaks
- any conversion properties that can be tested using just the
phony guests (but we don't go out of our way here)
- that up to date guests don't crash virt-v2v
Fix the Name and Packages database so it looks enough like a modern
RPM database to fool inspection. I'm not sure whether or not the old
version ever worked, but it doesn't work now.
The format is not documented, but in brief:
The Name database contains (Name, link) pairs. The link is an
arbitrary 32 bit integer.
The Packages database contains (link, RPM-blob) pairs.
The RPM-blob is a binary blob formatted like this:
* Number of fields 4 bytes, big endian
* Size in bytes of the store 4 bytes, big endian
* Field descriptions:
list of (RPMTAG, 0, offset in store, 0)
4 * 4 * nr_fields bytes, big endian
* The store binary data containing ASCIIZ strings
For the values of RPMTAG_* that we understand, see src/inspect-apps.c.
The calling code allows Fedora guests to be used when calling
Convert_linux. However Convert_linux itself would fail with an
assertion error when you did this.
Previous virt-v2v would write to a temporary destination file then
atomically rename everything at the end.
However this appears to be unnecessary since we write the OVF file
after copying the disks, so (I assume) RHEV-M won't see the disks
before then.
In any case nothing used ov_target_file_tmp so we can remove this
feature to simplify the code.
Currently various UUIDs used by `-o rhev' are generated randomly.
This means they are different on every run of the program.
Allow some of them to be specified on the command line. (If not
specified they are still generated randomly).
This allows RHEV-M to run virt-v2v several times (eg to examine the
OVF output) while having paths to files and OVF be stable across runs.
At the moment there is just one "miscellaneous RHEV command line
option" (--vmtype), but we may add more in future. Put them in a
single struct for convenience.
This is just code motion.
If the environment variables LIBGUESTFS_TRACE=1 or LIBGUESTFS_DEBUG=1
were set, then the current code would end up disabling trace/debug if
the command line arguments did not contain -x/-v.
Fix this so the effect of enabling trace/debug is cumulative.
Previously it was called 'libvirt.conf' which is more than a little
bit confusing.
This is only an internal name, so there is no visible change in
functionality.
It seems that Automake cannot handle properly the files generated by
flex and bison, leaving them in the dist tarball.
For now, add a dist-hook to manually prune them from distdir.
When using optional arguments of type OStringList, the code free'ing
the member in the optargs_s struct corresponding to that optional
argument would just check for a non-PyNone PyObject for that argument.
If before that optional argument there are other arguments which can
cause an earlier error return from that binding function, the free'ing
code will then act on garbage values.
Enhance the check by also checking whether the optargs struct has the
bitmask with the element for that argument, meaning that the
corresponding struct member was initialized.
The error is:
../src/.libs/libutils.a(
libutils_la-cleanup.o): In function `guestfs___cleanup_xmlBufferFree':
cleanup.c:(.text+0x79): undefined reference to `xmlBufferFree'
../src/.libs/libutils.a(libutils_la-cleanup.o): In function
`guestfs___cleanup_xmlFreeDoc':
cleanup.c:(.text+0x99): undefined reference to `xmlFreeDoc'
[etc.]
Thanks: Arnaud Gaboury
If pthread_cond_wait fails and the thread worker ends, make sure to
unlock retire_mutex, which has been locked for the condition.
The return value of pthread_mutex_unlock is ignored, as there's nothing
that can be done with it at that point.
Fix the memory issues in the implementation that uses no -p nor -i:
- use add_string_nodup to add results from get_blkid_tag (which returns
new strings), so those strings are not leaked
- use free_stringslen to clean the hash on error, as in such case the
stringbuf will not be terminated with a null entry, thus causing
free_strings to crash
When dealing with DeviceList parameters, the generator produces code
similar to the following:
CLEANUP_FREE_STRING_LIST char **devices = NULL;
[...]
devices = malloc (sizeof (char *) * (args.devices.devices_len+1));
{
size_t i;
for (i = 0; i < args.devices.devices_len; ++i)
RESOLVE_DEVICE (args.devices.devices_val[i], devices[i],
, goto done);
devices[i] = NULL;
}
The block hidden within the RESOLVE_DEVICE macro is supposed to
assign something to devices[i]; on the other hand, the code in
RESOLVE_DEVICE can cause to just end (with an error) the current RPC,
which would cause the cleanup of the "devices" array... whose members
from the i-th to the (args.devices.devices_len-1)-th would be garbage
pointers, causing random memory to be free'd (and thus crashing the
daemon).
Avoid the access to garbage memory just by having a cleaned "devices"
array, so there will be always a NULL element after the initialized
members.
Add a test for vgcreate which passes a wrong device path causing the
situation above, to test that vgcreate would fail gracefully.
Use %d to parse them as int (since the variables for them as int)
instead of %u, even if they both need to be at least > 0; the library
will check for the validity of them anyway.
Move the minimum memory check from the environment parsing to
set_memsize, so the limit is actually enforced also when using the API.
Adapt the rhbz557655.sh test to the invalid memsize values being
rejected now, and add a new test for checking invalid parameters
explicitly.
Even if luaL_error is a "no return" function for the Lua runtime, adopt
also in action functions the "return" idiom recommeded for it.
This also helps code analyzers in not thinking that "g" might still be
null after the null check followed by luaL_error.
Handy macro to reply the right way for an unavailable feature.
While generally used so far in generated code, it can shorten that a
bit, and avoid copy&paste when wanting to do manual feature checking.
This refactors existing code to close file descriptors in the recovery
process, and also adds code to close file descriptors between the
fork() and exec() of QEMU or User-Mode Linux.
The reason is to avoid leaking main process file descriptors where the
main process (or other libraries in the main process) are not setting
O_CLOEXEC at all or not setting it atomically. Python is a particular
culprit.
See also this OpenStack Nova bug report:
https://bugs.launchpad.net/nova/+bug/1313477
Thanks: Qin Zhao for identifying and characterizing the problem in Nova.
This is version 2 of this commit. This commit is identical to the
reverted commit 115fcc3432 except that
we don't close stderr.
This attempted fix for RHBZ#1123007 causes the qemu command line to be
lost when verbose mode is enabled. Since this is essential for
debugging many problems, I am reverting the patch.
This reverts commit 115fcc3432.
This is useful when analysing with `perf' since it allows us to
increase the amount of time in the test versus the amount of time
spent starting the appliance.
This refactors existing code to close file descriptors in the recovery
process, and also adds code to close file descriptors between the
fork() and exec() of QEMU or User-Mode Linux.
The reason is to avoid leaking main process file descriptors where the
main process (or other libraries in the main process) are not setting
O_CLOEXEC at all or not setting it atomically. Python is a particular
culprit.
See also this OpenStack Nova bug report:
https://bugs.launchpad.net/nova/+bug/1313477
Thanks: Qin Zhao for identifying and characterizing the problem in Nova.
We were mounting different filesystems on top of each other. In fact
this is not a problem, but it's nicer to unmount each filesystem
properly after examining it.
This test worked by uploading a 100MB file into a 100MB container and
seeing the inevitable crash. Unfortunately virtio-mmio (used on
aarch64) is quite slow. Since this test is not testing the speed of
virtio-mmio, use a smaller test file so the test finishes in a
reasonable time.
This library replaces Win::Hivex::Regedit, or at least enough for us
to be able to make the simple Registry modifications needed for
installing firstboot scripts.
To fix RHBZ#1056290, I prevented virt-sparsify being used if the
output device is a block device.
I have now retested this scenario and it does work (in both copying
and in-place mode), and does not delete the output device, and
therefore we can relax this restriction so only char devices are
banned.
This is useful for oVirt which uses a qcow2 formatted block device to
store virtual machines.
For example:
$ time ./run ./tests/qemu/qemu-boot -n 100
real 1m19.794s
user 0m10.001s
sys 0m5.928s
will run 100 appliance start/stops, from multiple threads.
This test has not been run since 2012 as it depends on specifics of
how supermin and new-style appliances work (see
commit 2d89aef897).
This commit removes the test entirely.
In OCaml 4.02, the 'string' type can be made immutable. All String.*
functions that are concerned with creating or mutating strings now
give a warning like this:
Warning 3: deprecated feature: String.unsafe_set
Disable this warning since we don't want to turn on immutable strings
yet.
This still only prints early console messages. Output stops
after:
bootconsole [earlycon0] disabled
but I have not found any way to get later kernel messages.
This option allows oVirt to pass a prebuilt qcow2 file to use as the
temporary overlay. The file must be qcow2, and must have indisk as a
backing file - the code does minimal checks to ensure this is correct.
Example usage:
qemu-img create -f qcow2 -b indisk overlay.qcow2
virt-sparsify indisk --tmp prebuilt:overlay.qcow2 outdisk
Note this only applies in copying mode.
Run ldconfig early in the init script, so libraries outside standard
library paths but with a proper ld.so configuration file pointing at
them can be found.
Just like no /etc/fstab is not an error, having one with no entries
shouldn't be an issue either.
With systemd, this could be a valid setup, with mount points set its own
way.
We were passing an int64_t which happens to be the same as time_t, but
only on 64 bit. This didn't work on 32 bit (or rather, it works on
little endian, but only by accident).
This fixes commit fb546eaee0.
sd_journal_get_realtime_usec is defined as:
int sd_journal_get_realtime_usec(sd_journal* j, uint64_t* usec);
Unfortunately we passed size_t* as the second argument. This happened
to work on 64 bit machines because size_t is 64 bit there, but failed
on 32 bit machines, where we didn't pass a sufficiently large result
buffer.
This fixes commit 2d996df66c.
These log files can get pretty big, and that causes problems
like this one:
http://linuxforum.ru/viewtopic.php?id=35381
and this one:
https://bugzilla.redhat.com/show_bug.cgi?id=1099172
In both cases the problem was /var/log/lastlog which is apparently a
large sparse file. Supermin cannot cope with sparseness, so it copies
the whole file non-sparse. But since this could affect other log
files this commit covers various common log file names and patterns.
Thanks: semperN
Sort the structs when generating code. Since the structs are
logically indepedent of each other, this should have no effect except
to make generated files list the structs in a different order.
However this also fixes the following build problem:
File "convert_linux.ml", line 322, characters 43-50:
Error: This expression has type G.stat = Guestfs.stat
but an expression was expected of type G.dirent = Guestfs.dirent
It turns out the OCaml bindings don't like the fact that we have
two structs with a common field name (dirent.ino and stat.ino).
In OCaml < 4.01, this means that any attempt to reference stat.ino
would fail because dirent.ino appears second in the file, overriding
stat.ino.
Sorting the structs has the side effect of making stat.ino appear
second, thus resolving the build failure above.
In OCaml >= 4.01 the compiler now uses some disambiguation rules based
on the known types to resolve this problem, so accessing either field
would work no matter what order they are listed in.
See:
http://ocaml.org/releases/4.01.0.htmlhttp://www.lexifi.com/blog/type-based-selection-label-and-constructorshttp://www.lexifi.com/blog/ocaml-extensions-lexifi-overidding-record-labels-and-constructors
virt-customize:
virt-sparsify:
virt-sysprep:
virt-v2v:
- These tools consistently used -v to mean verbose/debugging
and -x to mean enable libguestfs tracing.
virt-builder:
virt-resize:
- These two tools did not recognize -x at all, and used -v
to enable libguestfs tracing and general debugging.
- This commit changes these two tools to consume -v/-x
consistently with the other tools.
Unfortunately this has a cascade of effects through the code.
It turned out that Common_utils was exporting the 'G' module (an alias
for Guestfs). We want any code that uses G as a shortcut to declare:
module G = Guestfs
at the top, since that avoids confusion for newbie (or experienced)
OCaml programmers.
Old Xen PV guests had:
<driver name='tap' type='aio'/>
The previous xpath expression matched any driver type attribute in
order to pick up the format. However we only want to match:
<driver name='qemu' type='raw'/>
so we need to check the name attribute as well.
I'm not sure if this indicates that the xpath expression is wrong, but
in any case it can be that xpathobj->nodesetval is NULL. In this
case, return 0 from xpathobj_nr_nodes instead of segfaulting.
Commit c4dc70f8c4 broke the man pages
for the Perl scripts in the tools/ directory.
It inserted the =encoding line at the top of the file, instead of in
front of the first =head1 entry, and this meant that the #!/usr/bin/perl
and the prologue became a part of the POD, which was not intended.
Add the virt-sparsify --tmp option.
This works in two ways. Either you can specify a temporary directory,
in which case it's just the same as setting $TMPDIR before:
virt-sparsify indisk outdisk --tmp /var/tmp
Or, as a new feature, you can specify a block device which is directly
used (and OVERWRITTEN):
virt-sparsify indisk outdisk --tmp /dev/sdX
This is useful for oVirt nodes, where there is limited temporary
space, but a block device can be assigned to the node.
In both cases it is only used in copying mode. In-place
sparsification doesn't require large amounts of temporary space.
Previously if you were root, and you tried to change directory into a
directory which was not owned by you and not readable (eg. 0700
bin:bin), it would fail.
This doesn't fail on regular directories because when you are root the
kernel just ignores permissions.
Although libguestfs in general tries not to duplicate kernel code, in
the case where we emulate the FUSE access(2) system call,
unfortunately we have to do it by stat-ing the object and performing
some (half-arsed) heuristics.
This commit modifies the FUSE access(2) system call, so root is now
able to chdir to any directory.
It also adds some debugging so we can debug these complex permissions
checks in the field if some other problem arises in future.
This commit also removes the disk image signature. These have not
been needed since virt-builder 1.24.1. No one should be running that
version of virt-builder since there are newer 1.24 branch versions.
See:
https://www.redhat.com/archives/libguestfs/2014-June/thread.html#00045
Notes:
(1) Edit dracut configuration to make sure the change is permanent, in
case kickstart, virt-builder or something else installs a kernel or
runs dracut.
(2) We need to run dracut on the just-installed kernel, not on the
running kernel. The snippet to do this was copied from the Fedora
cloud kickstart.
Thanks: Olaf Hering, George Dunlap, Dario Faggioli.
This device is not present at least on ARM. Unfortunately libvirt
doesn't give us any way to query when the device is present.
This fixes commit bed592498d.
Setting environment variables such as PS1 for bash before starting it
might not be effective when the startup scripts provided by the
distribution unconditionally change it.
Hence, set PS1 and TERM in a ~/.bashrc, which will be source'd last and
thus be able to set them the way we want.
Augeas has been a mandatory dependency of libguestfs for quite some
time, so the aug_* functions could have been always called (without even
checking the augeas feature). Thus retire the feature, marking it as
always available with no more functions depending on it.
It seems recent (?) NetBSD versions have their kernel as /netbsd, so
also check for it to detect NetBSD installations.
The current detection so far basically relied on generic files and
directories which can potentially be in every UNIX system, misdetecting
them if a /etc/release file is present in them.
If the image is located on NFS, and the NFS mount is performing a
large copy, then all other NFS operations stop. This delay can be
large enough that we hit the current 10 minute timeout in udev.
Increasing this to 100 minutes is reported to cure the problem (in
reality we want udev to wait indefinitely).
This implements the guestmount --fd option to allow you to run
guestmount captive under another process (typically using
`guestmount --fd=<FD> --no-fork').
See: https://bugzilla.redhat.com/show_bug.cgi?id=1100498
Resolve the given path within the chroot, so scrub can be invoked
outside the chroot on an already-resolved path.
Add few tests in scrub-file for this and other similar issues.
commit a86eb0e0d2 made it an optional
feature, as on Windows it was not available; on the other hand, realpath
has been used unconditionally already for quite some time, so having
just the "realpath" command conditional on the presence of it does not
make much sense.
Drop the configure/build system handling of it, make the "realpath"
command no more optional, and keep the "realpath" feature as
unconditionally available now.
Support the possibility to have optional groups always enabled (e.g.
because they were present in the past, and they need to be kept for
users).
Add and use few helper optgroups-related functions to deal also with
them.
console=ttyS0 used to work. Now it's broken again, and the correct
setting appears to be console=hvc0. Since we can pass multiple
console settings, do that to try and catch the right console.
This is just a whitespace change. The new output looks like this:
Output file: fedora-20.img
Output size: 4.0G
Output format: raw
Total usable space: 5.2G
Free space: 4.5G (86%)
Previously it wasn't installed at all. However it makes sense to
install it somewhere so that it can be placed in a package (a
requirement if we're going to create a kickstart). By placing it in
libexecdir it is hopefully clear that it should not be used.
Note on Red Hat distros, libexecdir is usually /usr/libexec. On
Debian it is usually /usr/lib/libguestfs.
Bump to >= v196 the minimum version of libsystemd-journal needed for the
journal integration, since that version provides
sd_journal_get_data_threshold and sd_journal_set_data_threshold.
It is not possible to have the ISO be built automatically when the
user runs 'make'. The README file describes how you should go about
building the ISO or disk image.
This is a graphical standalone front-end to virt-v2v which can be run
on physical machines (usually linked into a ISO or PXE boot image) to
convert the physical machine to a virtual machine.
Copy the information about the proxy of a source in all the entries of
that source; this way it is possible to use it later when accessing to
the actual image of an entry.
Root partition is on /dev/sda2:
$ virt-filesystems -a rhel-5.10 --all --long -h
Name Type VFS Label MBR Size Parent
/dev/sda1 filesystem ext2 /boot - 510M -
/dev/sda2 filesystem ext4 / - 4.5G -
/dev/sda3 filesystem swap SWAP-hda3 - 1.0G -
/dev/sda1 partition - - 83 510M /dev/sda
/dev/sda2 partition - - 83 4.5G /dev/sda
/dev/sda3 partition - - 82 1.0G /dev/sda
/dev/sda device - - - 6.0G -
It would be better if we could mount by label in guestfish ...
There is another problem with RHEL 5 guests in that the
yum URL is slightly wrong.
Get rid of --enable-gcc-warnings and just add all the warnings in any
case; they are tested anyway, so only warnings supported by the compiler
will be used.
Regarding -Werror, add a new --enable-werror which enables its use.
Allow to skip testing the filesystem "foo" if the environment variable
SKIP_TEST_CHARSET_FIDELITY_foo=1 is set. This way it possible to not
test one or more filesystems without disabling the test altogether.
So far, passwords in URLs (eg http://user:password@host..) have been
handled as part of the username, and thus passing
add-drive path username:username:password ...
instead of
add-drive path username:username secret:password ...
Fix the parsing of URLs to handle passwords as separate elements,
properly passing it as "secret" parameter for add-drive, and properly
readd it when building URLs in the direct backend.
Furthmore, to keep curl- and ssh-based qemu drivers working with
authenticated resources, make sure they can accept secrets.
Reported in comment #1 of RHBZ#1092583.
This is useful for generating Linux initramfses from other types of
filesystems. For example:
guestfish --ro -a disk.img -i cpio-out / - | gzip -9 > initrd.img
Switch from xmlParse* to xmlRead* so we can explicitly specify parsing
flags, including no network usage and no entity resolution.
The two behaviours mentioned above were not done before either, so this
should not introduce behaviour changes in libguestfs.
qemu-system-arm 2.0 as currently available through Debian/unstable
doesn't want to tell us anything about available devices unless we
specify a machine type.
When parallel builds are enabled it was possible for index-parse.c to
be compiled before the file had been completely written by bison. The
usual symptom was that the 'do_parse' symbol was missing -- this
simply happened because that symbol is defined at the end of this
file, and the compiler compiled the file before the end part was
written out. But you could also see other strange & random compile
failures as you would expect.
This was tested by running this script on an 8 core server:
cd builder/
export MAKEFLAGS=-j9
while make clean >/dev/null; make all V=1 >& /tmp/log; do echo -n . ; done
Previously it would fail after <= 5 iterations. With this change it
runs for at least 100 iterations.
Parsing sources .conf files is not a fatal error (that file would just
be ignored), so explicitly state that such parsing errors are ignored.
This should address the last bit in RHBZ#1077817.
Convert the tests to the MiniTest test framework, but keeping the usage
of the old Test::Unit as fallback in case MiniTest is not available.
In the latter case, use a bit of "glue" to make the old API look like
the new API, so we can just rely on the newer MiniTest API.
Isolate in a common file all the standard boilerplate in tests, i.e. the
import of the test framework and the guestfs module (including the
import path hack needed for the latter).
Thanks to Vít Ondruch for the precious hints and suggestions.
Use parted to get the name of partitions in GPT layouts instead of
sgdisk, to reduce the possible discrepancy between output of tools.
The actual case here is that recent parted versions fixed/improved their
UTF-16 handling of partition names in GPT, and sgdisk seems to not be
properly handling them, returning also unicode control bytes.
Since parted can provide partition names already, just make use of it.
Since sgdisk is no more needed for part_get_name, the function is no
more optional on it.
Add the possibility to configure the proxy in each repository .conf
file, specifying whether use no proxy at all, follow the system
configuration or use a specific proxy.
Previous commit 97c57a8887 added a
potential bug in the sparsify build. If virt-sparsify had used a
*.mli file then it would have tried to run a command like:
ocamlc mlguestfs.cmxa -c foo.mli -o foo.cmi
This would fail because ocamlc doesn't know about native code
libraries (*.cmxa). In fact virt-sparsify was not affected by this
because it doesn't use any *.mli files.
Fix this by adjusting the automake variables.
If a disk has type 0x42 partition (which would indicate LDM), but LDM
is not available then try parsing the partition anyway. It might be
parseable as plain old NTFS.
There are two observations which make this possible: (1) virt-sparsify
contains at least one C unit, so automake thinks it is compiling a C
program. (2) Automake lets us add a *_LINK rule to override linking.
We list the C sources of virt-sparsify as virt_sparsify_SOURCES, so
automake thinks it is building a C program as normal.
We override virt_sparsify_LINK to make it actually link an OCaml
program.
We add virt_sparsify_DEPENDENCIES listing the OCaml objects so that
they get built before linking.
We need a small linker script (link.sh) which adds some options at the
end of the linker command line which are impossible to add using pure
automake.
Inspired by the same idea in supermin:
https://github.com/libguestfs/supermin/blob/master/src/Makefile.am
As long as over ~300MB is recovered, declare sparsification to be
successful in this test.
It looks as if the journal or other metadata takes up a variable
amount of space after sparsification.
We must avoid touching appliance/supermin.d unless it is necessary, so
that we avoid unnecessary rebuilds of the full appliance.
Unfortunately since we created temporary files there, even if we
didn't decide to keep those temporary files they would still end up
touching supermin.d. To stop this, move the temporary files out.
Exclude the arch-independent pkg-config files (no pkg-config available
in the appliance).
Exclude also the doc-base and reportbug files, typically found in Debian
systems.
Add a new lexer token, unused in the grammar, for the unknown lines in
index files; this should allow to better handle such kind of parsing
errors, removing the need to exit() directly (and leave things in an
unclean state).
Since HPET is specific to x86, we can safely add it its option only on
x86 and x86_64 when creating the libvirt XML (no more hitting the
launching failures due to that on other architectures).
Regarding the direct qemu launch, since qemu 1.1 (which is our current
minimum) "-ho-hpet" appears in the help only where actually supported,
so we could just checking for it and adding it only if present. This
should fix the architecture issues on this backend as well.
The boot partition name is /dev/ubda1 in this case, so use a small sed
script to canonicalize it for the test comparison.
This fixes commit f6a37740f2.
61323fa introduced a mandatory arch field within index files to
identify the architecture of the OS installed within a given template.
This change simply documents the requirement within the relevant
section of the virt-builder man page.
Signed-off-by: Lee Yarwood <lyarwood@redhat.com>
This removes the internal APIs internal_set_libvirt_selinux_label and
internal_set_libvirt_selinux_norelabel_disks. The communication
between the libvirt domain and the backend now uses the backend
settings.
Currently the backend settings are treated as a list of strings. You
can set the whole list (clearing any strings there previously), but
you cannot search for an individual string or replace an individual
string.
This adds further APIs allowing you to do that. We treat the backend
settings as a list of environment-like strings (ie. name=value), and
add the following functions:
- set-backend-setting (name, value)
Set name=value. Any previous settings of name are cleared.
- get-backend-setting (name)
Search for name or name=value and return the value.
- clear-backend-setting (name)
Remove any name or name=value settings.
This also adds a regression test.
This is not used by libguestfs right now, but it does allow you to
build a python "distribution" of libguestfs, like this:
make -C python sdist
The distribution tarball will be in python/dist/
You can copy the distribution tarball somewhere, unpack it, and use
regular 'python setup.py' commands such as:
python setup.py build
python setup.py install [--prefix=...]
python setup.py --name
In future we hope to be able to upload the distribution tarball to
PyPi, but licensing issues prevent this at present.
guestfs___validate_guid was a new function added to utils.c in
commit beef77403c.
However utils.c should not include <guestfs-internal.h> since the
other functions in this file can be used by all front-end code, not
just libguestfs.so.
This function is only needed in libguestfs.so, so move it to another
source file, and remove include of <guestfs-internal.h> from utils.c.
Also: use 'size_t' for counting, not 'int'.
This fixes commit beef77403c.
Breaks Python distutils which doesn't use C99:
utils.c: In function ‘guestfs___copy_string_list’:
utils.c:79:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
for (size_t j = 0; j < i; ++j)
^
utils.c:79:7: note: use option -std=c99 or -std=gnu99 to compile your code
[Gnulib tries to replace getopt because of the bug shown below.
However because -lgnu was not used in the Makefile, the replacement
failed. -- RWMJ]
from config.log:
/* This code dumps core on glibc 2.14. */
{
static char program[] = "program";
static char w[] = "-W";
static char dummy[] = "dummy";
char *argv[] = { program, w, dummy, NULL };
optind = opterr = 1;
if (getopt (3, argv, "W;") != 'W')
result |= 64;
}
gdb --quiet --readnow ./conftest -ex r -ex bt -ex detach -ex q
Reading symbols from /usr/src/packages/BUILD/libguestfs-1.26.0/conftest...expanding to full symbols...done.
Starting program: /usr/src/packages/BUILD/libguestfs-1.26.0/conftest
Missing separate debuginfo for /lib64/ld-linux-x86-64.so.2
Try: zypper install -C "debuginfo(build-id)=5d1a12e6f0e95331cc4e39df74ecbd5adb76a1f7"
Missing separate debuginfo for /lib64/libc.so.6
Try: zypper install -C "debuginfo(build-id)=a06caa12df8f953a453befa827c3145adaf6269a"
main(323) result 0
main(344) result 0
main(357) result 0
main(370) result 0
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b2dbf8 in _getopt_internal_r () from /lib64/libc.so.6
Detaching from program: /usr/src/packages/BUILD/libguestfs-1.26.0/conftest, process 31042
This change actually fixes build for me on sles11sp3 and 11.4.
The OCaml compiler isn't happy when two modules have the same name in
different directories. Since customize/ contains a library of modules
used in other directories, and since those directories may have
main.ml files too, we need to rename 'customize/main.ml' to
'customize/customize_main.ml'.
This adds the following options to virt-customize:
--attach
--attach-format
-m / --memsize
--network / --no-network
--smp
This fixes commit 2b208d84db.
The customize module includes an --selinux-label option which can run
'fixfiles restore' (on most SELinux guests) or set /.autorelabel (on
some older ones).
Commit 49014f81f3 renamed the old
--selinux-label option to --autorelabel, but note this was not
included in a stable version of libguestfs.
Note this change leaves a bunch of now redundant code for detecting if
we created a new file in the guest.
Split virt-builder into build and customize steps, so that we can spin
off a separate tool called 'virt-customize'. This commit does not in
fact create such a tool, but it moves all the common code into a
library, in the customize/ subdirectory of the source.
Although this is mostly refactoring, it does change the order in which
virt-builder command line arguments are processed, so they are now
processed in the order they appear, not the inflexible fixed order
used before.
Previously we ran them in essentially a random order, although it
might have looked alphabetical in some cases because the modules are
loaded in order.
Some tests might spawn an appliance, which will take time on slower
architectures and on some virtualized environments.
Hence raise the per-test timeout from the default of 60s to 300s (which
should be hopefully enough for now).
Commit f75142c577 disabled parallel
builds in the ocaml subdirectory (which was the correct thing to do).
This made building the OCaml tests in particular very slow. Therefore
fix things so that parallel builds can be used again.
See also discussion here about different approaches:
https://www.redhat.com/archives/libguestfs/2014-March/thread.html#00223
This reintroduces commit dce94f3e26.
When parsing the repository configuration files, track the file names
parsed, and in case of same file name in multiple location consider only
the one in the directory coming before others in the prioritised list of
paths.
This way it is possible to "shadow" a configuration file in a system
path with one with the same name in a local directory, for example.
Always close the file (ignoring its result) after a parsing, and cleanup
the parse_context object before any exit().
This eases the debugging of memory issues in the actual parser.
This changes podwrapper so that the input (POD) files should not
contain an =encoding directive. However they must be UTF-8.
Podwrapper then adds the '=encoding utf8' directive back during final
generation.
This in particular avoids problems with nested =encoding directives in
fragments. These break POD, and are undesirable anyway.
Switch the lex/yacc parser into reentrant mode, to ease the handling of
parsing-specific data; introduce a new parser_context struct for that,
which is added as extra data to the parser.
This should cause no behaviour changes in the parsing, just no more
global variables used for getting data in/out the parser.
So far the failure of some test would have not reported a non-zero
return value by run-tests.php. Since now all the PHP tests pass, we can
ask for failures to be fatal, by exporting REPORT_EXIT_STATUS=1 for
run-tests.php.
We have generally forbidden declaration-after-statement in our code
(although there are plenty of places that do it). In any case this
warning is not in the default set, so we don't need to turn off the
warning.
RHEL 6-era bash cannot handle the multi-line $(...) expression in the
original code. Rewrite the test so it is semantically the same but
doesn't have the expression that old bash can't handle.
The output of test-virt-rescue.pl contains non-ASCII chars which give
troubles when being XSLT-transformed, so just avoid its output to leave
the rest of the resulting XML well-formed.
If the RUN_OUTPUT_FILE environment variable is set, "run" will output
to the file indicated by that snippets of XML with the results and
outputs of the tests run.
Together with the run-xml-to-junit.sh (and its associated
run-xml-to-junit.xsl style sheet) it is possible to convert that output
file to a jUnit-like XML file, which could be used in CI systems.
It's reasonable to expect megabytes will be freed up and given back to
the host. Previously we just tested that at least 1 block was
released. This test checks that at least 1000 blocks are released
(since stat(2) returns blocks of 512 bytes: 512 bytes * 1000 = approx 0.5 MB).
Properly initialize the variables for them to NULL, so there is no crash
because of trying to use rubbish pointers.
Kind of followup of commit 289fd29f0b.
Since the default PHP test runner ignores a good number of environment
variables to potentially tampering the test suite execution, create a
custom php-for-tests.sh script which does nothing more than sourcing the
custom environment that our run-php-tests.sh outputs and running the
actual "php" executable (the one found by configure).
This fixes the loading of the guestfs_php.so module in the test suite,
as the libguestfs.so.0 library can be found by that module.
Commit 71dfd73042 broke 'make clean' in
the appliance subdirectory, so it would remove all the supermin.d/*
files, but without removing stamp-supermin so you couldn't rebuild
supermin.d/ with a 'make' afterwards.
In any case, 'make clean' is not supposed to delete the appliance
(which is expensive to rebuild).
Fix this so that 'make clean' doesn't remove supermin.d/* files.
'make distclean' can be used to delete this, and since it also removes
stamp-supermin, it will be recreated properly by 'make'.
Since virt-sysprep tends to delete a lot of files, adding discard
support to it makes some sense.
Note that this probably won't have any effect for most filesystems
since:
(a) ext4 mounts also need to use -o discard,
(b) ext4, and maybe others, require you to call fstrim explicitly,
they don't discard automatically (except for userspace tools like
mkfs.ext4 but that doesn't apply in this case).
Allow callers to specify that all the disks from a domain are added
with a specific cachemode (instead of always having to use the
default, writeback).
This adds a discard parameter to guestfs_add_drive_opts which
approximately maps to the discard=ignore|unmap parameter supported by
qemu.
If discard is set to "enable" then we force discard=unmap (and try to
fail if it is not possible). If discard is set to the more useful
"besteffort" option, then we enable discard if possible. The default
is "disable".
Add the possibility to choose which architecture use to build the wanted
image (--arch). Since this implies that running commands on the guest is
usually not possible when the architecture is different than the host
one, virt-builder will nevertheless try to check whether the host is
compatible with the guest, allowing to run commands only in that case.
The caching scheme is adapted to account for the architecture (with
--print-cache showing the architecture as well).
Group together all the various parameters (except the guestfs handle)
passed to the create_drive_*, to avoid passing all of them at every
function call.
This is mostly an internal refatoring with no behaviour change.
RWMJ:
- Patch taken from Debian downstream package.
- The default in the original patch was to disable the tests. I have
changed this to default to enable them instead.
- Removed AC_SUBST as the variable was not being used.
If appliance is disabled, the whole appliance/ subdirectory is
skipped.
If appliance is enabled, we can assume that the daemon is enabled too
(see previous commit) so we don't need the conditional code. That
conditional code didn't work since the supermin 5 rewrite.
See top of configure.ac:
> # NB: AC_CHECK_PROG(S) or AC_PATH_PROG(S)?
> # Use AC_CHECK_PROG(S) for programs which are only used during build.
> # Use AC_PATH_PROG(S) for program names which are compiled into the
> # binary and used at run time. The reason is so that we know which
> # programs the binary actually uses.
/proc/sys/kernel/hotplug is an obsolete interface for processing
hotplug events. It is not used by modern kernels, which should set
this path to the empty string (because they have
CONFIG_UEVENT_HELPER_PATH="").
Unfortunately our udev initialization code did:
echo -e '\000\000\000\000' > /proc/sys/kernel/hotplug
which was intended to set this file to \0 bytes. However on Debian,
where /bin/sh is "dash", not "bash", this writes "-e" to this file,
which breaks udev events.
Thanks: Peter Rajnoha for solving this.
https://bugzilla.redhat.com/show_bug.cgi?id=1020806#c10
Update the documentation according to the new .conf files structure, the
drop of VIRT_BUILDER_SOURCE and VIRT_BUILDER_FINGERPRINT, the drop of
hardcoded source and its key, etc.
Sigchecker can still import keys from the user's keyring, so there is no
need to hardcode fingerprint and key of the index hosted on
libguestfs.org (which is now pointed to using a .conf file).
Drop these two environment variables, and the implicit hardcoded source
hosted at libguestfs.org.
This means all the sources must be provided as .conf files, or at each
invocation with --source.
Introduce and use simple .conf files to configure the sources of indexes
for virt-builder. The location of these files is in XDG_CONFIG_DIRS /
XDG_CONFIG_HOME, so it can be easily overridden.
There are three .conf(.in) files shipped with this commit:
- "test-index.conf.in" (in "test-config"), which points to the
"test-index" index (used in tests only); the tests are adapted to
point to the hierarchy containing this .conf
- "libguestfs.conf.in" (in "test-website"), which points to the local
"index.asc" (i.e. the offline copy of the libguestfs.org index);
run(.in) will point to the hierarchy providing this .conf
- "libguestfs.conf.in" (directly among the other sources), which points
to the online "index.asc" and it is installed in sysconfdir, along
with the key of this repository
The tests are adapted, other than to the different way to pick sources,
to the different output of --list, as "test-index" is not signed.
Use Sigchecker.gpgkey_type instead of just string as type in the
sources list; adapt the listing code (and its expected output) to that.
No behaviour change which eases a bit the addition of new sources with
other key types.
Move in an own module the code which calls the C
virt_builder_parse_index and does the array -> list conversion of the
result. This way this code can be easily called also in places different
than Index_parser without the need to copy the types mapping, etc.
Just code motion, no actual behaviour changes.
Introduce a mandatory arch= key in all the entries of index files, to
identify which architecture is each. Adapt the long and JSON list
outputs to print also this new field.
This introduces an incompatibility with index files created with
virt-builder < 1.26, as they will be rejected until entries will have
the arch= key added (which is ignored by older virt-builder, so adding
it will not create backward-compatibility issues).
Extend Sigchecker so it allows both fingerprints (to be imported from
user's keyring, as before) and keys stored in files. To simplify this
process (and have the fingerprint always around), the key is imported
on Sigchecker.create time, instead of lazily at the first verification.
Create a temporary directory and tell gpg to use it as homedir, so
imported keys do not get into the user's keyring. This also avoid
importing the default key when a different one is needed to check the
signature.
The only exception is when a non-default fingerprint is used: in this
case, that key is read from the user's keyring, since it is where it is.
Even though so far there is just one possible setting to read, isolate
in an own function the code to parse a configuration file and read the
settings out of it.
Now there's a new config_t handle used every time, but since config_read
would reset an handle completely, there is no behaviour change.
Look for use of external_functions and fish_functions and replace with
use of external_functions_sorted and fish_functions_sorted where
possible. This ensures that the output of the generator is sorted as
far as possible.
I also checked for uses of internal_functions and documented_functions
but those are not used. The *_sorted versions are always used
instead.
CC virt_index_validate-index-validate.o
index-scan.c:1270:17: warning: 'yyunput' defined but not used [-Wunused-function]
static void yyunput (int c, register char * yy_bp )
^
CCLD virt-index-validate
By using %option nounput we can prevent this from being generated in
the first place.
This is just a whitespace change.
This was done by running the following command and then fixing a
couple of places where it got it wrong:
perl -pi.bak -e 'unless (m/#define/) { s/([a-zA-Z0-9])\(/$1 (/g }' \
generator/gobject.ml
The warning only appears when we set CFLAGS to be the same as used by
RPM, which you can do by setting:
CFLAGS="$(rpm --eval '%{__global_cflags}')"
before running ./configure.
FWIW the current value of that RPM expression (in Fedora 20) is:
-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches
It is not clear which option precisely enables the warning.
If calling guestfs_list_filesystems with a disk image containing a
corrupt btrfs volume, the library would segfault. There was a missing
check for a NULL return from guestfs_btrfs_subvolume_list.
This adds a check, returning the real error up through the stack and
out of guestfs_list_filesystems.
This is potentially a denial of service if processing disk images from
untrusted sources, but is not exploitable.
Thanks: Jeff Bastian for reporting the bug.
Small refactor of Downloader.download_to to allow different download
actions depending on the protocol of the URI (which is now parsed).
No actual behaviour changes, just mostly code motion.
Switch the type of the "guid" parameter from "String" to "GUID"; this
adds the validation of the GUID as such, rejecting straight away invalid
GUIDs which otherwise could be handled badly by low-level tools (such as
sgdisk).
Add a couple of easy tests (taken from RHBZ#1008417) to
part_set_gpt_type about this.
Isolate in an own function the code that runs sgdisk and parse a field
of it (using an extraction function passed as parameter), using it for
the GUID type.
This is just code motion, no actual behaviour changes.
If there is a GPT partition layout, then what should be read and
restored for each partition is the GPT type and not the MBR ID.
Related to RHBZ#1060404.
This adds:
- GUESTFISH_INIT
- GUESTFISH_OUTPUT
- GUESTFISH_RESTORE
which along with existing GUESTFISH_PS1 allow you to fine control the
colour of the output.
Output all the translations available for the notes in the "verbose"
output and the JSON output, while trying to match the system langauge in
the "show notes" output.
The JSON output is slightly changed to handle translations, with the
"untranslated" notes being matched as "C". The version is not bumped
though, since there have been no stable releases with the former output
yet.
virt-builder added a --selinux-relabel option with a slightly
different meaning.
This commit keeps the old --selinux-relabel / --no-selinux-relabel
options in virt-sysprep, but deprecates them and replaces them with
--autorelabel and --no-autorelabel.
The whole "COPYING AND CLONING" section really needs to be overhauled
too, but first we need to make virt-builder work properly for
customizing templates.
The previous translation from Perl slavishly followed the Perl code a
bit too much and used a named pipe to communicate between the
uncompressing subprocess and libguestfs tar-in. From C we can use an
anonymous pipe instead.
This updates commit d3512deb67.
Otherwise it moans about:
mkfs: ntfs: /dev/sda: /dev/sda is entire device, not just one partition.
Refusing to make a filesystem here!
This has been happening for a while, so I don't know why we didn't
notice it before. It even happens on RHEL 6.
A simple reproducer is:
virt-make-fs -t ntfs any-tarball.tar.gz test.img
Strictly speaking this reduces the number of formats that virt-make-fs
can output to, but it's likely that no one cares and if they do we can
add new formats in future.
The following commit managed to not actually add the --foreground
option to the timeout command, just test for it. Add it this time.
commit 6814888774
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Dec 19 08:21:53 2013 +0000
run: Use timeout --foreground option.
If timeout doesn't have this option (RHEL 6) don't use timeout at all.
Attempt to fix RHBZ#1025269.
Commit 72afcf450a was partially
incorrect. If the guest userspace is expecting /selinux to exist,
then we should bind-mount /sys/fs/selinux from the appliance kernel
there.
virt-builder and virt-sysprep may make use of
Common_utils.string_random8 (which uses Random.int) for constructing
temporary paths; not initialising the random generator means that every
invocation will reuse the same name used previously (!).
Thus just call Random.self_init, just like virt-sparsify already does.
Expand the test-virt-sysprep-script.sh test to ensure that virt-sysprep
is not affected again by this issue.
All it did was checking for a libvirt version, which is what
libvirt-is-version now does; hence remove the C part, and use guestfish,
ignoring the launch failure (as the C test did).
This adds the --selinux-relabel option which enables selinux in the
appliance and runs:
if load_policy && fixfiles restore; then
rm -f /.autorelabel
else
touch /.autorelabel
fi
at the end of installation.
When possible this fixes SELinux labels in the guest and makes the
autorelabel step unnecessary.
Notes:
- The previous commit is required so that load_policy works.
- During the build, SELinux is enabled but no policy is loaded. This
works because SELinux is in permissive mode.
- This flag does not work if the appliance kernel and the guest have
greatly differing versions, eg. a RHEL 6 guest with a Fedora 20
appliance. This is because SELinux changes the policy format and
breaks backwards compatibility. You would see errors like this:
libsepol.policydb_write: policy version 15 cannot support MLS
libsepol.policydb_to_image: could not compute policy length
libsepol.policydb_to_image: could not create policy image
SELinux: Could not downgrade policy file /etc/selinux/targeted/policy/policy.24, searching for an older version.
SELinux: Could not open policy file <= /etc/selinux/targeted/policy/policy.24: No such file or directory
These errors are ignored (they go to the log file) and relabelling
is done at boot instead.
- It's not clear if loading guest policy is safe. You should trust
the virt-builder templates and to use libguestfs confinement for
additional protection.
Even though we are already bind-mounting /sys, it is necessary to also
bind-mount /sys/fs/selinux in order for SELinux commands (in
particular, 'load_policy') to work.
This fixes/reverts commit 7367729ec7.
It is never normally valid to use the mount-local* APIs when you
haven't mounted some filesystems in the libguestfs namespace.
If you try it, it results in some odd errors. The mount-local-run
call is successful, but subsequent operations fail:
$ mkdir -p /tmp/mnt
$ guestfish -x -N fs mount-local /tmp/mnt : mount-local-run
libguestfs: error: lstat: lstat_stub: you must call 'mount' first to mount the root filesystem
libguestfs: error: lstat: lstat_stub: you must call 'mount' first to mount the root filesystem
With this commit, the mount-local-run call gives an error:
libguestfs: error: you must call 'guestfs_mount' first to mount a filesystem on '/'.
Note: '/tmp/mnt' is still mounted. Use 'guestunmount /tmp/mnt' to clean up.
The error is implemented by calling guestfs_exists (g, "/") which has
the side effect of running the NEED_ROOT macro in the daemon. (There
is no simple equivalent of NEED_ROOT on the library side.)
This fixes the handling of paths for the protocols named above, with
respect to leading '/' (or not) on the volume/export name.
See previous commits which did the same fixes for Ceph:
commit 53a3ff9c00
commit 992a6b2970
If we rely on OCaml's internal exception printing, then it will
truncate error messages like this:
Fatal error: exception Guestfs.Error("could not create appliance
through libvirt.
Try running qemu directly without libvirt using this environment
variable: export LIBGUESTFS_BACKEND=direct
Original error from libvirt: unable to set security context
'unconfined_u:object_r:svi
note the truncation here ^^^^^^^
Instead of using the internal exception printing, wrap the whole
program in a 'main ()' function and add an exception handler.
This large change is non-functional apart from the addition of the
exception handlers at the end.
This tests that libvirt domain XML (ie. guestfish -d option) is parsed
correctly and results in the correct qemu command line when using the
direct backend.
This is a good end-to-end test of various layers.
This is valid for some network drives, for example:
<source protocol='nbd'>
<host name='foo' port='1234'/>
</source>
We pass an empty string as path to the guestfs_add_drive_opts API in
this case.
This is valid for some network drives:
<source protocol='rbd' name='abc-def/ghi-jkl'/>
instead of this:
<source protocol='rbd' name='abc-def/ghi-jkl'>
<host name='foo' port='1234'/>
</source>
Allow both forms.
The path at the API level (for guestfs_add_drive_opts) is:
pool/disk
The URI syntax is either rbd:///pool/disk or rbd://server:port/pool/disk.
Because of the way URI parsing works we may need to remove a leading
'/' character before passing the path down to the API.
The path at the protocol level is:
pool/disk
(with no leading '/' character). This is now what you have to pass to
guestfs_add_drive_opts.
Also Ceph can be called with no explicit servers (it uses the contents
of /etc/ceph/ceph.conf instead). So allow zero servers to be used.
Switch the internal storage for the notes of each entry to a sorted list
with all the subkeys available (which should represent the translations
to various languages).
The current outputs are the same (i.e. still the untranslated notes), so
this is just internal refactoring/preparation.
Because EXTRA_DIST simply used $(TESTS), and because the list of tests
includes C binaries, these were included in the tarball (actually not
binaries, but libtool scripts, but it's still undesirable).
virt-sparsify doesn't work if the output is a block device, and cannot
possibly work if the output is a char device. Currently if you try
this it actually overwrites (deletes) the output device which is not
exactly desirable. Therefore throw an error and exit.
'chmod --reference' is not available on Mac OS X.
Keeping the original file and directing edits into it preserves permissions and removes the need to chmod.
(Tried 'sed -i' for inplace editing but unfortunately it isn't portable either.)
The index files already allowed the 'key[subkey]=...' syntax for keys,
but considering such string as whole key. Proper split the parsing and
the handling of the subkeys, so they can be searched a bit easier.
This causes no actual behaviour changes, it is just internal
micro-refactoring. (Thanks Rich for the hints, too.)
Previously callbacks would return a list of flags, such as []
or [`Created_files].
In this commit we introduce two new objects, filesystem_side_effects
and device_side_effects (the latter is not used yet).
The callbacks that create files now need to call
side_effects#created_file ()
instead of returning flags.
There is no functional change in this patch.
Improve the index grammar to allow more than one empty line between
sections, and to allow any number of empty lines at the end of the file
(after the last section).
0.10.2+ is still available on some recent-ish stable distros (like
Fedora 19, Ubuntu 13.04), and should work fine for guestfs.
This reverts commit 6b3e05e8da.
By using:
export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
you can force the libvirt backend to use TCG (software emulation)
instead of KVM (hardware accelerated virtualization).
By using:
export LIBGUESTFS_BACKEND=direct
export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
you can force the direct backend to use TCG (software emulation)
instead of KVM (hardware accelerated virtualization).
This is sometimes useful, especially if you are trying to use
nested KVM on Intel.
Allow settings (an arbitrary list of strings) to be passed to the
current backend. This will allow us to tweak how the backend works,
eg. by forcing TCG.
tests/nbd/test-nbd.pl contains a hack (a call to sleep) to wait
for qemu-nbd to start up in another process.
This sleep was previously located before the call to $g->launch()
since that is where (previous to commit 4a0f5ed382) the backend would
have connected to the NBD server.
However in the new code, an initial NBD connection test is done when
the drive is added (ie. $g->add_drive (..., protocol=>"nbd")), which
duly failed intermittently because the qemu-nbd server had not opened
its socket yet.
Move the wait earlier -- just after the fork -- to avoid this.
This updates commit 4a0f5ed382.
In some places we turn relative paths passed by the caller into
absolute paths: for example in the libvirt backend, all relative paths
to disk images have to be turned into absolute paths (by calling
realpath) because otherwise libvirt doesn't work.
This has the potential to break things, especially considering that
qemu uses the path passed to it in order to resolve the relative paths
of backing files.
For example consider the case where 'overlay' is a symbolic link to
/dev/dm-1, but /dev/dm-1 contains a qcow2 which references './backing'
as its backing file. If we rewrote 'overlay' to '/dev/dm-1', then
qemu would try to look up '/dev/backing' for the backing file and
fail.
See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1053684https://www.redhat.com/archives/libguestfs/2014-January/thread.html#00087
This fixes commit 4a0f5ed382.
A simple test of this:
guestfish <<EOF
add-drive "" readonly:1 format:raw protocol:nbd server:localhost:10000
EOF
which would core dump.
qemu has broken snapshot=on ... again.
Change the way that drives are created so that the backend no longer
has to use snapshot=on, <transient/> (which never worked), or UML's
corresponding COW-creation feature (also broken).
Instead of that, the src/drives.c code will create overlays when
required by calling into a new backend operation 'create_cow_overlay'.
This operation runs 'qemu-img create -b' or 'uml_mkcow' as determined
by the backend, and returns the name of the overlay.
The format of the overlay is still backend-specific because qemu needs
to use qcow2 and UML needs to use COW.
This patch also includes some factorization of the libvirt XML code.
This also drops the drv->priv (private per-drive data) field, since it
is no longer used by any backend.
This also moves the guestfs___drive_source_qemu_param utility
function, used & shared by the direct & libvirt backends only, into
src/launch-direct.c (from src/drives.c).
In some tests we were specifying qcow2 as the image format when adding
a disk, but actually passing a raw format image.
Libguestfs previously did not detect this until guestfs_launch, but it
was still a bug to pass an incorrect format to guestfs_add_drive_opts.
It only worked because these tests never call guestfs_launch.
A later commit in this series will cause this to be detected
(sometimes) during guestfs_add_drive_opts.
Switch from a boolean for the short/long list output to labels for the
actual format. Also, split the output of each list format to an own
function for easier maintaineance.
Not all the fields of struct stat are actually filled by us. This caused
rubbish to appear in the microseconds fields, which were then used as
base when changing atime/ctime (with e.g. touch), triggering EINVAL by
futimens/utimensat when those rubbish values were out of the range
allowed for microseconds.
Add a new --operation parameter which, similarly to --enable, can be
used to enable operations, but also to remove them, and to add/remove
the default operations and all the available ones.
Previously the list was reversed. Note this doesn't have any effect
on how the packages are installed since the guest package manager
doesn't care about the order.
This is too complex to implement directly in virt-builder.
Instead we just document how to do it for some common Linux distros
using --run-command, --edit etc.
Move in an own function the code that does the (l)listxattrs allocating
the buffer of the right legth, as it will be useful later.
No functional changes, just code motion.
Instead of enable them when having one of the two headers for it but
still checking for the HAVE_* availability of each *xattr() function
used, just enable the linuxxattrs as a whole when having any of the
needed headers (like before) and all the needed functions.
This might cause the linuxxattrs to not be available anymore on systems
without the whole set of *xattr() functions implemented, but OTOH it
simplifies the xattr.c implementations.
It now matches the documentation:
"Select the size of the output disk, [...]
Virt-builder will resize filesystems inside the disk image
automatically.
If the size is not specified, then one of two things happens. If
the output is a file, then the size is the same as the template. If
the output is a device, partition, etc then the size of that device
is used."
This also adds checks to make sure that users don't try to shrink a
disk (which is not possible), and they don't try to resize larger than
the output block device.
This means there will be no default hypervisor, and effectively the
user will always have to specify one (eg. by setting LIBGUESTFS_HV or
calling guestfs_set_hv).
This is useful on platforms where qemu doesn't work, or where qemu is
not needed (eg. if you want to use UML, or you just want to compile
libguestfs without using/testing it).
The .new method was unintentionally broken in
commit 9466060201.
This fixes the .new method and allows it to be called with multiple
parameters, so you can use:
Guestfs::Guestfs.new
Guestfs::Guestfs.new()
Guestfs::Guestfs.new(:close_on_exit => false)
etc.
For backwards compatibility, Guestfs::create may still be used.
This commit also adds regression tests:
- Use .new method in regular tests. (Because this was not done
before, we didn't catch the breakage.)
- Test that ::create still works.
- Test that args can be passed to .new method.
Added in 2392fae680 and removed in
be025369ad622867ecf08d5c9170e8e59d2d6294; now they should work with any
backend, as the device names are normalised and not change the guestfish
output depending on the used backend.
Thanks Richard for the hints.
Instead of:
libguestfs: error: could not create appliance through libvirt.
Try using the direct backend to run qemu directly without libvirt,
by setting the LIBGUESTFS_BACKEND=direct environment variable.: internal error: process exited while connecting to monitor: qemu-system-x86_64: -drive file=/tmp/libguestfs3RnJSd/snapshot2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=writeback: could not open disk image /tmp/libguestfs3RnJSd/snapshot2: Could not open backing file: Could not open '/home/lsm5/fimagebuilder/fedora-imagebuilder/fedora-imagebuilder-sda.qcow2': Permission denied
it will now say:
libguestfs: error: could not create appliance through libvirt.
Try running qemu directly without libvirt using this environment variable:
export LIBGUESTFS_BACKEND=direct
Original error from libvirt: internal error: process exited while connecting to monitor: qemu-system-x86_64: -drive file=/tmp/libguestfs3RnJSd/snapshot2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=writeback: could not open disk image /tmp/libguestfs3RnJSd/snapshot2: Could not open backing file: Could not open '/home/lsm5/fimagebuilder/fedora-imagebuilder/fedora-imagebuilder-sda.qcow2': Permission denied
NULL means that libvirt gets to guess what connection URI we meant.
Currently it guesses qemu:///session for non-root and qemu:///system
for root. Except if LIBVIRT_DEFAULT_URI is set in which case this is
used even if it's not appropriate. Except if Xen is installed in
which case it guesses xen:/// which is not helpful at all.
Instead use qemu:///session (or qemu:///system for root -- that's
likely to be wrong, but we can't help that).
If the user really wants to use NULL and let libvirt guess, then they
can now specify the libvirt:null backend.
See:
https://bugzilla.redhat.com/show_bug.cgi?id=1045033https://bugzilla.redhat.com/show_bug.cgi?id=886915
A good way to test this is:
LIBVIRT_DEFAULT_URI=iambroken libguestfs-test-tool
The libguestfs libvirt backend should (after this commit) ignore
LIBVIRT_DEFAULT_URI. However other parts of libguestfs will still use
it, eg. guestfs_add_drive_opts, guestfish '-d' option.
A single file can have multiple xattrs. Previously these were
returned in the same order as the kernel returns them. However it is
more useful if they are sorted by attribute name.
The wrong pointer was passed to the visitor_function, resulting in it
seeing the placeholder xattr entry (the one which stores the length of
the list of xattrs - see
http://libguestfs.org/guestfs.3.html#guestfs_lxattrlist ).
virt-ls doesn't list xattrs at the moment, hence this bug was not seen
before.
This is essentially just code motion, except:
(1) It cleans up a few variable declarations which were implicitly
used by the old macro that aren't needed any more.
(2) The options are reordered alphabetically.
The current add_cdrom way basically appends a new raw "-cdrom /path"
parameter to the qemu invocation (even when using libvirt as backend),
hence such images are seen as "CD-ROM drives" inside the appliance.
However, there is no need for such particular behaviour, as they need to
be handled as normal (read-only) drives.
Adding CD-ROM disk images as drives also changes the device names used
for them inside the appliance from /dev/srN to the usual e.g. /dev/sdX.
These changes fix different issues:
- it is possible to start guestfish without adding disks with -a, then
just add-cdrom and run
- list-devices does not cause guestfsd to crash when sorting the list
of devices (exposed by the test case in RHBZ#563450)
- the result of list-devices now reflects the order images were added
(RHBZ#563450)
add_cdrom is still deprecated, but now in favour of add_drive_ro
(instead of add_drive), with its documentation reflecting that.
Add two small regression tests for the fixes described above.
virt-builder --output /dev/null is tempting, but not possible, and
might result in /dev/null being deleted(!) Check for this case and
prevent it happening.
Previously we would use the virt-resize --no-sparse option if the
final output file was a block device. This is safe, but unnecessary
for example if virt-resize was used as an intermediate step. So only
use this option if virt-resize is actually writing to the block
device.
The original template might be, say, xz-compressed raw of a certain
size. We need to work out how to convert it to, say, qcow2 with a
larger size, in as few operations as possible.
Instead of using a nasty aggregation of special cases to do this, use
a breadth-first search over all possible trees to try to find an
optimal plan.
(Actually the current implementation isn't optimal, but it's fine for
virt-builder.)
It seems that FUSE can invoke flush to make sure the pending changes
(e.g. to the attributes) of a file are set. Since a missing flush
implementation is handled as if it were returning ENOSYS, this can cause
issues later.
To overcome this, just provide a stub implementation which does nothing,
since we have nothing to do and don't want to have FUSE error out.
Furthermore, uncomment the timestamp checks in test-fuse.sh, since now
they should be working fine.
This avoids:
xgettext: error while opening "libguestfs.pot-t" for reading: No such file or directory
which I observed with xgettext (GNU gettext-tools) 0.18.3
If you use virt-make-fs to create a partitionless FAT-formatted disk
image then currently you will get an error:
$ virt-make-fs --type=fat . /tmp/test.img
'mkfs' (create filesystem) operation failed.
Instead of 'fat', try 'vfat' (long filenames) or 'msdos' (short filenames).
mkfs: fat: /dev/sda: mkfs.fat: Device partition expected, not making filesystem on entire device '/dev/sda' (use -I to override) at /usr/bin/virt-make-fs line 508, <PIPE> line 1.
...propagated at /usr/bin/virt-make-fs line 518, <PIPE> line 1.
With this patch, the error goes away and partitionless disks can be
created.
When receive_file_data calls guestfs___recv_from_daemon, and that
function returns -1, it adds the following error to the handle:
receive_file_data: parse error in reply callback
However this hides a more specific error message which was already
added by guestfs___recv_from_daemon.
When chunked encoding fails, the following error is added to
the handle:
[filename]: error in chunked encoding
However all paths returning -1 from receive_file_data already set a
more specific error, which this error hides.
The latter is a better way to disable the qemu display output as we
need to, without enabling extra devices (which are disabled already,
anyway).
Also, related to the change above, ban the -display parameter from the
ones that can be supplied by the user.
Don't change "os-version" to "OS version". It's intended to be the
same as the virt-builder man page, which consistently calls this the
"os-version" string.
This partially reverts commit ee0e56f43e.
Fix the dependencies of the libguestfs.pot target: other than using the
right make variables holding the contents of the POTFILES, depend also
on the POTFILES themselves.
- when a command needs no parameters, tell that explicitly instead of
"command should have 0 parameters"
- use gettext's plural form when printing the number of required
arguments
- improve the error message for a variable number of parameters limited
only in the maximum number of them, using also a plural form
Extracting separately the pot for the various languages and then
creating manually the global pot (by manually joining the above ones
after having stripped their headers) is wrong, since other than being
an hack it can create an invalid pot when the same message appears in
sources written in different languages.
Instead, a cleaner and safer solution is to first let ocaml-gettext
(if available) extract the messages for the ml files, and then use
xgettext to extract the messages for the other languages, joining the
new messages to the existing (or not) pot file.
Check for /boot/loader.rc as "install disc" detection, using it to mark
FreeBSD install discs.
Also, check for /mfsroot.gz to see whether such disc is also a live one.
See also RHBZ#1033207.
/etc/fstab in installation discs of FreeBSD can have an entry pointing
to the mounted CD itself; skip it as it is done with other CD devices in
check_fstab.
Although this test only specifically tests the direct backend, the
same code is shared with the libvirt backend in some circumstances
(eg. creating readonly drives). So this is testing a bit of both.
This allows you to select both locked accounts and disabled
passwords. The two are subtly different concepts.
A locked account [cf. passwd -l] puts "!!" at the beginning of the
shadow password field. Locking is reversible, because the "!!" can
be removed, restoring the original password. Therefore "locked"
acts as a flag in front of an existing selector.
A disabled account has "*" in the password field. Therefore it has no
password.
Note that an account may be both locked and disabled, although this is
probably not useful. The shadow password field will contain "!!*".
It now appears in the respective man pages as:
--root-password selector
or:
--password selector
This avoids confusion from people who think these command line options
take the password directly.
If libselinux was detected, it was not added to the linker command
line. This still worked (at least on Fedora) because some other
library pulls in the dependency implicitly, possibly libvirt. However
this broke on Arch:
https://aur.archlinux.org/packages/libguestfs/
Reported by and thanks: Antoni Segura Puimedon.
Char.code (input_char chan) mod nr_chars has modulo bias because
the original interval is not a multiple of the destination interval,
i.e. 256 mod nr_chars != 0.
One way to fix this is to keep generating random numbers until they fall outside
the interval where modulo bias occurs, that is accept only c=[256 % nr_chars, 256).
That interval maps back to [0, nr_chars), and has a length of
(256 - 256 % nr_chars), which is a multiple of nr_chars.
RWMJ:
- Modify the code so it goes into a utility library.
- Use the same code across virt-builder and virt-sysprep.
OCaml's buffered 'in_channel' has a 64k buffer, so using it to read a
few bytes from /dev/urandom removes a lot of the system's entropy (for
example /proc/sys/kernel/random/entropy_avail goes from ~3000 to 128).
This patch was originally by Edwin Török for builder.ml. I
generalized it because there are two other places where we did
over-sized reads from /dev/urandom.
RWMJ:
- Renamed the script from sl.sh -> scientificlinux.sh
- The output file is called scientificlinux-$version
- I fixed some whitespace issues so it's now similar to the CentOS script.
index-parser-c.c depends on index-parse.h being created first.
However without an explicit dependency, this is not done (and implicit
deps don't work because automake doesn't sufficiently understand OCaml
programs).
This fixes commit a4800e2d4f.
Instead of calling out to the pxzcat program, use an internal
implementation. This requires liblzma to be available at build time.
If it's not available, fall back to using regular xzcat.
It is intended that eventually this code will go away when regular
xzcat / unxz is able to use threads.
The device name prefix for IDE hard drives used to be `ad' but now
is `ada' (http://www.freebsd.org/doc/handbook/disks-naming.html).
For virtio hard drives it is `vtbd'.
Under an MBR partition table a slice will be used, so the name of
the first partitions will be either `ada0s1a' or `vtbd0s1a'. Under a
GPT partition table, where no slice is needed, the name of the first
partition will be either `ada0p1' or `vtbd0p1'.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
This is for backwards compatibility with virt-builder 1.24.0 which
could not parse a key that contains square brackets and numbers.
This updates commit 4b73e0be29.
Users can now specify multiple source paths, eg:
virt-builder --source http://example.com/foo \
--source http://example.com/bar
to get templates from multiple places.
There is still only one built-in path, but we can add more later.
We don't need to sign individual templates, provided the signed index
file contains the cryptographically secure checksums of the templates.
Since the index file cannot be interfered with and the checksums
cannot be spoofed, this is sufficient to ensure the integrity of the
templates.
Note the template signatures are now deprecated and will be removed in
libguestfs 1.26. To avoid any backwards compatibility issues, I will
backport the relevant changes to the stable-1.24 branch.
Thanks Dennis Gilmore for pointing out that we could do this.
Instead of typing:
./run ./fish/guestfish ...
you can now use:
./run guestfish ...
This is shorter, but there are also two good reasons for this change:
It never made any sense to run the system-installed guestfish (or
other) binary. It would be unlikely to work. Also binaries in
libguestfs should be able to call other binaries freely without adding
special exceptions (which we had before this change).
For systemd:
Currently stdout+stderr go to the journal (these are effectively
empty unless there are gross errors). The output of the commands
goes to the log file.
With this change, the output goes to the journal, the console and
the log file.
For SysV-init:
Currently stdout+stderr go to the console (but these are effectively
empty). The output of the commands goes to the log file.
With this change, the output goes to the console and the log file.
If:
- the output is a regular file
- the output format is raw
- the user didn't specify the --size option
then we don't need to run virt-resize. Simply uncompress
the template directly to the output file.
When using the guestfish --remote or guestfish --listen options,
guestfish would create a socket in a known location
(/tmp/.guestfish-$UID/socket-$PID).
The location has to be a known one in order for both ends to
communicate. However no checking was done that the containing
directory (/tmp/.guestfish-$UID) is owned by the user. Thus another
user could create this directory and potentially modify sockets owned
by another user's guestfish client or server.
This commit fixes the issue by creating the directory unconditionally,
and then checking that the directory has the correct owner and
permissions, thus preventing another user from creating the directory
first.
If guestfish sees a suspicious socket directory it will print an error
like this and exit with an error status:
guestfish: '/tmp/.guestfish-1000' is not a directory or has insecure owner or permissions
Thanks: Michael Scherer for discovering this issue.
Version 2:
- Add assigned CVE number.
- Update documentation.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
This is not part of the automated test suite. It's a manual test run
by the maintainer which must be run on each new guest to ensure that
all the virt-builder features work on the new guest.
Since this is essential for security, be much more proactive about
trying to create a random seed file, even for guests which have buggy
installers that don't create this file during installation.
This commit removes a lot of cruft from old RHEL which was required to
start udev. Instead we run the daemon directly.
Also it removes the fallback path which created some device nodes in
/dev by hand. This likely did not work. This also removes the
dependency on MAKEDEV which is long gone from most distros.
This is required for Ubuntu, since the /etc/init.d/udev script didn't
actually work. The most common symptom was that /dev/virtio-ports/*
was not created because udev didn't process the cold start events.
This should be safe, and avoids the whole issue of whether a guest
uses systemd or not. Also it is possible to switch init systems
(eg. by specifying init= on the command line) so it's not even clear
if a guest "is" systemd.
Thanks: Tom Gundersen on #systemd.
This updates/fixes commit 596de56a43.
Ubuntu 13.10 has /etc/systemd but uses Upstart. This confuses the
script because it assumed that if you have /etc/systemd then you are
using systemd. Ubuntu includes systemd services (inherited from
Debian) but they just don't run.
If the output is a block device:
- Detect the size of the output device.
- Don't attempt to delete the output device on failure.
- Turn off sparse copying in virt-resize.
Some --install requests may pull in packages that start daemons (dbus
being a favourite daemon apparently). These daemons will prevent the
disks from being unmounted and everything being cleaned up at the end.
Add some experimental code to kill these daemons.
Because we're not running msgmerge but concatenating the PO files, you
can get duplicate messages (in this case "no"), resulting in this
error:
libguestfs.pot:6142: duplicate message definition...
libguestfs.pot:3285: ...this is the location of the first definition
Just work around this for now.
Commit c598e14052 exports
LD_PRELOAD=/lib64/libSegFault.so so that guestfsd and processes that
it runs will produce descriptive stack traces.
However it exports it to everything including the virt-rescue shell.
But if we are chrooting into a guest which doesn't have this file (or
indeed any guest) we should not use LD_PRELOAD.
Therefore unset LD_PRELOAD in the rescue shell.
See also related commit 21e5fc811e.
Previously a command such as:
virt-builder --cache /tmp/cache --delete-cache
would delete ~/.cache/virt-builder (the default cachedir).
It's obviously meant to delete /tmp/cache, which this commit fixes.
The third parameter (number of bytes to copy) was given as an offset
relative to dest, when it should be relative to src. This fixes some
valgrind warnings I happened across.
On Debian, /dev/kvm is mode 0660 and group kvm, so users need to add
themselves to the kvm group otherwise things are going to be very slow
(this is Debian bug 640328).
Since 'device' is assigned repeatedly, and because
device_name_translation allocates a new string each time, we have to
free it in the loop and along error paths.
Found by ./configure --enable-valgrind-daemon.
This adds a new internal API: internal_exit
Only when valgrinding the daemon, have the library call internal_exit
along the close path, and close the sockets first. This ensures we
will see normal valgrind messages (we were only seeing valgrind aborts
before).
Note this is not used in production builds.
The latter function is only available in OCaml >= 4.00 whereas
Filename.temp_dir_name has been around since OCaml 3.09.1. In this
case it doesn't make any difference which one we use.
When the user has enabled the network (not the default) we upload
/etc/resolv.conf from the host to the appliance /etc/resolv.conf
so that programs in the appliance can contact nameservers.
Commit 9521422ce6 previously changed the
behaviour to copy /etc/resolv.conf into the sysroot when running
commands.
Add a constant fixed timeout for launch (currently 20 minutes, we'll
make it configurable if needed in the future). If the appliance takes
longer than this to connect back, we time out assuming that the kernel
has hung during boot.
This call never did anything. Don't use it. Also I have submitted a
patch upstream to remove the corresponding option from blockdev.
See RHBZ#1002825 for an explanation of why this call was always
useless.
Thanks: Masayoshi Mizuma
When you try to run commands for an Ubuntu guest, they fail because in
Ubuntu /etc/resolv.conf is a symlink to /run/... and this turns out
to be a dangling symlink when the Ubuntu guest is mounted up under the
appliance.
Therefore even if the network is enabled, any command which tries to
do name resolution will fail.
Ideally we would like to bind-mount the appliance /etc/resolv.conf
into the sysroot. However this is not possible because mount is buggy
(see comment). So instead we use a complex hack to achieve the same
ends.
Note this is only done if the network is enabled and if /etc in the
guest actually exists. The original /etc/resolv.conf is restored
as soon as the command has run.
Using guestfs_exists on a symlink which existed but pointed to a
non-existent file was returning false. However exists obviously
should not be following the symlink in the first place.
Removing this directory breaks Ubuntu guests.
This change adds a utility function which removes only files from a
directory. This is a safer way to clean cache directories etc.
The 'virt-builder --format' option translates to the
'virt-resize --output-format' option, because it specifies the output
format.
On the contrary, the 'virt-resize --format' option (the input format)
is always 'raw'.
This also makes a minor rearrangement so that the random seed, root
password, hostname setting all happen before package installation.
This keeps these configuration changes together and means that the
packages that are installed can act differently based on hostname
(which is conceivable).
$ virt-sparsify a a
virt-resize: error: you cannot use the same disk image for input and
output
If reporting bugs, run virt-resize with the '-d' option and include the
complete output.
Note (a) it assumes the program is called "virt-resize" which it
isn't, and (b) it assumes the program has a debug option -d which it
doesn't.
This commit changes the error message and adds a -v option to
virt-resize.
Commit c598e14052 exports
LD_PRELOAD=/lib64/libSegFault.so so that guestfsd and processes that
it runs will produce descriptive stack traces.
However if we chroot into /sysroot (ie. CHROOT_IN/CHROOT_OUT) and if
the libSegFault.so library does not exist inside the chroot (as is the
case on Debian guests) then we cannot run any processes. In any case
we *don't* want to necessarily run this library from the guest.
The proper way to fix this is to confine all chrooting to a
subprocess, but that's a big change to guestfsd which we'll have to do
one day. For now, unset LD_PRELOAD once guestfsd starts up.
The old parser had several problems: firstly it called the error path
sometimes without calling reply_with_error causing a protocol hang.
More seriously it had hard-coded line numbers, and since Fedora 21 the
output of xfs_info has changed, moving lines around.
Change the parser to be more robust against added fields by using the
first name on the line as the section name, thus 'bsize=' is
interpreted differently depending on whether it appears in the "data"
section or the "naming" section.
Ensure also that we don't call the error path without calling
reply_with_error, which is a side-effect of the above change.
When 'mount -o loop' and similar commands are used, the loop module is
loaded automatically by the kernel when /dev/loop-control is accessed.
/dev/loop-control is created semi-statically by an unholy and
overcomplex combination of kmod static-nodes and systemd-tmpfiles
(instead of using, say, just udev or even just a simple series of
mknod commands).
tests/c-api would fail from time to time if --enable-valgrind-daemon
was configured. There was no obvious memory leak. It looks as if the
overhead of valgrind was sufficient to break long-running tests such
as this one.
<cpu model="host-passthrough"> really passes -cpu host to qemu, which
is what we want since we don't care about live migration or ABI
stability.
This should avoid http://bugzilla.redhat.com/870071 .
Note this "taints" the libvirt domain. We don't particularly care
about that, and the reason for the tainting doesn't seem to make much
sense anyway.
This updates commit 6f76fdb41e.
We have to include the right header so that guestfs___free_string_list
is declared. Unfortunately that means ensuring -I src is passed to
the compiler in every tool subdirectory.
Also fix (bogus) compiler warning about incorrect type of the
parameter to caml_copy_string_array.
qemu does in fact fix up the device tree with the correct memory size
supplied by -m <ramsize> so the comment was incorrect.
This updates commit 8d273a2715.
It is thought that qemu-system-arm -m <ramsize> could conflict with
the actual memory that Linux thinks is available (from the device tree).
Thanks: Marc Zyngier
guestfs_case_sensitive_path does not test for file existence. We have
to test for it explicitly.
This updates commit 9ea6e97014. See the
description of that commit for more details.
qemu-system-ppc64 lets you add the IDE interface to the appliance, but
the appliance kernel ignores it (and consequently can't find the
appliance disk so it all goes wrong). Best to just disallow this.
Also skip tests that try using iface = 'ide'.
ARM doesn't support IDE. (Well, that's not entirely true as I found a
reference to one extremely obscure ARM board that had IDE disks, but
qemu-system-arm doesn't appear to support them).
iface=virtio means use (old) virtio-blk. Since there's an existing
path for using virtio-blk, use that instead of adding if=virtio (which
will fail on ARM).
If supermin-helper >= 4.1.5 is found, use the new-style syntax and if
the architecture requires it (only ARM for now) implement device
trees.
This means we pass a supermin-helper --dtb option to find the right
device tree (currently Versatile Express A9, since that's what we pass
to qemu in the -M option). This makes supermin-helper find a
compatible device tree file.
Also that we pass the corresponding dtb file to qemu via the qemu -dtb
option, or to libvirt via the <dtb> element.
Currently this uses the Versatile Express A9 (vexpress-a9) machine
type, which is not ideal long-term. However it includes such things
as virtio-mmio and so virtio things should work.
Note that -M vexpress-a15 does not work with qemu (from git on
2013-09-05 + Fedora kernel 3.11.0). There are no boot messages.
For virtio-scsi, this is the same as always:
-drive ...,iface=none -device scsi-hd
For virtio-blk, this now uses:
-drive ...,iface=none -device virtio-blk
If we do not want to keep the original firewall rules, enable
this operation to remove these rules automatically.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Updated po-file.
- Change documentation to note that it's disabled by default
and in theory could make the guest exploitable.
Even though this doesn't stop the compile phase, I find it a bit
distracting that this is what I get with bootstrap:
../.gnulib/gnulib-tool: *** cannot find ./configure.ac - make sure you
run gnulib-tool from within your package's directory
../.gnulib/gnulib-tool: *** Stop.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
The macro _FORTIFY_SOURCE might be already defined by environment in
which case it shouldn't be overriden (might be if it has lower value).
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
It's an enum, not a macro, and in any case it has been present in
augeas.h since 0.10.0, and the minimum version that libguestfs
requires is 1.0.0.
This fixes commit 3d132f2920.
This commit adds an optional 'cachemode' parameter to the 'add_drive'
API to control caching. This corresponds approximately to the
'-drive ...,cache=' parameter in qemu, but the choices are much more
restrictive, just 'writeback' or 'unsafe', for reasons outlined below.
The caching modes supported by recent QEMU are:
writeback:
- Reports data writes completed when data is present in the host
page cache.
Only safe provided guest correctly issues flush operations.
writethrough:
- Reports data writes completed only when each write has been
flushed to disk. Performance is reported as not good.
none:
- Uses O_DIRECT (avoids all interaction with host cache), but does
not ensure every write is flushed to disk.
Only safe provided guest correctly issues flush operations.
directsync:
- Uses O_DIRECT (avoids all interaction with host cache), and
ensures every write has been flushed to disk.
unsafe:
- No special handling.
Since the libguestfs appliance kernel always issues flush operations
(eg. for filesystem journalling and for sync) the following modes can
be ignored: 'directsync', 'writethrough'.
That leaves 'writeback', 'none' and 'unsafe'. However 'none' is both
a constant source of pain (RHBZ#994517), is inefficient because it
doesn't use the host cache, and does not give us any safety guarantees
over and above 'writeback'. Therefore we should ignore 'none'.
This leaves 'writeback' (safe) and 'unsafe' (fast, useful for scratch
disks), which is what we implement in this patch.
Note that the previous behaviour was to use 'none' if possible, else
to use 'writeback'. The new behaviour is to use 'writeback' only
which is (in safety terms) equivalent to 'none', and also faster and
less painful (RHBZ#994517).
This patch also allows you to specify a cache mode for network drives
which also previously defaulted to 'writeback'.
There is a considerable performance benefit to using unsafe (for
scratch disks only, of course). The C API tests only use scratch
disks (since they are just tests, the final state of the disk doesn't
matter), and this decreases total run time from 202 seconds to 163
seconds, about 25% faster.
Test whether qcow2 features are possible in qemu-img instead of
assuming they are by keying off the version.
Also use the 'lazy_refcounts' flag, if available. It may make writing
to the overlay file marginally faster (however I tested it and there
was no measurable difference).
If you don't have a ^C (SIGINT) signal handler at all, then at_exit
handlers are not called so you end up leaving the large temporary
overlay file lying around. That was fixed, incorrectly, by
commit 7283a5a276.
However the code now would delete the overlay file in the SIGINT
handler but otherwise continue running until basically it tries to
read the overlay file (now deleted) and fails. So it kind of worked,
by accident.
Fix this so that the signal handler calls exit, thus ensuring both
that the exit handler is called (to delete the file) and that the
program actually exits as soon as possible.
This also refactors the unlink_on_exit function into a utility.
Libguestfs won't work if you try to use it with supermin < 4.1.4
because the logic for detecting if compressed images can be used was
wrong (it was always enabled, instead of disabled for supermin < 4.1.4).
You can now use virt-format or virt-make-fs --label option to set a
label for a DOS filesystem:
$ ./run ./format/virt-format -a /tmp/test.img --filesystem=vfat --label=BOOT
$ ./run ./cat/virt-filesystems -a /tmp/test.img --all --long -h
Name Type VFS Label MBR Size Parent
/dev/sda1 filesystem vfat BOOT - 1.0G -
/dev/sda1 partition - - 0b 1.0G /dev/sda
/dev/sda device - - - 1.0G -
This also contains a small code refactoring.
Thanks: Gerd Hoffmann (kraxel)
Code like:
CLEANUP_FREE char *buf;
/* some code which might return early */
buf = malloc (10);
is a potential bug because the free (*buf) might be called when buf is
an uninitialized pointer. Initialize buf = NULL to avoid this.
Several of these are bugs, most are not bugs (because there is no
early return statement before the variable gets initialized).
However the compiler can elide the initialization, and even if it does
not the performance "penalty" is miniscule, and correctness is better.
When using guestfish --remote, libguestfs is not the parent of qemu,
so waitpid is expected to return -ECHILD. And indeed guestfish --remote
would print a bogus error message in this case:
libguestfs: error: waitpid (qemu): No child processes
If the parent process forked into the background, then it should have
called 'set_recovery_proc 0' so we can use this to determine if we
need to wait for qemu.
Thanks: Kazuya Saito for reporting and identifying the issue.
This was accidentally moved by this commit:
commit b8b5ed65c2
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Wed May 23 11:46:23 2012 +0100
FAQ: Add section about using libguestfs in closed source programs.
Refactor the direct & uml backends to safely create the qemu & vmlinux
command lines.
Don't do malloc-after-fork, which is not safe. Do all the allocation
in the main process before the fork.
Use stringsbuf to simplify the code.
Instead of hard-coding the list of backends into the core of
libguestfs in many places, make each backend into a true module which
registers itself with the core when the library is loaded.
This is a step towards making backends into loadable modules, but it
doesn't actually do that.
This is just code motion.
It's fairly pointless to do this with TCG, since all we would be doing
is emulating a more complicated processor slowly. Also it may be the
cause of subtle problems we see during testing.
This updates commit 6f76fdb41e.
qemu -cpu \? documents this as:
host KVM processor with all supported host features (only available in KVM mode)
And indeed if you try it with TCG you'll get this error:
Unable to find CPU definition: host
This fixes commit 038ed0a08e.
These run the test suite using User-Mode Linux. Currently you have to
set LIBGUESTFS_QEMU to point to the vmlinux binary, but perhaps we
can solve that in future.
For reasons not fully understood, if md is linked into the kernel (and
not a module), a /dev/md0 device node is created. However this is not
a real RAID device. For example running mdadm --detail /dev/md0 will
fail on it.
Check the /dev/md<X> devices are real RAID devices before returning
them from the list-md-devices API.
This would have broken the function if we were comparing mixed
/dev/sda and /dev/ubda (ie different length) strings. Which we never
normally do, luckily.
If qemu goes away it is (more in theory than in reality) possible for
log messages to be lost in the kernel's socket buffer.
This turns out to matter more for User-Mode Linux.
Previously device name translation worked on the string in-place.
This worked fine because the device strings always come from XDR where
they are dynamically allocated. However it wouldn't work if the
translated name had to be longer than the original, specifically for
/dev/sd -> /dev/ubd (for User Mode Linux).
Therefore this commit changes the generator so that
device_name_translation and parse_btrfsvol (which depends on it)
allocate the new device name instead of overwriting it.
These macros are pretty horrible to use, with unexpected side-effects.
Move them exclusively into the generated code and rewrite the one
place in the general C code which used them.
There's no functional change in this code.
Using ./configure --enable-valgrind-daemon breaks valgrinding of the
library because guestfs_close wouldn't call guestfs_shutdown in that
case, resulting in some resources owned by the backend being leaked.
After this commit, --enable-valgrind-daemon will shut down the handle
normally.
This updates commit 55e3b8711f.
Don't add the "valgrind channel" to the appliance.
Just dump out the valgrind.log to stderr while the daemon is running.
Ensure that if valgrind tests fail in the appliance, that we don't
exit with success in the library by checking for a canary message in
the verbose daemon logs.
This allows the option to be used routinely by developers.
This is also called implicitly from internal_autosync, ensuring that
exit won't fail because of an open handle.
libguestfs: error: internal_autosync: umount: /sysroot: umount: /sysroot: target is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))
The code in do_part_get_bootable() assumes that if a partition has the
bootable flag set, then that is the only flag. It compares the entire
flags field with the string "boot". However, the boot flag isn't
always the only flag. For instance, POWER systems typically have a
bootable partition of type 0x41 (PPC PReP boot), which parted -m
displays as:
# parted -m -- f18.img unit b print
BYT;
/root/f18.img:16106127360B:file:512:512:msdos::;
1:1048576B:5242879B:4194304B:::boot, prep;
That is, the flags field contains "boot, prep", and thus libguestfs
fails to see that this partition is bootable. Ultimately this causes
virt-resize to fail to set the bootable flag on the boot partition of
the destination image, resulting in an image that won't boot.
This patch fixes the problem by searching for the string "boot" within
the flags field, instead of comparing the whole flags field.
'-cpu host' makes the CPU look like the host CPU. This is useful
because it means the appliance can use all the host CPU's features,
eg. for fast checksumming or MD XORing.
'kvmclock' is the paravirtualized clock for qemu/KVM. It's much more
stable than relying on tsc or a virtualized device such as HPET, and
clock instability is one of the problems we have encountered when
running libguestfs on a heavily loaded machine, especially in a nested VM.
Note that we require qemu >= 1.2 (and have for a while) which has long
supported both of these options.
- Use 'op' instead of 'xxx_op' for operation table.
- Add a 'defaults' for the operation table and remove any default
values.
This is just code motion and there is no functional change.
This adds a virt-sysprep 'password' operation which adds the
following command line options:
--root-password Set the guest's root password.
--password Set any guest password.
Disable this operation because it is more likely than not to break
Linux guests.
To fix this properly will involve finding all the places in the guest
where the UUIDs are used to locate and mount filesystems,
eg. /etc/fstab and possibly initramfs.
This updates commit 8965368eb8.
journal.c: In function 'do_journal_skip':
journal.c:134:5: error: format '%zd' expects argument of type 'signed size_t', but argument 3 has type 'int64_t' [-Werror=format=]
reply_with_perror_errno (-r, "failed to skip %zd journal entries", skip);
^
This fixes commit 5cb7f294f6.
This adds simple support for reading the journal files from guests
that use the systemd journal.
The new APIs are:
journal-open
journal-close
journal-next
journal-skip
journal-get
journal-get-data-threshold
journal-set-data-threshold
internal-journal-get
More complex journal support (eg. for seeking within the journal,
support for cursors) could be added later.
Although g->tmpdir is always set after launch, and none of these calls
should be used before launch, it's always possible that a caller would
use one of these APIs before launching the handle.
If guestfs___lazy_make_tmpdir is not called, then g->tmpdir is NULL,
and the constructed filename would look like "(null)/...".
In fact it's possible to observe this error (although it seems to have
no especially bad effects):
$ strace guestfish ls / |& fgrep '(null)'
unlink("(null)/ls1") = -1 ENOENT (No such file or directory)
The FileIn/FileOut parameters are not passed through to the daemon.
Previously we generated incorrect RPC code (an empty 'struct
guestfs_<fn>_args') because we didn't account for these FileIn/FileOut
parameters correctly.
Review every test(!) to ensure that it:
- Doesn't use a generic name (eg. "test1.img", "test.out") for any
temporary file it needs.
- Does instead use a unique name or a temporary name (eg. a name like
"name-of-the-test.img", or a scratch disk).
- Does not use 'rm -f' to clean up its temporary files (so we can
detect errors if the wrong temporary file is created or removed).
This allows tests to be run in parallel, so they don't stomp on each
other's temporary files.
Although autodetecting is safe (we always have just created a drive),
it saves a little bit of time if we don't have to run qemu-img to
detect the disk format with the libvirt backend.
Note that for prepared drives the format is always raw and we don't
anticipate creating drives in other formats.
Implement 'set-label' for XFS filesystems.
Fix the call for btrfs. Previous
commit d5817537fa added some bogus
documentation implying this call would work for btrfs, but it did
not.
Add tests.
Replaces code such as:
fd = open "test1.img"
ftruncate fd, size
close fd
g.add_drive "test1.img"
with the shorter and simpler:
g.add_drive_scratch size
Previously if the 'qemu-img info --output json' command failed, then
the stdout callback would be called first with a zero-length buffer.
Since a zero-length buffer cannot be valid JSON (in this context)
don't produce a separate error message.
However do catch the case when qemu-img info might produce no output
and no error (if that were to happen -- it does not happen at the
moment).
$ guestfish disk-format /etc/nosuchfile
Instead of this obscure double error message:
libguestfs: error: qemu-img info: JSON parse error: parse error: premature EOF
(right here) ------^
libguestfs: error: qemu-img info: /etc/nosuchfile: qemu-img info exited with error status 1.
To see full error messages you may need to enable debugging.
See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You will now see this simpler error:
libguestfs: error: disk info: /etc/nosuchfile: No such file or directory
Thanks: Bo Fan.
No single test should run longer than 1 hour even on the slowest of
hardware. We are having a problem in Koji where a test hangs and then
we end up losing the output completely, so a timeout + print the log
to that point is much better.
It was using the installed copy of libguestfs not the local copy.
We can't encode the paths in guestfs.go, since this source file gets
copied into the install directory where those paths would not make
sense. Instead we have to use CGO_CFLAGS and CGO_LDFLAGS to
communicate the right flags to go when doing the local build and test.
By specifying a cleanup function we can ensure that Augeas and hivex
functions can be tested.
There is no functional change here, verified by diffing the generated
file tests/c-api/tests.c before and after.
Add a utility function (fstype_is_extfs) to match ext2/3/4 filesystem
names. This is used in a couple of places.
When passing the mke2fs -t parameter, verify that the request is for
an ext2/3/4 filesystem. Previously we did not check this, and neither
did mke2fs when the -F flag was also used.
Since we don't usually move the first partition, if the first
partition has an unusually large offset from the start of the disk,
then the unpartitioned space in front of that partition counts as
partitioning overhead. However the previous surplus calculation was
not taking that into account.
This was a problem for certain Ubuntu images which are partitioned
with an 8 MB gap before the first partition.
Thanks: David Hart.
Also add an example from a real RHEL 6 guest.
Remove rules for building these XML files which are no longer
relevant. These files were built by running virt-inspector on real
guests, and then edited to remove excessive <application> sections.
Commit e71b2c11f5 broke inspection of
Fedora guests because guestfs_is_file returns false for
/etc/redhat-release (it's a symlink to a file, not a file).
We fix this by using the new followsymlinks flag added in the
previous commit. Thus guestfs_is_file becomes
guestfs_is_file_opts (g, filename, GUESTFS_IS_FILE_OPTS_FOLLOWSYMLINKS, 1, -1)
which checks if it's a file or a symlink to a file.
This fixes commit e71b2c11f5.
This adds an extra optional boolean 'followsymlinks' flag to those 6
is-* APIs. If the flag is true, then symlinks are followed, ie. we
use stat instead of lstat in the test.
For the rationale behind this change, see:
https://bugzilla.redhat.com/show_bug.cgi?id=974489
Commit 864ef706a8 (in Aug 2012) broke
the deprecated txz-out API. Because of a thinko during this commit,
instead of producing xz files it produces bzip2 files.
Fix this API again.
Thanks: Bo Fan / Red Hat QA team.
Fix a bogus warning introduced by
5abb196de8. If a non-windows partition
hasn't been detected as something else and falls through to windows
detection, inspection would display a bogus warning about missing
/boot.ini.
RWMJ: Test if guestfs_is_file > 0, in case it returns an error.
This change refactors guestfs___has_windows_systemroot to
guestfs___get_windows_systemroot. The new function returns a
dynamically allocated char * which must be freed.
The new function is no less efficient than before, as it returns the
result of guestfs___case_sensitive_path_silently, which is required
anyway. The new code is slightly more efficient than before, as it
re-uses the result of this testing in guestfs___check_windows_root
rather than running it again.
This file is mainly a central place to:
- include localenv if it exists, and
- define the RHEL 5 backwards compatibility macros, instead of
spreading them over every other file.
This has to run on every build, so:
- Use two grep commands instead of lots.
- Remove unnecessary use of sed.
- Remove some obsolete directories that haven't existed for a long time.
Previously, every subdirectory that contained generated files would
cause a recursion into the ../generator directory. This was slow and
unnecessary. Only recurse if the generator actually needs to be
rerun.
Note this changes the semantics slightly: For example if you added a
new action and just ran `make -C ocaml' then previously the generator
would be rerun, but now it would not. However I'm not convinced the
previous semantics were that useful, particularly considering that
with this change a simple `make' is much faster.
Yet another protocol bug!
If the daemon cancels, then the library should send a cancellation
chunk. It normally does this during the loop where it is uploading a
file. However due to an oversight or possibly a regression when we
modularized the socket code, the library did not send a cancellation
chunk if the daemon cancelled just before the end of file was sent.
This commit fixes this.
The public-facing functions like 'guestfs_*' are implemented as
wrappers around implementation functions like 'guestfs__*' (for
library-side functions only).
Don't call from one 'guestfs__*' function to another. It prevents
ENTER events from being caught, especially where a deprecated function
calls another implementation.
Most code was not doing this, but there were still some places left
where it was done.
The timeout lets us read remote images without timing out.
The noop scheduler lets the host do scheduling. This generally makes
sense for mixed workloads because the host has a global view of the
system, and often has more compute power too.
virtio-blk is obsolete so we don't need to bother optimizing it.
People should use virtio-scsi instead.
Also virtio-blk should choose the optimal settings itself.
This was originally added to fix this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=509383
This runs all of the check* rules. Since this includes 'make check',
'make check-all' is not quite equivalent to the old 'make extra-tests'
which was removed in the previous commit.
The following commit:
commit 5a3da36626
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Jan 24 17:07:38 2013 +0000
inspect: Use CLEANUP_* macros in inspection code.
can cause a double-free along an error path when certain guest files
are empty where we expected those files to contain at least one line.
This causes virt-inspector to crash when run on these guests.
The following is a test case which demonstrates the crash.
`f20rawhidex64' is a Fedora guest, but with small adjustments to the
test you could use any Linux guest for this test.
$ qemu-img create -f qcow2 -b f20rawhidex64 /tmp/test.qcow2
Formatting '/tmp/test.qcow2', fmt=qcow2 size=21474836480 backing_file='f20rawhidex64' encryption=off cluster_size=65536 lazy_refcounts=off
$ guestfish -i -a /tmp/test.qcow2 -- rm /etc/redhat-release : touch /etc/redhat-release
$ virt-inspector /tmp/test.qcow2
*** glibc detected *** virt-inspector: double free or corruption (fasttop): 0x00007f18bc9925a0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x34ecc7ca8e)[0x7f18b8e64a8e]
/lib64/libguestfs.so.0(+0x3f91898078)[0x7f18ba13c078]
/lib64/libguestfs.so.0(+0x3f91899761)[0x7f18ba13d761]
/lib64/libguestfs.so.0(+0x3f91896d12)[0x7f18ba13ad12]
/lib64/libguestfs.so.0(+0x3f91894140)[0x7f18ba138140]
/lib64/libguestfs.so.0(guestfs_inspect_os+0x35)[0x7f18ba0bcc35]
virt-inspector(main+0x547)[0x7f18ba7c57d7]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f18b8e09a05]
virt-inspector(+0x6665)[0x7f18ba7c7665]
This is a denial of service, but not likely to be exploitable.
(Found by Coverity)
Actually if you don't specify the port, currently libvirt gives
an error:
libguestfs: error: could not create appliance through libvirt: XML
error: missing port for host [code=27 domain=20]
However I think that is a bug in libvirt:
https://bugzilla.redhat.com/show_bug.cgi?id=965142
Because OStringList is an optarg, we must specify "a!" to
zend_parse_parameters.
However zend_parse_parameters still returns this as a *long* rather
than a NULL (or empty array) so positively check that it has returned
an array.
At least libpython2.7-dev and libpython3.3-dev on current
Debian/unstable ship with pkg-config files. As with the pkg-config
check for Lua, we check for versioned and an unversioned .pc files.
Modern qemu can now handle this properly. ':' is only special if what
precedes it looks like a transport, so:
qemu-system-x86_64 -drive foo:bar .. fails
qemu-system-x86_64 -drive ./foo:bar .. works
Thus by adding ./ in front of relative paths that contain ':' we can
work around this.
In addition, this broke iscsi:// URIs because iSCSI target names
routinely contain ':' characters.
This avoids confusion when using URIs in guestfish, since the path
will always start with a '/', and we don't otherwise know if we should
remove it or not. By forcing the '/' to always be there, we deal with
this problem in the API instead.
- Don't use fixed names for the disks. This will allow us to
parallelize this test.
- Add a new "GETKEY:<key>" String parameter which can retrieve keys
from the handle. The temporary disk names are stored as keys.
- Don't test the close callback. However this uses the close callback
mechanism to delete the temporary disks, and in any case close
callbacks are well tested by the language binding tests.
- The generated code now produces a static array of tests (instead of
a 'perform_tests' function), making it possible to parallelize.
Set ERL_LIBS and also create a phony 'ebin' directory for the
guestfs.beam (library) so that the tests run against the local copy
not the installed version, and so that separated builds work.
This was revealed by the regression test for RHBZ#664558.
Unfortunately because Ruby tests had been silently disabled,
this regression test was never run.
Despite this change, separated builds are still broken, apparently
because ExtUtils::MakeMaker doesn't support this sort of
configuration, or else we're using it wrong.
Since tests involving FileIn will often wish to read from local files,
it makes sense that they would want to open files in $srcdir.
Therefore allow such paths to be prefixed by "$srcdir" which is
substituted at run time by the same named environment variable (set by
automake).
This fixes separated builds in tests/c-api directory.
Previously the generator binary was (deliberately) placed into the
source directory. However there was no real reason for this to
happen. Always put the build products into the builddir.
Since stamp-generator is a Makefile thing, make it in the Makefile,
not in the generator itself.
Also this ensures stamp-generator is created in the builddir, not in
the source directory (fixing separated builds).
Because the generator subdirectory is cleaned first, and other
directories recurse into the generator directory, 'make distclean'
would fail (because generator/Makefile would be deleted by that
point).
Change subdirs-rules.mk so it doesn't fail if generator/Makefile has
been deleted already.
The previous test for grub-install hard-coded /dev/vda (ie. assuming
virtio-blk instead of more modern virtio-scsi).
This changes the test to hard-code /dev/sda instead. However this
change is still not correct since /dev/sda will be adjusted by block
device name translation in the call to grub-install, but not what is
written to /boot/grub/device.map.
Since we no longer support grub-install on Fedora, this won't affect
things, but Ubuntu still has a 'grub-install' command (although it's
actually from grub2).
This works around the following warning:
virt-ls.c: In function 'show_file':
virt-ls.c:574:1: error: stack protector not protecting function: all local arrays are less than 4 bytes long [-Werror=stack-protector]
With recent GCC, you get this warning:
tests-main.c: In function 'skipped':
tests-main.c:256:3: error: function might be possible candidate for 'gnu_printf' format attribute [-Werror=suggest-attribute=format]
This is safer, because otherwise a URI could contain some clever
"unix:..." string as the hostname, tricking qemu into opening a Unix
domain socket at an uncontrolled location.
This fixes commit 349300af08.
libxml2 is very commonly available on Linux distros and has also been
ported (and is widely available) on Mac OS X and Windows. Therefore
simply require libxml2, and remove a lot of conditional code.
Add a remote drive by doing:
guestfish -a ssh://example.com/path/to/disk.img
There are several different protocols supported, as explained in the
man page.
This affects all virt-* tools that use the common guestfish options
parsing code.
This adds helper C functions 'is_string_list' and 'is_device_list'
allowing these tests to be carried out in generic C code instead of as
specialized tests.
Instead of using the various 'TestOutput', 'TestOutputList' etc
macros, it makes better sense to let the tests contain fragments of C
code to perform the checks.
Several new macros are added:
- 'TestResult': This macro takes a C expression which is evaluated to
test the result of commands. For example to compare if a string
result has some value:
TestResult (* command sequence *), "STREQ (ret, \"abcd\")"
The variable "ret" contains the result from the last command in the
sequence. But also, variables "ret1", "ret2", etc contain the results
from the last but one command, the command before that, and so forth,
allowing much more flexible tests to be written.
- 'TestResultTrue', 'TestResultFalse': Wrappers that test the last
command for true/false.
It's simpler to use the glibc 'program_invocation_short_name(3)'
feature, and fall back to a generic solution. Also remove risky
assignments to argv[0].
These set or get the program name in the handle. Most programs
will never need to call this, since we set this, if possible,
using the glibc 'program_invocation_short_name(3)' feature.
- /usr/lib{,64}/gconv/*:
This is the iconv data, required for APIs like iconv_open to work. In
particular, the mtools 'mcopy' program fails completely if it cannot
use iconv. 'mcopy' is used by SYSLINUX to copy files to the disk.
The error you would have seen is:
Error converting to codepage 850 Invalid argument
Cannot initialize 'S:'
Bad target s:/ldlinux.sys
- /usr/bin/localedef and /usr/sbin/build-locale-archive:
These are not really needed, but both are simply small binaries so we
might as well not delete them.
Setting the 'sparse' optional boolean causes writes to be omitted if
the block to be written contains all zero bytes.
This should help with sparse backing files (eg. raw, qcow2, dm-thin, etc).
Also, modify virt-resize to use this option by default when copying
devices. The savings in virt-resize can be quite startling, eg
'du -sh' (ie. true size) of a resized disk image:
8.1G /tmp/f15x32-resized.img # before this change
3.2G /tmp/f15x32-resized.img # after this change
This API is an easier to use version of the existing guestfs_available,
because the new API returns true/false instead of throwing an error
when a feature from the list is not available.
In truth we've had this implementation internally in the library
and several tools and in Sys::Guestfs::Lib for a long time. This
change just turns it into a publicly consumable API.
This directory (containing HTML documentation) can be removed
completely when making clean. CLEANFILES cannot recursively remove a
directory, so use a clean-local rule instead.
Rules such as 'make check-valgrind' are implemented using a for-loop.
They would always exit after the first error, even if 'make -k' was
used at the top-level. Since 'make -k'-style behaviour is generally
more useful, change these for-loops so they run all the tests, and
report errors at the end.
Current code wrote a single valgrind.log in the top-level directory.
This was overwritten with each valgrinded program, which wasn't much
use. The new version writes multiple separate valgrind-*.log files
containing the date and PID (so each is unique).
It would nice if valgrind could delete log files that don't contain
errors, but this doesn't appear to be possible.
This large, but mainly mechanical commit, renames "attach method"
everywhere to "backend".
Backwards compatibility of the API (guestfs_{set,get}_attach_method)
and environment (LIBGUESTFS_ATTACH_METHOD) is maintained, but in new
code use guestfs_{set,get}_backend and LIBGUESTFS_BACKEND instead.
The default backend (launching qemu directly) is now called 'direct'
instead of 'appliance', although you can still use 'appliance' as a
synonym.
In virt-df and virt-alignment-scan, ensure that errors that happen in
worker threads are propagated all the way up and result in
exit(EXIT_FAILURE).
Note that this makes the align/test-virt-alignment-scan-guests.sh test
fail (for a genuine reason). This is fixed in the following commit.
This updates commit 8b90f55dc7.
If using (eg) virtio-blk, the canonical name won't work unless device
name translation is done first.
This fixes commit 47b929b7893b3a76ff22760d245cb80720ae6345..
In particular, building virt-sysprep is slow because there are so many
modules. Enable parallel builds. If it breaks, we should fix it, not
work around it.
We only have to serialize the two calls to ocamlmklib, since both will
try to create a file called 'libmlguestfs.a'. Apart from that,
parallel builds here should be fine.
For example:
$ guestfish --long-options
--add
--cmd-help
--connect
--csh
--domain
--echo-keys
[etc.]
The idea of this is to make it easier to write a bash completion
script that accurately expands --<TAB> options for each command.
ISO apparently specified a limit of 4096 bytes/characters for strings.
The description of the guestfish command 'add-drive' is now longer
than this limit, causing this warning.
Turn off this warning, since GCC has no problems with these strings.
This changes the existing (non-published-in-stable) API for
add_drives{_opt} so that instead of having separate 'server' and
'port' parameters, now we have a single 'server' parameter which is a
list of strings.
This is so we will be able to cope with protocols such as Ceph which
allows multiple servers, or Sheepdog which can use an implicit local
server (ie. zero servers specified).
NBD still requires exactly one server.
This warning is actively dangerous. Because it forces you to use a
'default:' case in every switch statement, it means you can no longer
detect missing cases in enums.
You can now add remote NBD drives using:
><fs> add-drive "" format:raw protocol:nbd server:localhost
(Note that you also need to add port:NNNN if the server is running on
a non-standard port).
The corresponding qemu-nbd service can be started by doing:
qemu-nbd disk.img -t
This commit also adds a test.
It causes plenty of problems with people have multiple parallel
versions of libguestfs installed, and there's no benefit because you
can easily run libguestfs and tools from the build directory.
As with Perl, you can now set INSTALLDIRS to control where Ruby places
its files (ie. in the site dir, which is the default, or in the vendor
dir).
The difference in file layout between 'make install' and
'make INSTALLDIRS=vendor install' is shown below (for Fedora 18).
--- /tmp/site 2013-03-14 12:14:35.740015694 +0000
+++ /tmp/vendor 2013-03-14 12:14:13.668093944 +0000
@@ -119,21 +119,20 @@
./usr/lib64/ocaml/guestfs/mlguestfs.cmxa
./usr/lib64/ocaml/stublibs/dllmlguestfs.so
./usr/lib64/ocaml/stublibs/dllmlguestfs.so.owner
-./usr/lib64/perl5/auto/Sys/Guestfs/Guestfs.bs
-./usr/lib64/perl5/auto/Sys/Guestfs/Guestfs.so
-./usr/lib64/perl5/auto/Sys/Guestfs/.packlist
./usr/lib64/perl5/perllocal.pod
-./usr/lib64/perl5/Sys/bindtests.pl
-./usr/lib64/perl5/Sys/Guestfs/Lib.pm
-./usr/lib64/perl5/Sys/Guestfs.pm
+./usr/lib64/perl5/vendor_perl/auto/Sys/Guestfs/Guestfs.bs
+./usr/lib64/perl5/vendor_perl/auto/Sys/Guestfs/Guestfs.so
+./usr/lib64/perl5/vendor_perl/auto/Sys/Guestfs/.packlist
+./usr/lib64/perl5/vendor_perl/Sys/bindtests.pl
+./usr/lib64/perl5/vendor_perl/Sys/Guestfs/Lib.pm
+./usr/lib64/perl5/vendor_perl/Sys/Guestfs.pm
./usr/lib64/php/modules/guestfs_php.so
./usr/lib64/pkgconfig/libguestfs-gobject-1.0.pc
./usr/lib64/pkgconfig/libguestfs.pc
./usr/lib64/python2.7/site-packages/guestfs.py
./usr/lib64/python2.7/site-packages/libguestfsmod.la
./usr/lib64/python2.7/site-packages/libguestfsmod.so
-./usr/local/lib64/ruby/site_ruby/_guestfs.so
-./usr/local/share/ruby/site_ruby/guestfs.rb
+./usr/lib64/ruby/vendor_ruby/_guestfs.so
./usr/sbin/libguestfs-make-fixed-appliance
./usr/share/doc/libguestfs/example-debian-netinst-cd.xml
./usr/share/doc/libguestfs/example-debian.xml
@@ -406,3 +405,4 @@
./usr/share/man/uk/man3/guestfs-perl.3
./usr/share/man/uk/man3/guestfs-python.3
./usr/share/man/uk/man3/guestfs-ruby.3
+./usr/share/ruby/vendor_ruby/guestfs.rb
For prompting me to fix this, thanks: Hilko Bengen.
This adds a common utility function (guestfs___exit_status_to_string)
and a common error function (guestfs___external_command_failed), and
uses them all over the library and tools when converting exit status
in error messages etc.
This allows custom parameters to modify parameters added by
libguestfs, eg. by doing:
-set drive.hd0.file=rbd:foo/bar
[infernix adds the comment below:]
This works for recent versions of QEMU. The way to use this is:
guestfish
><fs> add-drive /dev/null
><fs> config -set drive.hd0.file=rbd:pool/volume
><fs> run
This will probably work for more types (sheepdog, iscsi et al), but
with the caveat that the libguestfs layers are bypassed so you'll
*always* end up directly accessing the disk in rw mode.
Prefix functions in this file with old_parser_... to make it clear
this is the old / slightly unsafe human output parser, which is
shortly to be replaced by a safer JSON parser.
This is just code motion.
- Use ./run script to run the tests.
- Set environment variables correctly, including $PATH.
- Test the locally built, not installed, copy of libguestfs.
The values of CONFIG, LAUNCHING and READY and (kind of) exposed
through the API so shouldn't change.
Use a bad sentinel for NO_HANDLE since that indicates a serious error.
Ruby ships its own config.h which may or may not define the same
relevant constants as our autoconf-generated config.h. Instead of
trying to specify the exact path to the wanted header file we may just
as well simply use Ruby's autoconf-inspired checks and macros.
This creates an abstract layer below the protocol code which handles
the socket operations. This will make it easier to introduce libvirt
virSocketPtr operations in future.
In the handle, g->conn contains the connection to the appliance.
g->conn is NULL when we're not connected.
poll(2) is used instead of select(2).
All error messages about launch failing or the appliance unexpectedly
dying are handled by two common error message functions, and these
contain a better explanation of what to do.
The earlier (pipe-based) code never set this flag, but that was a bug,
potentially allowing the file descriptor to be leaked to subprocesses.
Set the FD_CLOEXEC flag, but also ensure it is cleared in the child
process just before qemu is exec'd (otherwise qemu would not have a
console).
The libvirt backend already uses a Unix socket for the appliance
console, and so for the libvirt backend the fields g->fd[0] == g->fd[1].
Change the appliance backend to use a socketpair, so we need just a
single file descriptor for qemu stdin/stdout (ie. appliance console).
Consequently we can remove the array int fd[2] in the handle and
replace it with a single file descriptor.
When debugging is enabled, this prints out the permissions (ie.
ls -laZ) of the appliance and sockets directories. This should be
helpful for debugging RHBZ#913774.
Since qemu connects to both, there was no point at all in having these
sockets be readable by other, nor executable by anyone.
The sockets should now end up as:
srw-rw---- root.qemu
Despite what the documentation may have said, it certainly was not
safe to call guestfs_umount_local from another thread.
guestfs_umount_local could generate events, call the error handler, or
access other fields in the handle, with no locking at all.
We should aim to make guestfs_umount_local thread-safe in future.
See also:
https://bugzilla.redhat.com/show_bug.cgi?id=917706
This function is now generated, so bindings in various languages
are made automatically.
Note that the function previously returned void, but now it returns
int (although always 0). We don't believe that this is an ABI break
since existing programs will continue to work.
Non-daemon functions normally have a wrapper function called
eg. guestfs_name. The "real" (ie. hand-written) function is called
eg. guestfs__name. The wrapper deals with checking parameters and
doing trace messages.
This commit allows the wrapper function to be omitted. The reason is
so that we can handle a few functions that have to be thread-safe
(currently just: guestfs_user_cancel). The wrapper is not thread safe
because it can call events and/or the error handler.
libxml2 xmlXPathEvalExpression can return an unusual nodeset:
$1 = {type = XPATH_NODESET, nodesetval = 0x0, boolval = 0, floatval = 0,
stringval = 0x0, user = 0x0, index = 0, user2 = 0x0, index2 = 0}
Note that the nodeset is non-NULL, but the nodesetval is NULL.
Check every call site and fix those that don't deal with this
correctly.
When adding a domain (ie. guestfs_add_domain), read the SELinux
<label/> and <imagelabel/> from the guest and use them for the
appliance. The appliance is statically labelled the same as the
guest, so it is able to read its disks.
However tell libvirt not to try relabelling the disks, to prevent
libvirt from disturbing the existing labels on the disks (in
particular when the libvirt connection is closed, we don't want
libvirt to try to restore some other label on the disks).
Updated with feedback from Matthew Booth.
When a disk is opened readonly, the libvirt attach-method privately
creates a qcow2 overlay on top.
This commit lets that overlay get an SELinux label, and sets it to the
imagelabel specified by guestfs_internal_set_libvirt_selinux_label.
The above only applies to the libvirt attach-method.
If set, this causes <seclabel model=selinux relabel=no> to be added to
the disk element in the libvirt XML.
It has no effect *except* on the libvirt attach method when SELinux
and sVirt is being used.
This internal API sets two SELinux labels in the handle (the process
label and the image label -- they are closely related).
If using the libvirt attach-method with SELinux and sVirt, then this
will cause the following XML to be added to the appliance definition:
<seclabel type=static model=selinux relabel=yes>
<label>[LABEL HERE]</label>
<imagelabel>[IMAGELABEL HERE]</imagelabel>
</seclabel>
It is ignored by other attach-methods.
libguestfs handles are not thread safe, and it's not safe even to read
settings from the handle from multiple threads (eg. guestfs_get_trace).
Stop doing this in the parallel library. This caused fairly
reproducible segfaults when you enabled '-x' and/or '-v'.
This fixes commit 34e77af1bf.
This function is no longer called from anywhere outside the library,
or even outside libvirt-domain.c.
This is just code motion, except for the size_t changes which are
required in order to avoid signed overflow optimization error:
assuming signed overflow does not occur when simplifying conditional
to constant [-Werror=strict-overflow]
You can now run virt-df and virt-alignment-scan on all of the test
guests by doing:
./run ./df/virt-df -c test:///path/to/tests/guests/guests.xml
./run ./align/virt-alignment-scan -c test:///path/to/tests/guests/guests.xml
which is kinda cool.
Previously when you asked virt-df to show stats for all your libvirt
guests, it used a single appliance and tried to attach as many disks
as possible to it, even disks from different guests.
However this has been problematic: Not only is the code to do this
horrendously complex, but it's also insecure, and it doesn't interact
well with sVirt labels (see RHBZ#912499 comment 7).
In the meantime we discovered that running parallel appliances gives
you most of the performance of using a single appliance, but with a
lot less complexity and better guest isolation (see the documentation
in commit 680450f3b4).
Therefore this is a partial rewrite of virt-df so that in this case it
now uses parallel appliances.
Notes:
The '--one-per-guest' option is now the default and only way to do
things; this option now does nothing.
By default, the amount of parallelism to use is controlled by the
amount of free memory seen when virt-df starts up (subject to some
minima and maxima). The user can control this through new command
line option '-P'.
This mini-library runs the 'free -m' command and greps the output to
estimate the max. number of libguestfs appliances we could run in
parallel in the remaining free memory.
(Not entirely sure whether using Gnulib to replace standard functions
is a good idea at all.)
link with libgnu:
CCLD libguestfs-test-tool
libguestfs_test_tool-test-tool.o: In function `main':
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:103: undefined reference to `rpl_getopt_long'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:113: undefined reference to `rpl_optarg'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:125: undefined reference to `rpl_optarg'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:126: undefined reference to `rpl_optarg'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:109: undefined reference to `rpl_optarg'
libguestfs_test_tool-test-tool.o: In function `set_qemu':
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:382: undefined reference to `rpl_perror'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:366: undefined reference to `rpl_perror'
libguestfs_test_tool-test-tool.o: In function `make_files':
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:416: undefined reference to `rpl_perror'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:428: undefined reference to `rpl_perror'
CC guestfsd-9p.o
In file included from ../../../daemon/9p.c:32:0:
../../../daemon/daemon.h:33:34: fatal error: guestfs-internal-all.h: No such file or directory
This involves adding a new test API which crashes the appliance in the
middle of a simulated upload, then a test which uses that API to test
for the libguestfs (library-side) crash.
Instead of the segfault you now get a more informative error:
*stdin*:0: libguestfs: error: connection to daemon was closed unexpectedly.
This usually means the libguestfs appliance crashed. Please enable
debugging (LIBGUESTFS_DEBUG=1) and rerun the command, then look at the
debug messages output prior to this error.
libguestfs: error: /dev/stdout: write: Broken pipe
libguestfs: error: file receive cancelled by daemon
It should be possible to add the guestfs___free_string_list to
dllmlguestfs.so, but I cannot work out exactly how to do this. As a
result we end up using src/utils.c directly.
The libutils convenience library is a place for code shared between
the main library, language bindings and virt tools. Note that the
code is statically linked into both the library, each binding and each
tool, but this is an improvement because (a) the source is shared and
(b) libguestfs.so can export fewer private functions.
Currently it contains the cleanup functions, and the functions
guestfs___free_string_list function and guestfs___for_each_disk.
guestfs___for_each_disk has changed so that it no longer
unconditionally sets the error in the guestfs handle. Instead callers
can control error handling.
These are internal functions. Very old versions of libguestfs used to
export them, but they haven't been (and shouldn't be) exported for a
long time.
Also remove the unused guestfs_error function.
Earlier versions of automake complain if they get a configuration
parameter which they don't understand. The error is:
configure.ac:27: error: option 'serial-tests' not recognized
Use some m4 hackery to work around this.
It was failing to clean up and close the 'struct command' buffer,
because commit 94d90f03e2 added an
incomplete patch to use CLEANUP_CMD_CLOSE.
This fixes commit 94d90f03e2.
btrfs_subvolume_list can now take either the path of a mounted btrfs
filesystem, or a mountable describing the location of a btrfs
filesystem, or one of its volumes. In the latter case, the filesystem
will be automatically mounted outside of /sysroot before running the
btrfs tool, and unmounted afterwards.
There are now two forms of the 'debug progress' command:
(1) debug progress <n> (the original form) generates ordinary
rate-limited progress messages for <n> seconds.
(2) debug progress <n> <rate> generates progress messages every <rate>
microseconds for <n> seconds.
The second form omit the usual rate-limiting, and so wouldn't
be generated like this from an ordinary API call. However this
is useful for testing events (see RHBZ#909624).
Note that you have to use this as follows:
#if defined(__GNUC__) && GUESTFS_GCC_VERSION >= x0y0z /* gcc >= x.y.z */
since GUESTFS_GCC_VERSION is not defined unless it's GCC.
A Mountable is passed from the library to the daemon as a string. The daemon
stub parses it into a mountable_t, which it passes to the implementation.
Update all implementations which now take a mountable_t.
This change updates the api style of all apis which should take Mountable
descriptions rather than block devices. It also updates the documentation
accordingly, but doesn't implement any functional changes.
Because of evolution of the code, if the user used the --qemu or
--qemudir options, libguestfs-test-tool would segfault because
guestfs_set_qemu was being called before the guestfs handle was
opened.
Change the code so this doesn't happen, and also remove the global 'g'
variable to make the code a bit more robust.
Bug found by Amit Shah.
The second parameter passed into getline(3) is the size of the
allocated buffer, *NOT* the length of the returned line. This can be
confusing, so rename this parameter as 'allocsize' consistently
throughout the code.
This is just code motion.
This was disturbing to some users, but other users saw the warning
when it was really something they should pay attention to. Make it
clear that it's a warning, unless people are really using SELinux +
sVirt.
Thanks: Richard Harman.
For example:
{
CLEANUP_FREE_STAT struct guestfs_stat *stat = guestfs_stat (g, "/foo");
// stat is freed automatically when we leave the scope.
}
The new header file "guestfs-internal-frontend-cleanups.h"
includes definitions for these. It should not be included
directly.
Partition number was being passed to guestfs___check_for_filesystem_on
based on an index in list_partition. However, this ignores the
possibility of multiple block devices.
This change makes guestfs___check_for_filesystem_on examine the
passed-in device directly to determine if it is a whole device, or
what its partition number is.
The output of btrfs subvolume list has changed in F18 to include generation,
which breaks the parsing in btrfs_subvolume_list. This change replaces sscanf
with a more robust regular expression. The new regular expression should also
handle the addition of future unexpected columns.
The visibility field in action replaces in_fish, in_docs and internal.
The defined types are:
VPublic:
A public API. This is exported and documented in all language
bindings, and in guestfish.
VStateTest:
A public API which queries the library state machine. It is exported
and documented in all language bindings, but not guestfish.
VBindTest:
An internal API used only for testing language bindings. It is
guarded by GUESTFS_PRIVATE in the C api, but exported by all other
language bindings as it is required for testing. If language
bindings offer any way to guard use of these apis, that mechanism
should be used. It is not documented anywhere.
VDebug:
A debugging API. It is exported by all language bindings, and in
guestfish, but is not documented anywhere.
VInternal:
An internal-only API. It is guarded by GUESTFS_PRIVATE in the C api,
and not exported at all in any other language binding. It is not
documented anywhere.
Replace selected calls to 'perror (filename)' with:
fprintf (stderr, "syscall: %s: %m\n", filename);
so that more information is available about precisely which syscall
failed.
Note this is *not* reply_with_perror. These messages are only printed
in verbose output, for the benefit of debugging.
Previously I noticed that bfan used this command without any error:
><fs> hivex-open /WINDOWS/system32/config/software write:ture
^^^^^^^^^^
This was because the code allowed any string to be evaluated as a
boolean.
The new code is stricter. It allows the following strings only case
insensitive (everything else is an error):
1
true
t
yes
y
on
0
false
f
no
n
off
Not to be confused with the libxml2 macro 'BAD_CAST' which converts
from 'signed char *' to 'unsigned char *'.
The 'bad_cast' function was defined and used all over the place as a
replacement for a '(char *)' cast. I think it is better to make these
casts explicit, instead of hiding them in an obscure function.
This commit rearranges the internal header files.
"src/guestfs-internal.h" is just for the library, as before.
"src/guestfs-internal-frontend.h" is for use by all library, bindings,
tools C code, but NOT the daemon.
"src/guestfs-internal-all.h" is for use by all C code including the
daemon.
This is just code motion, but it has some important consequences:
(1) We can use the CLEANUP_* macros in bindings and tools code.
(2) We can get rid of TMP_TEMPLATE_ON_STACK.
(3) We will (in future) be able to stop bindings and tools code from
using the safe_* allocation functions (which are NOT safe to use
outside the library alone).
qemu-wrapper isn't regenerated if QEMUDIR is changed, so just
delete it and force regeneration.
Additionally we can drop the silent binary check, since
check-with-upstream-qemu-1 already does a similar test with --version
that will actually report an error to the user.
Certain functions are intended to be internal only, but we currently
export them anyway. This change moves them into a separate section of
guestfs.h protected by a GUESTFS_PRIVATE variable. This change also
enables private structs, but doesn't implement any.
This change only affects the C api. Language bindings aren't affected,
but probably should be in the future.
Without this change all the tests in the haskell bindings are rebuilt
every time. The primary motivation for this change is to fix this.
The fix for the above also allows parallel builds to be re-enabled.
Use the macro like this to create temporary variables which are
automatically cleaned up when the scope is exited:
{
CLEANUP_FREE char *foo = safe_strdup (bar);
...
// no need to call free (foo)!
}
The following code is also valid. The initialization of foo as 'NULL'
prevents any chance of free being called on an uninitialized pointer.
It may not be required in all cases.
{
CLEANUP_FREE char *foo = NULL;
...
foo = safe_malloc (100);
...
// no need to call free (foo)!
}
This is also valid:
{
CLEANUP_FREE char *foo = ..., *bar = ...;
...
// no need to call free (foo) or free (bar)!
}
The CLEANUP_FREE_STRING_LIST macro calls guestfs___free_string_list
on its argument. The argument may be NULL.
The CLEANUP_HASH_FREE macro calls hash_free on its argument. The
argument may be NULL.
Important implementation note:
------------------------------
On GCC and LLVM, this is implemented using __attribute__((cleanup(...))).
There is no known way to implement this macro on other C compilers, so
this construct will cause a resource leak.
Important note about close/fclose:
----------------------------------
We did NOT implement 'CLEANUP_CLOSE' or 'CLEANUP_FCLOSE' macros. The
reason is that I am not convinced that these can be used safely. It
would be OK to use these to collect file handles along failure paths,
but you would still want a regular call to 'close'/'fclose' since you
must test for errors, and so you end up having to do:
if (close (fd) == -1) {
// failure case
// avoid double-close in cleanup handler:
fd = -1;
...
}
// avoid double-close in cleanup handler:
fd = -1;
...
This is just code motion, but it allows us to read this flag inside
the 'construct_libvirt_xml_seclabel' function in future (as a possible
way to fix RHBZ#890291).
When debugging is enabled, this produces output like below. This is
useful when diagnosing what URI libguestfs is using.
libguestfs: opening libvirt handle: URI = NULL, auth = virConnectAuthPtrDefault, flags = 0
libguestfs: successfully opened libvirt handle: conn = 0xb05580
guestfs_last_errno (g) == 0 doesn't mean "no error". It means the
errno was not captured. In this case we have to substitute some sort
of errno, so choose EINVAL arbitrarily.
The reasons to do this are twofold:
(a) It's probably a tiny bit faster.
(b) It lets us capture the real errno if the link(2) syscall fails.
The errno is also passed through guestmount, fixing RHBZ#895905:
+ guestmount -a test1.img -m /dev/sda1:/ -m /dev/sda2:/boot /tmp/mnt
+ touch /tmp/mnt/foo
+ cd /tmp/mnt
+ ln foo boot/foo
ln: failed to create hard link ‘boot/foo’ => ‘foo’: Invalid cross-device link
cp will fail if /etc/lvm is an empty directory. Copy the entire
directory and adjust environment variable.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
RWMJ:
- Fixed a couple of whitespace issues.
Using // coverity[...] or /* coverity[...] */ comments in the source
it is possible to suppress specific Coverity errors. The suppressed
error should occur in the line following the comment.
In this case I have suppressed two false positives from Coverity:
(a) We deliberately assign to a NULL pointer in order to cause a
segfault, for testing how the library reacts when this happens.
Coverity flags this, but it is not an error in this case.
(b) Coverity does not model global variables (a known shortcoming).
Therefore the code 'errno = posix_memalign (...)' cannot be modelled
by Coverity, even though the code is correct. Coverity raises a false
positive about this.
(Thanks Kamil Dudka, Coverity)
If there was no quoting character, the options could run off the end
of the string and into whatever string which happened to be following
in memory.
This bug was revealed when libguestfs was compiled on arm.
For the rationale behind this renaming, see:
b8cbe191d0
You can still, for the time being, use
./configure --with-febootstrap-yum-config
but it's preferred to switch to the new option. The new option is
identical, just renamed.
Qe still pass the deprecated --yum-config option to febootstrap
itself, since both old and new febootstrap support this.
For some reason the fs->device field was not set for swap devices.
Apparently this did not matter before (this bug has existed since at
least libguestfs 1.16), but in commit 6ee80b8dac we started to depend
on the fs->device field being non-NULL. This resulted in segfaults
when inspecting any guest which had a swap device, eg:
#0 __strcmp_sse42 () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:164
#1 0x00007ffff7d740cb in parent_device_already_probed (g=0x703a70,
partition=0x6f79b0 "/dev/vda4") at inspect.c:156
#2 0x00007ffff7d73e00 in guestfs__inspect_os (g=0x703a70) at inspect.c:83
#3 0x00007ffff7cee11d in guestfs_inspect_os (g=0x703a70) at actions-1.c:397
This commit sets fs->device to the device name.
(Thanks Matthew Booth for first noticing this problem.)
If ntfs-3g is not installed, then tests/guests creates a 'windows.img'
file which is zero-sized.
This is problematical because other tests which use this image will
fail at best, or hit RHBZ#847549 (qemu/virtio-scsi bug) at worst.
Change the tests which use windows.img so they skip if 'windows.img'
has zero size.
If you want to selectively run tests, or if the test suite fails half
way through, use:
make print-subdirs
to print the subdirectories, select the ones you want to run / the
remaining ones, and do:
make check SUBDIRS="..."
User Phill Bandelow noted that virt-resize fails with an e2fsck error
on a host where the system clock had been accidentally set in the
past.
Unfortunately this was hard to diagnose because guestfsd 'ate' the
stdout of the e2fsck program. I have verified by code inspection that
e2fsck prints messages on stdout.
Thus this changes the daemon to fold stdout and stderr together so we
get to see all error messages from e2fsck when it fails.
Rename guestfs_safe_malloc et al to guestfs___safe_malloc etc.
To use the private functions, code now has to define
-DGUESTFS_PRIVATE_FUNCTIONS=1. This will make it easier for us in
future to work out which programs are using these functions and to
minimize both the number of programs and the functions they are
calling.
Note that the Perl, Python, OCaml, Ruby and Java bindings use
guestfs_safe_* calls. None of the other bindings do. This is a bug
(in the bindings using those functions): these functions will call the
out of memory callback on failure. This function defaults to abort(),
and since this happens from a language binding, there is no way to
change this default.
I guess no one uses GCC 3 any more, because if they did, including
<guestfs.h> would not work since GUESTFS_DLL_PUBLIC would not have
been defined.
At the same time, ensure that __attribute__((noreturn)) is only used
when the compiler is GCC >= 3.1.
Currently the code is doing a redundant fork when passed the
COMMAND_FLAG_CHROOT_COPY_FILE_TO_STDIN flag. The additional
sub-process does a chroot() which has no effect because all file
handles are already open at that point, then simply copies its input
to its output.
This change simply replaces the above with a dup2 of the passed file
handle to STDIN of the command process.
RWMJ:
Don't initialize 'pid' variable.
Improve readability of commandrvf() by replacing bare int values for
file descriptors with their symbolic names STD{IN,OUT,ERR}_FILENO.
Also add PIPE_READ and PIPE_WRITE for referencing relevant ends of a pipe.
When executing a command, we temporarily chroot, fork and exec the
command, then chroot back. We intentionally don't chdir in the parent
process so that we can 'jailbreak' the chroot later. However, this has
the effect that commands are executed with a current working directory
which is outside the current root. This unusual state can cause
errors in executed commands which don't anticipate it.
This change does a chdir("/") before executing and command. This
happens inside the fork, so the jailbreak isn't affected in the
parent.
2012-12-13 17:15:01 +00:00
1174 changed files with 376323 additions and 115468 deletions
NOTE: This file is automatically generated from "update-bugs.sh".
Last updated: 2012-12-13
Last updated: 2014-09-22
This contains a local list of the bugs that are open against
libguestfs. Bugs are tracked in the Red Hat Bugzilla database
@@ -23,63 +23,30 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
507278 NEW https://bugzilla.redhat.com/show_bug.cgi?id=507278
libguestfs fails to build on Fedora sparc64
547488 NEW https://bugzilla.redhat.com/show_bug.cgi?id=547488
guestfish cannot tab complete filenames that contain spaces
554829 NEW https://bugzilla.redhat.com/show_bug.cgi?id=554829
SELinux handling could be done better.
563450 NEW https://bugzilla.redhat.com/show_bug.cgi?id=563450
list-devices returns devices of different types out of order
572337 NEW https://bugzilla.redhat.com/show_bug.cgi?id=572337
libguestfs should support gptsync
578103 NEW https://bugzilla.redhat.com/show_bug.cgi?id=578103
[RFE] Tool to Compare Windows Registry Entries
593511 NEW https://bugzilla.redhat.com/show_bug.cgi?id=593511
[RFE] function to get partition name
604041 NEW https://bugzilla.redhat.com/show_bug.cgi?id=604041
guestmount absolute symlinks don't work
624334 NEW https://bugzilla.redhat.com/show_bug.cgi?id=624334
blockdev-setbsz succeeds, but does not affect blockdev-getbsz
624335 NEW https://bugzilla.redhat.com/show_bug.cgi?id=624335
blockdev-setbsz succeeds, but does not affect blockdev-getbsz
637251 NEW https://bugzilla.redhat.com/show_bug.cgi?id=637251
virt-inspector fails to recognize data-only NTFS disk image
660687 NEW https://bugzilla.redhat.com/show_bug.cgi?id=660687
guestmount: "touch" command fails: touch: setting times of `timestamp': Invalid argument
672485 NEW https://bugzilla.redhat.com/show_bug.cgi?id=672485
[RFE] virt-edit/tar/inspector do not support encrypted system
693064 NEW https://bugzilla.redhat.com/show_bug.cgi?id=693064
Symbolic links on ntfs-3g are not followed correctly by some commands
696445 NEW https://bugzilla.redhat.com/show_bug.cgi?id=696445
Backport virt-inspector for virt-v2v
696451 NEW https://bugzilla.redhat.com/show_bug.cgi?id=696451
libguestfs: unknown filesystem label SWAP-sda2
700342 NEW https://bugzilla.redhat.com/show_bug.cgi?id=700342
virt-inspector resports unknown filesystem UUID
709326 NEW https://bugzilla.redhat.com/show_bug.cgi?id=709326
virt-inspector cannot detect ReactOS
737261 NEW https://bugzilla.redhat.com/show_bug.cgi?id=737261
libguestfs grub-install API needs grub1
745282 NEW https://bugzilla.redhat.com/show_bug.cgi?id=745282
[RFE] Support to use virt-filesystems with remote libvirt systems
745576 NEW https://bugzilla.redhat.com/show_bug.cgi?id=745576
libguestfs (or qemu?) hangs if sparse file runs out of disk space
@@ -95,33 +62,18 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
790837 NEW https://bugzilla.redhat.com/show_bug.cgi?id=790837
Use of atexit to clean up handles is wrong in multithreaded programs
803643 NEW https://bugzilla.redhat.com/show_bug.cgi?id=803643
inspect-is-multipart return false when inspection results should be true
803650 NEW https://bugzilla.redhat.com/show_bug.cgi?id=803650
inspect-is-live return false when inspection results should be true
803657 NEW https://bugzilla.redhat.com/show_bug.cgi?id=803657
[RFE] inspect-is-netinst : support more distributions
805417 NEW https://bugzilla.redhat.com/show_bug.cgi?id=805417
RFE: support inspection of installation ISOs of WinVista, Win7, Win2008 & Win2008r2
806179 NEW https://bugzilla.redhat.com/show_bug.cgi?id=806179
RFE: support inspection of icon for more Windows guests
808193 NEW https://bugzilla.redhat.com/show_bug.cgi?id=808193
g.launch() crashes on RHEL 5
812970 NEW https://bugzilla.redhat.com/show_bug.cgi?id=812970
virt-rescue cannot set ><rescue> prompt, on Ubuntu 12.04
813290 NEW https://bugzilla.redhat.com/show_bug.cgi?id=813290
mdadm (md-detail) test occasionally fails with 'md device /dev/md125 does not appear to be active.'
815149 NEW https://bugzilla.redhat.com/show_bug.cgi?id=815149
virt-alignment-scan gives error "part_list: could not parse row from output of parted print command: /dev/vda:4194304B:virtblk:512:512:msdos:Virtio Block Device"
819086 NEW https://bugzilla.redhat.com/show_bug.cgi?id=819086
look for qemu-kvm on /usr/libexec
822538 NEW https://bugzilla.redhat.com/show_bug.cgi?id=822538
libguestfs tools hang on qcow2 encrypted disks
@@ -140,41 +92,332 @@ Bugs in NEW or ASSIGNED state are open and waiting for someone to fix.
835622 NEW https://bugzilla.redhat.com/show_bug.cgi?id=835622
RFE: virt-sparsify should be able to sparsify onto a thin-provisioned LV
837941 NEW https://bugzilla.redhat.com/show_bug.cgi?id=837941
Data loss when writing to qcow2-format disk files
838081 NEW https://bugzilla.redhat.com/show_bug.cgi?id=838081
ocaml/t/guestfs_500_parallel_mount_local crashes in caml_thread_reinitialize
845234 NEW https://bugzilla.redhat.com/show_bug.cgi?id=845234
RFE: virt-ls on Windows guest doesn't support drive letters
846676 NEW https://bugzilla.redhat.com/show_bug.cgi?id=846676
guestfs_launch() hangs with Debian qemu-kvm 1.1.0
848464 NEW https://bugzilla.redhat.com/show_bug.cgi?id=848464
gobject javascript bindings cannot use 64 bit integers
848926 NEW https://bugzilla.redhat.com/show_bug.cgi?id=848926
virt-v2v should verify that the RHEV-M domain is an export storage domain
857763 NEW https://bugzilla.redhat.com/show_bug.cgi?id=857763
libguestfs 'file-architecture' returns 'ARM' for arm binaries
863696 NEW https://bugzilla.redhat.com/show_bug.cgi?id=863696
[F18] libguestfs fails to mount a disk image file(in this case qcow2) as 'root'
864871 NEW https://bugzilla.redhat.com/show_bug.cgi?id=864871
libvirt error: could not create appliance through libvirt: unable to set security context 'unconfined_u:object_r:svirt_image_t:s0:cXXX,cYYY' on '.../console.sock'
865923 NEW https://bugzilla.redhat.com/show_bug.cgi?id=865923
Check that new qemu -machine option doesn't break libguestfs.
866994 NEW https://bugzilla.redhat.com/show_bug.cgi?id=866994
tgz-out causes memory leak in guestfsd
869179 NEW https://bugzilla.redhat.com/show_bug.cgi?id=869179
libguestfs fail to startup on latest rhel7 tree
887826 NEW https://bugzilla.redhat.com/show_bug.cgi?id=887826
RFE: virt-sparsify cannot handle multi-disk virtual machines (eg. if they use LVM, MD, LDM, etc)
880806 NEW https://bugzilla.redhat.com/show_bug.cgi?id=880806
virt-df with two -a options displays incorrect disk image name
891909 NEW https://bugzilla.redhat.com/show_bug.cgi?id=891909
Can't make install with non-standard prefix as non-root
895898 NEW https://bugzilla.redhat.com/show_bug.cgi?id=895898
RFE: Allow interface to be specified with libvirt attach-method
895946 NEW https://bugzilla.redhat.com/show_bug.cgi?id=895946
Got incorrect block count number when resize ntfs file system via "ntfsresize_opts"
903987 NEW https://bugzilla.redhat.com/show_bug.cgi?id=903987
Command "acl-set-file" is inconvenient to use
906766 NEW https://bugzilla.redhat.com/show_bug.cgi?id=906766
virt-resize cannot resize RHEL 5 guest with GPT partition table type
906777 NEW https://bugzilla.redhat.com/show_bug.cgi?id=906777
In tests, set_label sometimes fails
910115 NEW https://bugzilla.redhat.com/show_bug.cgi?id=910115
RFE: support ntfsresize --bad-sectors option in virt-resize
910117 NEW https://bugzilla.redhat.com/show_bug.cgi?id=910117
virt-v2v removes serial console from securetty list
910269 NEW https://bugzilla.redhat.com/show_bug.cgi?id=910269
Tracker: Bugs in other packages that stop libguestfs working
913815 NEW https://bugzilla.redhat.com/show_bug.cgi?id=913815
febootstrap: no ext2 root device found when running libguestfs-test-tools
917706 NEW https://bugzilla.redhat.com/show_bug.cgi?id=917706
guestfs_umount_local is not thread safe
920617 NEW https://bugzilla.redhat.com/show_bug.cgi?id=920617
RFE: virt-inspector (or another tool) should produce OVF output for oVirt / RHEV
921604 NEW https://bugzilla.redhat.com/show_bug.cgi?id=921604
RFE: virt-v2v: add option to specify spice or VNC and whether to add USB connections
927447 NEW https://bugzilla.redhat.com/show_bug.cgi?id=927447
[suse] virt tools hang on disk image, but libguestfs-test-tool runs OK
963232 NEW https://bugzilla.redhat.com/show_bug.cgi?id=963232
RFE: v2v: a --no-cleanup would be useful for debugging of disk images that guestfs has issues converting
973138 NEW https://bugzilla.redhat.com/show_bug.cgi?id=973138
RFE: sh & debug sh should be able to capture all output and error return code
979392 NEW https://bugzilla.redhat.com/show_bug.cgi?id=979392
[abrt] virt-manager crash from libguestfs launch_libvirt
980790 NEW https://bugzilla.redhat.com/show_bug.cgi?id=980790
fstrim returns error "FITRIM ioctl failed: Inappropriate ioctl for device"
982516 NEW https://bugzilla.redhat.com/show_bug.cgi?id=982516
sh "rpm -Va" fails because apparently /dev/shm is not created/bind-mounted correctly
983969 NEW https://bugzilla.redhat.com/show_bug.cgi?id=983969
RFE: virt-sysprep should be SELinux-aware
986242 NEW https://bugzilla.redhat.com/show_bug.cgi?id=986242
golang bindings cannot be compiled if golang-guestfs package is installed
987829 NEW https://bugzilla.redhat.com/show_bug.cgi?id=987829
virt-resize error: "lvm_set_filter: vgchange: Couldn't find device with uuid .. Refusing activation of partial LV"
988100 NEW https://bugzilla.redhat.com/show_bug.cgi?id=988100
RFE: journal reader in guestfish
991641 NEW https://bugzilla.redhat.com/show_bug.cgi?id=991641
virt-sysprep fs-uuids plugin doesn't update UUIDs in fstab, elsewhere
991830 NEW https://bugzilla.redhat.com/show_bug.cgi?id=991830
virt-resize throws ambiguous error when /dev/mapper/VG-LV path is used
994965 NEW https://bugzilla.redhat.com/show_bug.cgi?id=994965
libguestfs doesn't work on armv7hl
1019388 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1019388
firstboot scripts (virt-builder, virt-sysprep) don't work for Debian 6 & 7 guests
1023630 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1023630
enable-valgrind-daemon option breaks UML
1025942 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1025942
Parallel virt-df and virt-alignment-scan tests sometimes deadlock
1027526 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1027526
RFE: Labels for logical partitions lost in Windows 2003 after virt-resize for primary partition
1028676 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1028676
Add virt-builder to RHEL 7.1
1029134 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1029134
Broken links in virt-builder web man page
1033573 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1033573
libguestfs inspection does not recognize NAS4Free OS installed on QCOW2 image
1040912 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1040912
[ARM] mount-local / FUSE support does not seem to work at all
1041552 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1041552
virt-sysprep should remove Pegasus certificates
1042780 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1042780
Support the virt-sysprep --delete option
1051694 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1051694
libguestfs FTBFS on Ubuntu saucy
1056045 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1056045
virt-v2v OVA import fails when no (optional) manifest file is included
1056047 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1056047
virt-df generates no output for Solaris 11 x86_64 guest.
1056126 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1056126
virt-v2v OVA/OVF import fails with 'MegaBytes' as memory allocation units
1056534 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1056534
virt-v2v OVF/OVA import fails when disks aren't attached to either IDE or SCSI controllers
1056640 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1056640
virt-v2v OVA/OVF import fails when no (optional) ovf:capacityAllocationUnits is specified
1056983 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1056983
virt-v2v OVA/OVF import fails when no (optional) ovf:populatedSize is specified
1057006 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1057006
virt-v2v OVA/OVF import fails when no (optional) Name under VirtualSystem is specified
1057873 NEW https://bugzilla.redhat.com/show_bug.cgi?id=1057873
part-get-name give 'libguestfs: error: part_get_name: parted does not support the machine output (-m)' error message when run 'part-get-name /dev/sda 1'
eprintf(f_"%s: no plan could be found for making a disk image with\nthe required size, format etc. This is a bug in libguestfs!\nPlease file a bug, giving the command line arguments you used.\n")prog;
eprintf(f_"%s: no guest operating systems or multiboot OS found in this disk image\nThis is a failure of the source repository. Use -v for more information.\n")prog;
exit1in
Customize_run.run~prog~verbose~quietgrootops;
(* Collect some stats about the final output file.
* Notes:
* - These are virtual disk stats.
* - Never fail here.
*)
letstats=
ifnotquietthen(
try
(* Calculate the free space (in bytes) across all mounted
elseifi=len1||j=len2thenfalse(* no match - different lengths *)
else(
letx1=getxdigitfp1.[i]andx2=getxdigitfp2.[j]in
matchx1,x2with
|Somex1,Somex2whenx1=x2->loop(i+1)(j+1)
|Somex1,Somex2->false(* no match - different content *)
|Some_,None->loopi(j+1)
|None,Some_->loop(i+1)j
|None,None->loop(i+1)(j+1)
)
in
loop00
andgetxdigit=function
|'0'..'9'asc->Some(Char.codec-Char.code'0')
|'a'..'f'asc->Some(Char.codec-Char.code'a')
|'A'..'F'asc->Some(Char.codec-Char.code'A')
|_->None
letrecverifytfilename=
ift.check_signaturethen(
letargs=quotefilenamein
do_verifytargs
)
andverify_detachedtfilenamesigfile=
ift.check_signaturethen(
matchsigfilewith
|None->
eprintf(f_"virt-builder: error: there is no detached signature file\nThis probably means the index file is missing a sig=... line.\nYou can use --no-check-signature to ignore this error, but that means\nyou are susceptible to man-in-the-middle attacks.\n");
eprintf(f_"virt-builder: error: GPG failure: could not verify digital signature of file\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!\n");
eprintf(f_"virt-builder: error: fingerprint of signature does not match the expected fingerprint!\n found fingerprint: %s\n expected fingerprint: %s\n")
eprintf(f_"virt-builder: error: could not run sha512sum command to verify checksum\n");
exit1
);
letcsum_actual=read_whole_filecsum_filein
letcsum_actual=
letlen=String.lengthcsum_actualin
iflen>0&&csum_actual.[len-1]='\n'then
String.subcsum_actual0(len-1)
else
csum_actualin
ifcsum<>csum_actualthen(
eprintf(f_"virt-builder: error: checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!\n")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
