By using:
export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
you can force the libvirt backend to use TCG (software emulation)
instead of KVM (hardware accelerated virtualization).
By using:
export LIBGUESTFS_BACKEND=direct
export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
you can force the direct backend to use TCG (software emulation)
instead of KVM (hardware accelerated virtualization).
This is sometimes useful, especially if you are trying to use
nested KVM on Intel.
Allow settings (an arbitrary list of strings) to be passed to the
current backend. This will allow us to tweak how the backend works,
eg. by forcing TCG.
tests/nbd/test-nbd.pl contains a hack (a call to sleep) to wait
for qemu-nbd to start up in another process.
This sleep was previously located before the call to $g->launch()
since that is where (previous to commit 4a0f5ed382) the backend would
have connected to the NBD server.
However in the new code, an initial NBD connection test is done when
the drive is added (ie. $g->add_drive (..., protocol=>"nbd")), which
duly failed intermittently because the qemu-nbd server had not opened
its socket yet.
Move the wait earlier -- just after the fork -- to avoid this.
This updates commit 4a0f5ed382.
In some places we turn relative paths passed by the caller into
absolute paths: for example in the libvirt backend, all relative paths
to disk images have to be turned into absolute paths (by calling
realpath) because otherwise libvirt doesn't work.
This has the potential to break things, especially considering that
qemu uses the path passed to it in order to resolve the relative paths
of backing files.
For example consider the case where 'overlay' is a symbolic link to
/dev/dm-1, but /dev/dm-1 contains a qcow2 which references './backing'
as its backing file. If we rewrote 'overlay' to '/dev/dm-1', then
qemu would try to look up '/dev/backing' for the backing file and
fail.
See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1053684https://www.redhat.com/archives/libguestfs/2014-January/thread.html#00087
This fixes commit 4a0f5ed382.
A simple test of this:
guestfish <<EOF
add-drive "" readonly:1 format:raw protocol:nbd server:localhost:10000
EOF
which would core dump.
qemu has broken snapshot=on ... again.
Change the way that drives are created so that the backend no longer
has to use snapshot=on, <transient/> (which never worked), or UML's
corresponding COW-creation feature (also broken).
Instead of that, the src/drives.c code will create overlays when
required by calling into a new backend operation 'create_cow_overlay'.
This operation runs 'qemu-img create -b' or 'uml_mkcow' as determined
by the backend, and returns the name of the overlay.
The format of the overlay is still backend-specific because qemu needs
to use qcow2 and UML needs to use COW.
This patch also includes some factorization of the libvirt XML code.
This also drops the drv->priv (private per-drive data) field, since it
is no longer used by any backend.
This also moves the guestfs___drive_source_qemu_param utility
function, used & shared by the direct & libvirt backends only, into
src/launch-direct.c (from src/drives.c).
In some tests we were specifying qcow2 as the image format when adding
a disk, but actually passing a raw format image.
Libguestfs previously did not detect this until guestfs_launch, but it
was still a bug to pass an incorrect format to guestfs_add_drive_opts.
It only worked because these tests never call guestfs_launch.
A later commit in this series will cause this to be detected
(sometimes) during guestfs_add_drive_opts.
Switch from a boolean for the short/long list output to labels for the
actual format. Also, split the output of each list format to an own
function for easier maintaineance.
Not all the fields of struct stat are actually filled by us. This caused
rubbish to appear in the microseconds fields, which were then used as
base when changing atime/ctime (with e.g. touch), triggering EINVAL by
futimens/utimensat when those rubbish values were out of the range
allowed for microseconds.
Add a new --operation parameter which, similarly to --enable, can be
used to enable operations, but also to remove them, and to add/remove
the default operations and all the available ones.
Previously the list was reversed. Note this doesn't have any effect
on how the packages are installed since the guest package manager
doesn't care about the order.
This is too complex to implement directly in virt-builder.
Instead we just document how to do it for some common Linux distros
using --run-command, --edit etc.
Move in an own function the code that does the (l)listxattrs allocating
the buffer of the right legth, as it will be useful later.
No functional changes, just code motion.
Instead of enable them when having one of the two headers for it but
still checking for the HAVE_* availability of each *xattr() function
used, just enable the linuxxattrs as a whole when having any of the
needed headers (like before) and all the needed functions.
This might cause the linuxxattrs to not be available anymore on systems
without the whole set of *xattr() functions implemented, but OTOH it
simplifies the xattr.c implementations.
It now matches the documentation:
"Select the size of the output disk, [...]
Virt-builder will resize filesystems inside the disk image
automatically.
If the size is not specified, then one of two things happens. If
the output is a file, then the size is the same as the template. If
the output is a device, partition, etc then the size of that device
is used."
This also adds checks to make sure that users don't try to shrink a
disk (which is not possible), and they don't try to resize larger than
the output block device.
This means there will be no default hypervisor, and effectively the
user will always have to specify one (eg. by setting LIBGUESTFS_HV or
calling guestfs_set_hv).
This is useful on platforms where qemu doesn't work, or where qemu is
not needed (eg. if you want to use UML, or you just want to compile
libguestfs without using/testing it).
The .new method was unintentionally broken in
commit 9466060201.
This fixes the .new method and allows it to be called with multiple
parameters, so you can use:
Guestfs::Guestfs.new
Guestfs::Guestfs.new()
Guestfs::Guestfs.new(:close_on_exit => false)
etc.
For backwards compatibility, Guestfs::create may still be used.
This commit also adds regression tests:
- Use .new method in regular tests. (Because this was not done
before, we didn't catch the breakage.)
- Test that ::create still works.
- Test that args can be passed to .new method.
Added in 2392fae680 and removed in
be025369ad622867ecf08d5c9170e8e59d2d6294; now they should work with any
backend, as the device names are normalised and not change the guestfish
output depending on the used backend.
Thanks Richard for the hints.
Instead of:
libguestfs: error: could not create appliance through libvirt.
Try using the direct backend to run qemu directly without libvirt,
by setting the LIBGUESTFS_BACKEND=direct environment variable.: internal error: process exited while connecting to monitor: qemu-system-x86_64: -drive file=/tmp/libguestfs3RnJSd/snapshot2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=writeback: could not open disk image /tmp/libguestfs3RnJSd/snapshot2: Could not open backing file: Could not open '/home/lsm5/fimagebuilder/fedora-imagebuilder/fedora-imagebuilder-sda.qcow2': Permission denied
it will now say:
libguestfs: error: could not create appliance through libvirt.
Try running qemu directly without libvirt using this environment variable:
export LIBGUESTFS_BACKEND=direct
Original error from libvirt: internal error: process exited while connecting to monitor: qemu-system-x86_64: -drive file=/tmp/libguestfs3RnJSd/snapshot2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=writeback: could not open disk image /tmp/libguestfs3RnJSd/snapshot2: Could not open backing file: Could not open '/home/lsm5/fimagebuilder/fedora-imagebuilder/fedora-imagebuilder-sda.qcow2': Permission denied
NULL means that libvirt gets to guess what connection URI we meant.
Currently it guesses qemu:///session for non-root and qemu:///system
for root. Except if LIBVIRT_DEFAULT_URI is set in which case this is
used even if it's not appropriate. Except if Xen is installed in
which case it guesses xen:/// which is not helpful at all.
Instead use qemu:///session (or qemu:///system for root -- that's
likely to be wrong, but we can't help that).
If the user really wants to use NULL and let libvirt guess, then they
can now specify the libvirt:null backend.
See:
https://bugzilla.redhat.com/show_bug.cgi?id=1045033https://bugzilla.redhat.com/show_bug.cgi?id=886915
A good way to test this is:
LIBVIRT_DEFAULT_URI=iambroken libguestfs-test-tool
The libguestfs libvirt backend should (after this commit) ignore
LIBVIRT_DEFAULT_URI. However other parts of libguestfs will still use
it, eg. guestfs_add_drive_opts, guestfish '-d' option.
A single file can have multiple xattrs. Previously these were
returned in the same order as the kernel returns them. However it is
more useful if they are sorted by attribute name.
The wrong pointer was passed to the visitor_function, resulting in it
seeing the placeholder xattr entry (the one which stores the length of
the list of xattrs - see
http://libguestfs.org/guestfs.3.html#guestfs_lxattrlist ).
virt-ls doesn't list xattrs at the moment, hence this bug was not seen
before.
This is essentially just code motion, except:
(1) It cleans up a few variable declarations which were implicitly
used by the old macro that aren't needed any more.
(2) The options are reordered alphabetically.
The current add_cdrom way basically appends a new raw "-cdrom /path"
parameter to the qemu invocation (even when using libvirt as backend),
hence such images are seen as "CD-ROM drives" inside the appliance.
However, there is no need for such particular behaviour, as they need to
be handled as normal (read-only) drives.
Adding CD-ROM disk images as drives also changes the device names used
for them inside the appliance from /dev/srN to the usual e.g. /dev/sdX.
These changes fix different issues:
- it is possible to start guestfish without adding disks with -a, then
just add-cdrom and run
- list-devices does not cause guestfsd to crash when sorting the list
of devices (exposed by the test case in RHBZ#563450)
- the result of list-devices now reflects the order images were added
(RHBZ#563450)
add_cdrom is still deprecated, but now in favour of add_drive_ro
(instead of add_drive), with its documentation reflecting that.
Add two small regression tests for the fixes described above.
virt-builder --output /dev/null is tempting, but not possible, and
might result in /dev/null being deleted(!) Check for this case and
prevent it happening.
Previously we would use the virt-resize --no-sparse option if the
final output file was a block device. This is safe, but unnecessary
for example if virt-resize was used as an intermediate step. So only
use this option if virt-resize is actually writing to the block
device.
The original template might be, say, xz-compressed raw of a certain
size. We need to work out how to convert it to, say, qcow2 with a
larger size, in as few operations as possible.
Instead of using a nasty aggregation of special cases to do this, use
a breadth-first search over all possible trees to try to find an
optimal plan.
(Actually the current implementation isn't optimal, but it's fine for
virt-builder.)
It seems that FUSE can invoke flush to make sure the pending changes
(e.g. to the attributes) of a file are set. Since a missing flush
implementation is handled as if it were returning ENOSYS, this can cause
issues later.
To overcome this, just provide a stub implementation which does nothing,
since we have nothing to do and don't want to have FUSE error out.
Furthermore, uncomment the timestamp checks in test-fuse.sh, since now
they should be working fine.
This avoids:
xgettext: error while opening "libguestfs.pot-t" for reading: No such file or directory
which I observed with xgettext (GNU gettext-tools) 0.18.3
If you use virt-make-fs to create a partitionless FAT-formatted disk
image then currently you will get an error:
$ virt-make-fs --type=fat . /tmp/test.img
'mkfs' (create filesystem) operation failed.
Instead of 'fat', try 'vfat' (long filenames) or 'msdos' (short filenames).
mkfs: fat: /dev/sda: mkfs.fat: Device partition expected, not making filesystem on entire device '/dev/sda' (use -I to override) at /usr/bin/virt-make-fs line 508, <PIPE> line 1.
...propagated at /usr/bin/virt-make-fs line 518, <PIPE> line 1.
With this patch, the error goes away and partitionless disks can be
created.
When receive_file_data calls guestfs___recv_from_daemon, and that
function returns -1, it adds the following error to the handle:
receive_file_data: parse error in reply callback
However this hides a more specific error message which was already
added by guestfs___recv_from_daemon.
When chunked encoding fails, the following error is added to
the handle:
[filename]: error in chunked encoding
However all paths returning -1 from receive_file_data already set a
more specific error, which this error hides.
The latter is a better way to disable the qemu display output as we
need to, without enabling extra devices (which are disabled already,
anyway).
Also, related to the change above, ban the -display parameter from the
ones that can be supplied by the user.
Don't change "os-version" to "OS version". It's intended to be the
same as the virt-builder man page, which consistently calls this the
"os-version" string.
This partially reverts commit ee0e56f43e.
Fix the dependencies of the libguestfs.pot target: other than using the
right make variables holding the contents of the POTFILES, depend also
on the POTFILES themselves.
- when a command needs no parameters, tell that explicitly instead of
"command should have 0 parameters"
- use gettext's plural form when printing the number of required
arguments
- improve the error message for a variable number of parameters limited
only in the maximum number of them, using also a plural form
Extracting separately the pot for the various languages and then
creating manually the global pot (by manually joining the above ones
after having stripped their headers) is wrong, since other than being
an hack it can create an invalid pot when the same message appears in
sources written in different languages.
Instead, a cleaner and safer solution is to first let ocaml-gettext
(if available) extract the messages for the ml files, and then use
xgettext to extract the messages for the other languages, joining the
new messages to the existing (or not) pot file.
Check for /boot/loader.rc as "install disc" detection, using it to mark
FreeBSD install discs.
Also, check for /mfsroot.gz to see whether such disc is also a live one.
See also RHBZ#1033207.
/etc/fstab in installation discs of FreeBSD can have an entry pointing
to the mounted CD itself; skip it as it is done with other CD devices in
check_fstab.
Although this test only specifically tests the direct backend, the
same code is shared with the libvirt backend in some circumstances
(eg. creating readonly drives). So this is testing a bit of both.
This allows you to select both locked accounts and disabled
passwords. The two are subtly different concepts.
A locked account [cf. passwd -l] puts "!!" at the beginning of the
shadow password field. Locking is reversible, because the "!!" can
be removed, restoring the original password. Therefore "locked"
acts as a flag in front of an existing selector.
A disabled account has "*" in the password field. Therefore it has no
password.
Note that an account may be both locked and disabled, although this is
probably not useful. The shadow password field will contain "!!*".
It now appears in the respective man pages as:
--root-password selector
or:
--password selector
This avoids confusion from people who think these command line options
take the password directly.
If libselinux was detected, it was not added to the linker command
line. This still worked (at least on Fedora) because some other
library pulls in the dependency implicitly, possibly libvirt. However
this broke on Arch:
https://aur.archlinux.org/packages/libguestfs/
Reported by and thanks: Antoni Segura Puimedon.
Char.code (input_char chan) mod nr_chars has modulo bias because
the original interval is not a multiple of the destination interval,
i.e. 256 mod nr_chars != 0.
One way to fix this is to keep generating random numbers until they fall outside
the interval where modulo bias occurs, that is accept only c=[256 % nr_chars, 256).
That interval maps back to [0, nr_chars), and has a length of
(256 - 256 % nr_chars), which is a multiple of nr_chars.
RWMJ:
- Modify the code so it goes into a utility library.
- Use the same code across virt-builder and virt-sysprep.
OCaml's buffered 'in_channel' has a 64k buffer, so using it to read a
few bytes from /dev/urandom removes a lot of the system's entropy (for
example /proc/sys/kernel/random/entropy_avail goes from ~3000 to 128).
This patch was originally by Edwin Török for builder.ml. I
generalized it because there are two other places where we did
over-sized reads from /dev/urandom.
RWMJ:
- Renamed the script from sl.sh -> scientificlinux.sh
- The output file is called scientificlinux-$version
- I fixed some whitespace issues so it's now similar to the CentOS script.
index-parser-c.c depends on index-parse.h being created first.
However without an explicit dependency, this is not done (and implicit
deps don't work because automake doesn't sufficiently understand OCaml
programs).
This fixes commit a4800e2d4f.
Instead of calling out to the pxzcat program, use an internal
implementation. This requires liblzma to be available at build time.
If it's not available, fall back to using regular xzcat.
It is intended that eventually this code will go away when regular
xzcat / unxz is able to use threads.
The device name prefix for IDE hard drives used to be `ad' but now
is `ada' (http://www.freebsd.org/doc/handbook/disks-naming.html).
For virtio hard drives it is `vtbd'.
Under an MBR partition table a slice will be used, so the name of
the first partitions will be either `ada0s1a' or `vtbd0s1a'. Under a
GPT partition table, where no slice is needed, the name of the first
partition will be either `ada0p1' or `vtbd0p1'.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
This is for backwards compatibility with virt-builder 1.24.0 which
could not parse a key that contains square brackets and numbers.
This updates commit 4b73e0be29.
Users can now specify multiple source paths, eg:
virt-builder --source http://example.com/foo \
--source http://example.com/bar
to get templates from multiple places.
There is still only one built-in path, but we can add more later.
We don't need to sign individual templates, provided the signed index
file contains the cryptographically secure checksums of the templates.
Since the index file cannot be interfered with and the checksums
cannot be spoofed, this is sufficient to ensure the integrity of the
templates.
Note the template signatures are now deprecated and will be removed in
libguestfs 1.26. To avoid any backwards compatibility issues, I will
backport the relevant changes to the stable-1.24 branch.
Thanks Dennis Gilmore for pointing out that we could do this.
Instead of typing:
./run ./fish/guestfish ...
you can now use:
./run guestfish ...
This is shorter, but there are also two good reasons for this change:
It never made any sense to run the system-installed guestfish (or
other) binary. It would be unlikely to work. Also binaries in
libguestfs should be able to call other binaries freely without adding
special exceptions (which we had before this change).
For systemd:
Currently stdout+stderr go to the journal (these are effectively
empty unless there are gross errors). The output of the commands
goes to the log file.
With this change, the output goes to the journal, the console and
the log file.
For SysV-init:
Currently stdout+stderr go to the console (but these are effectively
empty). The output of the commands goes to the log file.
With this change, the output goes to the console and the log file.
If:
- the output is a regular file
- the output format is raw
- the user didn't specify the --size option
then we don't need to run virt-resize. Simply uncompress
the template directly to the output file.
When using the guestfish --remote or guestfish --listen options,
guestfish would create a socket in a known location
(/tmp/.guestfish-$UID/socket-$PID).
The location has to be a known one in order for both ends to
communicate. However no checking was done that the containing
directory (/tmp/.guestfish-$UID) is owned by the user. Thus another
user could create this directory and potentially modify sockets owned
by another user's guestfish client or server.
This commit fixes the issue by creating the directory unconditionally,
and then checking that the directory has the correct owner and
permissions, thus preventing another user from creating the directory
first.
If guestfish sees a suspicious socket directory it will print an error
like this and exit with an error status:
guestfish: '/tmp/.guestfish-1000' is not a directory or has insecure owner or permissions
Thanks: Michael Scherer for discovering this issue.
Version 2:
- Add assigned CVE number.
- Update documentation.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
This is not part of the automated test suite. It's a manual test run
by the maintainer which must be run on each new guest to ensure that
all the virt-builder features work on the new guest.
Since this is essential for security, be much more proactive about
trying to create a random seed file, even for guests which have buggy
installers that don't create this file during installation.
This commit removes a lot of cruft from old RHEL which was required to
start udev. Instead we run the daemon directly.
Also it removes the fallback path which created some device nodes in
/dev by hand. This likely did not work. This also removes the
dependency on MAKEDEV which is long gone from most distros.
This is required for Ubuntu, since the /etc/init.d/udev script didn't
actually work. The most common symptom was that /dev/virtio-ports/*
was not created because udev didn't process the cold start events.
This should be safe, and avoids the whole issue of whether a guest
uses systemd or not. Also it is possible to switch init systems
(eg. by specifying init= on the command line) so it's not even clear
if a guest "is" systemd.
Thanks: Tom Gundersen on #systemd.
This updates/fixes commit 596de56a43.
Ubuntu 13.10 has /etc/systemd but uses Upstart. This confuses the
script because it assumed that if you have /etc/systemd then you are
using systemd. Ubuntu includes systemd services (inherited from
Debian) but they just don't run.
If the output is a block device:
- Detect the size of the output device.
- Don't attempt to delete the output device on failure.
- Turn off sparse copying in virt-resize.
Some --install requests may pull in packages that start daemons (dbus
being a favourite daemon apparently). These daemons will prevent the
disks from being unmounted and everything being cleaned up at the end.
Add some experimental code to kill these daemons.
Because we're not running msgmerge but concatenating the PO files, you
can get duplicate messages (in this case "no"), resulting in this
error:
libguestfs.pot:6142: duplicate message definition...
libguestfs.pot:3285: ...this is the location of the first definition
Just work around this for now.
Commit c598e14052 exports
LD_PRELOAD=/lib64/libSegFault.so so that guestfsd and processes that
it runs will produce descriptive stack traces.
However it exports it to everything including the virt-rescue shell.
But if we are chrooting into a guest which doesn't have this file (or
indeed any guest) we should not use LD_PRELOAD.
Therefore unset LD_PRELOAD in the rescue shell.
See also related commit 21e5fc811e.
Previously a command such as:
virt-builder --cache /tmp/cache --delete-cache
would delete ~/.cache/virt-builder (the default cachedir).
It's obviously meant to delete /tmp/cache, which this commit fixes.
The third parameter (number of bytes to copy) was given as an offset
relative to dest, when it should be relative to src. This fixes some
valgrind warnings I happened across.
On Debian, /dev/kvm is mode 0660 and group kvm, so users need to add
themselves to the kvm group otherwise things are going to be very slow
(this is Debian bug 640328).
Since 'device' is assigned repeatedly, and because
device_name_translation allocates a new string each time, we have to
free it in the loop and along error paths.
Found by ./configure --enable-valgrind-daemon.
This adds a new internal API: internal_exit
Only when valgrinding the daemon, have the library call internal_exit
along the close path, and close the sockets first. This ensures we
will see normal valgrind messages (we were only seeing valgrind aborts
before).
Note this is not used in production builds.
The latter function is only available in OCaml >= 4.00 whereas
Filename.temp_dir_name has been around since OCaml 3.09.1. In this
case it doesn't make any difference which one we use.
When the user has enabled the network (not the default) we upload
/etc/resolv.conf from the host to the appliance /etc/resolv.conf
so that programs in the appliance can contact nameservers.
Commit 9521422ce6 previously changed the
behaviour to copy /etc/resolv.conf into the sysroot when running
commands.
Add a constant fixed timeout for launch (currently 20 minutes, we'll
make it configurable if needed in the future). If the appliance takes
longer than this to connect back, we time out assuming that the kernel
has hung during boot.
This call never did anything. Don't use it. Also I have submitted a
patch upstream to remove the corresponding option from blockdev.
See RHBZ#1002825 for an explanation of why this call was always
useless.
Thanks: Masayoshi Mizuma
When you try to run commands for an Ubuntu guest, they fail because in
Ubuntu /etc/resolv.conf is a symlink to /run/... and this turns out
to be a dangling symlink when the Ubuntu guest is mounted up under the
appliance.
Therefore even if the network is enabled, any command which tries to
do name resolution will fail.
Ideally we would like to bind-mount the appliance /etc/resolv.conf
into the sysroot. However this is not possible because mount is buggy
(see comment). So instead we use a complex hack to achieve the same
ends.
Note this is only done if the network is enabled and if /etc in the
guest actually exists. The original /etc/resolv.conf is restored
as soon as the command has run.
Using guestfs_exists on a symlink which existed but pointed to a
non-existent file was returning false. However exists obviously
should not be following the symlink in the first place.
Removing this directory breaks Ubuntu guests.
This change adds a utility function which removes only files from a
directory. This is a safer way to clean cache directories etc.
The 'virt-builder --format' option translates to the
'virt-resize --output-format' option, because it specifies the output
format.
On the contrary, the 'virt-resize --format' option (the input format)
is always 'raw'.
This also makes a minor rearrangement so that the random seed, root
password, hostname setting all happen before package installation.
This keeps these configuration changes together and means that the
packages that are installed can act differently based on hostname
(which is conceivable).
$ virt-sparsify a a
virt-resize: error: you cannot use the same disk image for input and
output
If reporting bugs, run virt-resize with the '-d' option and include the
complete output.
Note (a) it assumes the program is called "virt-resize" which it
isn't, and (b) it assumes the program has a debug option -d which it
doesn't.
This commit changes the error message and adds a -v option to
virt-resize.
Commit c598e14052 exports
LD_PRELOAD=/lib64/libSegFault.so so that guestfsd and processes that
it runs will produce descriptive stack traces.
However if we chroot into /sysroot (ie. CHROOT_IN/CHROOT_OUT) and if
the libSegFault.so library does not exist inside the chroot (as is the
case on Debian guests) then we cannot run any processes. In any case
we *don't* want to necessarily run this library from the guest.
The proper way to fix this is to confine all chrooting to a
subprocess, but that's a big change to guestfsd which we'll have to do
one day. For now, unset LD_PRELOAD once guestfsd starts up.
The old parser had several problems: firstly it called the error path
sometimes without calling reply_with_error causing a protocol hang.
More seriously it had hard-coded line numbers, and since Fedora 21 the
output of xfs_info has changed, moving lines around.
Change the parser to be more robust against added fields by using the
first name on the line as the section name, thus 'bsize=' is
interpreted differently depending on whether it appears in the "data"
section or the "naming" section.
Ensure also that we don't call the error path without calling
reply_with_error, which is a side-effect of the above change.
When 'mount -o loop' and similar commands are used, the loop module is
loaded automatically by the kernel when /dev/loop-control is accessed.
/dev/loop-control is created semi-statically by an unholy and
overcomplex combination of kmod static-nodes and systemd-tmpfiles
(instead of using, say, just udev or even just a simple series of
mknod commands).
tests/c-api would fail from time to time if --enable-valgrind-daemon
was configured. There was no obvious memory leak. It looks as if the
overhead of valgrind was sufficient to break long-running tests such
as this one.
<cpu model="host-passthrough"> really passes -cpu host to qemu, which
is what we want since we don't care about live migration or ABI
stability.
This should avoid http://bugzilla.redhat.com/870071 .
Note this "taints" the libvirt domain. We don't particularly care
about that, and the reason for the tainting doesn't seem to make much
sense anyway.
This updates commit 6f76fdb41e.
We have to include the right header so that guestfs___free_string_list
is declared. Unfortunately that means ensuring -I src is passed to
the compiler in every tool subdirectory.
Also fix (bogus) compiler warning about incorrect type of the
parameter to caml_copy_string_array.
qemu does in fact fix up the device tree with the correct memory size
supplied by -m <ramsize> so the comment was incorrect.
This updates commit 8d273a2715.
It is thought that qemu-system-arm -m <ramsize> could conflict with
the actual memory that Linux thinks is available (from the device tree).
Thanks: Marc Zyngier
guestfs_case_sensitive_path does not test for file existence. We have
to test for it explicitly.
This updates commit 9ea6e97014. See the
description of that commit for more details.
qemu-system-ppc64 lets you add the IDE interface to the appliance, but
the appliance kernel ignores it (and consequently can't find the
appliance disk so it all goes wrong). Best to just disallow this.
Also skip tests that try using iface = 'ide'.
ARM doesn't support IDE. (Well, that's not entirely true as I found a
reference to one extremely obscure ARM board that had IDE disks, but
qemu-system-arm doesn't appear to support them).
iface=virtio means use (old) virtio-blk. Since there's an existing
path for using virtio-blk, use that instead of adding if=virtio (which
will fail on ARM).
If supermin-helper >= 4.1.5 is found, use the new-style syntax and if
the architecture requires it (only ARM for now) implement device
trees.
This means we pass a supermin-helper --dtb option to find the right
device tree (currently Versatile Express A9, since that's what we pass
to qemu in the -M option). This makes supermin-helper find a
compatible device tree file.
Also that we pass the corresponding dtb file to qemu via the qemu -dtb
option, or to libvirt via the <dtb> element.
Currently this uses the Versatile Express A9 (vexpress-a9) machine
type, which is not ideal long-term. However it includes such things
as virtio-mmio and so virtio things should work.
Note that -M vexpress-a15 does not work with qemu (from git on
2013-09-05 + Fedora kernel 3.11.0). There are no boot messages.
For virtio-scsi, this is the same as always:
-drive ...,iface=none -device scsi-hd
For virtio-blk, this now uses:
-drive ...,iface=none -device virtio-blk
If we do not want to keep the original firewall rules, enable
this operation to remove these rules automatically.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Updated po-file.
- Change documentation to note that it's disabled by default
and in theory could make the guest exploitable.
Even though this doesn't stop the compile phase, I find it a bit
distracting that this is what I get with bootstrap:
../.gnulib/gnulib-tool: *** cannot find ./configure.ac - make sure you
run gnulib-tool from within your package's directory
../.gnulib/gnulib-tool: *** Stop.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
The macro _FORTIFY_SOURCE might be already defined by environment in
which case it shouldn't be overriden (might be if it has lower value).
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
It's an enum, not a macro, and in any case it has been present in
augeas.h since 0.10.0, and the minimum version that libguestfs
requires is 1.0.0.
This fixes commit 3d132f2920.
This commit adds an optional 'cachemode' parameter to the 'add_drive'
API to control caching. This corresponds approximately to the
'-drive ...,cache=' parameter in qemu, but the choices are much more
restrictive, just 'writeback' or 'unsafe', for reasons outlined below.
The caching modes supported by recent QEMU are:
writeback:
- Reports data writes completed when data is present in the host
page cache.
Only safe provided guest correctly issues flush operations.
writethrough:
- Reports data writes completed only when each write has been
flushed to disk. Performance is reported as not good.
none:
- Uses O_DIRECT (avoids all interaction with host cache), but does
not ensure every write is flushed to disk.
Only safe provided guest correctly issues flush operations.
directsync:
- Uses O_DIRECT (avoids all interaction with host cache), and
ensures every write has been flushed to disk.
unsafe:
- No special handling.
Since the libguestfs appliance kernel always issues flush operations
(eg. for filesystem journalling and for sync) the following modes can
be ignored: 'directsync', 'writethrough'.
That leaves 'writeback', 'none' and 'unsafe'. However 'none' is both
a constant source of pain (RHBZ#994517), is inefficient because it
doesn't use the host cache, and does not give us any safety guarantees
over and above 'writeback'. Therefore we should ignore 'none'.
This leaves 'writeback' (safe) and 'unsafe' (fast, useful for scratch
disks), which is what we implement in this patch.
Note that the previous behaviour was to use 'none' if possible, else
to use 'writeback'. The new behaviour is to use 'writeback' only
which is (in safety terms) equivalent to 'none', and also faster and
less painful (RHBZ#994517).
This patch also allows you to specify a cache mode for network drives
which also previously defaulted to 'writeback'.
There is a considerable performance benefit to using unsafe (for
scratch disks only, of course). The C API tests only use scratch
disks (since they are just tests, the final state of the disk doesn't
matter), and this decreases total run time from 202 seconds to 163
seconds, about 25% faster.
Test whether qcow2 features are possible in qemu-img instead of
assuming they are by keying off the version.
Also use the 'lazy_refcounts' flag, if available. It may make writing
to the overlay file marginally faster (however I tested it and there
was no measurable difference).
If you don't have a ^C (SIGINT) signal handler at all, then at_exit
handlers are not called so you end up leaving the large temporary
overlay file lying around. That was fixed, incorrectly, by
commit 7283a5a276.
However the code now would delete the overlay file in the SIGINT
handler but otherwise continue running until basically it tries to
read the overlay file (now deleted) and fails. So it kind of worked,
by accident.
Fix this so that the signal handler calls exit, thus ensuring both
that the exit handler is called (to delete the file) and that the
program actually exits as soon as possible.
This also refactors the unlink_on_exit function into a utility.
Libguestfs won't work if you try to use it with supermin < 4.1.4
because the logic for detecting if compressed images can be used was
wrong (it was always enabled, instead of disabled for supermin < 4.1.4).
You can now use virt-format or virt-make-fs --label option to set a
label for a DOS filesystem:
$ ./run ./format/virt-format -a /tmp/test.img --filesystem=vfat --label=BOOT
$ ./run ./cat/virt-filesystems -a /tmp/test.img --all --long -h
Name Type VFS Label MBR Size Parent
/dev/sda1 filesystem vfat BOOT - 1.0G -
/dev/sda1 partition - - 0b 1.0G /dev/sda
/dev/sda device - - - 1.0G -
This also contains a small code refactoring.
Thanks: Gerd Hoffmann (kraxel)
Code like:
CLEANUP_FREE char *buf;
/* some code which might return early */
buf = malloc (10);
is a potential bug because the free (*buf) might be called when buf is
an uninitialized pointer. Initialize buf = NULL to avoid this.
Several of these are bugs, most are not bugs (because there is no
early return statement before the variable gets initialized).
However the compiler can elide the initialization, and even if it does
not the performance "penalty" is miniscule, and correctness is better.
When using guestfish --remote, libguestfs is not the parent of qemu,
so waitpid is expected to return -ECHILD. And indeed guestfish --remote
would print a bogus error message in this case:
libguestfs: error: waitpid (qemu): No child processes
If the parent process forked into the background, then it should have
called 'set_recovery_proc 0' so we can use this to determine if we
need to wait for qemu.
Thanks: Kazuya Saito for reporting and identifying the issue.
This was accidentally moved by this commit:
commit b8b5ed65c2
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Wed May 23 11:46:23 2012 +0100
FAQ: Add section about using libguestfs in closed source programs.
Refactor the direct & uml backends to safely create the qemu & vmlinux
command lines.
Don't do malloc-after-fork, which is not safe. Do all the allocation
in the main process before the fork.
Use stringsbuf to simplify the code.
Instead of hard-coding the list of backends into the core of
libguestfs in many places, make each backend into a true module which
registers itself with the core when the library is loaded.
This is a step towards making backends into loadable modules, but it
doesn't actually do that.
This is just code motion.
It's fairly pointless to do this with TCG, since all we would be doing
is emulating a more complicated processor slowly. Also it may be the
cause of subtle problems we see during testing.
This updates commit 6f76fdb41e.
qemu -cpu \? documents this as:
host KVM processor with all supported host features (only available in KVM mode)
And indeed if you try it with TCG you'll get this error:
Unable to find CPU definition: host
This fixes commit 038ed0a08e.
These run the test suite using User-Mode Linux. Currently you have to
set LIBGUESTFS_QEMU to point to the vmlinux binary, but perhaps we
can solve that in future.
For reasons not fully understood, if md is linked into the kernel (and
not a module), a /dev/md0 device node is created. However this is not
a real RAID device. For example running mdadm --detail /dev/md0 will
fail on it.
Check the /dev/md<X> devices are real RAID devices before returning
them from the list-md-devices API.
This would have broken the function if we were comparing mixed
/dev/sda and /dev/ubda (ie different length) strings. Which we never
normally do, luckily.
If qemu goes away it is (more in theory than in reality) possible for
log messages to be lost in the kernel's socket buffer.
This turns out to matter more for User-Mode Linux.
Previously device name translation worked on the string in-place.
This worked fine because the device strings always come from XDR where
they are dynamically allocated. However it wouldn't work if the
translated name had to be longer than the original, specifically for
/dev/sd -> /dev/ubd (for User Mode Linux).
Therefore this commit changes the generator so that
device_name_translation and parse_btrfsvol (which depends on it)
allocate the new device name instead of overwriting it.
These macros are pretty horrible to use, with unexpected side-effects.
Move them exclusively into the generated code and rewrite the one
place in the general C code which used them.
There's no functional change in this code.
Using ./configure --enable-valgrind-daemon breaks valgrinding of the
library because guestfs_close wouldn't call guestfs_shutdown in that
case, resulting in some resources owned by the backend being leaked.
After this commit, --enable-valgrind-daemon will shut down the handle
normally.
This updates commit 55e3b8711f.
Don't add the "valgrind channel" to the appliance.
Just dump out the valgrind.log to stderr while the daemon is running.
Ensure that if valgrind tests fail in the appliance, that we don't
exit with success in the library by checking for a canary message in
the verbose daemon logs.
This allows the option to be used routinely by developers.
This is also called implicitly from internal_autosync, ensuring that
exit won't fail because of an open handle.
libguestfs: error: internal_autosync: umount: /sysroot: umount: /sysroot: target is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))
The code in do_part_get_bootable() assumes that if a partition has the
bootable flag set, then that is the only flag. It compares the entire
flags field with the string "boot". However, the boot flag isn't
always the only flag. For instance, POWER systems typically have a
bootable partition of type 0x41 (PPC PReP boot), which parted -m
displays as:
# parted -m -- f18.img unit b print
BYT;
/root/f18.img:16106127360B:file:512:512:msdos::;
1:1048576B:5242879B:4194304B:::boot, prep;
That is, the flags field contains "boot, prep", and thus libguestfs
fails to see that this partition is bootable. Ultimately this causes
virt-resize to fail to set the bootable flag on the boot partition of
the destination image, resulting in an image that won't boot.
This patch fixes the problem by searching for the string "boot" within
the flags field, instead of comparing the whole flags field.
'-cpu host' makes the CPU look like the host CPU. This is useful
because it means the appliance can use all the host CPU's features,
eg. for fast checksumming or MD XORing.
'kvmclock' is the paravirtualized clock for qemu/KVM. It's much more
stable than relying on tsc or a virtualized device such as HPET, and
clock instability is one of the problems we have encountered when
running libguestfs on a heavily loaded machine, especially in a nested VM.
Note that we require qemu >= 1.2 (and have for a while) which has long
supported both of these options.
- Use 'op' instead of 'xxx_op' for operation table.
- Add a 'defaults' for the operation table and remove any default
values.
This is just code motion and there is no functional change.
This adds a virt-sysprep 'password' operation which adds the
following command line options:
--root-password Set the guest's root password.
--password Set any guest password.
Disable this operation because it is more likely than not to break
Linux guests.
To fix this properly will involve finding all the places in the guest
where the UUIDs are used to locate and mount filesystems,
eg. /etc/fstab and possibly initramfs.
This updates commit 8965368eb8.
journal.c: In function 'do_journal_skip':
journal.c:134:5: error: format '%zd' expects argument of type 'signed size_t', but argument 3 has type 'int64_t' [-Werror=format=]
reply_with_perror_errno (-r, "failed to skip %zd journal entries", skip);
^
This fixes commit 5cb7f294f6.
This adds simple support for reading the journal files from guests
that use the systemd journal.
The new APIs are:
journal-open
journal-close
journal-next
journal-skip
journal-get
journal-get-data-threshold
journal-set-data-threshold
internal-journal-get
More complex journal support (eg. for seeking within the journal,
support for cursors) could be added later.
Although g->tmpdir is always set after launch, and none of these calls
should be used before launch, it's always possible that a caller would
use one of these APIs before launching the handle.
If guestfs___lazy_make_tmpdir is not called, then g->tmpdir is NULL,
and the constructed filename would look like "(null)/...".
In fact it's possible to observe this error (although it seems to have
no especially bad effects):
$ strace guestfish ls / |& fgrep '(null)'
unlink("(null)/ls1") = -1 ENOENT (No such file or directory)
The FileIn/FileOut parameters are not passed through to the daemon.
Previously we generated incorrect RPC code (an empty 'struct
guestfs_<fn>_args') because we didn't account for these FileIn/FileOut
parameters correctly.
Review every test(!) to ensure that it:
- Doesn't use a generic name (eg. "test1.img", "test.out") for any
temporary file it needs.
- Does instead use a unique name or a temporary name (eg. a name like
"name-of-the-test.img", or a scratch disk).
- Does not use 'rm -f' to clean up its temporary files (so we can
detect errors if the wrong temporary file is created or removed).
This allows tests to be run in parallel, so they don't stomp on each
other's temporary files.
Although autodetecting is safe (we always have just created a drive),
it saves a little bit of time if we don't have to run qemu-img to
detect the disk format with the libvirt backend.
Note that for prepared drives the format is always raw and we don't
anticipate creating drives in other formats.
Implement 'set-label' for XFS filesystems.
Fix the call for btrfs. Previous
commit d5817537fa added some bogus
documentation implying this call would work for btrfs, but it did
not.
Add tests.
Replaces code such as:
fd = open "test1.img"
ftruncate fd, size
close fd
g.add_drive "test1.img"
with the shorter and simpler:
g.add_drive_scratch size
Previously if the 'qemu-img info --output json' command failed, then
the stdout callback would be called first with a zero-length buffer.
Since a zero-length buffer cannot be valid JSON (in this context)
don't produce a separate error message.
However do catch the case when qemu-img info might produce no output
and no error (if that were to happen -- it does not happen at the
moment).
$ guestfish disk-format /etc/nosuchfile
Instead of this obscure double error message:
libguestfs: error: qemu-img info: JSON parse error: parse error: premature EOF
(right here) ------^
libguestfs: error: qemu-img info: /etc/nosuchfile: qemu-img info exited with error status 1.
To see full error messages you may need to enable debugging.
See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You will now see this simpler error:
libguestfs: error: disk info: /etc/nosuchfile: No such file or directory
Thanks: Bo Fan.
No single test should run longer than 1 hour even on the slowest of
hardware. We are having a problem in Koji where a test hangs and then
we end up losing the output completely, so a timeout + print the log
to that point is much better.
It was using the installed copy of libguestfs not the local copy.
We can't encode the paths in guestfs.go, since this source file gets
copied into the install directory where those paths would not make
sense. Instead we have to use CGO_CFLAGS and CGO_LDFLAGS to
communicate the right flags to go when doing the local build and test.
By specifying a cleanup function we can ensure that Augeas and hivex
functions can be tested.
There is no functional change here, verified by diffing the generated
file tests/c-api/tests.c before and after.
Add a utility function (fstype_is_extfs) to match ext2/3/4 filesystem
names. This is used in a couple of places.
When passing the mke2fs -t parameter, verify that the request is for
an ext2/3/4 filesystem. Previously we did not check this, and neither
did mke2fs when the -F flag was also used.
Since we don't usually move the first partition, if the first
partition has an unusually large offset from the start of the disk,
then the unpartitioned space in front of that partition counts as
partitioning overhead. However the previous surplus calculation was
not taking that into account.
This was a problem for certain Ubuntu images which are partitioned
with an 8 MB gap before the first partition.
Thanks: David Hart.
Also add an example from a real RHEL 6 guest.
Remove rules for building these XML files which are no longer
relevant. These files were built by running virt-inspector on real
guests, and then edited to remove excessive <application> sections.
Commit e71b2c11f5 broke inspection of
Fedora guests because guestfs_is_file returns false for
/etc/redhat-release (it's a symlink to a file, not a file).
We fix this by using the new followsymlinks flag added in the
previous commit. Thus guestfs_is_file becomes
guestfs_is_file_opts (g, filename, GUESTFS_IS_FILE_OPTS_FOLLOWSYMLINKS, 1, -1)
which checks if it's a file or a symlink to a file.
This fixes commit e71b2c11f5.
This adds an extra optional boolean 'followsymlinks' flag to those 6
is-* APIs. If the flag is true, then symlinks are followed, ie. we
use stat instead of lstat in the test.
For the rationale behind this change, see:
https://bugzilla.redhat.com/show_bug.cgi?id=974489
Commit 864ef706a8 (in Aug 2012) broke
the deprecated txz-out API. Because of a thinko during this commit,
instead of producing xz files it produces bzip2 files.
Fix this API again.
Thanks: Bo Fan / Red Hat QA team.
Fix a bogus warning introduced by
5abb196de8. If a non-windows partition
hasn't been detected as something else and falls through to windows
detection, inspection would display a bogus warning about missing
/boot.ini.
RWMJ: Test if guestfs_is_file > 0, in case it returns an error.
This change refactors guestfs___has_windows_systemroot to
guestfs___get_windows_systemroot. The new function returns a
dynamically allocated char * which must be freed.
The new function is no less efficient than before, as it returns the
result of guestfs___case_sensitive_path_silently, which is required
anyway. The new code is slightly more efficient than before, as it
re-uses the result of this testing in guestfs___check_windows_root
rather than running it again.
This file is mainly a central place to:
- include localenv if it exists, and
- define the RHEL 5 backwards compatibility macros, instead of
spreading them over every other file.
This has to run on every build, so:
- Use two grep commands instead of lots.
- Remove unnecessary use of sed.
- Remove some obsolete directories that haven't existed for a long time.
Previously, every subdirectory that contained generated files would
cause a recursion into the ../generator directory. This was slow and
unnecessary. Only recurse if the generator actually needs to be
rerun.
Note this changes the semantics slightly: For example if you added a
new action and just ran `make -C ocaml' then previously the generator
would be rerun, but now it would not. However I'm not convinced the
previous semantics were that useful, particularly considering that
with this change a simple `make' is much faster.
Yet another protocol bug!
If the daemon cancels, then the library should send a cancellation
chunk. It normally does this during the loop where it is uploading a
file. However due to an oversight or possibly a regression when we
modularized the socket code, the library did not send a cancellation
chunk if the daemon cancelled just before the end of file was sent.
This commit fixes this.
The public-facing functions like 'guestfs_*' are implemented as
wrappers around implementation functions like 'guestfs__*' (for
library-side functions only).
Don't call from one 'guestfs__*' function to another. It prevents
ENTER events from being caught, especially where a deprecated function
calls another implementation.
Most code was not doing this, but there were still some places left
where it was done.
The timeout lets us read remote images without timing out.
The noop scheduler lets the host do scheduling. This generally makes
sense for mixed workloads because the host has a global view of the
system, and often has more compute power too.
virtio-blk is obsolete so we don't need to bother optimizing it.
People should use virtio-scsi instead.
Also virtio-blk should choose the optimal settings itself.
This was originally added to fix this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=509383
This runs all of the check* rules. Since this includes 'make check',
'make check-all' is not quite equivalent to the old 'make extra-tests'
which was removed in the previous commit.
The following commit:
commit 5a3da36626
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Thu Jan 24 17:07:38 2013 +0000
inspect: Use CLEANUP_* macros in inspection code.
can cause a double-free along an error path when certain guest files
are empty where we expected those files to contain at least one line.
This causes virt-inspector to crash when run on these guests.
The following is a test case which demonstrates the crash.
`f20rawhidex64' is a Fedora guest, but with small adjustments to the
test you could use any Linux guest for this test.
$ qemu-img create -f qcow2 -b f20rawhidex64 /tmp/test.qcow2
Formatting '/tmp/test.qcow2', fmt=qcow2 size=21474836480 backing_file='f20rawhidex64' encryption=off cluster_size=65536 lazy_refcounts=off
$ guestfish -i -a /tmp/test.qcow2 -- rm /etc/redhat-release : touch /etc/redhat-release
$ virt-inspector /tmp/test.qcow2
*** glibc detected *** virt-inspector: double free or corruption (fasttop): 0x00007f18bc9925a0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x34ecc7ca8e)[0x7f18b8e64a8e]
/lib64/libguestfs.so.0(+0x3f91898078)[0x7f18ba13c078]
/lib64/libguestfs.so.0(+0x3f91899761)[0x7f18ba13d761]
/lib64/libguestfs.so.0(+0x3f91896d12)[0x7f18ba13ad12]
/lib64/libguestfs.so.0(+0x3f91894140)[0x7f18ba138140]
/lib64/libguestfs.so.0(guestfs_inspect_os+0x35)[0x7f18ba0bcc35]
virt-inspector(main+0x547)[0x7f18ba7c57d7]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f18b8e09a05]
virt-inspector(+0x6665)[0x7f18ba7c7665]
This is a denial of service, but not likely to be exploitable.
(Found by Coverity)
Actually if you don't specify the port, currently libvirt gives
an error:
libguestfs: error: could not create appliance through libvirt: XML
error: missing port for host [code=27 domain=20]
However I think that is a bug in libvirt:
https://bugzilla.redhat.com/show_bug.cgi?id=965142
Because OStringList is an optarg, we must specify "a!" to
zend_parse_parameters.
However zend_parse_parameters still returns this as a *long* rather
than a NULL (or empty array) so positively check that it has returned
an array.
At least libpython2.7-dev and libpython3.3-dev on current
Debian/unstable ship with pkg-config files. As with the pkg-config
check for Lua, we check for versioned and an unversioned .pc files.
Modern qemu can now handle this properly. ':' is only special if what
precedes it looks like a transport, so:
qemu-system-x86_64 -drive foo:bar .. fails
qemu-system-x86_64 -drive ./foo:bar .. works
Thus by adding ./ in front of relative paths that contain ':' we can
work around this.
In addition, this broke iscsi:// URIs because iSCSI target names
routinely contain ':' characters.
This avoids confusion when using URIs in guestfish, since the path
will always start with a '/', and we don't otherwise know if we should
remove it or not. By forcing the '/' to always be there, we deal with
this problem in the API instead.
- Don't use fixed names for the disks. This will allow us to
parallelize this test.
- Add a new "GETKEY:<key>" String parameter which can retrieve keys
from the handle. The temporary disk names are stored as keys.
- Don't test the close callback. However this uses the close callback
mechanism to delete the temporary disks, and in any case close
callbacks are well tested by the language binding tests.
- The generated code now produces a static array of tests (instead of
a 'perform_tests' function), making it possible to parallelize.
Set ERL_LIBS and also create a phony 'ebin' directory for the
guestfs.beam (library) so that the tests run against the local copy
not the installed version, and so that separated builds work.
This was revealed by the regression test for RHBZ#664558.
Unfortunately because Ruby tests had been silently disabled,
this regression test was never run.
Despite this change, separated builds are still broken, apparently
because ExtUtils::MakeMaker doesn't support this sort of
configuration, or else we're using it wrong.
Since tests involving FileIn will often wish to read from local files,
it makes sense that they would want to open files in $srcdir.
Therefore allow such paths to be prefixed by "$srcdir" which is
substituted at run time by the same named environment variable (set by
automake).
This fixes separated builds in tests/c-api directory.
Previously the generator binary was (deliberately) placed into the
source directory. However there was no real reason for this to
happen. Always put the build products into the builddir.
Since stamp-generator is a Makefile thing, make it in the Makefile,
not in the generator itself.
Also this ensures stamp-generator is created in the builddir, not in
the source directory (fixing separated builds).
Because the generator subdirectory is cleaned first, and other
directories recurse into the generator directory, 'make distclean'
would fail (because generator/Makefile would be deleted by that
point).
Change subdirs-rules.mk so it doesn't fail if generator/Makefile has
been deleted already.
The previous test for grub-install hard-coded /dev/vda (ie. assuming
virtio-blk instead of more modern virtio-scsi).
This changes the test to hard-code /dev/sda instead. However this
change is still not correct since /dev/sda will be adjusted by block
device name translation in the call to grub-install, but not what is
written to /boot/grub/device.map.
Since we no longer support grub-install on Fedora, this won't affect
things, but Ubuntu still has a 'grub-install' command (although it's
actually from grub2).
This works around the following warning:
virt-ls.c: In function 'show_file':
virt-ls.c:574:1: error: stack protector not protecting function: all local arrays are less than 4 bytes long [-Werror=stack-protector]
With recent GCC, you get this warning:
tests-main.c: In function 'skipped':
tests-main.c:256:3: error: function might be possible candidate for 'gnu_printf' format attribute [-Werror=suggest-attribute=format]
This is safer, because otherwise a URI could contain some clever
"unix:..." string as the hostname, tricking qemu into opening a Unix
domain socket at an uncontrolled location.
This fixes commit 349300af08.
libxml2 is very commonly available on Linux distros and has also been
ported (and is widely available) on Mac OS X and Windows. Therefore
simply require libxml2, and remove a lot of conditional code.
Add a remote drive by doing:
guestfish -a ssh://example.com/path/to/disk.img
There are several different protocols supported, as explained in the
man page.
This affects all virt-* tools that use the common guestfish options
parsing code.
This adds helper C functions 'is_string_list' and 'is_device_list'
allowing these tests to be carried out in generic C code instead of as
specialized tests.
Instead of using the various 'TestOutput', 'TestOutputList' etc
macros, it makes better sense to let the tests contain fragments of C
code to perform the checks.
Several new macros are added:
- 'TestResult': This macro takes a C expression which is evaluated to
test the result of commands. For example to compare if a string
result has some value:
TestResult (* command sequence *), "STREQ (ret, \"abcd\")"
The variable "ret" contains the result from the last command in the
sequence. But also, variables "ret1", "ret2", etc contain the results
from the last but one command, the command before that, and so forth,
allowing much more flexible tests to be written.
- 'TestResultTrue', 'TestResultFalse': Wrappers that test the last
command for true/false.
It's simpler to use the glibc 'program_invocation_short_name(3)'
feature, and fall back to a generic solution. Also remove risky
assignments to argv[0].
These set or get the program name in the handle. Most programs
will never need to call this, since we set this, if possible,
using the glibc 'program_invocation_short_name(3)' feature.
- /usr/lib{,64}/gconv/*:
This is the iconv data, required for APIs like iconv_open to work. In
particular, the mtools 'mcopy' program fails completely if it cannot
use iconv. 'mcopy' is used by SYSLINUX to copy files to the disk.
The error you would have seen is:
Error converting to codepage 850 Invalid argument
Cannot initialize 'S:'
Bad target s:/ldlinux.sys
- /usr/bin/localedef and /usr/sbin/build-locale-archive:
These are not really needed, but both are simply small binaries so we
might as well not delete them.
Setting the 'sparse' optional boolean causes writes to be omitted if
the block to be written contains all zero bytes.
This should help with sparse backing files (eg. raw, qcow2, dm-thin, etc).
Also, modify virt-resize to use this option by default when copying
devices. The savings in virt-resize can be quite startling, eg
'du -sh' (ie. true size) of a resized disk image:
8.1G /tmp/f15x32-resized.img # before this change
3.2G /tmp/f15x32-resized.img # after this change
This API is an easier to use version of the existing guestfs_available,
because the new API returns true/false instead of throwing an error
when a feature from the list is not available.
In truth we've had this implementation internally in the library
and several tools and in Sys::Guestfs::Lib for a long time. This
change just turns it into a publicly consumable API.
This directory (containing HTML documentation) can be removed
completely when making clean. CLEANFILES cannot recursively remove a
directory, so use a clean-local rule instead.
Rules such as 'make check-valgrind' are implemented using a for-loop.
They would always exit after the first error, even if 'make -k' was
used at the top-level. Since 'make -k'-style behaviour is generally
more useful, change these for-loops so they run all the tests, and
report errors at the end.
Current code wrote a single valgrind.log in the top-level directory.
This was overwritten with each valgrinded program, which wasn't much
use. The new version writes multiple separate valgrind-*.log files
containing the date and PID (so each is unique).
It would nice if valgrind could delete log files that don't contain
errors, but this doesn't appear to be possible.
This large, but mainly mechanical commit, renames "attach method"
everywhere to "backend".
Backwards compatibility of the API (guestfs_{set,get}_attach_method)
and environment (LIBGUESTFS_ATTACH_METHOD) is maintained, but in new
code use guestfs_{set,get}_backend and LIBGUESTFS_BACKEND instead.
The default backend (launching qemu directly) is now called 'direct'
instead of 'appliance', although you can still use 'appliance' as a
synonym.
In virt-df and virt-alignment-scan, ensure that errors that happen in
worker threads are propagated all the way up and result in
exit(EXIT_FAILURE).
Note that this makes the align/test-virt-alignment-scan-guests.sh test
fail (for a genuine reason). This is fixed in the following commit.
This updates commit 8b90f55dc7.
If using (eg) virtio-blk, the canonical name won't work unless device
name translation is done first.
This fixes commit 47b929b7893b3a76ff22760d245cb80720ae6345..
In particular, building virt-sysprep is slow because there are so many
modules. Enable parallel builds. If it breaks, we should fix it, not
work around it.
We only have to serialize the two calls to ocamlmklib, since both will
try to create a file called 'libmlguestfs.a'. Apart from that,
parallel builds here should be fine.
For example:
$ guestfish --long-options
--add
--cmd-help
--connect
--csh
--domain
--echo-keys
[etc.]
The idea of this is to make it easier to write a bash completion
script that accurately expands --<TAB> options for each command.
ISO apparently specified a limit of 4096 bytes/characters for strings.
The description of the guestfish command 'add-drive' is now longer
than this limit, causing this warning.
Turn off this warning, since GCC has no problems with these strings.
This changes the existing (non-published-in-stable) API for
add_drives{_opt} so that instead of having separate 'server' and
'port' parameters, now we have a single 'server' parameter which is a
list of strings.
This is so we will be able to cope with protocols such as Ceph which
allows multiple servers, or Sheepdog which can use an implicit local
server (ie. zero servers specified).
NBD still requires exactly one server.
This warning is actively dangerous. Because it forces you to use a
'default:' case in every switch statement, it means you can no longer
detect missing cases in enums.
You can now add remote NBD drives using:
><fs> add-drive "" format:raw protocol:nbd server:localhost
(Note that you also need to add port:NNNN if the server is running on
a non-standard port).
The corresponding qemu-nbd service can be started by doing:
qemu-nbd disk.img -t
This commit also adds a test.
It causes plenty of problems with people have multiple parallel
versions of libguestfs installed, and there's no benefit because you
can easily run libguestfs and tools from the build directory.
As with Perl, you can now set INSTALLDIRS to control where Ruby places
its files (ie. in the site dir, which is the default, or in the vendor
dir).
The difference in file layout between 'make install' and
'make INSTALLDIRS=vendor install' is shown below (for Fedora 18).
--- /tmp/site 2013-03-14 12:14:35.740015694 +0000
+++ /tmp/vendor 2013-03-14 12:14:13.668093944 +0000
@@ -119,21 +119,20 @@
./usr/lib64/ocaml/guestfs/mlguestfs.cmxa
./usr/lib64/ocaml/stublibs/dllmlguestfs.so
./usr/lib64/ocaml/stublibs/dllmlguestfs.so.owner
-./usr/lib64/perl5/auto/Sys/Guestfs/Guestfs.bs
-./usr/lib64/perl5/auto/Sys/Guestfs/Guestfs.so
-./usr/lib64/perl5/auto/Sys/Guestfs/.packlist
./usr/lib64/perl5/perllocal.pod
-./usr/lib64/perl5/Sys/bindtests.pl
-./usr/lib64/perl5/Sys/Guestfs/Lib.pm
-./usr/lib64/perl5/Sys/Guestfs.pm
+./usr/lib64/perl5/vendor_perl/auto/Sys/Guestfs/Guestfs.bs
+./usr/lib64/perl5/vendor_perl/auto/Sys/Guestfs/Guestfs.so
+./usr/lib64/perl5/vendor_perl/auto/Sys/Guestfs/.packlist
+./usr/lib64/perl5/vendor_perl/Sys/bindtests.pl
+./usr/lib64/perl5/vendor_perl/Sys/Guestfs/Lib.pm
+./usr/lib64/perl5/vendor_perl/Sys/Guestfs.pm
./usr/lib64/php/modules/guestfs_php.so
./usr/lib64/pkgconfig/libguestfs-gobject-1.0.pc
./usr/lib64/pkgconfig/libguestfs.pc
./usr/lib64/python2.7/site-packages/guestfs.py
./usr/lib64/python2.7/site-packages/libguestfsmod.la
./usr/lib64/python2.7/site-packages/libguestfsmod.so
-./usr/local/lib64/ruby/site_ruby/_guestfs.so
-./usr/local/share/ruby/site_ruby/guestfs.rb
+./usr/lib64/ruby/vendor_ruby/_guestfs.so
./usr/sbin/libguestfs-make-fixed-appliance
./usr/share/doc/libguestfs/example-debian-netinst-cd.xml
./usr/share/doc/libguestfs/example-debian.xml
@@ -406,3 +405,4 @@
./usr/share/man/uk/man3/guestfs-perl.3
./usr/share/man/uk/man3/guestfs-python.3
./usr/share/man/uk/man3/guestfs-ruby.3
+./usr/share/ruby/vendor_ruby/guestfs.rb
For prompting me to fix this, thanks: Hilko Bengen.
This adds a common utility function (guestfs___exit_status_to_string)
and a common error function (guestfs___external_command_failed), and
uses them all over the library and tools when converting exit status
in error messages etc.
This allows custom parameters to modify parameters added by
libguestfs, eg. by doing:
-set drive.hd0.file=rbd:foo/bar
[infernix adds the comment below:]
This works for recent versions of QEMU. The way to use this is:
guestfish
><fs> add-drive /dev/null
><fs> config -set drive.hd0.file=rbd:pool/volume
><fs> run
This will probably work for more types (sheepdog, iscsi et al), but
with the caveat that the libguestfs layers are bypassed so you'll
*always* end up directly accessing the disk in rw mode.
Prefix functions in this file with old_parser_... to make it clear
this is the old / slightly unsafe human output parser, which is
shortly to be replaced by a safer JSON parser.
This is just code motion.
- Use ./run script to run the tests.
- Set environment variables correctly, including $PATH.
- Test the locally built, not installed, copy of libguestfs.
The values of CONFIG, LAUNCHING and READY and (kind of) exposed
through the API so shouldn't change.
Use a bad sentinel for NO_HANDLE since that indicates a serious error.
Ruby ships its own config.h which may or may not define the same
relevant constants as our autoconf-generated config.h. Instead of
trying to specify the exact path to the wanted header file we may just
as well simply use Ruby's autoconf-inspired checks and macros.
This creates an abstract layer below the protocol code which handles
the socket operations. This will make it easier to introduce libvirt
virSocketPtr operations in future.
In the handle, g->conn contains the connection to the appliance.
g->conn is NULL when we're not connected.
poll(2) is used instead of select(2).
All error messages about launch failing or the appliance unexpectedly
dying are handled by two common error message functions, and these
contain a better explanation of what to do.
The earlier (pipe-based) code never set this flag, but that was a bug,
potentially allowing the file descriptor to be leaked to subprocesses.
Set the FD_CLOEXEC flag, but also ensure it is cleared in the child
process just before qemu is exec'd (otherwise qemu would not have a
console).
The libvirt backend already uses a Unix socket for the appliance
console, and so for the libvirt backend the fields g->fd[0] == g->fd[1].
Change the appliance backend to use a socketpair, so we need just a
single file descriptor for qemu stdin/stdout (ie. appliance console).
Consequently we can remove the array int fd[2] in the handle and
replace it with a single file descriptor.
When debugging is enabled, this prints out the permissions (ie.
ls -laZ) of the appliance and sockets directories. This should be
helpful for debugging RHBZ#913774.
Since qemu connects to both, there was no point at all in having these
sockets be readable by other, nor executable by anyone.
The sockets should now end up as:
srw-rw---- root.qemu
Despite what the documentation may have said, it certainly was not
safe to call guestfs_umount_local from another thread.
guestfs_umount_local could generate events, call the error handler, or
access other fields in the handle, with no locking at all.
We should aim to make guestfs_umount_local thread-safe in future.
See also:
https://bugzilla.redhat.com/show_bug.cgi?id=917706
This function is now generated, so bindings in various languages
are made automatically.
Note that the function previously returned void, but now it returns
int (although always 0). We don't believe that this is an ABI break
since existing programs will continue to work.
Non-daemon functions normally have a wrapper function called
eg. guestfs_name. The "real" (ie. hand-written) function is called
eg. guestfs__name. The wrapper deals with checking parameters and
doing trace messages.
This commit allows the wrapper function to be omitted. The reason is
so that we can handle a few functions that have to be thread-safe
(currently just: guestfs_user_cancel). The wrapper is not thread safe
because it can call events and/or the error handler.
libxml2 xmlXPathEvalExpression can return an unusual nodeset:
$1 = {type = XPATH_NODESET, nodesetval = 0x0, boolval = 0, floatval = 0,
stringval = 0x0, user = 0x0, index = 0, user2 = 0x0, index2 = 0}
Note that the nodeset is non-NULL, but the nodesetval is NULL.
Check every call site and fix those that don't deal with this
correctly.
When adding a domain (ie. guestfs_add_domain), read the SELinux
<label/> and <imagelabel/> from the guest and use them for the
appliance. The appliance is statically labelled the same as the
guest, so it is able to read its disks.
However tell libvirt not to try relabelling the disks, to prevent
libvirt from disturbing the existing labels on the disks (in
particular when the libvirt connection is closed, we don't want
libvirt to try to restore some other label on the disks).
Updated with feedback from Matthew Booth.
When a disk is opened readonly, the libvirt attach-method privately
creates a qcow2 overlay on top.
This commit lets that overlay get an SELinux label, and sets it to the
imagelabel specified by guestfs_internal_set_libvirt_selinux_label.
The above only applies to the libvirt attach-method.
If set, this causes <seclabel model=selinux relabel=no> to be added to
the disk element in the libvirt XML.
It has no effect *except* on the libvirt attach method when SELinux
and sVirt is being used.
This internal API sets two SELinux labels in the handle (the process
label and the image label -- they are closely related).
If using the libvirt attach-method with SELinux and sVirt, then this
will cause the following XML to be added to the appliance definition:
<seclabel type=static model=selinux relabel=yes>
<label>[LABEL HERE]</label>
<imagelabel>[IMAGELABEL HERE]</imagelabel>
</seclabel>
It is ignored by other attach-methods.
libguestfs handles are not thread safe, and it's not safe even to read
settings from the handle from multiple threads (eg. guestfs_get_trace).
Stop doing this in the parallel library. This caused fairly
reproducible segfaults when you enabled '-x' and/or '-v'.
This fixes commit 34e77af1bf.
This function is no longer called from anywhere outside the library,
or even outside libvirt-domain.c.
This is just code motion, except for the size_t changes which are
required in order to avoid signed overflow optimization error:
assuming signed overflow does not occur when simplifying conditional
to constant [-Werror=strict-overflow]
You can now run virt-df and virt-alignment-scan on all of the test
guests by doing:
./run ./df/virt-df -c test:///path/to/tests/guests/guests.xml
./run ./align/virt-alignment-scan -c test:///path/to/tests/guests/guests.xml
which is kinda cool.
Previously when you asked virt-df to show stats for all your libvirt
guests, it used a single appliance and tried to attach as many disks
as possible to it, even disks from different guests.
However this has been problematic: Not only is the code to do this
horrendously complex, but it's also insecure, and it doesn't interact
well with sVirt labels (see RHBZ#912499 comment 7).
In the meantime we discovered that running parallel appliances gives
you most of the performance of using a single appliance, but with a
lot less complexity and better guest isolation (see the documentation
in commit 680450f3b4).
Therefore this is a partial rewrite of virt-df so that in this case it
now uses parallel appliances.
Notes:
The '--one-per-guest' option is now the default and only way to do
things; this option now does nothing.
By default, the amount of parallelism to use is controlled by the
amount of free memory seen when virt-df starts up (subject to some
minima and maxima). The user can control this through new command
line option '-P'.
This mini-library runs the 'free -m' command and greps the output to
estimate the max. number of libguestfs appliances we could run in
parallel in the remaining free memory.
(Not entirely sure whether using Gnulib to replace standard functions
is a good idea at all.)
link with libgnu:
CCLD libguestfs-test-tool
libguestfs_test_tool-test-tool.o: In function `main':
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:103: undefined reference to `rpl_getopt_long'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:113: undefined reference to `rpl_optarg'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:125: undefined reference to `rpl_optarg'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:126: undefined reference to `rpl_optarg'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:109: undefined reference to `rpl_optarg'
libguestfs_test_tool-test-tool.o: In function `set_qemu':
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:382: undefined reference to `rpl_perror'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:366: undefined reference to `rpl_perror'
libguestfs_test_tool-test-tool.o: In function `make_files':
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:416: undefined reference to `rpl_perror'
/home/bengen/src/deb/pkg-libvirt/libguestfs/debian/build-default/test-tool/../../../test-tool/test-tool.c:428: undefined reference to `rpl_perror'
CC guestfsd-9p.o
In file included from ../../../daemon/9p.c:32:0:
../../../daemon/daemon.h:33:34: fatal error: guestfs-internal-all.h: No such file or directory
This involves adding a new test API which crashes the appliance in the
middle of a simulated upload, then a test which uses that API to test
for the libguestfs (library-side) crash.
Instead of the segfault you now get a more informative error:
*stdin*:0: libguestfs: error: connection to daemon was closed unexpectedly.
This usually means the libguestfs appliance crashed. Please enable
debugging (LIBGUESTFS_DEBUG=1) and rerun the command, then look at the
debug messages output prior to this error.
libguestfs: error: /dev/stdout: write: Broken pipe
libguestfs: error: file receive cancelled by daemon
It should be possible to add the guestfs___free_string_list to
dllmlguestfs.so, but I cannot work out exactly how to do this. As a
result we end up using src/utils.c directly.
The libutils convenience library is a place for code shared between
the main library, language bindings and virt tools. Note that the
code is statically linked into both the library, each binding and each
tool, but this is an improvement because (a) the source is shared and
(b) libguestfs.so can export fewer private functions.
Currently it contains the cleanup functions, and the functions
guestfs___free_string_list function and guestfs___for_each_disk.
guestfs___for_each_disk has changed so that it no longer
unconditionally sets the error in the guestfs handle. Instead callers
can control error handling.
These are internal functions. Very old versions of libguestfs used to
export them, but they haven't been (and shouldn't be) exported for a
long time.
Also remove the unused guestfs_error function.
Earlier versions of automake complain if they get a configuration
parameter which they don't understand. The error is:
configure.ac:27: error: option 'serial-tests' not recognized
Use some m4 hackery to work around this.
It was failing to clean up and close the 'struct command' buffer,
because commit 94d90f03e2 added an
incomplete patch to use CLEANUP_CMD_CLOSE.
This fixes commit 94d90f03e2.
btrfs_subvolume_list can now take either the path of a mounted btrfs
filesystem, or a mountable describing the location of a btrfs
filesystem, or one of its volumes. In the latter case, the filesystem
will be automatically mounted outside of /sysroot before running the
btrfs tool, and unmounted afterwards.
There are now two forms of the 'debug progress' command:
(1) debug progress <n> (the original form) generates ordinary
rate-limited progress messages for <n> seconds.
(2) debug progress <n> <rate> generates progress messages every <rate>
microseconds for <n> seconds.
The second form omit the usual rate-limiting, and so wouldn't
be generated like this from an ordinary API call. However this
is useful for testing events (see RHBZ#909624).
Note that you have to use this as follows:
#if defined(__GNUC__) && GUESTFS_GCC_VERSION >= x0y0z /* gcc >= x.y.z */
since GUESTFS_GCC_VERSION is not defined unless it's GCC.
A Mountable is passed from the library to the daemon as a string. The daemon
stub parses it into a mountable_t, which it passes to the implementation.
Update all implementations which now take a mountable_t.
This change updates the api style of all apis which should take Mountable
descriptions rather than block devices. It also updates the documentation
accordingly, but doesn't implement any functional changes.
Because of evolution of the code, if the user used the --qemu or
--qemudir options, libguestfs-test-tool would segfault because
guestfs_set_qemu was being called before the guestfs handle was
opened.
Change the code so this doesn't happen, and also remove the global 'g'
variable to make the code a bit more robust.
Bug found by Amit Shah.
The second parameter passed into getline(3) is the size of the
allocated buffer, *NOT* the length of the returned line. This can be
confusing, so rename this parameter as 'allocsize' consistently
throughout the code.
This is just code motion.
This was disturbing to some users, but other users saw the warning
when it was really something they should pay attention to. Make it
clear that it's a warning, unless people are really using SELinux +
sVirt.
Thanks: Richard Harman.
For example:
{
CLEANUP_FREE_STAT struct guestfs_stat *stat = guestfs_stat (g, "/foo");
// stat is freed automatically when we leave the scope.
}
The new header file "guestfs-internal-frontend-cleanups.h"
includes definitions for these. It should not be included
directly.
Partition number was being passed to guestfs___check_for_filesystem_on
based on an index in list_partition. However, this ignores the
possibility of multiple block devices.
This change makes guestfs___check_for_filesystem_on examine the
passed-in device directly to determine if it is a whole device, or
what its partition number is.
The output of btrfs subvolume list has changed in F18 to include generation,
which breaks the parsing in btrfs_subvolume_list. This change replaces sscanf
with a more robust regular expression. The new regular expression should also
handle the addition of future unexpected columns.
The visibility field in action replaces in_fish, in_docs and internal.
The defined types are:
VPublic:
A public API. This is exported and documented in all language
bindings, and in guestfish.
VStateTest:
A public API which queries the library state machine. It is exported
and documented in all language bindings, but not guestfish.
VBindTest:
An internal API used only for testing language bindings. It is
guarded by GUESTFS_PRIVATE in the C api, but exported by all other
language bindings as it is required for testing. If language
bindings offer any way to guard use of these apis, that mechanism
should be used. It is not documented anywhere.
VDebug:
A debugging API. It is exported by all language bindings, and in
guestfish, but is not documented anywhere.
VInternal:
An internal-only API. It is guarded by GUESTFS_PRIVATE in the C api,
and not exported at all in any other language binding. It is not
documented anywhere.
Replace selected calls to 'perror (filename)' with:
fprintf (stderr, "syscall: %s: %m\n", filename);
so that more information is available about precisely which syscall
failed.
Note this is *not* reply_with_perror. These messages are only printed
in verbose output, for the benefit of debugging.
Previously I noticed that bfan used this command without any error:
><fs> hivex-open /WINDOWS/system32/config/software write:ture
^^^^^^^^^^
This was because the code allowed any string to be evaluated as a
boolean.
The new code is stricter. It allows the following strings only case
insensitive (everything else is an error):
1
true
t
yes
y
on
0
false
f
no
n
off
Not to be confused with the libxml2 macro 'BAD_CAST' which converts
from 'signed char *' to 'unsigned char *'.
The 'bad_cast' function was defined and used all over the place as a
replacement for a '(char *)' cast. I think it is better to make these
casts explicit, instead of hiding them in an obscure function.
This commit rearranges the internal header files.
"src/guestfs-internal.h" is just for the library, as before.
"src/guestfs-internal-frontend.h" is for use by all library, bindings,
tools C code, but NOT the daemon.
"src/guestfs-internal-all.h" is for use by all C code including the
daemon.
This is just code motion, but it has some important consequences:
(1) We can use the CLEANUP_* macros in bindings and tools code.
(2) We can get rid of TMP_TEMPLATE_ON_STACK.
(3) We will (in future) be able to stop bindings and tools code from
using the safe_* allocation functions (which are NOT safe to use
outside the library alone).
qemu-wrapper isn't regenerated if QEMUDIR is changed, so just
delete it and force regeneration.
Additionally we can drop the silent binary check, since
check-with-upstream-qemu-1 already does a similar test with --version
that will actually report an error to the user.
Certain functions are intended to be internal only, but we currently
export them anyway. This change moves them into a separate section of
guestfs.h protected by a GUESTFS_PRIVATE variable. This change also
enables private structs, but doesn't implement any.
This change only affects the C api. Language bindings aren't affected,
but probably should be in the future.
Without this change all the tests in the haskell bindings are rebuilt
every time. The primary motivation for this change is to fix this.
The fix for the above also allows parallel builds to be re-enabled.
Use the macro like this to create temporary variables which are
automatically cleaned up when the scope is exited:
{
CLEANUP_FREE char *foo = safe_strdup (bar);
...
// no need to call free (foo)!
}
The following code is also valid. The initialization of foo as 'NULL'
prevents any chance of free being called on an uninitialized pointer.
It may not be required in all cases.
{
CLEANUP_FREE char *foo = NULL;
...
foo = safe_malloc (100);
...
// no need to call free (foo)!
}
This is also valid:
{
CLEANUP_FREE char *foo = ..., *bar = ...;
...
// no need to call free (foo) or free (bar)!
}
The CLEANUP_FREE_STRING_LIST macro calls guestfs___free_string_list
on its argument. The argument may be NULL.
The CLEANUP_HASH_FREE macro calls hash_free on its argument. The
argument may be NULL.
Important implementation note:
------------------------------
On GCC and LLVM, this is implemented using __attribute__((cleanup(...))).
There is no known way to implement this macro on other C compilers, so
this construct will cause a resource leak.
Important note about close/fclose:
----------------------------------
We did NOT implement 'CLEANUP_CLOSE' or 'CLEANUP_FCLOSE' macros. The
reason is that I am not convinced that these can be used safely. It
would be OK to use these to collect file handles along failure paths,
but you would still want a regular call to 'close'/'fclose' since you
must test for errors, and so you end up having to do:
if (close (fd) == -1) {
// failure case
// avoid double-close in cleanup handler:
fd = -1;
...
}
// avoid double-close in cleanup handler:
fd = -1;
...
This is just code motion, but it allows us to read this flag inside
the 'construct_libvirt_xml_seclabel' function in future (as a possible
way to fix RHBZ#890291).
When debugging is enabled, this produces output like below. This is
useful when diagnosing what URI libguestfs is using.
libguestfs: opening libvirt handle: URI = NULL, auth = virConnectAuthPtrDefault, flags = 0
libguestfs: successfully opened libvirt handle: conn = 0xb05580
guestfs_last_errno (g) == 0 doesn't mean "no error". It means the
errno was not captured. In this case we have to substitute some sort
of errno, so choose EINVAL arbitrarily.
The reasons to do this are twofold:
(a) It's probably a tiny bit faster.
(b) It lets us capture the real errno if the link(2) syscall fails.
The errno is also passed through guestmount, fixing RHBZ#895905:
+ guestmount -a test1.img -m /dev/sda1:/ -m /dev/sda2:/boot /tmp/mnt
+ touch /tmp/mnt/foo
+ cd /tmp/mnt
+ ln foo boot/foo
ln: failed to create hard link ‘boot/foo’ => ‘foo’: Invalid cross-device link
cp will fail if /etc/lvm is an empty directory. Copy the entire
directory and adjust environment variable.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
RWMJ:
- Fixed a couple of whitespace issues.
Using // coverity[...] or /* coverity[...] */ comments in the source
it is possible to suppress specific Coverity errors. The suppressed
error should occur in the line following the comment.
In this case I have suppressed two false positives from Coverity:
(a) We deliberately assign to a NULL pointer in order to cause a
segfault, for testing how the library reacts when this happens.
Coverity flags this, but it is not an error in this case.
(b) Coverity does not model global variables (a known shortcoming).
Therefore the code 'errno = posix_memalign (...)' cannot be modelled
by Coverity, even though the code is correct. Coverity raises a false
positive about this.
(Thanks Kamil Dudka, Coverity)
If there was no quoting character, the options could run off the end
of the string and into whatever string which happened to be following
in memory.
This bug was revealed when libguestfs was compiled on arm.
For the rationale behind this renaming, see:
b8cbe191d0
You can still, for the time being, use
./configure --with-febootstrap-yum-config
but it's preferred to switch to the new option. The new option is
identical, just renamed.
Qe still pass the deprecated --yum-config option to febootstrap
itself, since both old and new febootstrap support this.
For some reason the fs->device field was not set for swap devices.
Apparently this did not matter before (this bug has existed since at
least libguestfs 1.16), but in commit 6ee80b8dac we started to depend
on the fs->device field being non-NULL. This resulted in segfaults
when inspecting any guest which had a swap device, eg:
#0 __strcmp_sse42 () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:164
#1 0x00007ffff7d740cb in parent_device_already_probed (g=0x703a70,
partition=0x6f79b0 "/dev/vda4") at inspect.c:156
#2 0x00007ffff7d73e00 in guestfs__inspect_os (g=0x703a70) at inspect.c:83
#3 0x00007ffff7cee11d in guestfs_inspect_os (g=0x703a70) at actions-1.c:397
This commit sets fs->device to the device name.
(Thanks Matthew Booth for first noticing this problem.)
If ntfs-3g is not installed, then tests/guests creates a 'windows.img'
file which is zero-sized.
This is problematical because other tests which use this image will
fail at best, or hit RHBZ#847549 (qemu/virtio-scsi bug) at worst.
Change the tests which use windows.img so they skip if 'windows.img'
has zero size.
If you want to selectively run tests, or if the test suite fails half
way through, use:
make print-subdirs
to print the subdirectories, select the ones you want to run / the
remaining ones, and do:
make check SUBDIRS="..."
User Phill Bandelow noted that virt-resize fails with an e2fsck error
on a host where the system clock had been accidentally set in the
past.
Unfortunately this was hard to diagnose because guestfsd 'ate' the
stdout of the e2fsck program. I have verified by code inspection that
e2fsck prints messages on stdout.
Thus this changes the daemon to fold stdout and stderr together so we
get to see all error messages from e2fsck when it fails.
Rename guestfs_safe_malloc et al to guestfs___safe_malloc etc.
To use the private functions, code now has to define
-DGUESTFS_PRIVATE_FUNCTIONS=1. This will make it easier for us in
future to work out which programs are using these functions and to
minimize both the number of programs and the functions they are
calling.
Note that the Perl, Python, OCaml, Ruby and Java bindings use
guestfs_safe_* calls. None of the other bindings do. This is a bug
(in the bindings using those functions): these functions will call the
out of memory callback on failure. This function defaults to abort(),
and since this happens from a language binding, there is no way to
change this default.
I guess no one uses GCC 3 any more, because if they did, including
<guestfs.h> would not work since GUESTFS_DLL_PUBLIC would not have
been defined.
At the same time, ensure that __attribute__((noreturn)) is only used
when the compiler is GCC >= 3.1.
Currently the code is doing a redundant fork when passed the
COMMAND_FLAG_CHROOT_COPY_FILE_TO_STDIN flag. The additional
sub-process does a chroot() which has no effect because all file
handles are already open at that point, then simply copies its input
to its output.
This change simply replaces the above with a dup2 of the passed file
handle to STDIN of the command process.
RWMJ:
Don't initialize 'pid' variable.
Improve readability of commandrvf() by replacing bare int values for
file descriptors with their symbolic names STD{IN,OUT,ERR}_FILENO.
Also add PIPE_READ and PIPE_WRITE for referencing relevant ends of a pipe.
When executing a command, we temporarily chroot, fork and exec the
command, then chroot back. We intentionally don't chdir in the parent
process so that we can 'jailbreak' the chroot later. However, this has
the effect that commands are executed with a current working directory
which is outside the current root. This unusual state can cause
errors in executed commands which don't anticipate it.
This change does a chdir("/") before executing and command. This
happens inside the fork, so the jailbreak isn't affected in the
parent.
Commit ed7fda161e changed the g->drives
linked list into an array, and also made a subtle change to the
(undocumented) semantics of drives in the rare case when qemu is
relaunched.
Every time we go back to the CONFIG state, we should free the drives
array. However there was one place where this was not happening.
This was caught by the tests of the protocol, but only when using the
default (appliance) method, not the libvirt method which we were more
interested in testing in Fedora.
When testing on Debian using the appliance method, we therefore hit
this bug for the first time.
This relates to or fixes the following commits:
commit 9df159b34a
commit 33f49d85c2
commit ed7fda161e
Since we as developers rarely test the case where some library is
statically not available, that side of the code was hardly tested,
except by unfortunate users in the field who often hit cases where
functions were missing or misdeclared. In fact, when making this
change I noticed several bugs like that.
Change it so that this code is autogenerated, and therefore always
correct and up to date.
Previous code which looked like this:
int
optgroup_acl_available (void)
{
return 0;
}
char * __attribute__((noreturn))
do_acl_get_file (const char *path, const char *acltype)
{
abort ();
}
/* etc */
is replaced by a single line:
OPTGROUP_ACL_NOT_AVAILABLE
Since I made lots of changes to the Erlang bindings (prompted by
Coverity), I wasn't confident that something didn't break because
there were no real tests before.
Comment from Coverity:
libguestfs-1.19.66/erlang/erl-guestfs-proto.c:115: sign_extension:
Suspicious implicit sign extension: "buf[0]" with type "unsigned char"
(8 bits, unsigned) is promoted in "(buf[0] << 24) | (buf[1] << 16) |
(buf[2] << 8) | buf[3]" to type "int" (32 bits, signed), then
sign-extended to type "unsigned long" (64 bits, unsigned). If
"(buf[0] << 24) | (buf[1] << 16) | (buf[2] << 8) | buf[3]" is greater
than 0x7FFFFFFF, the upper bits of the result will all be 1.
In the previous code, appliance was unconditionally set to NULL and so
we never added the dummy appliance slot to g->drives. This probably
doesn't matter, since hotplugging doesn't work for the 'appliance'
attach-method. Despite that, it's better to get this right.
This fixes commit ed7fda161e.
For some guestfs_set_* calls, add checks for error, when error might
possibly occur. eg. It's plausible that guestfs_set_network might
fail if the attach-method being used doesn't support it (although this
doesn't happen at the moment).
In other cases, don't check for errors, eg. if the error doesn't
matter or there's nothing we could plausibly do about it.
These configure flags enable code profiling (with gprof) and code
coverage (with gcov) respectively.
Although this is a nice idea, it's not currently very useful.
Libtool mangles filenames in such a way that gcov cannot locate its
datafiles.
Profiling is of dubious utility with libguestfs which is not CPU-bound
and relies extensively on running external programs (oprofile-like
system profiling that took into account libguestfs + qemu or
libguestfs + qemu + the appliance + filesystem tools *would* be
useful).
Also neither flag will help in capturing data from the appliance.
latest archlinux has some small changes:
btrfs-progs-unstable --> btrfs-progs
genisoimage --> cdrkit
iproute --> iproute2
module-init-tools --> kmod
procps --> procps-ng
udev --> systemd
util-linux-ng --> util-linux
and could you plz add the below packages
lrzip
less
sed
RWMJ: I took this opportunity to make some general cleanups to the file:
- add 'less' and 'sed' for everyone
- 'module-init-tools' & 'kmod'[1]
- 'procps' & 'procps-ng'[1]
- 'util-linux' & 'util-linux-ng'[1]
- sort the ArchLinux packages into order
[1] Note that febootstrap is supposed to ignore packages it doesn't
know about.
The way the existing configure script worked, @VG@ would be empty, and
it would run all the way through the check-valgrind rule successfully
without actually running anything under valgrind.
See the comment from the code:
NB about Windows 8: No location we've found so far contains a
suitable icon for Win8. In particular, explorer.exe definitely
does *not* contain any Windows logo as a resource (I checked).
Therefore the "basket icon" that this produces is just a stand-in
until we have a better idea for solving this problem.
and:
https://bugzilla.redhat.com/show_bug.cgi?id=801117
This is very well explained in the bugzilla comments:
https://bugzilla.redhat.com/show_bug.cgi?id=880801#c1
With this change, the disk name has a one or more plus signs ('+')
appended to show that there are additional disks:
$ virt-df -a Win7x32TwoDisks-a -a Win7x32TwoDisks-b
Filesystem 1K-blocks Used Available Use%
Win7x32TwoDisks-a+:/dev/sda1 102396 24712 77684 25%
Win7x32TwoDisks-a+:/dev/sda2 12478460 7403416 5075044 60%
Win7x32TwoDisks-a+:/dev/sdb1 521212 55728 465484 11%
The read_all function is used as a callback for
guestfs___cmd_set_stdout_callback (cmd, read_all, [str],
CMD_STDOUT_FLAG_WHOLE_BUFFER);
As noted in the documentation for CMD_STDOUT_FLAG_WHOLE_BUFFER, the
buffer returned is not \0-terminated, and so using memdup will create
an unterminated string, and therefore potentially a memory overrun
when reading or searching the string.
Use strndup instead so the final string is \0-terminated.
Try to get the host's loops_per_jiffy value and pass this on the
command line to the guest. In theory this should avoid the appliance
having to recalculate this value in a VM with TCG (which is generally
error-prone). This should avoid timing issues.
We only do this when we are certain that the guest will be TCG.
Currently we only have enough information to do this from the libvirt
attach-method. So mostly this will only affect people using Fedora in
a VM.
The host loops_per_jiffy value is not exported by the kernel. It is
only printed by the kernel early during boot, so if boot messages have
"scrolled off" the kernel ring buffer, it won't be available. Some
operating systems save early kernel messages in /var/log/dmesg but (a)
Fedora 18+ seem to have abandoned this file and (b) on Ubuntu this
file is unreadable for spurious "security" reasons.
I have submitted a patch to make lpj available through /proc/cpuinfo.
Fix the following memory leak found by valgrind:
==13629== 498 (112 direct, 386 indirect) bytes in 1 blocks are definitely lost in loss record 99 of 110
==13629== at 0x4A06B2F: calloc (vg_replace_malloc.c:593)
==13629== by 0x4CA564E: guestfs_safe_calloc (alloc.c:71)
==13629== by 0x4CA9B02: guestfs___new_command (command.c:143)
==13629== by 0x4CA66E9: guestfs___build_appliance (appliance.c:690)
==13629== by 0x4CBD1B9: launch_libvirt (launch-libvirt.c:188)
==13629== by 0x402E7E: main (virt-filesystems.c:349)
Also adjust the command code in several places to make it simpler. We
can almost always call guestfs___cmd_close right after
guestfs___cmd_run, avoiding any need to close the handle along error
paths.
Tested by running the test suite under valgrind.
Because the shell script was missing a space, the final output test
was failing, although for some reason this didn't cause the whole test
to fail.
Fixing the missing space also revealed a bug in the test output, which
is also fixed.
See http://article.gmane.org/gmane.comp.lang.lua.general/95065
Note that this is incompatible with existing code. You
have to do:
local G = require "guestfs"
local g = G.create ()
ie. give the module your own name ("G" in that example).
This is faster, but more importantly it avoids the strange error
'lua: attempt to index a string value' which appears with (some)
single element lists.
- add support for events (with test)
- test progress messages
- update documentation to describe events
- refactor handle closing code
- refactor error code
- use 'assert' in test code instead of 'if ... then error end'
This file was present in git, but missing from the tarball.
Don't actually run the test: it depends on details of how the
appliance is constructed which would fail for old-style appliances.
'make extra-tests' was a monolithic set of tests that did all sorts of
things: valgrind, tests over local guests, tests with upstream qemu,
tests with upstream libvirt, tests with the appliance attach method.
This made it hard to perform individual tests, eg. just valgrind
testing. It was also hard to maintain because the tests were not
located in the same directories as the programs and sometimes
duplicated tests that were run elsewhere.
This commit splits up 'make extra-tests' into 5 separate targets:
make check-valgrind # run a subset of tests under valgrind
make check-valgrind-local-guests # test under valgrind with local guests
make check-with-appliance # test with attach-method == appliance
make check-with-upstream-qemu # test with an alternate/upstream qemu
make check-with-upstream-libvirt # test with an alternate/upstream libvirt
(You can also still run 'make extra-tests' which is now simply
a rule that runs the above 5 targets in order).
This replaces everything that was in the tests/extra directory,
so that has now gone.
This is mostly just code motion, but it also changes the default
location where valgrind.log is written to be the top build directory
(instead of tests/extra/valgrind.log).
This file had grown by accretion to include:
- code related to handles (now in src/handle.c)
- safe allocation (src/alloc.c)
- debug, errors, warnings (src/errors.c)
- private data (src/private-data.c)
- miscellaneous functions (src/canonical-name.c, src/utils.c)
This commit also removes about a dozen #include files which were
probably not really used.
This is just code motion.
If you have a function which is in the X optgroup, and also has
IfAvailable X, then two tests for the optgroup are added to the
output.
Check for this case and give an error.
This also fixes existing APIs.
guestfs_pop_error_handler.
The addition of these calls makes it much simpler to temporarily
disable or ignore errors:
guestfs_push_error_handler (g, NULL, NULL);
guestfs_mkdir (g, "/foo"); /* We don't care if this fails. */
guestfs_pop_error_handler (g);
Note these calls are used by the C API only. They are not necessary
in other languages, because those map errors to exceptions.
The subsequent commits change existing code to use push/pop instead of
laboriously saving and restoring the error handler.
New APIs: set-tmpdir, get-tmpdir, set-cachedir, get-cachedir.
The current code has evolved over time and has a number of problems:
(a) A single environment variable ($TMPDIR) controls the
location of several directories.
(b) It's hard for the library user to predict which directory
libguestfs will use, unless the user simulates the same internal steps
that libguestfs performs.
This commit fixes these issues.
(a) Now three environment variables control the location of all small
temporary files, and the appliance cache:
For temporary files: $LIBGUESTFS_TMPDIR or $TMPDIR or /tmp.
For the appliance cache: $LIBGUESTFS_CACHEDIR or $TMPDIR or /var/tmp.
The user can also set these directories explicitly through API calls
(guestfs_set_tmpdir and guestfs_set_cachedir).
(b) The user can also retrieve the actual directories that libguestfs
will use, by calling guestfs_get_tmpdir and guestfs_get_cachedir.
These functions are also used internally.
This commit also:
- reworks the internal tmpdir code
- removes the internal (undocumented) guestfs_tmpdir call (replacing
it with calls to the documented guestfs_get_tmpdir API instead)
- changes the ./run script to set LIBGUESTFS_TMPDIR and
LIBGUESTFS_CACHEDIR
- adds a test
- fixes a few places like libguestfs-make-fixed-appliance which
depended on $TMPDIR
Previously wipefs (the utility) allowed this, but it's obviously a bad
thing to do and the new wipefs gives an error if you try it:
wipefs: error: /dev/VG/LV: probing initialization failed: Device or resource busy
Split the functions according to a simple hash across
C.nr_actions_files files (this number can be increased later if
necessary).
This is just code motion.
This is currently done implicitly because of the linker script.
However in order to do symbol versioning, we will have to do
this explicitly at each definition instead.
Use <cpu mode="host-model"> (not model=... which was a typo).
Also set fallback=allow so that libvirt knows it can use another
CPU model if it wants.
This fixes commit 890a4fbc87.
Commit 2e90f43129 added a debug message
which isn't filtered out by the set of grep -v's in this test.
Therefore this test would fail when run with LIBGUESTFS_DEBUG=1.
Commit a0722c7ad8 introduced a
dependency on the Unix module. This was not listed in the list of
'-package's, but as long as you had ocaml-gettext installed it would
still work because that pulled in Unix implicitly.
Thanks Olaf Hering.
This is a wrapper or mini-library for running external command,
loosely based on libvirt's virCommand interface.
Amongst the advantages are:
- Can redirect errors into the error log (RHBZ#713678).
- Can redirect output into a callback function.
- Handles shell quoting properly.
- Safely resets signal handlers, closes file descriptors, etc.
- Single place where we can implement other improvements in future.
Used to create temporary directory or file with an optional suffix.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Split this out into a new file (daemon/mktemp.c).
- I don't see a reason to deprecate the mkdtemp function which
works fine. Instead remove complex dir-making code from the
new function.
- Test and fix the patch (missing close(fd)).
The cleanup of package managers cache is already implemented
by sprep_operation_package_manager_cache.ml, so remove this
TODO item.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
In languages like Python where we release a global lock around
long-running libguestfs functions, it is also useful to *not* release
this lock for small, non-blocking functions.
Therefore mark all functions with a 'blocking' boolean flag. It
defaults to true, and is true by definition for all daemon functions.
For non-daemon functions, I have classified them manually.
Only when the blocking flag is set do we generate the code to release
and reacquire the lock around libguestfs calls.
guestfs_parse_environment_list.
Add a new function for creating a handle:
guestfs_h *guestfs_create_flags (unsigned flags [, ...]);
This variant lets you supply flags and extra arguments, although extra
arguments are not used at the moment.
Of particular interest is the ability to separate the creation of the
handle from the parsing of environment variables like
LIBGUESTFS_DEBUG. guestfs_create does both together, which prevents
us from propagating errors from parsing environment variables back to
the caller (guestfs_create has always printed any errors on stderr and
then just ignored them).
If you are interested in these errors, you can now write:
g = guestfs_create_flags (GUESTFS_CREATE_NO_ENVIRONMENT);
if (!g)
exit (EXIT_FAILURE);
r = guestfs_parse_environment (g);
if (!r)
exit (EXIT_FAILURE);
Also you can *omit* the call to guestfs_parse_environment, which
creates a handle unaffected by the environment (which was not possible
before).
This commit also includes new (backwards compatible) changes to the
OCaml, Perl, Python, Ruby and Java constructors that let you use the
flags.
This commit models libvirt authentication events through the API,
adding one new event (GUESTFS_EVENT_LIBVIRT_AUTH) and several new
APIs:
guestfs_set_libvirt_supported_credentials
guestfs_get_libvirt_requested_credentials
guestfs_get_libvirt_requested_credential_prompt
guestfs_get_libvirt_requested_credential_challenge
guestfs_get_libvirt_requested_credential_defresult
guestfs_set_libvirt_requested_credential
See the documentation and example which shows how to use the new API.
This commit also changes existing calls to virConnectOpen* within the
library so that the new API is used.
Also included is an example (but not a test, because it's hard to see
how to automatically test the libvirt API).
Normally we check if libvirt is installed, and the default is still to
do this.
However having this option makes it simpler to check if a change to
the code has broken the case where libvirt is not installed.
In fact, this change revealed one such place, which is also fixed in
this commit.
This means that we can use virConnectPtr and virDomainPtr instead of
using void* types. In the case where libvirt support is disabled or
not present, effectively nothing changes.
If TMPDIR is not set, we must choose one, because otherwise libvirt
will use a random TMPDIR: http://bugzilla.redhat.com/865464
The convenient guestfs___persistent_tmpdir function does everything
needed in this case.
The code was identical -- just copied with s/resize/sparsify/.
Instead of duplicating identical code, cause the Makefile.am to use
the code from the ../resize/ directory.
Unfortunately because there are two Utils modules (which are
different), this means we had to rename those modules to Resize_utils
and Sparsify_utils respectively. So this is a rather larger change
than intended. However it's just code motion.
Revert "btrfs: Add an extended workaround for btrfs failures seen with kernel 3.7.0."
Reverted these workaround, since we may have found a fix for the btrfs
bug itself (for details see RHBZ#863978).
This reverts commit d9e5b514aa
and commit a03f536f0d.
Commit ed7fda161e changed the
way that the drives are handled across appliance shutdowns.
Previously during the following sequence of calls:
create the handle
add drive(s)
launch
kill subprocess
launch
the added drives are still in the handle at the second launch.
After the above commit, the added drives are removed from the handle,
which means the second launch happens with no drives (which could be
an error).
This was never actually defined either way, so in this case fix the
test to re-add the drive before the second launch.
Since hotplugging was added, it isn't really feasible to return to the
original semantics, since users might remove drives, in which case we
have lost information about those drives so we cannot restore them on
the second launch.
NOTE: PLEASE CALLERS DON'T DO THIS! Always use a new handle for each
launch of the appliance.
When libvirt is used, we can allow disks to be hotplugged.
guestfs_add_drive can be called after launch to hot-add a disk.
When a disk is hot-added, we first ask libvirt to add the disk to the
appliance, then we make an internal call into the appliance to get it
to wait for the disk to appear (ie. udev_settle ()).
Hot-added disks are tracked in the g->drives array.
This also adds a test.
New API: list-disk-labels
Allow the user to pass an optional disk label when adding a drive.
This is passed through to qemu / libvirt using the disk serial field,
and from there to the appliance which exposes it through udev,
creating a special alias of the device /dev/disk/guestfs/<label>.
Partitions are named /dev/disk/guestfs/<label><partnum>.
virtio-blk and virtio-scsi limit the serial field to 20 bytes. We
further limit the name to maximum 20 ASCII characters in [a-zA-Z].
list-devices and list-partitions are not changed: these calls still
return raw block device names. However a new call, list-disk-labels,
returns a hash table allowing callers to map between disk labels, and
block device and partition names.
This commit also includes a test.
Instead of adding the snapshot=on option via <qemu:arg>, create qcow2
overlays for any read-only drives and the appliance using 'qemu-img
create' + a temporary file.
This is a workaround for missing support for <transient/> in libvirt's
qemu driver. Also for the unpredictable way that libvirtd handles
$TMPDIR: we want to control where the temporary disk is created.
Currently it is also much slower, because qemu-img is slow. However
we hope to fix qemu upstream.
(RHBZ#858128).
If compare_device_names was given two devices with devices with
different interfaces (eg. /dev/sda and /dev/vda) then it would try to
compare the partition numbers, and fail when it could parse them.
It's arguable what we should be doing in this case (except for
strongly discouraging people from using the interface feature), but
let's at least not cause the daemon to assert-fail.
Found by Red Hat QA, thanks Mohua Li.
Don't assume that if guestfs_case_sensitive_path returns NULL, that it
means the file does not exist.
The (previously undefined) behaviour of case_sensitive_path was that a
NULL return meant "either the file doesn't exist or some other error".
However in commit 973581780d this was
changed so that if the last element of the path didn't exist, it was
assumed to be a new file and the (non-NULL) path of the new file is
returned.
This change breaks code (including in libguestfs) which tries to use
case_sensitive_path as a dual-purpose call to fix-up a path for
Windows and test if the file exists. Such code should be rewritten so
that it explicitly tests for file existence after calling
case_sensitive_path.
I examined all the calls to case_sensitive_path in libguestfs and
modified them where necessary.
The 'windows_path' function was blindly copied from virt-cat. In
virt-cat, errors are checked by the caller to 'windows_path'. But
virt-edit lacks this check. Change the function in virt-edit to add a
check and exit on error.
There's not really any point just documenting that people
should label the tmp directory, when we can just label it.
So change the ./run script so it calls chcon.
However it's not a failure if chcon doesn't work. If SELinux
is not installed or is disabled, then labelling will fail.
Everyone ignores the documentation. Everyone ignores me even
when I specifically tell them to include the complete output.
Place a prominent notice at the beginning of the output.
Let tests/bigdirs/test-big-dirs.pl uses mke2fs to create small
inode ratio files.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Fix invocation of mke2fs with optional arguments.
- Run the test to check it works.
New api mke2fs for full configuration of filesystem.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Update description.
- Run and fix the tests.
- Remove bogus filename from po/POTFILES.
Without this empty rule, automake doesn't generate the rest of the
testing infrastructure, so you can't enable the test yourself by doing
'make check TESTS=test-big-dirs.pl'.
Since generator source files were renamed, the comment at the
top of each generated file was wrong.
Unfortunately we cannot allow /* to appear within a comment,
so the space is necessary.
cron-spool claims to remove at jobs, but it has no code to actually do
that. Add patterns to remove files in known at spool locations.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Currently firstboot would only work on redhat-based images.
Handle redhat-based, suse-based and debian guests, error out in case of an
unknown distro.
Update firstboot.sh:
- make sure scripts exists and can be executed
- add LSB header to avoid insserv warnings later on
- run script only if called with "start"
Update functions, pass only required options.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
The second parameter to 'config' may be NULL.
In commit 52fa23d74f (refactoring of
guestfs_config) the code this got lost, and guestfs_config would
segfault if qemu_value was NULL.
Also this fixes the libvirt method to handle the same case.
I checked libguestfs-1.18 and -1.16 branches, and this problem does
NOT affect them.
Update SuSE Linux detection. Up to now everything with a
/etc/SuSE-release file was treated as openSuSE. With this change SLES
based distributions such as "SUSE Linux Enterprise Server", "SUSE
Linux Enterprise Desktop" and "Novell Linux Desktop" will show up as
"sles". The 'opensuse' detection was updated to handle older openSuSE
releases as well as the well known SuSE Linux releases, starting from
6.0. Everything else with a /etc/SuSE-release file will show up as
"suse-based".
Here is a collection of release files:
==> Dist/6.0/etc/SuSE-release <==
SuSE Linux 6.0 (i386)
VERSION = 6.0
==> Dist/6.1/etc/SuSE-release <==
SuSE Linux 6.1 (i386)
VERSION = 6.1
==> Dist/6.2/etc/SuSE-release <==
SuSE Linux 6.2 (i386)
VERSION = 6.2
==> Dist/6.3/etc/SuSE-release <==
SuSE Linux 6.3 (i386)
VERSION = 6.3
==> Dist/6.4/etc/SuSE-release <==
SuSE Linux 6.4 (i386)
VERSION = 6.4
==> Dist/7.0/etc/SuSE-release <==
SuSE Linux 7.0 (i386)
VERSION = 7.0
==> Dist/7.1/etc/SuSE-release <==
SuSE Linux 7.1 (i386)
VERSION = 7.1
==> Dist/7.2/etc/SuSE-release <==
SuSE Linux 7.2 (i386)
VERSION = 7.2
==> Dist/7.3/etc/SuSE-release <==
SuSE Linux 7.3 (i386)
VERSION = 7.3
==> Dist/8.0/etc/SuSE-release <==
SuSE Linux 8.0 (i386)
VERSION = 8.0
==> Dist/8.1/etc/SuSE-release <==
SuSE Linux 8.1 (i386)
VERSION = 8.1
==> Dist/8.2/etc/SuSE-release <==
SuSE Linux 8.2 (x86-64)
VERSION = 8.2
==> Dist/9.0/etc/SuSE-release <==
SuSE Linux 9.0 (x86-64)
VERSION = 9.0
==> Dist/9.1/etc/SuSE-release <==
SuSE Linux 9.1 (x86-64)
VERSION = 9.1
==> Dist/9.2/etc/SuSE-release <==
SuSE Linux 9.2 (x86-64)
VERSION = 9.2
==> Dist/9.3/etc/SuSE-release <==
SuSE Linux 9.3 (x86-64)
VERSION = 9.3
==> Dist/10.0/etc/SuSE-release <==
SUSE LINUX 10.0 (X86-64) OSS
VERSION = 10.0
==> Dist/10.1-remastered/etc/SuSE-release <==
SUSE LINUX 10.1 (X86-64)
VERSION = 10.1
==> Dist/10.1/etc/SuSE-release <==
SUSE Linux 10.1 (X86-64) Beta8
VERSION = 10.1
==> Dist/10.2/etc/SuSE-release <==
openSUSE 10.2 (X86-64)
VERSION = 10.2
==> Dist/10.3/etc/SuSE-release <==
openSUSE 10.3 (X86-64)
VERSION = 10.3
==> Dist/11.0/etc/SuSE-release <==
openSUSE 11.0 (X86-64)
VERSION = 11.0
==> Dist/11.1/etc/SuSE-release <==
openSUSE 11.1 (x86_64)
VERSION = 11.1
==> Dist/11.2/etc/SuSE-release <==
openSUSE 11.2 (x86_64)
VERSION = 11.2
==> Dist/11.3/etc/SuSE-release <==
openSUSE 11.3 (x86_64)
VERSION = 11.3
==> Dist/11.4/etc/SuSE-release <==
openSUSE 11.4 (x86_64)
VERSION = 11.4
CODENAME = Celadon
==> Dist/12.1/etc/SuSE-release <==
openSUSE 12.1 (x86_64)
VERSION = 12.1
CODENAME = Asparagus
==> Dist/12.2/etc/SuSE-release <==
openSUSE 12.2 (x86_64)
VERSION = 12.2
CODENAME = Mantis
==> Dist/sles8/etc/SuSE-release <==
SuSE SLES-8 (AMD64)
VERSION = 8.1
==> Dist/sles9/etc/SuSE-release <==
SUSE LINUX Enterprise Server 9 (x86_64)
VERSION = 9
==> Dist/sles9sp2/etc/SuSE-release <==
SUSE LINUX Enterprise Server 9 (x86_64)
VERSION = 9
PATCHLEVEL = 2
==> Dist/sles9sp3/etc/SuSE-release <==
SUSE LINUX Enterprise Server 9 (x86_64)
VERSION = 9
PATCHLEVEL = 3
==> Dist/sles9sp4/etc/SuSE-release <==
SUSE LINUX Enterprise Server 9 (x86_64)
VERSION = 9
PATCHLEVEL = 4
==> Dist/sled10/etc/SuSE-release <==
SUSE Linux Enterprise Desktop 10 (x86_64)
VERSION = 10
==> Dist/sled10sp1/etc/SuSE-release <==
SUSE Linux Enterprise Desktop 10 (x86_64)
VERSION = 10
PATCHLEVEL = 1
==> Dist/sled10sp2/etc/SuSE-release <==
SUSE Linux Enterprise Desktop 10 (x86_64)
VERSION = 10
PATCHLEVEL = 2
==> Dist/sled10sp3/etc/SuSE-release <==
SUSE Linux Enterprise Desktop 10 (x86_64)
VERSION = 10
PATCHLEVEL = 3
==> Dist/sled10sp4/etc/SuSE-release <==
SUSE Linux Enterprise Desktop 10 (x86_64)
VERSION = 10
PATCHLEVEL = 4
==> Dist/sled11/etc/SuSE-release <==
SUSE Linux Enterprise Desktop 11 (x86_64)
VERSION = 11
PATCHLEVEL = 0
==> Dist/sled11sp1/etc/SuSE-release <==
SUSE Linux Enterprise Desktop 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1
==> Dist/sled11sp2/etc/SuSE-release <==
SUSE Linux Enterprise Desktop 11 (x86_64)
VERSION = 11
PATCHLEVEL = 2
==> Dist/sles10/etc/SuSE-release <==
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
==> Dist/sles10sp1/etc/SuSE-release <==
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 1
==> Dist/sles10sp2/etc/SuSE-release <==
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 2
==> Dist/sles10sp3/etc/SuSE-release <==
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 3
==> Dist/sles10sp4/etc/SuSE-release <==
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 4
==> Dist/sles11/etc/SuSE-release <==
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 0
==> Dist/sles11sp1/etc/SuSE-release <==
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1
==> Dist/sles11sp2/etc/SuSE-release <==
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 2
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Since this is the most common error seen by people who have
installation problems, buggy qemu, etc, and since no one reads the
FAQ, describe in this error message what resources are available to
debug launch problems.
libzypp and zypper store their cache files in
/var/cache/zypp/{packages,raw,solv}/ and /var/cache/zypper/RPMS/
Add a single pattern to match both cases.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
SuSE based installations store the hostname in /etc/HOSTNAME.
Add code to handle both opensuse and sles. Code to properly detect
the latter will be added with another patch.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
<file>: error: jump skips variable initialization [-Werror=jump-misses-init]
This has only just appeared, possibly related to previous gnulib
update. In any case, this is just code motion / cleanup.
For some reason, argument declarations were handled
inconsistently for strings and non-string arguments.
Non-string arguments were all declared at the top of the
function stub, where as string arguments were declared
in place.
Be consistent.
This is just code motion.
* cfg.mk (sc_prohibit_trailing_blank_lines): Remove rule. It is now
subsumed by the much more efficient sc_prohibit_empty_lines_at_EOF
rule in gnulib's maint.mk.
Passing the $upstream_qemu environment variable is not reliable when
libvirt is involved: Environment variables are passed to the first
instance of the session libvirtd, but because libvirt reuses the
session libvirtd, subsequent environment variable settings are lost
(this is filed as RHBZ#856619).
Bypass all of this by writing a custom qemu-wrapper.sh which contains
the hard-coded values we want.
On Linux PATH_MAX is 4096, but on some platforms it can be much larger
or even not defined (ie. unlimited). Therefore using a PATH_MAX-sized
stack buffer is not a great idea for portable programs.
This change removes use of PATH_MAX-sized stack-allocated buffers.
This change only applies to the library and standalone programs.
Inside the daemon, memory allocation is much more complicated so I
have not changed those (yet).
Found by 'make syntax-check'.
If these are not defined in the header files, it's a problem on
the platform which we should fix with gnulib. Our replacement
definitions might be wrong here.
Having a separate directory means:
(1) It's easy to clean up orphaned temporary files, the appliance,
etc.
(2) You can put an SELinux label on this directory so that qemu can
write to it when you're using sVirt and SELinux is enforcing:
chcon --reference=/tmp tmp
This is a fix for multiprogramming: If two instances of libguestfs
share the same appliance disk, then libvirt would unlabel the disk
when one of the instances closes the handle, resulting in the other
qemu being unable to continue accessing the appliance.
Adding the flag makes libvirt understand that the disk is shared so it
doesn't do this, and it apparently handles locking correctly too if we
were using sanlock.
Since 'QEMU' is set (overwritten) by the autoconf-generated Makefile,
this test would always test installed qemu, not upstream qemu,
making it pretty useless. Use another name for this variable.
If you go back in time in git (eg. git reset, git bisect) then you can
end up in a situation where a file that was generated by a later
version is left around unused in the earlier version.
This isn't a problem for most things, but gobject documentation
generation picks up any file in a directory, even unreferenced ones,
and breaks. So the correct thing to do is to remove these files.
I: Program returns random data in a function
E: libguestfs no-return-in-nonvoid-function guestmount.c:75
The function fuse_opt_add_opt_escaped has only one caller and a return
code is not checked.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
This hint tells the backend whether anyone cares about errors when the
appliance is shut down.
Currently this only has any effect on the libvirt backend, where it
controls whether or not we use the VIR_DOMAIN_DESTROY_GRACEFUL flag.
This command was not tested at all. As a result we didn't notice that
it was broken for a long time (RHBZ#853159).
This adds a test that drives the command through a pty. It uses the
perl 'Expect' module, although this is not required.
https://bugzilla.redhat.com/show_bug.cgi?id=853159
git bisect pointed to the following commit:
commit ec8e3b6cad
Author: Richard W.M. Jones <rjones@redhat.com>
Date: Fri Jul 20 14:24:10 2012 +0100
launch: Abstract attach method operations.
g->attach_ops points to a structure which contains the
operations supported by each attach method backend
(ie. appliance, unix, etc.).
Since that commit was essentially just code motion, it wasn't clear
why virt-rescue should be affected by it.
In fact the reason is as follows:
(1) In direct mode, we don't need g->fd[] (which would normally be
connected to the stdin/stdout of qemu). So we opened them on
/dev/null so they had some value.
(2) accept_from_daemon / read_log_message_or_eof reads from g->fd[1].
Since this is connected to /dev/null, it always reads EOF.
(3) This would cause child_cleanup to be called. This is completely
unintentional: we don't want to cleanup the child at this point, even
in direct mode.
(4) Prior to the commit above, child_cleanup first waited for the
process to exit (ie. waitpid). This happened to work, since we are
effectively waiting for the user to exit virt-rescue.
(5) After the commit above, the order of operations was changed so
that we first killed qemu before waiting for it. This broke
virt-rescue.
The fix is to change direct mode so that it leaves g->fd[]'s as -1.
The rest of the protocol code can deal with this situation -- it
ignores the log fd instead of trying to read from it.
The libvirt backend currently doesn't support direct mode. We should
make that work in future. As a workaround, force the attach-method to
be 'appliance' in this case.
We set the error handler to NULL in order to mask "normal" error
messages that we expect to see because of the unusual way that
virt-rescue runs the appliance. However if the user selected -v /
enabled debugging, then it is reasonable to expect they want to see
every message, so do not mask anything.
guestfs_disk_image makes a symbolic link to the real filename in order
to sanitize the filename. However this fails if the filename is a
relative path. Call realpath(3) to make the filename canonical.
This fixes commit 20902e7ce0.
udevadm is included in all reasonable recent distributions.
This avoids 'command not found' errors in verbose mode.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
This is a simple renaming of the files/modules.
Note that in OCaml, module names are derived from filenames by
capitalizing the first letter. Thus the old module names had the form
"Generator_api_versions". The new modules names have the form
"Api_versions".
This is almost just code motion. While doing this I discovered two
prototypes for functions which no longer exist:
* guestfs___launch_appliance
* guestfs___launch_unix
so these prototypes have been removed. The rest are rearranged more
logically.
guestfsd calls many different tools. Keeping track of all of them is
error prone. This patch introduces a new helper macro to put the command
string into its own ELF section:
GUESTFSD_EXT_CMD(C_variable, command_name);
This syntax makes it still possible to grep for used command names.
The actual usage of the collected list could be like this:
objcopy -j .guestfsd_ext_cmds -O binary daemon/guestfsd /dev/stdout |
tr '\0' '\n' | sort -u
The resulting output will be used to tell mkinitrd which programs to
copy into the initrd.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
RWMJ:
- Move str_vgchange at request of author.
- Fix snprintf call in daemon/debug.c
Add a new api xfs_repair for repairing an XFS filesystem.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Fix non-error return path so it doesn't send two replies.
- Document return code.
I tested this by comparing the output of virt-inspector over Windows
guests before and after the change, which was identical:
$ md5sum `ls -1 /tmp/*.before /tmp/*.after`
c292d6629b5a761eccb4a279754399b4 /tmp/Win2003.after
c292d6629b5a761eccb4a279754399b4 /tmp/Win2003.before
eb1e1ff29208a9ee46e9c100dfec26b2 /tmp/Win2012.after
eb1e1ff29208a9ee46e9c100dfec26b2 /tmp/Win2012.before
d060a95d7ffe5dce6c4e66feb80c2837 /tmp/Win7x32.after
d060a95d7ffe5dce6c4e66feb80c2837 /tmp/Win7x32.before
8914eee70ac4f8a0317659e09e00dcdc /tmp/Win7x32Dynamic.after
8914eee70ac4f8a0317659e09e00dcdc /tmp/Win7x32Dynamic.before
a2dcdfc0f9d64054640875aa791889e0 /tmp/Win7x32TwoDisks.after
a2dcdfc0f9d64054640875aa791889e0 /tmp/Win7x32TwoDisks.before
5ed49568a5147dce7517c99de41ebf2e /tmp/Win8previewx64.after
5ed49568a5147dce7517c99de41ebf2e /tmp/Win8previewx64.before
fdfc7d272b79a665ae3313ae1ae30660 /tmp/WinXP.after
fdfc7d272b79a665ae3313ae1ae30660 /tmp/WinXP.before
3c705444be664f1316b21c5d8d3cb0be /tmp/WinXPRecConsole.after
3c705444be664f1316b21c5d8d3cb0be /tmp/WinXPRecConsole.before
Transscribe many hivex(3) APIs into the libguestfs API.
There is one hive handle per libguestfs handle, as with Augeas.
Note that hivex uses iconv_open for some APIs (eg. hivex_value_string).
But since we delete all the i18n files from the appliance, this
doesn't work -- iconv_open returns EINVAL. Therefore hivex APIs which
require iconv cannot be bound in the daemon.
The library doesn't actually use libaugeas, except indirectly via the
libguestfs API. The libguestfs API implements this in the daemon, so
there's no need for the library to link to augeas at all.
libguestfs fails to build with --enable-fuse on openSuSE 11.4 and
earlier because the included fuse version does not include
libulockmgr.so. configure already used pkgconfig to retrieve the correct
CFLAGS, so there is no need to hardcode -lulockmgr.
With this change the build succeeds again.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Without this patch, if JAVAC points to a valid executable, --with-java-home=no
is not respected but the build simply fails.
Furthermore, --with{,out}-java is a more common form and more in-line with all
other options.
The assert (conn != NULL) was being triggered with this
stack trace:
at launch-libvirt.c:1305
fd=<optimized out>, error_if_eof=error_if_eof@entry=0) at proto.c:222
size_rtn=size_rtn@entry=0x7fffffffdb34,
buf_rtn=buf_rtn@entry=0x7fffffffdb58) at proto.c:548
libvirt_uri=<optimized out>) at launch-libvirt.c:391
--disable-perl isn't currently respected because $PERL correctly points
to the perl executable and $missing_perl_modules doesn't prevent the
perl parts to be installed either. Thus, a check for $enable_perl is
needed.
These don't contain literal strings like "AUTHORS" (they have the
translated versions instead) and so checking for these English strings
is rather useless.
Also we don't usually have the '--license' parameter conveniently
around so don't enforce this section.
This adds standard LICENSE and BUGS sections to all of the man pages
that are processed by podwrapper.
Modify all the calls to $(PODWRAPPER) to add the right --license
parameter according to the content. Note that this relaxes the
license on some code example pages, making them effectively BSD-style
licensed.
section.
Ensure each man page contains consistent COPYRIGHT and AUTHOR
sections.
Remove the LICENSE section. We will add that back in podwrapper in a
later commit.
Add new api xfs_admin to change parameters of an XFS filesystem.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Remove printuuid, printlabel, since they don't change any settings.
- Adjusted the documentation.
- Fix the tests.
Since our minimum supported version is now 1.16 and mount was fixed in
1.13.16, it is now safe to replace mount-options + empty options with
mount wherever it occurs.
Note that we keep the old daemon calls, but rename them as
"internal_write" and "internal_write_append". This lets us implement
the new library-side calls more efficiently in the common case when
the uploaded content is smaller than the message buffer. In most
cases the new calls won't end up using a temporary file.
These internal (testing) APIs allow the bindtests output to
be sent to some other place than stdout.
This is necessary for Erlang, since stdout is used to communicate with
the Erlang interpreter.
Previously with Python it was impossible to set a boolean or integer
optarg to -1 because that was used as a special sentinel value to
indicate that the optarg was not set.
Instead, use None as the sentinel value, since that cannot be a
boolean or integer type.
The compress flag can be used to control compression, one of: (none),
"compress", "gzip", "bzip2", "xz", "lzop". Thus these calls can now
be used instead of tgz-in/tgz-out/txz-in/txz-out, and also support
more compression types.
Mark these APIs as once_had_no_optargs so that compatibility code is
generated.
Use the safer, higher level Store_field macro when constructing arrays
of structs to return.
I don't know if it is strictly necessary in this case, but it's safer.
We just remove the process accounting files previously without
touch a empty file, this will cause psacct runs error.
Restart the service can't help us create this file auto.
couldn't open file '/var/account/pacct': No such file or directory
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
In libguestfs 1.20, you will be able to use 'add_drive'
instead of 'add_drive_opts' (except in the C bindings).
However until libguestfs 1.20 is the minimum stable version
people will still be using old versions where you have to use
'add_drive_opts'. This makes the examples confusing.
Therefore continue to use 'add_drive_opts' in the examples
for now.
This commit makes grep into an optargs API, with flags for extended,
fixed, [case-]insensitive and compressed.
At the same time it deprecates: egrep, fgrep, grepi, egrepi, fgrepi,
zgrep, zegrep, zfgrep, zgrepi, zegrepi and zfgrepi.
NB: The patch to implement this feature in qemu is not upstream, and
may never make it upstream. However this is so useful for
virt-sparsify that I decided to add this to libguestfs while we see
what qemu decides to do.
Allow optargs functions to be called as either:
g.mkfs_opts(fs, device)
or:
g.mkfs_opts(fs, device, { optargs hash })
This also preserves backwards compatibility with once_has_no_optargs
functions such as mkfs.
qcow2 v3 is generally more efficient, although unfortunately it
doesn't support automatically sparsifying zero writes.
Note this *only* uses qcow2 v3 for the intermediate overlay file, not
for the final output (since we want the output to be broadly
compatible with old hypervisors).
These APIs will allow sysprep to change the UUIDs of all PVs and VGs
in the system.
LVs don't have UUIDs AFAICT, or at least there seems to be no way to
change them if they do have them.
Operations that need to work directly on guest block devices will fail
because the block devices are busy. Therefore add a phase with the
filesystems unmounted, and allow operations to specify that they need
to work in this phase.
remove the local machine ID, leave it empty, let it be generated
during next booting.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Whitespace change.
- Remove completed items from TODO.
Note this does not yet enable sVirt confinement of the appliance.
That requires at least that SELinux policy goes upstream, plus there
may still be bugs.
Use Dev_or_Path instead of String.
Remove the RESOLVE_DEVICE since Dev_or_Path will generate
REQUIRE_ROOT_OR_RESOLVE_DEVICE instead.
RWMJ:
Note a change in semantics: this now requires root. However this is
OK and still works with mkmountpoint and friends because
'is_root_mounted' works even if something is mounted below the root.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
If ./configure --with-default-attach-method is set to something other
than 'appliance', then this will legitimately return a different
string. Simply test that it runs, rather than testing the output.
This fixes commit 20a5b4de7d.
We were using format= which was bogus and libvirt was ignoring it and
forcing raw format instead.
Also in its default configuration libvirt won't do disk format
autodetection at all, so we must do it instead.
And in any case there is no point, since proto.c already sets the
SOCK_CLOEXEC flag (using accept4) on this socket.
The code now matches the code in launch-appliance.c.
You can now choose the default attach method in two ways:
(1) Set the LIBGUESTFS_ATTACH_METHOD environment variable.
(2) ./configure --with-default-attach-method=appliance|libvirt|...
Note that (1) overrides (2).
We need to select the <domain type=...> attribute depending on whether
KVM is detected in the libvirt capabilities.
This is a hack because it doesn't try to parse the libvirt
capabilities XML.
For compatibility with the appliance backend, devices added readonly
are in fact added writable with snapshots on top. This is necessary
because options like 'mount -o ro' usually require write access to the
disk (eg. to replay journals).
Complete the attach-method libvirt backend.
This backend uses libvirt to create a transient KVM domain to run the
appliance.
Note that this still will only work with local libvirt URIs since the
<kernel>, <initrd> and appliance links in the libvirt XML refer to
local files, and virtio serial only works locally (limitation of
libvirt). Remote support will be added later.
Since we will be calling guestfs___build_appliance from the libvirt
code in future, there's no point having two places where we have to
acquire the lock. Push the lock down into this function instead.
Because "glthread/lock.h" includes <errno.h> we have to add this
header to the file too.
Move and rewrite guestfs_config so it accumulates a list of qemu
parameters in the handle. These are added to the appliance at launch
time (with attach method == unix:... you'll now get an error).
launch-appliance.c contains the code associated with the 'appliance'
attach-method. Mostly. In fact there are a few APIs which don't fit
so nicely:
- config: deprecated API which fiddles with the qemu command
line directly
- max-disks: depends on the qemu implementation (virtio-scsi
or not)
- debug-drives: used for testing only
launch-unix.c contains the code associated with 'unix:<path>'.
launch.c is the common code for launching, along with a few other APIs
such as guestfs_add_drive_opts.
This commit also reduces the number of headers to just those
which are required.
Unfortunately the parallel test keeps hitting this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=838081
which could be a bug in the OCaml runtime.
Just test simple mount-local. We will write a parallel test in C to
replace this.
When using tar-in or tools built around it such as virt-make-fs, if
the target filesystem is vfat then unpacking the tarball would fail
because tar tries to chown(2) files and fails.
You would see errors such as:
tar: <file>: Cannot change ownership to uid 500, gid 500: Operation not permitted
This change detects whether the target filesystem supports chown(2).
If not, it adds the --no-same-owner flag to tar to stop it from trying
to change the ownership of newly created files.
Add proper dependencies for these files, but also use 'cmp' to ensure
they only get overwritten if the new files have actually changed, so
we don't rebuild the appliance unnecessarily.
Add xfs_info to show the geometry of the xfs filesystem.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Updated po/POTFILES.
- Use xfs_ prefix for all struct fields.
- Return uninitialized fields as -1 / empty string.
- Copyedit the description.
case_sensitive_path is undefined when the final path element doesn't
exist. Currently it returns an error, but this means that creating a
new file doesn't work as expected:
$ guestfish --rw -i -d windows touch 'win:c:\blah'
libguestfs: error: case_sensitive_path: blah no file or directory found with this name
We should allow this case (provided there is no trailing slash) so
that new files or directories can be created.
This uses Pod::Simple so it properly parses the input POD and can
generate proper custom output as required specifically by libguestfs.
One immediate benefit is that links between and within manual pages
now work mostly correctly.
By using the once_had_no_optargs flag, this change is backwards
compatible for callers (except Haskell, PHP and GObject as discussed
in earlier commit).
In C, a function called 'func' which has once_had_no_optargs=true will
(because of the previous commit) generate 'func_opts' and a
backwards-compatibility function called 'func'.
This commit changes some of the non-C bindings so that they also
generate 'func_opts' which is merely a wrapper that calls 'func'.
This avoids incompatibility when we rename 'mkfs_opts' etc back to
plain 'mkfs', and it also makes it easier to translate between other
language bindings and C code.
NB: Some bindings do not include aliases:
PHP: There's no way to easily alias methods in PHP < 5.3, and we
can't assume everyone has this minimum version.
GObject: Very complex to add aliases, but we should probably do this
at some point.
Haskell: No support for optargs in these bindings. Unfortunately
this means that we can no longer bind 'Guestfs.add_drive'
(since it will be changed to add optional arguments) making
the Haskell bindings even less useful than they were already.
This commit adds a flag (once_had_no_optargs) which can be used to add
optargs to functions that currently don't have any.
The idea is that if 'func' currently has no optargs, we can safely add
optargs provided we are backwards compatible for existing callers.
In C that means we leave 'guestfs_func' alone and provide an extra
function 'guestfs_func_opts' that takes the optargs ('guestfs_func'
becomes a wrapper that calls 'guestfs_func_opts').
In the C generator this means there are two names for each function
(although the two names are normally identical). 'c_name' is the name
that we export publicly (eg. [guestfs_] 'func_opts'). 'name' is the
internal name of the function (eg. 'func') which is used for
everything apart from the public interface, and also to generate the
no-optargs compat function.
In other languages that can add optional arguments safely, we simply
add the arguments to the existing 'func', so for example in Perl:
$g->func (required_args)
$g->func (required_args, optional_args)
can be used.
Note that this commit does not cause any change to the output of the
generator. I verified this by diffing the output before and after.
For example the existing method:
public void mkfs_opts (String fstype, String device, Map<..> optargs);
is now accompanied by this overloaded method which is a simple wrapper:
public void mkfs_opts (String fstype, String device)
throws LibGuestFSException
{
mkfs_opts (fstype, device, null);
}
This field, which is generated internally by the generator, is the
name of the C function corresponding to each action.
For actions that have NO optional arguments, it's just "guestfs_<name>".
For actions that have any optional arguments, it is
"guestfs_<name>_argv" (since any binding has to construct the optional
argument struct explicitly).
In a future commit, this mapping may become more complex.
This commit also "fixes" the C# bindings which didn't handle optional
arguments properly at all. In fact, it doesn't fix this, it just
changes it enough that it probably now compiles. We should either
compile and test the bindings routinely with Mono, or drop them, since
they are starting to bit-rot.
In the GObject bindings, I have added a space between the C function
name and the first paren.
Apart from the C# and GObject changes, this is just code motion. It
was verified by diffing the output of the generator before and after.
Store the camel-case name directly in the struct instead
of generating it on the fly in only the GObject bindings.
This is just code motion. Tested by verifying that the generator
output is identical.
Each action changes from a tuple like this:
("cat", (RString "content", [Pathname "path"], []), 4,
[ProtocolLimitWarning],
[InitISOFS, Always, TestOutput (
[["cat"; "/known-2"]], "abcdef\n")],
"list the contents of a file",
"[...]");
to a slightly longer but more readable struct:
{ defaults with
name = "cat";
style = RString "content", [Pathname "path"], [];
proc_nr = Some 4;
protocol_limit_warning = true;
tests = [
InitISOFS, Always, TestOutput (
[["cat"; "/known-2"]], "abcdef\n")
];
shortdesc = "list the contents of a file";
longdesc = "[...]" };
["defaults" is a struct which contains the defaults for every field,
allowing us to use the "{ defaults with ... }" syntax to just update
the fields we want to be different from the defaults.]
This is a mechanical change and there is no change to the output of
the generator. I checked the output before and after with diff to
verify this. There are no changes in the output apart from UUIDs
which are expected to change with each run.
Earlier versions of qemu contained a bug in the qcow2 code which
causes qemu to segfault when shutting down and flushing its internal
cache, and this can result in data loss.
The new API splits orderly close into a two-step process:
if (guestfs_shutdown (g) == -1) {
/* handle the error, eg. qemu error */
}
guestfs_close (g);
Note that the explicit shutdown step is only necessary in the case
where you have made changes to the disk image and want to handle write
errors. Read the documentation for further information.
This change also:
- deprecates guestfs_kill_subprocess
- turns guestfs_kill_subprocess into the same as guestfs_shutdown
- changes guestfish and other tools to call shutdown + close
where necessary (not for read-only tools)
- updates documentation
- updates examples
The order is now:
- remove the handle from the list of handles
- send close trace message
- sync and shutdown qemu
- run user close callback
- free temporary directory
- free memory
This commit ought to be no functional change.
On Linux, sync(2) does not actually issue a write barrier, thus it
doesn't force a flush of the underlying hardware write cache (or
qemu's disk cache in the virtual case).
This can be a problem, because libguestfs relies on running sync in
the appliance, followed by killing qemu (using SIGTERM).
In most cases, this is fine, because killing qemu with SIGTERM should
cause it to flush out the disk cache before it exits. However we have
found various bugs in qemu which cause qemu to crash while doing the
flush, leaving the data unwritten (see RHBZ#836913).
The solution is to issue fsync(2) to the block devices. This has a
write barrier, so it ensures that qemu writes out its cache long
before we get around to killing qemu.
Replace:
cp tests/guests/fedora.img test.img
with the longer but possibly more space-efficient equivalent:
qemu-img create -F raw -b tests/guests/fedora.img -f qcow2 test.qcow2
This returns the number of whole block devices added. It is usually
simpler to call this than to list the devices and count them, which
is what we do in some places in the current codebase.
On Debian, the Ruby C extensions library isn't '-lruby', it's
something like '-lruby1.8' or '-lruby-1.9.1' and these can even be
parallel-installed.
Fix detection so we use Ruby's own rbconfig.rb file to find the right
library to use.
MALLOC_PERTURB_ is a glibc feature which causes malloc to wipe memory
before and after it is used, allowing both use-after-free and
uninitialized reads to be detected with relatively little performance
penalty:
http://udrepper.livejournal.com/11429.html?nojs=1
Modify the ./run script so that it always sets this.
We were already using MALLOC_PERTURB_ in most tests. Since ./run is
now setting this, we can remove it from individual Makefiles. Most
TESTS_ENVIRONMENT will now simply look like this:
TESTS_ENVIRONMENT = $(top_builddir)/run --test
On RHEL 5, because _GNU_SOURCE was not defined, open_memstream was not
being declared, resulting in miscompilation and a segfault in the
trace code whenever open_memstream returned a pointer >= 0x80000000
(which would be truncated to a 32 bit int and then sign-extended).
When libguestfs calls 'error (g, "error")', gjs in F17 throws
error.message == "Error invoking Guestfs.test0rinterr: error"
In F18, error.message is simply the string "error".
Fix the test so it works for both cases.
This option, when added via
TESTS_ENVIRONMENT = [...] $(top_builddir)/run --test
allows us to run the tests and only print the full output (including
debugging etc) when the test fails.
The stdin and stdout of the qemu process are aliased to g->fd:
g->fd[0] = wfd[1];
g->fd[1] = rfd[0];
However if the child exits early, then child_cleanup closes g->fd[0],
g->fd[1], AND the code at the cleanup1 label closes wfd[1], rfd[0],
resulting in a double-close.
Avoid this case by setting wfd[1], rfd[0] to -1. In the cleanup1
label, only close wfd[1], rfd[0] if they are not -1, and add the same
for g->fd[0], g->fd[1].
Old KVM can't add /dev/null readonly. Treat /dev/null as a special
case.
We also fix a few tests where /dev/null was being used with
format=qcow2. This was always incorrect behaviour, but qemu appears
to tolerate it.
Old <libvirt.h> had a conflicting definition of virDomainGetXMLDesc:
fake_libvirt_xml.c:36: error: conflicting types for 'virDomainGetXMLDesc'
/usr/include/libvirt/libvirt.h:715: error: previous declaration of 'virDomainGetXMLDesc' was here
The difference is not material ('int' vs. 'unsigned int').
Avoid the error by not including <libvirt.h>.
The original fix for this in
commit 511c82df46 was not complete, in
that it did not fix the case of the old (pre '-m' option) parted.
This doesn't matter for Fedora, but it matters for RHEL 5 which has
this ancient parted.
If -lruby was not available, this used to define HAVE_LIBRUBY=0.
However this meant that the later test -n "$HAVE_LIBRUBY" would
be successful, whereas it should fail in this case.
RHEL 5-era autoconf did not define these, so define them manually
when they are missing.
Define builddir as '.' The scripts require this. It won't work
in the srcdir != builddir case, but we don't care about that for
RHEL 5.
This commit also moves the builddir / abs_srcdir variable setting
above the include of subdir-rules.mk, in case that include uses
these variables.
Useful script:
for f in $(find -name Makefile.am | xargs fgrep '$(abs_srcdir)' -l) ; do
if ! grep -q '^abs_srcdir' $f; then
echo missing in $f
fi
done
This returns the index of the device, eg. /dev/sdb => 1.
Or you can think of it as the order that the device was
added, or the index of the device in guestfs_list_devices.
Apparently e2fsprogs only knows that "/dev/sda" is a whole device, but
doesn't think that "/dev/vda" is. On switching the default device
over to virtio-scsi, that causes mke2fs without -F option to complain
and ask for an interactive prompt. Adding -F forces it to go ahead
anyway.
This caused several less-used APIs to break with virtio-scsi.
QEMU 1.0 was released at the end of 2011.
Remove all the cruft about detecting broken -machine type which
was only required for QEMU 0.15.
This also reverts commit 30ecbf3ec2.
Even on ARM you can pass -machine accel=kvm:tcg and qemu does the
right thing, so I'm not sure why we wanted to disable that.
These were used to select the default drive and network interface.
They both default to 'virtio'.
These were added back in the day when virtio was buggy, so that
packagers could revert to using ide/ne2k_pci to work around distro
bugs. However virtio has been stable in qemu for a very long time, so
it seems unlikely that any packager would need to use these, and in
any case it would be better to do this detection at runtime (cf. for
virtio-scsi).
The test used the external 'od' command to compare the output of
guestfish with what it's supposed to be. Unfortunately by default
this outputs groups of 2-byte words, with the words' endianness
affected by the current hardware endianness. For example:
x86-64$ echo -n ab | od
0000000 061141
0000002
ppc64$ echo -n ab | od
0000000 060542
0000002
By using 'od -b' instead we can output bytes instead of words, and
there is no endianness issue, and the output is clearer:
x86-64$ echo -n ab | od -b
0000000 141 142
0000002
ppc64$ echo -n ab | od -b
0000000 141 142
0000002
This is closer to the real meaning of "availability of btrfs", since
just having the btrfs tool doesn't help much if it's not supported by
the kernel too.
Since RHBZ was updated, the old python-bugzilla program broke. When
it was fixed, I found the (undocumented) ordering of the results from
the old tool was no longer true.
This commit adds a small Perl script to sort the output predictably.
Now bugs are sorted by both bug state and bug number, so the output
should be more stable than before.
Re-enable creation of the BUGS file in 'make dist'
(this reverts commit 91f3456244).
Also the BUGS file has been updated.
Previously the code would drop the first character in any sequence of
non-printing characters that occurred within a string.
Fix for commit e6f18c59d5.
Change guestfs___first_line_of_file so that if the file is empty this
returns an empty string instead of an error. This is consistent with
the behaviour of this function in the case where the file starts with
a \n character, where it would previously have returned an empty
string.
Change all callers so that they handle this case.
Then change the hostname parsing code so that it doesn't give up when
/etc/HOSTNAME is empty, but falls through to the next alternative, and
similarly for /etc/hostname.
Thanks Todd Mummert for finding and diagnosing this bug.
parse_release_file should be used to set fs->product_name, instead of
calling guestfs___first_line_of_file directory, although currently the
two are equivalent.
This is code motion and fixes
commit b648b1813f.
Missing package caused this error in compress-device-out:
libguestfs: error: compress_device_out: compression type bzip2 is not supported
Thanks Mohua Li.
Add the new API btrfs-fsck to check the btrfs filesystem.
Btrfs is currently under heavy development, and not suitable for
any uses other than benchmarking and review. But it'll be useful
in the near future.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
I used scsi_debug to create a 4k sector virtual disk:
modprobe scsi_debug dev_size_mb=128 sector_size=4096
I then used 'gdisk' to create lots of partitions, and used 'hexdump'
to examine what was written to disk.
* Move MOUNT_OPTIONS to #define at top.
* Move SIZE_MB to #define at top.
* Set PS1 in subshell (doesn't work - why?).
* Hide libguestfs errors from guestfs_mount_local_run.
This fixes commit 46b6766156.
Presently KVM is only applicable to x86 and x86-64 (although that will
change in future, and there are rumoured to be implementations for
some current non-x86 architectures). In any case having these options
breaks ARM, so disable them for non-x86 architectures at the moment.
For ARM Versatile ("versatilepb") architecture, memsize must be set to
256 (megabytes) else the appliance is unable to boot. Previously
setting LIBGUESTFS_MEMSIZE=256 was not allowed.
Versatile Express has larger limits.
'./configure --disable-probes' will disable these probes. Otherwise
they are autodetected as before.
The <sys/sdt.h> produces lots of errors when you try to compile with
these probes enabled under clang, so it is necessary to provide a way
to disable them for clang users.
This commit adds a tests/xml directory, and an LD_PRELOAD module which
can fake arbitrary libvirt XML from an external file (and is therefore
a much more flexible test than using the libvirt test:// driver
alone).
Also added is one regression test for:
https://bugzilla.redhat.com/show_bug.cgi?id=701814
Loading the given libvirt XML using Sys::Guestfs::Lib::open_guest
used to fail with the error:
format parameter is empty or contains disallowed characters at /home/rjones/d/libguestfs/perl/blib/lib/Sys/Guestfs/Lib.pm line 256.
Thanks to Tom Horsley for supplying the test data.
Old versions of libvirt allowed you to define disks like this:
<disk type='file' device='disk'>
<driver name='qemu'/>
...
Since the <driver> element does not have a 'type' attribute (which
defines the format), we are supposed to do autodetection, so the
format should be undefined.
However what actually happened was that the code in
Sys::Guestfs::Lib::open_guest received format as an empty string from
the xpath query, causing libguestfs to give an error.
If the xpath query returns the format as an empty string, undefine it.
This commit tidies up the code for the 'glob' command. It also makes
the command return an error if malloc fails (previously it would just
print a message but not return any error indication).
For a test case, see:
https://bugzilla.redhat.com/show_bug.cgi?id=803533
In guestfish, we use the gnulib closeout module which sets up an
atexit handler to detect if there were any errors on stdout when the
program exits. This is so we can fail correctly in out of disk space
cases like:
guestfish [...] > output
However the atexit handler just checks that there was any error on
stdout (using ferror). If a pipe command such as:
><fs> command_with_lots_of_output | head
ran at any time during the session, the error flag would be set
(because the underlying writes failed with EPIPE errors).
So the commit first adds a test for ferror (stdout) after each command
that we issue. This brings error handling closer to the point of
failure, and so is generally a good thing.
Secondly we reset the error flag after detecting and dealing with the
error, so that avoids the redundant 'guestfish: write error' message
produced by gnulib closeout.
Thirdly we add a --pipe-error flag which causes guestfish commands to
fail on pipe commands line the one above. The default is off for
backwards compatibility reasons.
Note that this support is optional: To enable it, install the
ocaml-gettext library from
http://forge.ocamlcore.org/projects/ocaml-gettext . If this library
is not installed, then configure detects this and inserts dummy
gettext functions that do nothing.
gettextize provides a local file called "gettext.h". Remove this and
use <libintl.h> from glibc headers instead.
Most of this change is mechanical: #include <libintl.h> in every C
file which uses any gettext function. But also we remove the
gettext.h file, and adjust the "_" macros.
Note that this effectively removes the ./configure --disable-nls
option, although we don't know if that ever worked.
Remove the logfiles configured by /etc/logrotate.d/*.
Omit the logfile of "samba" and "sssd" which we removed
them separately .
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
Originally this state was intended so that in some way you could find
out if the appliance was running a command. However there was never a
thread-safe way to access the state of the handle, so in effect you
could never do anything useful safely with this information.
This commit completely removes the BUSY state.
The only visible change is to the guestfs_is_busy API. Previously you
could never call this safely from another thread. If you called it
from the same thread it would always return false (since the current
thread can't be running a libguestfs command at that point by
definition). Now it always returns false.
The gobject bindings generate a large number of header files, which pollute
/usr/include when installed. This patch moves them all into a guestfs-gobject/
subdirectory. guestfs-gobject.h remains in the same place.
This change also moves generated source files into src/, because it makes the
gobject directory a bit tidier.
Implement libguestfs events as GObject signals. Callback arguments are passed in
a boxed object.
Note that this patch fixes the length of the uint64_t array in the callback
arguments at 16, whereas it is actually arbitrary length. This is to make it
introspectable. There is currently no way to pass an arbitrary length array to a
callback, and have its type introspected.
virt-rescue prints errors such as:
rm: cannot remove `/proc': Is a directory
mkdir: cannot create directory `/proc': File exists
rm: cannot remove `/sys': Is a directory
mkdir: cannot create directory `/sys': File exists
People have reported these errors (which are nothing to worry about)
as bugs in the past, so avoid them where possible.
The two tarballs:
8252 bytes tests/data/filesanddirs-10M.tar.xz
41152 bytes tests/data/filesanddirs-100M.tar.xz
contain, as their names suggest, highly compressed files and
directories, with unpacked size respectively around 10M and 100M (the
precise size after unpacking depends on the filesystem).
These can be used for testing purposes.
Bind the easy parts of the 'btrfs' program.
The new APIs are:
btrfs-device-add: add devices to a btrfs filesystem
btrfs-device-delete: remove devices from a btrfs filesystem
btrfs-filesystem-sync: sync a btrfs filesystem
btrfs-filesystem-balance: balance a btrfs filesystem
btrfs-subvolume-create: create a btrfs snapshot
btrfs-subvolume-delete: delete a btrfs snapshot
btrfs-subvolume-list: list btrfs snapshots and subvolumes
btrfs-subvolume-set-default: set default btrfs subvolume
btrfs-subvolume-snapshot: create a writable btrfs snapshot
Flag the system for reconfiguration by touching /.unconfigured .
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Updated so this is disabled by default, since it requires
user intervention at next boot.
Remove user accounts from /etc/passwd, /etc/group,
/etc/shadow, and the home directory of the user,
except the root user.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
RWMJ:
- Updated to use Augeas to delete accounts.
- Disable this by default, since it is very invasive.
All current operations are enabled by default.
Also:
* The POD description is split into a heading and the description body.
* An 'enabled_by_default' flag is added to the structure.
'del g' is a trap for the unwary. If the handle has any other
references, it does nothing (in fact, it can be actively dangerous if
the user was expecting the appliance to go away). In non-CPython it
can be delayed arbitrarily long.
Using 'g.close()' on the other hand is always safe.
Colon characters are not allowed in qemu/qemu-img filenames. There's
no way to quote them.
Comma characters CAN be used. However they must be quoted (by
doubling) when used in the '-o' option.
Fix general quoting problems in the external command.
The output looks like this:
F16x64:/dev/sda1 1048576 1024K ok
F16x64:/dev/sda2 2097152 2048K ok
F16x64:/dev/sda3 526385152 2048K ok
If the --uuid option is used, then UUIDs are shown instead of names.
The new APIs are:
get-e2attrs: List ext2 file attributes of a file.
set-e2attrs: Set or clear ext2 file attributes of a file.
get-e2generation: Get ext2 file generation of a file.
set-e2generation: Set ext2 file generation of a file.
These are implemented using the lsattr and chattr programs from
e2fsprogs.
Remove the .ssh directory of user "root" and any other
users who have a .ssh directory in their home directory.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
Remove the bogus check_path function and move the functionality into
the two places where it was being used.
qemu -cdrom ,
works fine, I tested it.
Colon cannot be used in a block device filename anywhere, since the
qemu block driver interprets it as a prefix. There is no known way to
work around this problem. I checked this is true with kwolf.
Comma is fine in -drive options, provided it is escaped by doubling it.
On the new faster computer, Thread.yield wasn't yielding, so the
second thread would block the main test from proceeding (only when run
under valgrind however).
The first time the directory is checked out from git, there are no
*.pod files. In a parallel build, the first thread notices this and
runs the 'make -C .. update-po' rule, but the other threads also try
to run at the same time and fail.
This means renaming various temporary files (eg. fstab.tmp) that
several of the scripts were using so they don't conflict
(eg. fstab.tmp.$$).
Also the rule:
fedora-md1.img fedora-md2.img: guest-aux/make-fedora-img.pl
causes the script to run twice in parallel when using 'make -jN'.
Replace this with a stamp file so it runs once:
fedora-md1.img fedora-md2.img: stamp-fedora-md.img
stamp-fedora-md.img: guest-aux/make-fedora-img.pl
Add FUSE support directly to the API. Instead of needing to use the
external 'guestmount' command, you can mount the libguestfs filesystem
space on a local mountpoint using an API call from any language.
Note that although mount-local-run is marked as Cancellable, the
current implementation does not support it, but it would be relatively
simple to add it.
guestfs-gobject.h (in the source directory) was not found.
In file included from <stdin>:4:0:
/path/to/libguestfs/gobject/guestfs-gobject.h:23:37: fatal error: guestfs-gobject-session.h: No such file or directory
compilation terminated.
Error while processing the source.
make[2]: *** [Guestfs-1.0.gir] Error 1
This greatly improves the usability of the generated gtk-doc.
Although there is a lot of churn in generator_gobject.ml, this is almost
exclusively code motion.
javah from old GNU classpath won't overwrite the target *.h file,
instead leaving the old one which results in a predictable build
failure. Delete the target so this won't happen.
virt_df-domains.o: In function `add_domain':
/home/feeliwood/Downloads/libguestfs-1.17.21/df/domains.c:274: undefined reference to `guestfs___for_each_disk'
guestfs___for_each_disk is only available when libvirt _and_ libxml2
dependencies are available at compile time.
Thanks Tho Huynh.
This function allows you to pass an explicit errno back to the
library. reply_with_error is redefined as a macro that calls
reply_with_error_errno with errno == 0.
Previously there was no programmatic way to tell if inspection APIs
were unavailable because they are not compiled in (because hivex isn't
around). This contrasts with daemon APIs where the availability is
covered by the guestfs_available API.
Change the inspection APIs so that when they are not available, the
last errno is set to ENOTSUP. (Note that ENOTSUP must be defined on
all POSIX platforms).
This allows programs to detect if they are using a version of
libguestfs that was compiled without support for inspection, without
having to parse error messages.
We were not using an unsigned type in the XDR. This doesn't
affect data integrity or the protocol, but it makes it more
complicated for the daemon to set these fields.
Add an API for doing what virt-sparsify was doing: freeing up free
space in a filesystem.
The current implementation is simple-minded: we create a file, fill it
with zeroes until we run out of space, then delete the file. However
the description leaves it open to do a better implementation, eg.
using sparsification support that is currently being worked on in ext4
and qemu.
The implementation also sends progress notifications, which is an
advantage over the old 'dd' method.
When a Windows guest doesn't have a HKLM\SYSTEM\MountedDevices node,
inspection fails. However inspection should not completely fail just
because we cannot get the drive letter mapping from a guest.
The presumption is that all file descriptors should be created with
the close-on-exec flag set. The only exception are file descriptors
that we want passed through to exec'd subprocesses (mainly pipes and
stdin/stdout/stderr).
For open calls, we pass O_CLOEXEC as an extra flag, eg:
fd = open ("foo", O_RDONLY|O_CLOEXEC);
This is a Linux-ism, but using a macro we can easily make it portable.
For sockets, similarly:
sock = socket (..., SOCK_STREAM|SOCK_CLOEXEC, ...);
For accepted sockets, we use the Linux accept4 system call which
allows flags to be supplied, but we use the Gnulib 'accept4' module to
make this portable.
For dup, dup2, we use the Linux dup3 system call, and the Gnulib
modules 'dup3' and 'cloexec'.
Previously a lot of daemon code used three variables (a string list,
'int size' and 'int alloc') to track growable strings buffers. This
commit implements a simple struct containing the same variables, but
using size_t instead of int:
struct stringsbuf {
char **argv;
size_t size;
size_t alloc;
};
Use it like this:
DECLARE_STRINGSBUF (ret);
//...
if (add_string (&ret, str) == -1)
return NULL;
//...
if (end_stringsbuf (&ret) == -1)
return NULL;
return ret.argv;
In two places, we were counting things in an array using an 'int'. In
theory, the int could overflow, so gcc determines this to be undefined
behaviour.
The fix is to use size_t or ssize_t instead.
When these tests were originally in the old regressions/ directory,
they used to refer to guestfish via the path '../fish/guestfish'.
Some of the tests were also called 'test-guestfish-*'.
Now that the tests have been moved into the fish/ directory, neither
of these things make sense. So change the relative path to
'./guestfish' and rename all 'test-guestfish-*' as 'test-*'.
If the parent process uses a pipe (or any fd, but pipes are a
particular problem), then the recovery process would hold open the
file descriptor(s) of the pipe, meaning that it could not be fully
closed in the parent. Because the recovery process doesn't use
exec(2), this wasn't avoidable even using FD_CLOEXEC.
Avoid this by closing all file descriptors when starting the recovery
process.
After discussion with Dan Berrange, he points out that it's also a
good idea to set signal handlers to the default after forking, so that
any signal handlers set up in the parent don't affect the child.
Error: UNINIT:
/builddir/build/BUILD/libguestfs-1.16.5/daemon/glob.c:32: var_decl: Declaring variable "buf" without initializer.
/builddir/build/BUILD/libguestfs-1.16.5/daemon/glob.c:36: uninit_use_in_call: Using uninitialized value "buf.gl_offs" when calling "rpl_glob".
/builddir/build/BUILD/libguestfs-1.16.5/gnulib/lib/glob.c:557: read_parm_fld: Reading a parameter field.
/builddir/build/BUILD/libguestfs-1.16.5/daemon/glob.c:36: uninit_use_in_call: Using uninitialized value "buf.gl_pathc" when calling "rpl_glob".
/builddir/build/BUILD/libguestfs-1.16.5/gnulib/lib/glob.c:557: read_parm_fld: Reading a parameter field.
The Coverity error is this (which I think is wrong):
Error: TAINTED_SCALAR:
/builddir/build/BUILD/libguestfs-1.16.5/daemon/inotify.c:211: tainted_data_argument: Calling function "read" taints argument "inotify_buf".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/inotify.c:232: var_assign_var: Assigning: "event" = "(struct inotify_event *)&inotify_buf[n]". Both are now tainted.
/builddir/build/BUILD/libguestfs-1.16.5/daemon/inotify.c:258: lower_bounds: Checking lower bounds of unsigned scalar "event->len" by "event->len > 0U".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/inotify.c:272: var_assign_var: Compound assignment involving tainted variable "16UL + event->len" to variable "n" taints "n".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/inotify.c:228: lower_bounds: Checking lower bounds of unsigned scalar "n" by "n < inotify_posn".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/inotify.c:281: tainted_data: Using tainted variable "n" as an index into an array "inotify_buf".
Adding a sanity check of event->len is prudent.
Error: RESOURCE_LEAK:
/builddir/build/BUILD/libguestfs-1.16.5/daemon/debug.c:469: open_fn: Calling opening function "open".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/debug.c:469: var_assign: Assigning: "fd" = handle returned from "open("/proc/sys/kernel/core_pattern", 1)".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/debug.c:474: noescape: Variable "fd" is not closed or saved in function "write".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/debug.c:476: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
This also includes some tidying up of the generated code.
Error: RESOURCE_LEAK:
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13254: alloc_fn: Calling allocation function "parse_string_list".
/builddir/build/BUILD/libguestfs-1.16.5/fish/fish.c:1386: alloc_fn: Storage is returned from allocation function "realloc".
/builddir/build/BUILD/libguestfs-1.16.5/fish/fish.c:1386: var_assign: Assigning: "argv_new" = "realloc(argv, 8UL * argv_len)".
/builddir/build/BUILD/libguestfs-1.16.5/fish/fish.c:1392: var_assign: Assigning: "argv" = "argv_new".
/builddir/build/BUILD/libguestfs-1.16.5/fish/fish.c:1396: return_alloc: Returning allocated memory "argv".
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13254: var_assign: Assigning: "devices" = storage returned from "parse_string_list(argv[i++])".
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13271: leaked_storage: Variable "devices" going out of scope leaks the storage it points to.
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13288: leaked_storage: Variable "devices" going out of scope leaks the storage it points to.
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13293: leaked_storage: Variable "devices" going out of scope leaks the storage it points to.
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13311: leaked_storage: Variable "devices" going out of scope leaks the storage it points to.
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13316: leaked_storage: Variable "devices" going out of scope leaks the storage it points to.
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13334: leaked_storage: Variable "devices" going out of scope leaks the storage it points to.
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13349: leaked_storage: Variable "devices" going out of scope leaks the storage it points to.
/builddir/build/BUILD/libguestfs-1.16.5/fish/cmds.c:13355: leaked_storage: Variable "devices" going out of scope leaks the storage it points to.
Error: RESOURCE_LEAK:
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:894: open_fn: Calling opening function "open".
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:894: var_assign: Assigning: "fd" = handle returned from "open(filename, 0)".
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:903: noescape: Variable "fd" is not closed or saved in function "read".
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:911: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:918: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
Error: RESOURCE_LEAK:
/builddir/build/BUILD/libguestfs-1.16.5/daemon/upload.c:225: open_fn: Calling opening function "open".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/upload.c:225: var_assign: Assigning: "fd" = handle returned from "open(filename, 0)".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/upload.c:233: noescape: Variable "fd" is not closed or saved in function "lseek".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/upload.c:235: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
Error: RESOURCE_LEAK:
/builddir/build/BUILD/libguestfs-1.16.5/daemon/9p.c:130: open_fn: Calling opening function "open".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/9p.c:130: var_assign: Assigning: "fd" = handle returned from "open(filename, 0)".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/9p.c:142: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
/builddir/build/BUILD/libguestfs-1.16.5/daemon/9p.c:149: noescape: Variable "fd" is not closed or saved in function "read".
/builddir/build/BUILD/libguestfs-1.16.5/daemon/9p.c:153: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
Error: RESOURCE_LEAK:
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:1125: open_fn: Calling opening function "open".
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:1125: var_assign: Assigning: "fd" = handle returned from "open(filename, 833, 438)".
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:1133: noescape: Variable "fd" is not closed or saved in function "xwrite".
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:1146: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
/builddir/build/BUILD/libguestfs-1.16.5/src/proto.c:1173: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
Error: NULL_RETURNS:
/builddir/build/BUILD/libguestfs-1.16.5/examples/virt-dhcp-address.c:129: var_assigned: Assigning: "guest_distro" = null return value from "guestfs_inspect_get_distro".
/builddir/build/BUILD/libguestfs-1.16.5/examples/virt-dhcp-address.c:131: dereference: Dereferencing a pointer that might be null "guest_distro" when calling "__coverity_strcmp".
[...]
/builddir/build/BUILD/libguestfs-1.16.5/examples/virt-dhcp-address.c:126: var_assigned: Assigning: "guest_type" = null return value from "guestfs_inspect_get_type".
/builddir/build/BUILD/libguestfs-1.16.5/examples/virt-dhcp-address.c:128: dereference: Dereferencing a pointer that might be null "guest_type" when calling "__coverity_strcmp".
Error: DEADCODE:
/builddir/build/BUILD/libguestfs-1.16.5/fish/event-names.c:65: dead_error_condition: On this path, the condition "comma" cannot be true.
/builddir/build/BUILD/libguestfs-1.16.5/fish/event-names.c:57: const: After this line, the value of "comma" is equal to 0.
/builddir/build/BUILD/libguestfs-1.16.5/fish/event-names.c:57: assignment: Assigning: "comma" = "0".
/builddir/build/BUILD/libguestfs-1.16.5/fish/event-names.c:65: dead_error_line: Execution cannot reach this statement "fputc(44, fp);".
Remove dead code; however only comment it out since if we change the
preceeding code we may need this line again.
Error: DEADCODE:
/builddir/build/BUILD/libguestfs-1.16.5/src/listfs.c:107: dead_error_condition: On this path, the condition "lvs" cannot be true.
/builddir/build/BUILD/libguestfs-1.16.5/src/listfs.c:54: const: After this line, the value of "lvs" is equal to 0.
/builddir/build/BUILD/libguestfs-1.16.5/src/listfs.c:91: const: After this line, the value of "lvs" is equal to 0.
/builddir/build/BUILD/libguestfs-1.16.5/src/listfs.c:54: assignment: Assigning: "lvs" = "NULL".
/builddir/build/BUILD/libguestfs-1.16.5/src/listfs.c:91: new_values: Noticing condition "lvs == NULL".
/builddir/build/BUILD/libguestfs-1.16.5/src/listfs.c:107: dead_error_line: Execution cannot reach this statement "guestfs___free_string_list(...".
Error: CHECKED_RETURN:
/builddir/build/BUILD/libguestfs-1.16.5/java/com_redhat_et_libguestfs_GuestFS.c:9552: example_assign: Assigning: "r" = return value from "guestfs_internal_autosync(g)".
/builddir/build/BUILD/libguestfs-1.16.5/java/com_redhat_et_libguestfs_GuestFS.c:9555: example_checked: "r" has its value checked in "r == -1".
/builddir/build/BUILD/libguestfs-1.16.5/ocaml/guestfs_c_actions.c:5584: example_assign: Assigning: "r" = return value from "guestfs_internal_autosync(g)".
/builddir/build/BUILD/libguestfs-1.16.5/ocaml/guestfs_c_actions.c:5586: example_checked: "r" has its value checked in "r == -1".
/builddir/build/BUILD/libguestfs-1.16.5/perl/Guestfs.xs:5990: example_assign: Assigning: "r" = return value from "guestfs_internal_autosync(g)".
/builddir/build/BUILD/libguestfs-1.16.5/perl/Guestfs.xs:5991: example_checked: "r" has its value checked in "r == -1".
/builddir/build/BUILD/libguestfs-1.16.5/python/guestfs-py.c:13702: example_assign: Assigning: "r" = return value from "guestfs_internal_autosync(g)".
/builddir/build/BUILD/libguestfs-1.16.5/python/guestfs-py.c:13707: example_checked: "r" has its value checked in "r == -1".
/builddir/build/BUILD/libguestfs-1.16.5/ruby/ext/guestfs/_guestfs.c:16000: example_assign: Assigning: "r" = return value from "guestfs_internal_autosync(g)".
/builddir/build/BUILD/libguestfs-1.16.5/ruby/ext/guestfs/_guestfs.c:16001: example_checked: "r" has its value checked in "r == -1".
/builddir/build/BUILD/libguestfs-1.16.5/src/guestfs.c:191: check_return: Calling function "guestfs_internal_autosync" without checking return value (as is done elsewhere 5 out of 6 times).
/builddir/build/BUILD/libguestfs-1.16.5/src/guestfs.c:191: unchecked_value: No check of the return value of "guestfs_internal_autosync(g)".
Netpbm and icoutils (wrestool) have always been dependencies. Since
they are not always present, make these into optional dependencies
(which they were, sort of, before).
Also document these dependencies in the README file.
guestfs_set_autosync sets a flag which affects guestfs_close, and so
this call can be made at any time before the handle is closed, not
just in the config state.
Function guestfs_mkdtemp uses c++ keyword "template" as a parameter
name. In result, attempt to use guestfs.h header in c++ program
results in compile error.
This is just the 'kernel', 'initrd' and 'root' files, copied from one
machine to another, along with a 'README.fixed' file which is also
used for identification.
This allows the appliance to be copied from one machine to another,
making it easier for us to distribute a starter appliance for people
who cannot get febootstrap or appliance-building working.
--20cf303ea4a84a7a7c04ba331375
Content-Type: text/plain; charset=ISO-8859-1
Hello Richard,
I found what the problem was and the udev symlinks where not created. The
init script in the appliance uses a full paths for udevadm (/sbin/udevadm)
which in my case was wrong. In Arch Linux udevadm is hosted under /usr/bin,
not /sbin. When I fixed this, libguestfs was able to communicate with the
appliance VM.
Since PATH variable is defined and exported in init, there is no need in
using full paths for external programs. As far as I've seen this affects
all the git branches in libguestfs's repository. Please find attached a
patch for the master branch.
Nikos Skalkotos,
Athens, Greece
On 24 February 2012 10:25, Richard W.M. Jones <rjones@redhat.com> wrote:
> On Thu, Feb 23, 2012 at 08:50:12PM +0200, Nikos Skalkotos wrote:
> [...]
>
> I don't know specifically why it fails with ArchLinux, but the problem
> is caused by the /dev/virtio-ports/* symlinks not getting created by
> udev.
>
> /dev/vport0p1 exists:
>
> > crw------- 1 root root 252, 1 Feb 23 18:17 vport0p1
>
> but udev doesn't make the corresponding /dev/virtio-ports symlink:
>
> > /dev/virtio-ports/org.libguestfs.channel.0: No such file or directory
>
> The symlink is supposed to be created by this udev rule:
>
> /lib/udev/rules.d/50-udev-default.rules:KERNEL=="vport*",
> ATTR{name}=="?*", SYMLINK+="virtio-ports/$attr{name}"
>
> So I'd start by looking to see if that rule exists in the udev rules
> that Arch is using. Secondly if it does exist, is the corresponding
> *.rules being copied into the appliance? (Check appliance/supermin.d/
> hostfiles)
>
> In an old Ubuntu that has udev that predates having this rule, we add
> the following patch:
>
>
> http://libguestfs.org/download/binaries/ubuntu1004-packages/0002-ubuntu-10.04-Use-dev-vport0p1.patch
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> virt-p2v converts physical machines to virtual machines. Boot with a
> live CD or over the network (PXE) and turn machines into Xen guests.
> http://et.redhat.com/~rjones/virt-p2v
>
Hello Richard,<br><br>I found what the problem was and the udev symlinks where not created. The init script in the appliance uses a full paths for udevadm (/sbin/udevadm) which in my case was wrong. In Arch Linux udevadm is hosted under /usr/bin, not /sbin. When I fixed this, libguestfs was able to communicate with the appliance VM.<br>
<br>Since PATH variable is defined and exported in init, there is no need in using full paths for external programs. As far as I've seen this affects all the git branches in libguestfs's repository. Please find attached a patch for the master branch.<br>
<br>Nikos Skalkotos,<br>Athens, Greece<br><br><div class="gmail_quote">On 24 February 2012 10:25, Richard W.M. Jones <span dir="ltr"><<a href="mailto:rjones@redhat.com">rjones@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Thu, Feb 23, 2012 at 08:50:12PM +0200, Nikos Skalkotos wrote:<br>
[...]<br>
<br>
I don't know specifically why it fails with ArchLinux, but the problem<br>
is caused by the /dev/virtio-ports/* symlinks not getting created by<br>
udev.<br>
<br>
/dev/vport0p1 exists:<br>
<div class="im"><br>
> crw------- 1 root root 252, 1 Feb 23 18:17 vport0p1<br>
<br>
</div>but udev doesn't make the corresponding /dev/virtio-ports symlink:<br>
<div class="im"><br>
> /dev/virtio-ports/org.libguestfs.channel.0: No such file or directory<br>
<br>
</div>The symlink is supposed to be created by this udev rule:<br>
<br>
/lib/udev/rules.d/50-udev-default.rules:KERNEL=="vport*", ATTR{name}=="?*", SYMLINK+="virtio-ports/$attr{name}"<br>
<br>
So I'd start by looking to see if that rule exists in the udev rules<br>
that Arch is using. Secondly if it does exist, is the corresponding<br>
*.rules being copied into the appliance? (Check appliance/supermin.d/<br>
hostfiles)<br>
<br>
In an old Ubuntu that has udev that predates having this rule, we add<br>
the following patch:<br>
<br>
<a href="http://libguestfs.org/download/binaries/ubuntu1004-packages/0002-ubuntu-10.04-Use-dev-vport0p1.patch" target="_blank">http://libguestfs.org/download/binaries/ubuntu1004-packages/0002-ubuntu-10.04-Use-dev-vport0p1.patch</a><br>
<br>
Rich.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Richard Jones, Virtualization Group, Red Hat <a href="http://people.redhat.com/%7Erjones%0Avirt-p2v" target="_blank">http://people.redhat.com/~rjones<br>
virt-p2v</a> converts physical machines to virtual machines. Boot with a<br>
live CD or over the network (PXE) and turn machines into Xen guests.<br>
<a href="http://et.redhat.com/%7Erjones/virt-p2v" target="_blank">http://et.redhat.com/~rjones/virt-p2v</a><br>
</font></span></blockquote></div><br>
Add a flag "ConfigOnly" to make sure that some non-daemon functions
should be called only at CONFIG state (RHBZ#796520).
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
Appliance building can be called from multiple processes, but this is
only safe if each process holds a lock on the 'checksum' file.
However threads within a process are not excluded by a file lock, and
so this strategy completely failed for a multithreaded program calling
guestfs_launch in parallel.
Since it makes no sense for threads in a single program to race each
other to try to create the appliance, add a lock around appliance
building.
This serialises building the appliance, but the rest of guestfs_launch
(eg. starting up qemu) can run in parallel.
rhbz727178.sh exists, but historically was never added to the list of
tests, so it doesn't run. Create a special variable for this test and
ensure it is added to EXTRA_DIST.
Test was originally added in
commit d7356a2801.
In the case where the caller attempts to mount the "hidden"
appliance root device (eg. /dev/vdb if /dev/vda is the only
normal block device added), we were calling reply_with_error
but not actually returning immediately, resulting in protocol
desynchronization.
This commit fixes this obvious mistake.
For stubs of functions that had arguments, code did this:
static void
mount_stub (XDR *xdr_in)
{
int r;
struct guestfs_mount_args args;
if (optargs_bitmask != 0) {
//...
goto done;
}
// possibly other tests here
memset (&args, 0, sizeof args);
[...]
done:
xdr_free ((xdrproc_t) xdr_guestfs_mount_args, (char *) &args);
return;
}
This caused xdr_free to be called on uninitialized 'args' struct,
causing a segfault.
The fix is to add another label, so the code looks like:
static void
mount_stub (XDR *xdr_in)
{
int r;
struct guestfs_mount_args args;
if (optargs_bitmask != 0) {
//...
goto done_no_free;
}
// possibly other tests here
memset (&args, 0, sizeof args);
[...]
done:
xdr_free ((xdrproc_t) xdr_guestfs_mount_args, (char *) &args);
done_no_free:
return;
}
This fixes commit 330fbea5b2
and commit 0344248af5.
erase the filesystem signatures on each device, then erase
the partitions, avoid to list all the partitions here.
Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com>
Check /sbin, /usr/sbin, /bin and /usr/bin to ensure that neither the
directory nor the parent (eg. /usr) is a symlink, and use this
directory for the virtual directory used to store guestfsd in the
appliance.
This file shouldn't be included in the tarball as it contains
hard-coded paths from the maintainer's machine which cannot be
relevant to the target machine.
Callers are supposed to use the availability API to check for
functions that may not be available in particular builds of
libguestfs. If they don't do this, currently they tend to get obscure
error messages, eg:
libguestfs: error: zerofree: /dev/vda1: zerofree: No such file or directory
This commit changes the error message to explain what callers ought to
be doing instead:
libguestfs: error: zerofree: feature 'zerofree' is not available in this
build of libguestfs. Read 'AVAILABILITY' in the guestfs(3) man page for
how to check for the availability of features.
This patch makes the stubs check for availability. The stub code
changes to:
static void
zerofree_stub (XDR *xdr_in)
{
[...]
/* The caller should have checked before calling this. */
if (! optgroup_zerofree_available ()) {
reply_with_error ("feature '%s' is not available in this\n"
"build of libguestfs. Read 'AVAILABILITY' in the guestfs(3) man page for\n"
"how to check for the availability of features.",
"zerofree");
goto done;
}
[...]
Set LD_LIBRARY_PATH so we link against the just-built library, not
the installed library.
Use OCAMLCFLAGS instead of OCAMLOPTFLAGS where appropriate.
Remove unnecessary -cclib option for bytecode linking (gcc is not used
in this case).
This fixes commit eb68a31413.
The error messages would be prefixed by the path to the
alternate binary, eg:
../fish/guestfish: invalid escape sequence in string (starting at offset 0)
Fix this by normalizing the messages further.
RConstOptString cannot return an error in the C api. This makes it a special
case for the GObject api, as all other return types have a corresponding GError
**err argument to return an error. This change removes this special case, and
includes the possibility of an error return in the API. An error is indicated by
setting *err to a non-NULL value.
This change is in preparation for adding a close api. An attempt to call any
api, even RConstOptString, on a closed handle must return an error.
Change 'ifconfig' and 'netstat' commands to use 'ip' instead.
'iproute' was already included in the appliance, so this reduces the
size of the appliance accordingly.
This commit tidies up the code by splitting out the path
element-searching code into a separate function.
Valgrind found that 'closedir' frees the 'struct dirent *', which
wasn't immediately obvious. So now we do the 'closedir' after all
operations which touch 'd->d_name'.
This commit allows you to run the daemon under valgrind. You have to
enable it at configure time:
./configure --enable-valgrind-daemon
This should *not* be done for production builds.
When this feature is enabled, valgrind is added to the appliance and
the daemon is run under valgrind. Log messages from valgrind are
passed back over a virtio-serial channel into a file called
'valgrind.log.$PID' in the top build directory.
Running 'make check', 'make extra-tests' etc causes many
valgrind.log.* files to be created which must be examined by hand.
elseifi=len1||j=len2thenfalse(* no match - different lengths *)
else(
letx1=getxdigitfp1.[i]andx2=getxdigitfp2.[j]in
matchx1,x2with
|Somex1,Somex2whenx1=x2->loop(i+1)(j+1)
|Somex1,Somex2->false(* no match - different content *)
|Some_,None->loopi(j+1)
|None,Some_->loop(i+1)j
|None,None->loop(i+1)(j+1)
)
in
loop00
andgetxdigit=function
|'0'..'9'asc->Some(Char.codec-Char.code'0')
|'a'..'f'asc->Some(Char.codec-Char.code'a')
|'A'..'F'asc->Some(Char.codec-Char.code'A')
|_->None
letrecverifytfilename=
ift.check_signaturethen(
letargs=quotefilenamein
do_verifytargs
)
andverify_detachedtfilenamesigfile=
ift.check_signaturethen(
matchsigfilewith
|None->
eprintf(f_"virt-builder: error: there is no detached signature file\nThis probably means the index file is missing a sig=... line.\nYou can use --no-check-signature to ignore this error, but that means\nyou are susceptible to man-in-the-middle attacks.\n");
eprintf(f_"virt-builder: error: GPG failure: could not verify digital signature of file\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!\n");
eprintf(f_"virt-builder: error: fingerprint of signature does not match the expected fingerprint!\n found fingerprint: %s\n expected fingerprint: %s\n")
eprintf(f_"virt-builder: error: could not run sha512sum command to verify checksum\n");
exit1
);
letcsum_actual=read_whole_filecsum_filein
letcsum_actual=
letlen=String.lengthcsum_actualin
iflen>0&&csum_actual.[len-1]='\n'then
String.subcsum_actual0(len-1)
else
csum_actualin
ifcsum<>csum_actualthen(
eprintf(f_"virt-builder: error: checksum of template did not match the expected checksum!\n found checksum: %s\n expected checksum: %s\nTry:\n - Use the '-v' option and look for earlier error messages.\n - Delete the cache: virt-builder --delete-cache\n - Check no one has tampered with the website or your network!\n")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
