If the guest uses SELinux, then make sure to run a relabel (or at least
schedule one) after the image build: this way the template is
relabelled, or at least it will do that at the next boot, without the
need for the user to ask for a relabel.
This just covers the case of building a new image with no additional
operations on it though.
Since we started to use the parallel tests framework in automake,
'make check-slow' has been broken. This is because parallel tests
doesn't allow you to run 'make check TESTS=...' with a set of test
scripts which do not also appear in the static list of tests in the
Makefile.am. We would like to list and run only "fast" tests in the
Makefile.am, and have other scripts for slow tests.
The solution is to add the slow tests to Makefile.am, but condition
those tests on an environment variable SLOW=1 being set.
This commit fixes all the existing slow tests in this way, and updates
the documentation (guestfs-hacking(1)) to document how slow tests
should be written in future.
On Linux this will load the whole file into the page cache. However
the output file is empty and zero sized just after it is opened, so
this has no effect. Note that the advice is not persistent, so this
really does nothing.
I considered adding the call back after the file has been written,
just before the close, but:
- If we do a virt-resize next then we will open and read the file mostly
sequentially, so readahead will deal with any missing pages.
- If we do a virt-customize next then we will only access a small
part of the disk image, so loading it all into the page cache adds
extra work.
- In any case, since we have just written the file it's likely to
still be in the page cache.
Setting POSIX_FADV_RANDOM makes no measurable difference, but at least
it's the right thing to do.
POSIX_FADV_WILLNEED makes no measurable difference either.
Moving the calls to posix_fadvise to just after the open() makes no
measurable difference, but does make the code a bit clearer.
Changing POSIX_FADV_NOREUSE on the input file descriptor to
POSIX_FADV_DONTNEED slows things down by about 10%.
Add wrappers around posix_fadvise and use them in places we were
calling posix_fadvise directly before.
Also in virt-builder we were doing this (and ignoring the result):
posix_fadvise (fd, 0, 0, POSIX_FADV_RANDOM|POSIX_FADV_DONTNEED);
However the POSIX_FADV_* flags are _not_ bitmasks! In fact
POSIX_FADV_RANDOM|POSIX_FADV_DONTNEED == POSIX_FADV_NOREUSE so we were
giving a completely different hint from what we thought we were
giving.
Like with the previous commit, this replaces instances of:
if (something_bad) {
fprintf (stderr, "%s: error message\n", guestfs_int_program_name);
exit (EXIT_FAILURE);
}
with:
if (something_bad)
error (EXIT_FAILURE, 0, "error message");
(except in a few cases were errno was incorrectly being ignored, in
which case I have fixed that).
It's slightly more complex than the previous commit because we must be
careful to:
- Remove the program name (since error(3) prints it).
- Remove any trailing \n character from the message.
Candidates for replacement were found using:
pcregrep --buffer-size 10M -M '\bfprintf\b.*\n.*\bexit\b' `git ls-files`
Wherever we had code which did:
if (something_bad) {
perror (...);
exit (EXIT_FAILURE);
}
replace this with use of the error(3) function:
if (something_bad)
error (EXIT_FAILURE, errno, ...);
The error(3) function is supplied by glibc, or by gnulib on platforms
which don't have it, and is much more flexible than perror(3). Since
we already use error(3), there seems to be no downside to mandating it
everywhere.
Note there is one nasty catch with error(3): error (EXIT_SUCCESS, ...)
does *not* exit! This is also the reason why error(3) cannot be
marked as __attribute__((noreturn)).
Because the examples can't use gnulib, I did not change them.
To search for multiline patterns of the above form, pcregrep -M turns
out to be very useful:
pcregrep --buffer-size 10M -M '\bperror\b.*\n.*\bexit\b' `git ls-files`
Update the system at the end of the installation, so the generated image
is up-to-date. This also removes the need to manually update dnf on
i686 and x86_64.
GCC has two warnings related to large stack frames. We were already
using the -Wframe-larger-than warning, but this reduces the threshold
from 10000 to 5000 bytes.
However that warning only covers the static part of frames (not
alloca). So this change also enables -Wstack-usage=10000 which covers
both the static and dynamic usage (alloca and variable length arrays).
Multiple changes are made throughout the code to reduce frames to fit
within these new limits.
Note that stack allocation of large strings can be a security issue.
For example, we had code like:
size_t len = strlen (fs->windows_systemroot) + 64;
char software[len];
snprintf (software, len, "%s/system32/config/software",
fs->windows_systemroot);
where fs->windows_systemroot is guest controlled. It's not clear what
the effects might be of allowing the guest to allocate potentially
very large stack frames, but at best it allows the guest to cause
libguestfs to segfault. It turns out we are very lucky that
fs->windows_systemroot cannot be set arbitrarily large (see checks in
is_systemroot).
This commit changes those to large heap allocations instead.
Introduce and use a new inspect_mount_root function to mount all the
mountpoints of a root in the guest, replacing the same code doing that
in different tools.
inspect_mount_root_ro is inspect_mount_root with readonly mount option.
Allow the user to specify a template in --list mode, which will be the
only result in the resulting output (instead of all the available
templates).
This makes it easier to find out the details of a specific template.
Require qemu >= 1.3.0, the first version that supported
`qemu-img --output=json'.
This means we require yajl (for parsing the JSON output of qemu-img),
and that in turn has consequences elsewhere.
If uncompressing a template to a filesystem which is nearly full,
virt-builder displays an error which points to the wrong (source) file:
$ virt-builder fedora-22
[ 1.2] Downloading: http://libguestfs.org/download/builder/fedora-22.xz
[ 1.7] Planning how to build this image
[ 1.7] Uncompressing
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
/home/rjones/.cache/virt-builder/fedora-22.x86_64.1: No space left on device
(The error message is usually printed once by each thread, so it is
printed several times.)
Change the error message to point to the output file, which might be a
temporary or the final file depending on the build plan:
$ virt-builder fedora-22
[ 1.1] Downloading: http://libguestfs.org/download/builder/fedora-22.xz
[ 1.6] Planning how to build this image
[ 1.6] Uncompressing
fedora-22.img: No space left on device
fedora-22.img: No space left on device
fedora-22.img: No space left on device
fedora-22.img: No space left on device
Instead of creating Guestfs handles and manually apply common options
(e.g. debug and trace), use the open_guestfs in Common_utils.
This also applies the common options to handles which didn't set them
before, so we can inspect also their messages if needed.
Various tests cannot be run in parallel just because they happen to
use conflicting names for temporary output files (eg. "test.out").
Change these tests to use unique temporary names, so the tests could
be run in parallel.
Remove man pages and other pages which 'make clean' did not remove
before.
To evaluate which pages could be removed, I did a full build and
check, and then ran 'make clean' followed by 'git clean -xdf'. By
examining the output of the git clean command I could see which files
were being missed.
Files that are _not_ removed by make clean or make distclean:
- generator-built files
- Makefile, Makefile.in, .deps, .depend
- any ./configure output files (maybe they should be?)
Move the random set of HTML files we build from html/ into
the website/ directory.
Also in the website/ directory, put the index.html file from
http://libguestfs.org, which was previously not under version control.
It is generated from index.html.in so we can automatically add the
current version and release date.
Also in the website/ directory, put various CSS file, images, etc.
which are required by the website and were also previously not under
version control.
Change the 'make website' rule to 'make maintainer-upload-website'.
As the name suggests, it is only useful for the maintainer, and will
fail with an error for anyone else.
Create a new top-level directory called test-data, which will carry
all the test data which is large and/or shared between multiple tests.
There are actually several new subdirectories created:
test-data/binaries: The pre-built binary and library files for random
architectures that we use to test various architecture detection
features (was part of tests/data).
test-data/blank-disks: The blank disks which are used for disk format
detection (was part of tests/data).
test-data/files: Other miscellaneous test files from tests/data that
are not included in the above.
test-data/phony-guests: The phony guests (was tests/guests).
test-data: The top-level directory builds the 'test.iso' image file
that is used for testing the C API and in miscellaneous other tests.
