Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
GCC has two warnings related to large stack frames. We were already
using the -Wframe-larger-than warning, but this reduces the threshold
from 10000 to 5000 bytes.
However that warning only covers the static part of frames (not
alloca). So this change also enables -Wstack-usage=10000 which covers
both the static and dynamic usage (alloca and variable length arrays).
Multiple changes are made throughout the code to reduce frames to fit
within these new limits.
Note that stack allocation of large strings can be a security issue.
For example, we had code like:
size_t len = strlen (fs->windows_systemroot) + 64;
char software[len];
snprintf (software, len, "%s/system32/config/software",
fs->windows_systemroot);
where fs->windows_systemroot is guest controlled. It's not clear what
the effects might be of allowing the guest to allocate potentially
very large stack frames, but at best it allows the guest to cause
libguestfs to segfault. It turns out we are very lucky that
fs->windows_systemroot cannot be set arbitrarily large (see checks in
is_systemroot).
This commit changes those to large heap allocations instead.
Updating gnulib has caused -Wformat-signedness to be enabled. This
has revealed many problems in C format strings. The fixes here fall
into the following main categories:
- Using %d with an unsigned parameter.
- %x and %o expect an unsigned argument.
- uid_t and gid_t are unsigned on Linux. The safe way to print these
is to cast them to uintmax_t and then print them using the %ju
modifier (see http://stackoverflow.com/a/1401581).
- Using %d to print an enum. Since enums may be either char or int,
I fixed this by casting the enum to int.
- strtol_error & lzma_ret are both unsigned types.
guestfsd calls many different tools. Keeping track of all of them is
error prone. This patch introduces a new helper macro to put the command
string into its own ELF section:
GUESTFSD_EXT_CMD(C_variable, command_name);
This syntax makes it still possible to grep for used command names.
The actual usage of the collected list could be like this:
objcopy -j .guestfsd_ext_cmds -O binary daemon/guestfsd /dev/stdout |
tr '\0' '\n' | sort -u
The resulting output will be used to tell mkinitrd which programs to
copy into the initrd.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
RWMJ:
- Move str_vgchange at request of author.
- Fix snprintf call in daemon/debug.c
Add an API for doing what virt-sparsify was doing: freeing up free
space in a filesystem.
The current implementation is simple-minded: we create a file, fill it
with zeroes until we run out of space, then delete the file. However
the description leaves it open to do a better implementation, eg.
using sparsification support that is currently being worked on in ext4
and qemu.
The implementation also sends progress notifications, which is an
advantage over the old 'dd' method.
The presumption is that all file descriptors should be created with
the close-on-exec flag set. The only exception are file descriptors
that we want passed through to exec'd subprocesses (mainly pipes and
stdin/stdout/stderr).
For open calls, we pass O_CLOEXEC as an extra flag, eg:
fd = open ("foo", O_RDONLY|O_CLOEXEC);
This is a Linux-ism, but using a macro we can easily make it portable.
For sockets, similarly:
sock = socket (..., SOCK_STREAM|SOCK_CLOEXEC, ...);
For accepted sockets, we use the Linux accept4 system call which
allows flags to be supplied, but we use the Gnulib 'accept4' module to
make this portable.
For dup, dup2, we use the Linux dup3 system call, and the Gnulib
modules 'dup3' and 'cloexec'.
This code modifies zero, zero-device, is-zero, is-zero-device.
zero and zero-device are modified so that if the blocks of the device
already contain zeroes, then we don't write zeroes. The reason for
this is to avoid unnecessarily making the underlying storage
non-sparse or (in the qcow2 case) growing it.
is-zero and is-zero-device are modified so that zero detection is
faster. This is a nice side effect of making the first change.
Since avoiding unnecessary zeroing involves reading the blocks before
writing them, whereas before we just blindly wrote, this can be
slower. As you can see from the tests below, in the case where the
disk is sparse, it actually turns out to be faster, because we avoid
allocating the underlying blocks.
However in the case where the disk is non-sparse and full of existing
data, it is much slower. There might be a case for an API flag to
adjust whether or not we perform the zero check. I did not add this
flag because it is unlikely that the caller would have enough
information to be able to set the flag correctly.
(Elapsed time in seconds)
Format Test case Before After
Raw Sparse 16.4 5.3
Preallocated zero 17.0 18.8
Preallocated random 16.0 41.3
Qcow2 preallocation=off 18.7 5.6
preallocation=metadata 17.4 5.8
The current code uses a fixed block size of 4K for reading and
writing. I also tried the same tests with a block size of 64K but it
didn't make any significant difference.
(Thanks to Federico Simoncelli for suggesting this change)
Nearly every file-related function in daemons/*.c is affected:
Remove this pair of statements from each affected do_* function:
- NEED_ROOT (return -1);
- ABS_PATH (dir, return -1);
and change the type of the corresponding parameter to "const char *".
* src/generator.ml: Emit NEED_ROOT just once, even when there are two or
more Pathname args.
