common/mlutils: Unix_utils.StatVFS.statvfs: This commit implements a
full-featured binding for the statvfs(3) function.
We then use this to reimplement the daemon statvfs API in OCaml.
Note that the Gnulib fallback is dropped in this commit. It
previously referenced non-existent field names in the fs_usage struct
so it didn't work. Also it's not necessary as POSIX has supported
statvfs(3) since 2001, it's supported in *BSD, macOS > 10.4, and there
is already a Windows fallback.
The previous code:
fcntl (fd, F_SETFL, O_NONBLOCK)
was technically incorrect, because it would have reset any
other flags on the file descriptor.
Thanks: Eric Blake
Note this requires either the following fix in autoconf:
http://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=e17a30e98
OR gnulib sys_types module plus gnulib
commit a512e041120e9012e69afa2f5c3adc196ec4999a (any gnulib more
recent than Sep 2016) which corrects the AC_HEADER_MAJOR macro in a
similar way.
GCC 7.0.1 can determine if there is likely to be sufficient space in
the output buffer when using sprintf/snprintf, based on the format
string.
The errors were all either of this form:
bindtests.c:717:29: error: '%zu' directive output may be truncated writing between 1 and 19 bytes into a region of size 16 [-Werror=format-truncation=]
snprintf (strs[i], 16, "%zu", i);
^~~
bindtests.c:717:28: note: directive argument in the range [0, 2305843009213693951]
snprintf (strs[i], 16, "%zu", i);
^~~~~
or this form:
sync.c: In function 'fsync_devices':
sync.c:108:50: error: '%s' directive output may be truncated writing up to 255 bytes into a region of size 251 [-Werror=format-truncation=]
snprintf (dev_path, sizeof dev_path, "/dev/%s", d->d_name);
^~
Fixed by converting these into dynamic allocation, or making the
output buffer larger, whichever was easier.
There is a gnulib macro we can use to make this simpler for integers.
It requires a new gnulib module (intprops), but it turns out that we
were already pulling that in through dependencies, so the change to
bootstrap is a no-op. (thanks: Dan Berrange)
Make use of the recently added 'getprogname' module in gnulib: replace
our guestfs_int_program_name with the getprogname() provided by the
module, since it does the same thing, and in a portable way.
As consequence of the above, use gnulib in a couple of tests that use
getprogname().
Since guestfs_int_program_name is gone, drop the configure checks
associated with it.
Commit 96158d42f5 enabled the previously
private guestfs_add_libvirt_dom API. It also tried to enable the
existing test for this API, but failed to do that correctly. Also the
test was broken. Fix all of this.
This fixes commit 96158d42f5.
Since the daemon has long since used the same build system as the rest
of libguestfs, it no longer needs its own gnulib.
This arcane bit of code was left over from
commit e05ddc70f1 (added 2009-08-03 !)
Implement the GUESTFS_ADD_DRIVE_OPTS_SECRET argument of
guestfs_add_drive_opts. For libvirt we have to save the secret in
libvirtd first, get a UUID, and then pass the UUID back through the
domain XML.
Since the signature of iconv() changes between implementations (the
constness of the second parameter, in particular), make use of the iconv
module of gnulib to handle these potential differences, including an
external (out of libc) iconv implementation.
Even though this doesn't stop the compile phase, I find it a bit
distracting that this is what I get with bootstrap:
../.gnulib/gnulib-tool: *** cannot find ./configure.ac - make sure you
run gnulib-tool from within your package's directory
../.gnulib/gnulib-tool: *** Stop.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
It's simpler to use the glibc 'program_invocation_short_name(3)'
feature, and fall back to a generic solution. Also remove risky
assignments to argv[0].
We were using format= which was bogus and libvirt was ignoring it and
forcing raw format instead.
Also in its default configuration libvirt won't do disk format
autodetection at all, so we must do it instead.
The presumption is that all file descriptors should be created with
the close-on-exec flag set. The only exception are file descriptors
that we want passed through to exec'd subprocesses (mainly pipes and
stdin/stdout/stderr).
For open calls, we pass O_CLOEXEC as an extra flag, eg:
fd = open ("foo", O_RDONLY|O_CLOEXEC);
This is a Linux-ism, but using a macro we can easily make it portable.
For sockets, similarly:
sock = socket (..., SOCK_STREAM|SOCK_CLOEXEC, ...);
For accepted sockets, we use the Linux accept4 system call which
allows flags to be supplied, but we use the Gnulib 'accept4' module to
make this portable.
For dup, dup2, we use the Linux dup3 system call, and the Gnulib
modules 'dup3' and 'cloexec'.
Combine the two Gnulib instances together.
Add checks from old daemon/configure.ac into configure.ac.
Fix daemon/Makefile.am so it is like a normal subdirectory
Makefile.am.
Because we are now using the replacement strerror_r function from
Gnulib (instead of the one from glibc directly), this requires a small
change to src/guestfs.c.
This tool replaces virt-list-filesystems and virt-list-partitions with
a new tool written in C with a more uniform command line structure
and output.
This existing Perl tools are deprecated but remain indefinitely.
This reverts commit 064569bcbf.
This commit does the wrong thing: creating an empty ocaml/.depend
file is wrong because building the OCaml bindings will fail.
Not having this file will prevent automake from running. Therefore
this file has to exist with the correct content in git.
I don't see a reason to autogenerate po/Makevars, and in the
earlier commit which changed this file to being autogenerated
we accidentally lost the special Perl keywords, copyright notice
and bug reporting address. Fix all of that.
This partially reverts commit febff9d2a3.
libguestfs-supermin-helper was previously a shell script. Although
we had steadily optimized it, there were a number of intractable
hot spots:
(1) cpio still reads input files in 512 byte chunks; this is *very*
pessimal behaviour, particularly when SELinux is enabled.
(2) the hostfiles globbing was done very inefficiently by the shell,
with the shell rereading the same directory over and over again.
This is a rewrite of this shell script in C. It is approximately
3 times faster without SELinux, and has an even greater speed difference
with SELinux.
The main features are:
(a) It never frees memory, making it simpler. The program is designed
to run and exit in sub-second times, so this is acceptable.
(b) It caches directory reads, making the globbing of host files much
faster (measured this as ~ 4 x speed up).
(c) It doesn't use external cpio, but instead contains code to write
newc format cpio files, which is all that the kernel can read. Unlike
cpio, this code uses large buffers for reads and writes.
(d) Ignores missing or unreadable hostfiles, whereas cpio gave a
warning.
(e) Checks all return values from system calls.
(f) With --verbose flag, it will print messages timing itself.
This passes all tests.
Updated with feedback from Jim Meyering.
If this flag is omitted (as in the case for all existing callers)
then the hive is still opened read-only.
We add a 'writable' flag to the hive handle, and we change the way
that the hive file (data) is stored. The data is still mmapped if
the file is opened read-only, since that is more efficient and allows
us to handle larger hives. However if we need to write to the file
then we have to read it all into memory, since if we had to extend the
file we need to realloc that data.
Note the manpage section L</WRITING TO HIVE FILES> comes in a later
commit.
Current code uses atoi to parse the generator Int type and
atoll to parse the generator Int64 type. The problem with the
ato* functions is that they don't cope with errors very well,
and they cannot parse numbers that begin with 0.. or 0x..
for octal and hexadecimal respectively.
This replaces the atoi call with a call to Gnulib xstrtol
and the atoll call with a call to Gnulib xstrtoll.
The generated code looks like this for all Int arguments:
{
strtol_error xerr;
long r;
xerr = xstrtol (argv[0], NULL, 0, &r, "");
if (xerr != LONGINT_OK) {
fprintf (stderr,
_("%s: %s: invalid integer parameter (%s returned %d)\n"),
cmd, "memsize", "xstrtol", xerr);
return -1;
}
/* The Int type in the generator is a signed 31 bit int. */
if (r < (-(2LL<<30)) || r > ((2LL<<30)-1)) {
fprintf (stderr, _("%s: %s: integer out of range\n"), cmd, "memsize");
return -1;
}
/* The check above should ensure this assignment does not overflow. */
memsize = r;
}
and like this for all Int64 arguments (note we don't need the
range check for these):
{
strtol_error xerr;
long long r;
xerr = xstrtoll (argv[1], NULL, 0, &r, "");
if (xerr != LONGINT_OK) {
fprintf (stderr,
_("%s: %s: invalid integer parameter (%s returned %d)\n"),
cmd, "size", "xstrtoll", xerr);
return -1;
}
size = r;
}
Note this also fixes an unrelated bug in guestfish handling of
RBufferOut. We were using 'fwrite' without checking the return
value, and this could have caused silent failures, eg. in the case
where there was not enough disk space to store the resulting file,
or even if the program was interrupted (but continued) during the
write.
Replace this with Gnulib 'full-write', and check the return value
and report errors.
This commit uses the Gnulib 'lock' module to implement a mutex on
the global list of handles which is stored by the library.
Note that Gnulib nicely avoids explicitly linking with -lpthread
unless the application program itself links to -lpthread. Locks
are only enabled in multithreaded applications.
$ ldd src/.libs/libguestfs.so.0.217.0
linux-vdso.so.1 => (0x00007fffcb7ff000)
libc.so.6 => /lib64/libc.so.6 (0x00007f96a4e6c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f96a544d000)
Please enter the commit message for your changes. Lines starting
This implements FUSE filesystem support so that any libguestfs-
accessible disk image can be mounted as a local filesystem.
Note: file writes (ie. write(2) system call) is not yet implemented.
The API needs more test coverage, particularly lesser-used system
calls.
The big unresolved issue is UID/GID mapping between guest filesystem
IDs and the host. It's not easy to automate this because you need
extra details about the guest itself in order to get to its
UID->username map (eg. /etc/passwd from the guest).
* bootstrap: Don't use autoreconf's --norecursive
option. We require the default --recursive behavior in order
to create daemon/configure. Reported by Matthew Booth.
* Makefile.am (ACLOCAL_AMFLAGS): Specify only one include dir: m4.
* bootstrap: Tell gnulib-tool to put .m4 files in m4/, not gnulib/m4.
* autogen.sh: Move autoreconf from here into...
* bootstrap: ...here, so that it is run only when gnulib-tool is.
Also, tell it to skip the usual autopoint and libtoolize runs.
* m4/.gitignore: Update.
* bootstrap: Invoke autopoint with --force, to avoid warning
about existing build-aux/config.rpath.
Invoke libtoolize before gnulib-tool, to avoid spurious warnings.
* autogen.sh: Add comments.
Remove build-aux/config.rpath before running autoreconf.
Now that we're using gnulib in earnest, any manual definition
would provoke a redefinition warning.
* fish/fish.c (_GNU_SOURCE): Don't define.
* fish/destpaths.c (_GNU_SOURCE): Likewise.
* src/guestfs.c (_GNU_SOURCE): Likewise.
* bootstrap (modules): Add asprintf, strchrnul, strerror, strndup
and vasprintf.
* fish/fish.c (main): Set argv[0] to sanitized program_name, so
functions like getopt_long that use argv[0] use the clean name.
Use gnulib's closeout module to ensure any failure to write to
stdout is detected and reported.
* fish/fish.c: Include "closeout.h".
(main): Call atexit (close_stdout);
* bootstrap (modules): Add closeout.
* fish/fish.c: Include "progname.h".
(main): Call set_program_name to initialize.
Don't hard-code guestfish everywhere. Use program_name.
However, be careful when modifying argv[0], since it is used
in the hopes that it is an absolute file name.
(usage): Don't spew all of --help for a mis-typed option.
Split long lines.
...because it probably didn't work, and even if it did, we've
discovered that using a separate git repo like that can lead
to subtle mix-ups.
Also, fix invocation of gnulib-tool in daemon/.
* daemon/Makefile.am (SUBDIRS): Define.
(AM_CPPFLAGS): Define, to include from gnulib's lib/
(LDADD): Define, to link with gnulib's libgnu.a.
* daemon/configure.ac: Use AC_CONFIG_AUX_DIR([build-aux]),
gl_EARLY and gl_INIT.
(AC_CONFIG_FILES): Add lib/Makefile and tests/Makefile
* daemon/m4/gnulib-cache.m4: New file, generated by running
../.gnulib/gnulib-tool --import --with-tests hash
* daemon/.gitignore: Ignore all of the imported files.
build: tell bootstrap about daemon/
* bootstrap: Run gnulib-tool --update in daemon/.
Remove bootstrap's --gnulib-srcdir option, because it probably
didn't work, and even if it did, we've discovered that using
a separate git repo like that can lead to subtle mix-ups.
* .gitmodules: New file, to track gnulib.
* .gnulib: Submodule directory.
* Makefile.am (EXTRA_DIST): Don't list config.rpath or
gitlog-to-changelog.
* autogen.sh: Adapt to use the new submodule.
* cfg.mk: New file.
(SUBDIRS): Add gnulib/lib and gnulib/tests.
(dist-hook): Reflect new location of getlog-to-changelog.
* configure.ac: Set build-aux/ as AUX_DIR.
Invoke gl_EARLY and gl_INIT.
(AC_CONFIG_FILES): Add gnulib/lib/Makefile and gnulib/tests/Makefile.
