This commit deprecates luks-open/luks-open-ro/luks-close for the more
generic sounding names cryptsetup-open/cryptsetup-close, which also
correspond directly to the cryptsetup commands.
The optional cryptsetup-open readonly flag is used to replace the
functionality of luks-open-ro.
The optional cryptsetup-open crypttype parameter can be used to select
the type (corresponding to cryptsetup open --type), which allows us to
open BitLocker-encrypted disks with no extra effort. As a convenience
the crypttype parameter may be omitted, and libguestfs will use a
heuristic (based on vfs-type output) to try to determine the correct
type to use.
The deprecated functions and the new functions are all (re-)written in
OCaml.
There is no new test here, unfortunately. It would be nice to test
Windows BitLocker support in this new API, however the Linux tools do
not support creating BitLocker disks, and while it is possible to
create one under Windows, the smallest compressed disk I could create
is 37M because of a mixture of the minimum support size for BitLocker
disks and the fact that encrypted parts of NTFS cannot be compressed.
Also synchronise with common module.
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.
It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory. Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up. This is
certainly the case for examples, and some language bindings and some
tests.
In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
Since the introduction of the TEST_FUNCTIONS framework in tests, with
commit 2cb0413049, using a non-bash shell
will not work anymore, since the framework currently relies on bash.
Hence, switch the interpreter of these tests to bash, so they can run
also on systems where /bin/sh is not bash (e.g. default Ubuntu
installations).
The gobject bindings are adequately covered in the usual manual pages:
guestfs(3). There is no need for separate generation of gtk-doc.
Also generating gtk documentation is the slowest part of the build,
and the tooling around gtk-doc is broken
(https://bugzilla.redhat.com/show_bug.cgi?id=1465665).
Note this removes the configure ‘--enable-gtk-doc’ option. Using this
option now gives a warning, but is otherwise ignored:
configure: WARNING: unrecognized options: --enable-gtk-doc
The internal_yara_scan runs the Yara engine with the previously loaded
rules against the given file.
For each rule matching against the scanned file, a struct containing
the file name and the rule identifier is returned.
The gathered list of yara_detection structs is serialised into XDR format
and written to a file.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Introduce a new API to create a new squashfs filesystem out of a path
in the guest. It can be configured to exclude paths based on patterns,
and to select which compression use for the filesystem.
The advantage of running mksquashfs directly in the appliance is that
ownerships are properly saved, as opposed to tar_out + local untar.
The error is:
gobject/docs/gtk-doc.make:52: warning: CLEANFILES multiply defined in condition TRUE ...
gobject/docs/Makefile.am:95: 'gobject/docs/gtk-doc.make' included from here
common-rules.mk:28: ... 'CLEANFILES' previously defined here
gobject/docs/Makefile.am:18: 'subdir-rules.mk' included from here
subdir-rules.mk:20: 'common-rules.mk' included from here
Run the following command over the source:
perl -pi.bak -e 's/(20[01][0-9])-2016/$1-2017/g' `git ls-files`
(Thanks Rich for the perl snippet, as used in past years.)
By adding common CLEANFILES and DISTCLEANFILES variables to
common-rules.mk, we can remove these from most other Makefiles, and
also clean files more consistently.
Note that bin_PROGRAMS are already cleaned by 'make clean', so I
removed cases where these were unnecessarily added to CLEANFILES.
Expose the aug_transform API through the library, so it's possible to
add/remove Augeas transformations to handle files in custom places using
existing lenses.
This function allows to download file system data units (blocks) from
the given partition.
The API can be used to detect data hidden within filesystem bad blocks
or slack space.
Moreover for filesystems such as Ext3 and Ext4, this function is the
only way to retrieve deleted files. An example is given in the function
tests.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
- generator: Added tsk_dirent struct
The tsk_dirent struct contains the information gathered via TSK APIs.
The struct contains the following fields:
* tsk_inode: inode of a file
* tsk_type: type of file such as for dirwalk command
* tsk_size: file size in bytes
* tsk_name: path relative to its disk partition
* tsk_flags: bitfield containing extra information
* tsk_spare[1-5]: extra space for future usage
- configure: Added libtsk compile-time check
Ensure libtsk is available at compile time.
If not, daemon routines depending on it won't be available.
- API: internal_filesystem_walk
The internal_filesystem_walk command walks through the FS structures
of a disk partition and returns all the files or directories
which could be found.
The command is able to retrieve information regarding deleted
or unaccessible files where other commands such as stat or find
would fail.
The gathered list of tsk_dirent structs is serialised into XDR format
and written to a file by the appliance.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
Introduce a new kind of bindings tests, 090-retvalues, to check all the
possible return values in bindings; start implementing them for
scripting languages such as GObject introspection, Perl, PHP, Python,
and Ruby, reusing existing implementations where existing.
Add a new optional bool "directoryslash" to indicate whether the caller
wants trailing slashes in names of directories, defaulting to true (the
current behaviour); this helps with interoperability with other tools
(such as rm).
Related to RHBZ#1293271.
Create a new top-level directory called test-data, which will carry
all the test data which is large and/or shared between multiple tests.
There are actually several new subdirectories created:
test-data/binaries: The pre-built binary and library files for random
architectures that we use to test various architecture detection
features (was part of tests/data).
test-data/blank-disks: The blank disks which are used for disk format
detection (was part of tests/data).
test-data/files: Other miscellaneous test files from tests/data that
are not included in the above.
test-data/phony-guests: The phony guests (was tests/guests).
test-data: The top-level directory builds the 'test.iso' image file
that is used for testing the C API and in miscellaneous other tests.
Use the generator_built variable to list any files which are built by
the generator. This ensures they get automatically rebuilt.
Three files were missing from EXTRA_DIST, so they were not included in
the tarball, meaning you would have needed OCaml to build from the
tarball. You would have seen this when building:
cd .. && /builddir/build/BUILD/libguestfs-1.31.9/generator/generator
written gobject/docs/guestfs-title.sgml
written customize/customize-synopsis.pod
written customize/customize-options.pod
generated 476921 lines of code
This API already existed (as guestfs___add_libvirt_dom), and was used
by a few tools.
This commit changes it to a public API.
Note that for reasons outlined in the previous commit message, it is
impossible to call this from guestfish or from non-C language
bindings.
The existing APIs guestfs_stat, guestfs_lstat and guestfs_lstatlist
return a stat structure that contains atime, mtime and ctime fields
that store only the timestamp in seconds.
Modern filesystems can store timestamps down to nanosecond
granularity, and the ordinary glibc stat(2) wrapper will return these
in "hidden" stat fields:
struct timespec st_atim; /* Time of last access. */
struct timespec st_mtim; /* Time of last modification. */
struct timespec st_ctim; /* Time of last status change. */
with the following macros defined for backwards compatibility:
#define st_atime st_atim.tv_sec
#define st_mtime st_mtim.tv_sec
#define st_ctime st_ctim.tv_sec
It is not possible to redefine guestfs_stat to return a longer struct
guestfs_stat with room for the extra nanosecond fields, because that
would break the ABI of guestfs_lstatlist as it returns an array
containing consecutive stat structs (not pointers). Changing the
return type of guestfs_stat would break API. Changing the generator
to support symbol versioning is judged to be too intrusive.
Therefore this adds a new struct (guestfs_statns) and new APIs:
guestfs_statns
guestfs_lstatns
guestfs_lstatnslist
which return the new struct (or array of structs in the last case).
The old APIs may of course still be used, forever, but are deprecated
and shouldn't be used in new programs.
Because virt tools are compiled with -DGUESTFS_WARN_DEPRECATED=1, I
have updated all the places calling the deprecated functions. This
has revealed some areas for improvement: in particular virt-diff and
virt-ls could be changed to print the nanosecond fields.
FUSE now returns nanoseconds in stat calls where available, fixing
https://bugzilla.redhat.com/show_bug.cgi?id=1144891
Notes about the implementation:
- guestfs_internal_lstatlist has been removed and replaced by
guestfs_internal_lstatnslist. As the former was an internal API no
one should have been calling it, or indeed can call it unless they
start defining their own header files.
- guestfs_stat and guestfs_lstat have been changed into library-side
functions. They, along with guestfs_lstatlist, are now implemented
as wrappers around the new functions which just throw away the
nanosecond fields.
Sort the structs when generating code. Since the structs are
logically indepedent of each other, this should have no effect except
to make generated files list the structs in a different order.
However this also fixes the following build problem:
File "convert_linux.ml", line 322, characters 43-50:
Error: This expression has type G.stat = Guestfs.stat
but an expression was expected of type G.dirent = Guestfs.dirent
It turns out the OCaml bindings don't like the fact that we have
two structs with a common field name (dirent.ino and stat.ino).
In OCaml < 4.01, this means that any attempt to reference stat.ino
would fail because dirent.ino appears second in the file, overriding
stat.ino.
Sorting the structs has the side effect of making stat.ino appear
second, thus resolving the build failure above.
In OCaml >= 4.01 the compiler now uses some disambiguation rules based
on the known types to resolve this problem, so accessing either field
would work no matter what order they are listed in.
See:
http://ocaml.org/releases/4.01.0.htmlhttp://www.lexifi.com/blog/type-based-selection-label-and-constructorshttp://www.lexifi.com/blog/ocaml-extensions-lexifi-overidding-record-labels-and-constructors
This is useful for generating Linux initramfses from other types of
filesystems. For example:
guestfish --ro -a disk.img -i cpio-out / - | gzip -9 > initrd.img
Look for use of external_functions and fish_functions and replace with
use of external_functions_sorted and fish_functions_sorted where
possible. This ensures that the output of the generator is sorted as
far as possible.
I also checked for uses of internal_functions and documented_functions
but those are not used. The *_sorted versions are always used
instead.
This adds an extra optional boolean 'followsymlinks' flag to those 6
is-* APIs. If the flag is true, then symlinks are followed, ie. we
use stat instead of lstat in the test.
For the rationale behind this change, see:
https://bugzilla.redhat.com/show_bug.cgi?id=974489
This file is mainly a central place to:
- include localenv if it exists, and
- define the RHEL 5 backwards compatibility macros, instead of
spreading them over every other file.
