Commit Graph

112 Commits

Author SHA1 Message Date
Richard W.M. Jones
dbc2fd8dc8 lib: Remove libguestfs live
This experimental feature allowed you (in theory) to connect to an
existing instance of the libguestfs daemon.  (Again, in theory) it
allowed you to attach to running guests.  This didn't work well in
practice.  If you want to do this, install qemu-guest-agent inside
your guest instead.

This also disables the --live options in guestfish and guestmount.
(The option now prints an error).

This was never supported in RHEL.

The daemon tests relied on this connection method to perform tests on
a bare daemon, so this removes those tests.  They were not especially
valuable.

See-also: https://bugzilla.redhat.com/798980
2022-03-09 09:27:19 +00:00
Richard W.M. Jones
536b5ed993 Version 1.45.5. 2021-04-09 11:39:10 +01:00
Richard W.M. Jones
efb8a766ca daemon: Allow xorriso as an alternative to isoinfo.
Currently the guestfs_isoinfo and guestfs_isoinfo_device APIs run
isoinfo inside the appliance to extract the information.

isoinfo is part of genisoimage which is somewhat dead upstream.
xorriso is supposedly the new thing.  (For a summary of the situation
see: https://wiki.debian.org/genisoimage).

This commit rewrites the parsing from C to OCaml to make it easier to
deal with, and allows you to use either isoinfo or xorriso.

Mostly the same fields are available from either tool, but xorriso is
a bit more awkward to parse.
2021-03-30 15:21:54 +01:00
Richard W.M. Jones
c9ee831aff inspection: Fix inspection of recent RPM guests using non-BDB.
Recent RPM-based guests have switched from using Berkeley DB (BDB) to
sqlite.  In order to inspect these guests (and earlier ones) we need
to stop using the hokey parsing of the BDB and use librpm APIs
instead.

This commit adds a new internal API so we can call librpm from the
daemon, and changes the library part to use the new API for RPM-based
guests.

This change removes the requirement for BDB tools like db_dump.

See also:
http://lists.rpm.org/pipermail/rpm-ecosystem/2021-March/000751.html
http://lists.rpm.org/pipermail/rpm-ecosystem/2021-March/000754.html
https://blog.fpmurphy.com/2011/08/programmatically-retrieve-rpm-package-details.html

This breaks the virt-inspector test (now in the separate guestfs-tools
repository).  However this is not a bug in libguestfs, but a bug in
the phoney Fedora guest that we use for testing - we created a
BDB-style RPM database which was supposed to be just enough to make
the old code work.  The new code using real librpm needs
/usr/lib/rpm/rpmrc (not present in the phoney image) and also cannot
parse the phoney database, so we will need to separately rework that
test.

Thanks: Panu Matilainen
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1766487
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1409024
2021-03-26 16:26:00 +00:00
Richard W.M. Jones
09639387aa Version 1.45.2. 2021-03-22 16:41:30 +00:00
Richard W.M. Jones
733d2182b6 Remove the tools.
These have now moved to a new repository.  Provisionally it is here:

https://github.com/rwmjones/guestfs-tools/

but this is not the final location, as it will eventually be hosted on
gitlab.com.

The tarballs are here:

https://download.libguestfs.org/guestfs-tools/
2021-03-11 13:58:41 +00:00
Richard W.M. Jones
75abec1f70 include: Move lib/guestfs.h to include/guestfs.h
This brings libguestfs into line with other projects which have a
separate include/ directory for the public header.

It's also the case that <guestfs.h> has never particularly belonged in
the lib/ subdirectory.  Some tools add -Ilib/ but they only need
<guestfs.h> and not any other headers from that directory, and
separating out the public header allows us to clean those up.  This is
certainly the case for examples, and some language bindings and some
tests.

In future I'm hopeful we can use this as the basis to tease out other
dependencies, as a prelude to separating them out from the repo.
2020-09-21 18:38:28 +01:00
Pino Toscano
2c4aec6dcf build: update miscellaneous files
C_SOURCE_FILES
POTFILES

Followup of commit cb0edd4354.
2020-01-09 15:47:56 +01:00
Pino Toscano
98700b6d8c docs: exclude dummy.c sources
They are empty sources used to build OCaml-only targets using automake,
so there is no documentation to extract.
2019-12-18 11:01:58 +01:00
Pino Toscano
957549cd2d Remove mlaugeas and mllibvirt leftovers
Update according to the removal of the embedded copies of augeas-ocaml,
and libvirt-ocaml from the common submodule.
2019-12-16 18:55:01 +01:00
Richard W.M. Jones
58ab39ba34 Remove boot and qemu analysis tools into a separate project.
These have now moved here:
https://github.com/libguestfs/libguestfs-analysis-tools

This commit removes them and updates guestfs-performance(1) to point
to the new location.
2019-11-28 15:37:31 +00:00
Richard W.M. Jones
85c99edec1 v2v: Remove virt-v2v.
It has moved to a new repository:
https://github.com/libguestfs/virt-v2v
2019-11-13 12:49:55 +00:00
Richard W.M. Jones
8097dc16e7 Version 1.41.6. 2019-10-15 13:34:40 +01:00
Pino Toscano
dea9636c59 Remove remaining virt-p2v bits
Remove (almost) all the remaining bits related to virt-p2v.
2019-09-10 17:52:16 +02:00
Richard W.M. Jones
cce016dfe3 docs: Remove p2v/ directory from generated docs files.
make[2]: *** No rule to make target '../p2v/about-authors.c', needed by 'internal-documentation.pod'.  Stop.

Since we're going to remove the whole p2v subdirectory shortly anyway,
it was simplest to ignore the whole directory.
2019-09-02 12:34:07 +01:00
Richard W.M. Jones
0d0c3eb1dd docs: Remove generated file common/mllibvirt/libvirt_c.c.
Avoids this error when building from git:

make[2]: *** No rule to make target '../common/mllibvirt/libvirt_c.c', needed by 'internal-documentation.pod'.  Stop.

This file doesn't have any generated docs or gettext annotations in it
so adding it to these files is useless anyway.
2019-09-02 12:22:25 +01:00
Richard W.M. Jones
2a5ea2892e Version 1.41.1. 2019-09-02 10:12:38 +01:00
Richard W.M. Jones
c873e04b26 Update docs/C_SOURCE_FILES. 2019-08-20 16:11:49 +01:00
Richard W.M. Jones
ab09bc25c7 build: Update miscellaneous files.
BUGS
docs/C_SOURCE_FILES
po/POTFILES
po/POTFILES-ml

These files are normally updated by ‘make dist’ but as we've not had a
release in a while they had not been updated for some time.
2019-07-29 12:57:24 +01:00
Pino Toscano
fe96e24001 p2v: generate C about data authors from AUTHORS file
Create a small Perl script to generate about-authors.c from the p2v
AUTHORS file at build time, instead of generating in the generator
at dist time.
2019-07-03 12:41:02 +02:00
Pino Toscano
796ce74f31 p2v: move kernel config to perl script
Instead of generating the p2v kernel config using the OCaml generator,
create a Perl script to do this job, mostly at build time.  This is done
to rely less on the generator for p2v, and because the generation of
these sources is quick enough that it can be done at build time (instead
of shipping the generated sources in dist tarballs).

The generate-p2v-config.pl mimics what generator/p2v_config.ml --
namings, and general structure are kept close to that for comparison.

The two C sources are created at build time by the script; however, the
p2v-config.h header is generated at configure time: this is done because
p2v-config.h is included by p2v.h (another header), which in turn is
included by all the p2v C sources -- automake is not able to properly
resolve the dependency, and thus it would not be generated properly.
2019-07-01 13:58:13 +02:00
Pino Toscano
a871f6c251 v2v: switch to ocaml-libvirt
Currently virt-v2v has few custom C-based functions for libvirt
operations, which are limited in what they do, and there is a lot of
duplicated code.

Instead, switch to ocaml-libvirt for all the libvirt interaction
currently done by the Libvirt_utils module.  This has few advantages:
- each input & output module now opens a libvirt connection only once,
  only when needed
- no need to pass URIs and passwords around, if not needed
- a wider range of libvirt APIs can now be used, with no need to create
  bindings manually

The hierarchy of input_libvirt* classes is changed to take a Lazy object
with the libvirt connection, accessing it through a "proctected" method:
this way, the connection is opened only at the first access.
Also, the Libvirt_utils module now is just helpers around the Libvirt
module, to centralize error handling, and few common operations.
2019-05-20 13:40:19 +02:00
Richard W.M. Jones
6d5959a045 common/utils: Move libxml2 writer macros to a common header file.
In some places when generating XML output in C code we use some clever
macros:

  start_element ("memory") {
    attribute ("unit", "MiB");
    string_format ("%d", g->memsize);
  } end_element ();

This commit which is mostly refactoring moves the repeated definitions
of these macros into a common header file.

I also took this opportunity to change / clean up the macros:

 - The macros are now documented properly.

 - comment() and empty_element() macros are now available everywhere.

 - Error handling has been made generic.

 - Added do..while(0) around some of the macros to make them safe to
   use in all contexts.
2018-11-02 15:15:04 +00:00
Richard W.M. Jones
38f688b8b5 build: Rebuild docs/C_SOURCE_FILES and po/POTFILES.
Remove:

common/utils/libxml2-utils.c
common/utils/libxml2-utils.h

These were accidentally added by
commit a63d02f8f1 because of a bad
interactive rebase.
2018-08-22 15:28:50 +01:00
Richard W.M. Jones
a63d02f8f1 mltools: JSON: Rename Yajl module as JSON_parser and move to common/mltools.
Commit bd1c5c9f4d changed all the code
to use Jansson instead of yajl.  However it didn't change the OCaml
module name (still Yajl).

This commit changes the module to a neutral name ("JSON_parser") and
moves it into common/mltools so it can be used by other tools.

This leaves us in a slightly awkward situation of having two JSON-ish
OCaml modules (JSON for creating trees and JSON_parser for parsing
them) with incompatible types.  That is left for future work to
resolve.  (It should be easier to do now that both modules live in the
same directory.)

This is just renaming and general refactoring.  There should be no
change in functionality.
2018-08-22 15:18:57 +01:00
Richard W.M. Jones
040cb93717 Version 1.39.8. 2018-07-27 12:06:04 +01:00
Pino Toscano
aa078b362e Update documentation/translation file indexes
Followup of commits cc04573927, and
commit d00e860ef0.
2018-04-20 13:04:29 +02:00
Richard W.M. Jones
55b347d71c Version 1.39.1. 2018-03-13 12:17:28 +00:00
Richard W.M. Jones
bbd54295d3 daemon: Reimplement statvfs API in OCaml, drop gnulib fallback.
common/mlutils: Unix_utils.StatVFS.statvfs: This commit implements a
full-featured binding for the statvfs(3) function.

We then use this to reimplement the daemon statvfs API in OCaml.

Note that the Gnulib fallback is dropped in this commit.  It
previously referenced non-existent field names in the fs_usage struct
so it didn't work.  Also it's not necessary as POSIX has supported
statvfs(3) since 2001, it's supported in *BSD, macOS > 10.4, and there
is already a Windows fallback.
2017-10-04 15:25:32 +01:00
Richard W.M. Jones
cd304ad838 common/mltools: Rename Common_utils to Tools_utils.
Reflecting the purpose of this module now, which is to act as a place
for utility functions shared only by the OCaml virt tools.
2017-09-28 14:39:23 +01:00
Richard W.M. Jones
80fa8a91e3 Rename mllib -> common/mltools.
This directory which previously contained random modules and functions
now has an official purpose: to be the place for any OCaml utility
needed by the OCaml virt tools.

This is just code movement, I didn't (yet) rename or move any of the
modules.
2017-09-28 14:39:23 +01:00
Richard W.M. Jones
2b00983f23 common/mlgettext: Move common_gettext.ml{,i} to common/mlgettext.
Mostly just code motion, but common_gettext.mli was the same whether
or not ocaml-gettext exists, so instead of generating it, add the file
to git.
2017-09-28 14:39:22 +01:00
Richard W.M. Jones
41df9aaf90 lib: Move guestfs_int_download_to_tmp and remove inspect.c.
Move the last remaining function ‘guestfs_int_download_to_tmp’ to
lib/inspect-icon.c (the main, but not only user of this function).
Then remove lib/inspect.c entirely.

This is not quite code motion because I updated the comment for the
function to reflect what it does in reality.
2017-09-21 18:05:07 +01:00
Richard W.M. Jones
3a00c4d179 Remove inspection from the C library and switch to daemon/OCaml implementation. 2017-09-16 22:27:16 +01:00
Richard W.M. Jones
19940fc2bd New API: Deprecate hivex_value_utf8 and replace with hivex_value_string.
hivex has a function hivex_value_string.  We were not calling it under
the mistaken belief that because hivex implements this using iconv,
the function wouldn't work inside the daemon.  Instead we
reimplemented the functionality in the library.

This commit deprecates hivex_value_utf8 and removes the library side
code.  It replaces it with a plain wrapper around hivex_value_string.

Thanks: Pino Toscano
2017-09-16 22:27:16 +01:00
Richard W.M. Jones
2ffb8a6b25 common: Bundle the ocaml-augeas library for use by the daemon.
This commit bundles the ocaml-augeas library (upstream here:
http://git.annexia.org/?p=ocaml-augeas.git;a=summary).  It's identical
to the upstream version and should remain so.

We can work towards using system ocaml-augeas, when it's more widely
available.
2017-09-16 22:27:16 +01:00
Richard W.M. Jones
65cfecb0f5 inspection: Deprecate APIs and remove support for inspecting installer CDs.
This just duplicated libosinfo information, and because it was never
tested it didn't work most of the time.
2017-09-16 22:27:16 +01:00
Richard W.M. Jones
0bf521cfcb daemon: Deduplicate some utility functions.
The following functions were previously reimplemented in OCaml.  This
commit replaces them with calls to the C functions:

 - is_root_device
 - prog_exists
 - udev_settle

plus the internal get_verbose_flag function.

However note that we cannot do this for every utility function.  In
particular the C function must not call any reply* functions.
2017-08-08 16:23:42 +01:00
Richard W.M. Jones
089ade6f95 Version 1.37.20. 2017-08-03 18:49:01 +01:00
Richard W.M. Jones
1e956dfc7d daemon: Reimplement ‘realpath’ API in OCaml. 2017-07-27 22:31:22 +01:00
Richard W.M. Jones
c37c8e770d daemon: Reimplement ‘findfs_uuid’ and ‘findfs_label’ APIs in OCaml.
This also reimplements the lv_canonical function in OCaml.  We cannot
call the original C function because it calls reply_with_perror which
would break the OCaml bindings.
2017-07-27 22:31:22 +01:00
Richard W.M. Jones
4052f613d2 daemon: Reimplement ‘list_filesystems’ API in the daemon, in OCaml.
Move the list_filesystems API into the daemon, reimplementing it in
OCaml.  Since this API makes many other API calls, it runs a lot
faster in the daemon.
2017-07-27 22:31:22 +01:00
Richard W.M. Jones
b48da89dd6 daemon: Reimplement ‘file_architecture’ API in OCaml.
The previously library-side ‘file_architecture’ API is reimplemented
in the daemon, in OCaml.

There are some significant differences compared to the C
implementation:

 - The C code used libmagic.  That is replaced by calling the ‘file’
   command (because that is simpler than using the library).

 - The C code had extra cases to deal with compressed files.  This is
   not necessary since the ‘file’ command supports the ‘-z’ option
   which transparently looks inside compressed content (this is a
   consequence of the change above).

This commit demonstrates a number of techniques which will be useful
for moving inspection code to the daemon:

 - Moving an API from the C library to the OCaml daemon.

 - Calling from one OCaml API inside the daemon to another (from
   ‘Filearch.file_architecture’ to ‘File.file’).  This can be done and
   is done with C daemon APIs but correct reply_with_error handling is
   more difficult in C.

 - Use of Str for regular expression matching within the appliance.
2017-07-27 22:31:22 +01:00
Richard W.M. Jones
7a084269ef daemon: Add unit tests of the ‘Utils’ module. 2017-07-27 22:31:22 +01:00
Richard W.M. Jones
d5b6f1df5f daemon: Allow parts of the daemon and APIs to be written in OCaml.
This change allows parts of the daemon to be written in the OCaml
programming language.  I am using the ‘Main Program in C’ method along
with ‘-output-obj’ to create an object file from the OCaml code /
runtime, as described here:
https://caml.inria.fr/pub/docs/manual-ocaml/intfc.html

Furthermore, change the generator to allow individual APIs to be
implemented in OCaml.  This is picked by setting:

  impl = OCaml <ocaml_function>;

The generator creates ‘do_function’ (the same one you would have to
write by hand in C), with the function calling the named
‘ocaml_function’ and dealing with marshalling/unmarshalling the OCaml
parameters.
2017-07-27 22:31:22 +01:00
Richard W.M. Jones
f60f8c47cc lib: Move implementation of ‘hivex_value_utf8’ to new file ‘lib/hivex.c’.
Just a code movement, no change.
2017-07-14 14:35:37 +01:00
Richard W.M. Jones
09b14a6076 Version 1.37.17. 2017-07-10 22:06:45 +01:00
Richard W.M. Jones
707f5bcfe0 daemon: Link guestfsd with libutils.
After the previous refactoring, we are able to link the daemon to
common/utils, and also remove some of the "duplicate" functions that
the daemon carried ("duplicate" in quotes because they were often not
exact duplicates).

Also this removes the duplicate reimplementation of (most) cleanup
functions in the daemon, since those are provided by libutils now.

It also allows us in future (but not in this commit) to move utility
functions from the daemon into libutils.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
e6c89f9631 utils: Rename ‘guestfs-internal-frontend.h’ to ‘guestfs-utils.h’.
The reason it's not just ‘utils.h’ is because Pino is worried that we
might pick up /usr/include/utils.h from a rogue library.
2017-07-10 17:01:59 +01:00
Richard W.M. Jones
8be72935dc mllib, v2v: Split out OCaml utils bindings ‘common/mlutils’.
Create a module ‘C_utils’ containing functions like ‘drive_name’ and
‘shell_unquote’ which come from the C utilities.

The new directory ‘common/mlutils’ also contains the ‘Unix_utils’
wrappers around POSIX functions missing from the OCaml stdlib.
2017-07-10 17:01:59 +01:00