This change was done almost entirely automatically using the script
below. This uses the OCaml lexer to read the source files and extract
the strings and locations. Strings which are "candidates" (in this
case, longer than 3 lines) are replaced in the output with quoted
string literals.
Since the OCaml lexer is used, it already substitutes all escape
sequences correctly. I diffed the output of the generator and it is
identical after this change, except for UUIDs, which change because of
how Utils.stable_uuid is implemented.
Thanks: Nicolas Ojeda Bar
$ ocamlfind opt -package unix,compiler-libs.common find_strings.ml \
-o find_strings.opt -linkpkg
$ for f in $( git ls-files -- \*.ml ) ; do ./find_strings.opt $f ; done
open Printf
let read_whole_file path =
let buf = Buffer.create 16384 in
let chan = open_in path in
let maxlen = 16384 in
let b = Bytes.create maxlen in
let rec loop () =
let r = input chan b 0 maxlen in
if r > 0 then (
Buffer.add_substring buf (Bytes.to_string b) 0 r;
loop ()
)
in
loop ();
close_in chan;
Buffer.contents buf
let count_chars c str =
let count = ref 0 in
for i = 0 to String.length str - 1 do
if c = String.unsafe_get str i then incr count
done;
!count
let subs = ref []
let consider_string str loc =
let nr_lines = count_chars '\n' str in
if nr_lines > 3 then
subs := (str, loc) :: !subs
let () =
Lexer.init ();
let filename = Sys.argv.(1) in
let content = read_whole_file filename in
let lexbuf = Lexing.from_string content in
let rec loop () =
let token = Lexer.token lexbuf in
(match token with
| Parser.EOF -> ();
| STRING (s, loc, sopt) ->
consider_string s loc; (* sopt? *)
loop ();
| token ->
loop ();
)
in
loop ();
(* The list of subs is already reversed, which is convenient
* because we must the file substitutions in reverse order.
*)
let subs = !subs in
let new_content = ref content in
List.iter (
fun (str, loc) ->
let { Location.loc_start = { pos_cnum = p1 };
loc_end = { pos_cnum = p2 } } = loc in
let len = String.length !new_content in
let before = String.sub !new_content 0 (p1-1) in
let after = String.sub !new_content (p2+1) (len - p2 - 1) in
new_content := before ^ "{|" ^ str ^ "|}" ^ after
) subs;
let new_content = !new_content in
if content <> new_content then (
(* Update the file in place. *)
let new_filename = filename ^ ".new"
and backup_filename = filename ^ ".bak" in
let chan = open_out new_filename in
fprintf chan "%s" new_content;
close_out chan;
Unix.rename filename backup_filename;
Unix.rename new_filename filename
)
This acts just like FString except that we do reverse device name
translation on it. The only use is in the 'pvs-full' API where we
will use it (in a subsequent commit) to reverse translate the pv_name
field (a device name) before returning it from the daemon.
Compare this to the 'pvs' API which also returns a list of device
names, but using the generator's 'RStructList (RDevice,...)' return
type, where RDevice is similarly reverse translated.
Note in the library-side bindings, because the name has already been
translated in the daemon, we just treat it exactly the same as
FString. The vast majority of this patch is this mechanical change.
Run this command across the source:
perl -pi.bak -e 's/(20[012][0-9])-20[12][012]/$1-2023/g' `git ls-files`
and remove changes to po{,-docs}/*.po{,t} (these will be regenerated
later when we run 'make dist').
The macOS rpcgen actually generates calls to xdr_uint64_t but doesn't
define them. Despite this we can just use xdr_int64_t instead since
it's just byte swapping.
The new module ‘Std_utils’ contains only functions which are pure
OCaml and depend only on the OCaml stdlib. Therefore these functions
may be used by the generator.
The new module is moved to ‘common/mlstdutils’.
This also removes the "<stdlib>" hack, and the code which copied the
library around.
Also ‘Guestfs_config’, ‘Libdir’ and ‘StringMap’ modules are moved
since these are essentially the same.
The bulk of this change is just updating files which use
‘open Common_utils’ to add ‘open Std_utils’ where necessary.
Previously the generator did not change any string returned from the
daemon. Thus guestfs_list_devices (for example) might return internal
device names like /dev/vda (if virtio-blk was in use).
This changes calls to the daemon so that returned strings are
annotated as plain strings, devices or mountables:
old ---> new
RString "uuid" RString (RPlainString "uuid")
RString "device" RString (RDevice "device")
RString "fs" RString (RMountable "fs")
For hash tables, keys and values must be annotated separately. For
example a hash table of mountables (keys) -> plain strings (values)
would be annotated like this:
old ---> new
RHashtable "fses" RHashtable (RMountable, RPlainString, "fses")
The daemon calls reverse_device_name_translation (currently a no-op)
for devices and mountables.
Note that this has no effect for calls which are handled on the
library side.
(cherry picked from commit 6b77cc196ecb8d7e1d73592ef65a189a7412c97c)
Previously we had lots of types like String, Device, StringList,
DeviceList, etc. where Device was just a String with magical
properties (but only inside the daemon), and DeviceList was just a
list of Device strings.
Replace these with some simple top-level types:
String
StringList
and move the magic into a subtype.
The change is mechanical, for example:
old ---> new
FileIn "filename" String (FileIn, "filename")
DeviceList "devices" StringList (Device, "devices")
Handling BufferIn is sufficiently different from a plain String
throughout all the bindings that it still uses a top-level type.
(Compare with FileIn/FileOut where the only difference is in the
protocol, but the bindings can uniformly treat it as a plain String.)
There is no semantic change, and the generated files are identical
except for a minor change in the (deprecated) Perl
%guestfs_introspection table.
glibc in Fedora is currently configured with `--enable-obsolete-rpc',
so I guess we can see which way the wind is blowing.
(1) This changes our configure script to prefer libtirpc if it is
available.
If libtirpc is _not_ available then:
(a) Headers must be located in <rpc/xdr.h>, or the user must supply
the right CFLAGS.
(b) XDR functions must be located in one of -lportablexdr, -lrpc,
-lxdr, -lnsl or no library at all (ie. -lc), and the user must set
LDFLAGS if needed.
(2) We no longer add these paths automatically to $(CFLAGS)/$(LIBS).
Any part of libguestfs which needs <rpc/*.h> or the xdr_* functions
must use $(RPC_CFLAGS)/$(RPC_LIBS) explicitly.
(3) Previously Mac OS X had a workaround for the broken 64 bit support
in the supplied rpcgen. This workaround "activates" all the time if
you use tirpc, so breaking Linux after the above changes. tirpc is
supported on OS X, so I think it's just better to use that rather than
the broken rpcgen. For that reason I removed the workaround
completely.
Thanks: Roy Keene
Daemon 'proc_nr's have to be assigned monotonically and uniquely to
each daemon function. However in practice it can be difficult to work
out which is the next free proc_nr. Placing all of them into a single
table in a new file (proc_nr.ml) should make this easier.
Sort the functions so the output is stable.
This changes the order in which the C API tests run. Previously we
ran the newest tests first, which was useful when we were frequently
adding new APIs. Now we run them in sorted order.
- Make module name explicit, so it's more obvious which module a
function is defined in.
- Group lines of code by feature.
- Capitalize some module names properly.
Just code motion, no functional change.
