Ubuntu Manpage: guestfs-hacking - extending and contributing to libguestfs

Provided by: libguestfs0t64_1.52.2-3ubuntu2_amd64

名前

       guestfs-hacking - extending and contributing to libguestfs

説明

       This manual page is for hackers who want to extend libguestfs itself.

THE SOURCE CODE

Libguestfs source is located in the github repository https://github.com/libguestfs/libguestfs

Large amounts of boilerplate code in libguestfs (RPC, bindings, documentation) are generated. This means
that many source files will appear to be missing from a straightforward git checkout. You have to run
the generator ("./configure && make -C generator") in order to create those files.

Libguestfs uses an autotools-based build system, with the main files being configure.ac and Makefile.am.
See "THE BUILD SYSTEM".

The generator subdirectory contains the generator, plus files describing the API. The lib subdirectory
contains source for the library. The appliance and daemon subdirectories contain the source for the code
that builds the appliance, and the code that runs in the appliance respectively. Other directories are
covered in the section "SOURCE CODE SUBDIRECTORIES" below.

Apart from the fact that all API entry points go via some generated code, the library is straightforward.
(In fact, even the generated code is designed to be readable, and should be read as ordinary code). Some
actions run entirely in the library, and are written as C functions in files under lib. Others are
forwarded to the daemon where (after some generated RPC marshalling) they appear as C functions in files
under daemon.

To build from source, first read the guestfs-building(1).

ソースコードのサブディレクトリー
There are a lot of subdirectories in the source tree! Which ones should you concentrate on first? lib and
daemon which contain the source code of the core library. generator is the code generator described
above, so that is important. The Makefile.am in the root directory will tell you in which order the
subdirectories get built. And then if you are looking at a particular tool (eg. customize) or language
binding (eg. python), go straight to that subdirectory, but remember that if you didn't run the generator
yet, then you may find files which appear to be missing.

align
virt-alignment-scan(1) のコマンドおよびドキュメント。

appliance
libguestfs アプライアンス、スクリプトなどを構築します。

bash
Bash tab-completion scripts.

build-aux
autotools により使用されるさまざまなビルドスクリプト。

builder
virt-builder(1) コマンドおよびドキュメント。

cat The virt-cat(1), virt-filesystems(1), virt-log(1), virt-ls(1) and virt-tail(1) commands and
documentation.

common
Various libraries of internal code can be found in the common subdirectory:

common/edit
Common code for interactively and non-interactively editing files within a libguestfs filesystem.

common/errnostring
The communication protocol used between the library and the daemon running inside the appliance
has to encode errnos as strings, which is handled by this library.

common/mlcustomize
Library code associated with "virt-customize" but also used in other tools.

common/mlgettext
Small, generated wrapper which allows libguestfs to be compiled with or without ocaml-gettext.
This is generated by ./configure.

common/mlpcre
Lightweight OCaml bindings for Perl Compatible Regular Expressions (PCRE). Note this is not
related in any way to Markus Mottl's ocaml-pcre library.

common/mlprogress
OCaml bindings for the progress bar functions (see common/progress).

common/mlstdutils
A library of pure OCaml utility functions used in many places.

common/mltools
OCaml utility functions only used by the OCaml virt tools (like "virt-sysprep", "virt-customize"
etc.)

common/mlutils
OCaml bindings for C functions in "common/utils", and some POSIX bindings which are missing from
the OCaml stdlib.

common/mlvisit
OCaml bindings for the visit functions (see common/visit).

common/mlxml
OCaml bindings for the libxml2 library.

common/options
Common options parsing for guestfish, guestmount and some virt tools.

common/parallel
A framework used for processing multiple libvirt domains in parallel.

common/progress
Common code for printing progress bars.

common/protocol
The XDR-based communication protocol used between the library and the daemon running inside the
appliance is defined here.

common/qemuopts
Mini-library for writing qemu command lines and qemu config files.

common/structs
Common code for printing and freeing libguestfs structs, used by the library and some tools.

common/utils
Various utility functions used throughout the library and tools.

common/visit
Recursively visit a guestfs filesystem hierarchy.

common/windows
Utility functions for handling Windows drive letters.

contrib
外部の貢献、実験的な部分です。

customize
virt-customize(1) command and documentation.

daemon
libguestfs アプライアンスの中で実行され、アクションを実行しているデーモン。

df virt-df(1) のコマンドおよびドキュメント。

dib virt-dib(1) command and documentation.

diff
virt-diff(1) command and documentation.

docs
Miscellaneous manual pages.

edit
virt-edit(1) のコマンドおよびドキュメント。

examples
C API のコード例。

fish
guestfish(1) コマンドラインシェル、および virt-copy-in(1), virt-copy-out(1), virt-tar-in(1),
virt-tar-out(1) のようなさまざまなシェルスクリプト。

format
virt-format(1) のコマンドおよびドキュメント。

fuse
guestmount(1), libguestfs の上に組み立てられた FUSE (ユーザー空間ファイルシステム)。

generator
The crucially important generator, used to automatically generate large amounts of boilerplate C code
for things like RPC and bindings.

get-kernel
virt-get-kernel(1) command and documentation.

inspector
virt-inspector(1), 仮想マシンイメージ検査ツール。

lib C ライブラリーのソースコード。

logo
Logo used on the website. The fish is called Arthur by the way.

m4 M4 macros used by autoconf. See "THE BUILD SYSTEM".

make-fs
virt-make-fs(1) command and documentation.

po シンプルな gettext 文字列の翻訳。

po-docs
The build infrastructure and PO files for translations of manpages and POD files. Eventually this
will be combined with the po directory, but that is rather complicated.

rescue
virt-rescue(1) のコマンドおよびドキュメント。

resize
virt-resize(1) のコマンドおよびドキュメント。

sparsify
virt-sparsify(1) のコマンドおよびドキュメント。

sysprep
virt-sysprep(1) コマンドおよびドキュメント。

tests
テストします。

test-data
Files and other test data used by the tests.

test-tool
エンドユーザーが QEMU/カーネルの組み合わせが libguestfs で動作するかどうかを確認するためのテストツー
ルです。

tmp Used for temporary files when running the tests (instead of /tmp etc). The reason is so that you can
run multiple parallel tests of libguestfs without having one set of tests overwriting the appliance
created by another.

tools
Perl で書かれたコマンドラインツール (virt-win-reg(1) および他の多くのもの)。

utils
Miscellaneous utilities, such as "boot-benchmark".

v2v Up to libguestfs > 1.42 this contained the virt-v2v(1) tool, but this has now moved into a separate
repository: https://github.com/libguestfs/virt-v2v

website
The http://libguestfs.org website files.

csharp
erlang
gobject
golang
haskell
java
lua
ocaml
php
perl
python
ruby
言語バインディング。

THE BUILD SYSTEM
Libguestfs uses the GNU autotools build system (autoconf, automake, libtool).

The ./configure script is generated from configure.ac and m4/guestfs-*.m4. Most of the configure script
is split over many m4 macro files by topic, for example m4/guestfs-daemon.m4 deals with the dependencies
of the daemon.

The job of the top level Makefile.am is mainly to list the subdirectories ("SUBDIRS") in the order they
should be compiled.

common-rules.mk is included in every Makefile.am (top level and subdirectories). subdir-rules.mk is
included only in subdirectory Makefile.am files.

There are many make targets. Use this command to list them all:

make help

EXTENDING LIBGUESTFS

ADDING A NEW API
Because large amounts of boilerplate code in libguestfs are generated, this makes it easy to extend the
libguestfs API.

To add a new API action there are two changes:

1. You need to add a description of the call (name, parameters, return type, tests, documentation) to
generator/actions_*.ml and possibly generator/proc_nr.ml.

There are two sorts of API action, depending on whether the call goes through to the daemon in the
appliance, or is serviced entirely by the library (see "ARCHITECTURE" in guestfs-internals(1)).
"guestfs_sync" in guestfs(3) is an example of the former, since the sync is done in the appliance.
"guestfs_set_trace" in guestfs(3) is an example of the latter, since a trace flag is maintained in
the handle and all tracing is done on the library side.

Most new actions are of the first type, and get added to the "daemon_functions" list. Each function
has a unique procedure number used in the RPC protocol which is assigned to that action when we
publish libguestfs and cannot be reused. Take the latest procedure number and increment it.

For library-only actions of the second type, add to the "non_daemon_functions" list. Since these
functions are serviced by the library and do not travel over the RPC mechanism to the daemon, these
functions do not need a procedure number, and so the procedure number is set to -1.

2. Implement the action (in C):

For daemon actions, implement the function "do_<name>" in the "daemon/" directory.

For library actions, implement the function "guestfs_impl_<name>" in the "lib/" directory.

In either case, use another function as an example of what to do.

3. As an alternative to step 2: Since libguestfs 1.38, daemon actions can be implemented in OCaml. You
have to set the "impl = OCaml ..." flag in the generator. Take a look at daemon/file.ml for an
example.

これらの変更をした後、コンパイルするために "make" を使用してください。

Note that you don’t need to implement the RPC, language bindings, manual pages or anything else. It’s
all automatically generated from the OCaml description.

Adding tests for an API

You can supply zero or as many tests as you want per API call. The tests can either be added as part of
the API description (generator/actions_*.ml), or in some rarer cases you may want to drop a script into
"tests/*/". Note that adding a script to "tests/*/" is slower, so if possible use the first method.

The following describes the test environment used when you add an API test in actions_*.ml.

テスト環境は 4 個のブロックデバイスを持ちます:

/dev/sda 2 GB
テスト用の一般的なブロックデバイス。

/dev/sdb 2 GB
/dev/sdb1 is an ext2 filesystem used for testing filesystem write operations.

/dev/sdc 10 MB
2 つのブロックデバイスが必要となるいくつかのテストにおいて使用されます。

/dev/sdd
ISO with fixed content (see images/test.iso).

To be able to run the tests in a reasonable amount of time, the libguestfs appliance and block devices
are reused between tests. So don't try testing "guestfs_kill_subprocess" in guestfs(3) :-x

Each test starts with an initial scenario, selected using one of the "Init*" expressions, described in
generator/types.ml. These initialize the disks mentioned above in a particular way as documented in
types.ml. You should not assume anything about the previous contents of other disks that are not
initialized.

You can add a prerequisite clause to any individual test. This is a run-time check, which, if it fails,
causes the test to be skipped. Useful if testing a command which might not work on all variations of
libguestfs builds. A test that has prerequisite of "Always" means to run unconditionally.

In addition, packagers can skip individual tests by setting environment variables before running "make
check".

SKIP_TEST_<CMD>_<NUM>=1

eg: "SKIP_TEST_COMMAND_3=1" skips test #3 of "guestfs_command" in guestfs(3).

または:

SKIP_TEST_<CMD>=1

eg: "SKIP_TEST_ZEROFREE=1" skips all "guestfs_zerofree" in guestfs(3) tests.

Packagers can run only certain tests by setting for example:

TEST_ONLY="vfs_type zerofree"

See tests/c-api/tests.c for more details of how these environment variables work.

Debugging new APIs

Test new actions work before submitting them.

新しいコマンドを試すために guestfish を使うことができます。

Debugging the daemon is a problem because it runs inside a minimal environment. However you can fprintf
messages in the daemon to stderr, and they will show up if you use "guestfish -v".

ADDING A NEW LANGUAGE BINDING
All language bindings must be generated by the generator (see the generator subdirectory).

There is no documentation for this yet. We suggest you look at an existing binding, eg.
generator/ocaml.ml or generator/perl.ml.

Adding tests for language bindings

Language bindings should come with tests. Previously testing of language bindings was rather ad-hoc, but
we have been trying to formalize the set of tests that every language binding should use.

Currently only the OCaml and Perl bindings actually implement the full set of tests, and the OCaml
bindings are canonical, so you should emulate what the OCaml tests do.

This is the numbering scheme used by the tests:

- 000+ basic tests:

010 load the library
020 create
030 create-flags
040 create multiple handles
050 test setting and getting config properties
060 explicit close
065 implicit close (in GC'd languages)
070 optargs
080 version
090 retvalues

- 100 launch, create partitions and LVs and filesystems

- 400+ events:

410 close event
420 log messages
430 progress messages

- 800+ regression tests (specific to the language)

- 900+ any other custom tests for the language

To save time when running the tests, only 100, 430, 800+, 900+ should launch the handle.

FORMATTING CODE
Our C source code generally adheres to some basic code-formatting conventions. The existing code base is
not totally consistent on this front, but we do prefer that contributed code be formatted similarly. In
short, use spaces-not-TABs for indentation, use 2 spaces for each indentation level, and other than that,
follow the K&R style.

If you use Emacs, add the following to one of your start-up files (e.g., ~/.emacs), to help ensure that
you get indentation right:

;;; In libguestfs, indent with spaces everywhere (not TABs).
;;; Exceptions: Makefile and ChangeLog modes.
(add-hook 'find-file-hook
'(lambda () (if (and buffer-file-name
(string-match "/libguestfs\\>"
(buffer-file-name))
(not (string-equal mode-name "Change Log"))
(not (string-equal mode-name "Makefile")))
(setq indent-tabs-mode nil))))

;;; When editing C sources in libguestfs, use this style.
(defun libguestfs-c-mode ()
"C mode with adjusted defaults for use with libguestfs."
(interactive)
(c-set-style "K&R")
(setq c-indent-level 2)
(setq c-basic-offset 2))
(add-hook 'c-mode-hook
'(lambda () (if (string-match "/libguestfs\\>"
(buffer-file-name))
(libguestfs-c-mode))))

変更のテスト方法
Turn warnings into errors when developing to make warnings hard to ignore:

./configure --enable-werror

有用なターゲットは次のとおりです:

"make check"
一般的なテスト群を実行します。

This is implemented using the regular automake "TESTS" target. See the automake documentation for
details.

"make check-valgrind"
valgrind にあるテスト群のサブセットを実行します。

See "VALGRIND" below.

"make check-valgrind-local-guests"
ローカルにインストールされた libvirt 仮想マシン (読み込み専用) を使用して、valgrind にあるテスト群の
サブセットを実行します。

"make check-direct"
Runs all tests using default appliance back-end. This only has any effect if a non-default backend
was selected using "./configure --with-default-backend=..."

"make check-valgrind-direct"
Run a subset of the test suite under valgrind using the default appliance back-end.

"make check-with-upstream-qemu"
Runs all tests using a local qemu binary. It looks for the qemu binary in QEMUDIR (defaults to
$HOME/d/qemu), but you can set this to another directory on the command line, eg:

make check-with-upstream-qemu QEMUDIR=/usr/src/qemu

"make check-with-upstream-libvirt"
Runs all tests using a local libvirt. This only has any effect if the libvirt backend was selected
using "./configure --with-default-backend=libvirt"

It looks for libvirt in LIBVIRTDIR (defaults to $HOME/d/libvirt), but you can set this to another
directory on the command line, eg:

make check-with-upstream-libvirt LIBVIRTDIR=/usr/src/libvirt

"make check-slow"
Runs some slow/long-running tests which are not run by default.

To mark a test as slow/long-running:

• Add it to the list of "TESTS" in the Makefile.am, just like a normal test.

• Modify the test so it checks if the "SLOW=1" environment variable is set, and if not set it skips
(ie. returns with exit code 77). If using $TEST_FUNCTIONS, you can call the function "slow_test"
for this.

• Add a variable "SLOW_TESTS" to the Makefile.am listing the slow tests.

• Add a rule to the Makefile.am:

check-slow:
$(MAKE) check TESTS="$(SLOW_TESTS)" SLOW=1

"sudo make check-root"
Runs some tests which require root privileges. These are supposed to be safe, but take care. You
have to run this as root (eg. using sudo(8) explicitly).

To mark a test as requiring root:

• Add it to the list of "TESTS" in the Makefile.am, just like a normal test.

• Modify the test so it checks if euid == 0, and if not set it skips (ie. returns with exit code
77). If using $TEST_FUNCTIONS, you can call the function "root_test" for this.

• Add a variable "ROOT_TESTS" to the Makefile.am listing the root tests.

• Add a rule to the Makefile.am:

check-root:
$(MAKE) check TESTS="$(ROOT_TESTS)"

"make check-all"
Equivalent to running all "make check*" rules except "check-root".

"make check-release"
Runs a subset of "make check*" rules that are required to pass before a tarball can be released.
Currently this is:

• check

• check-valgrind

• check-direct

• check-valgrind-direct

• check-slow

"make installcheck"
Run "make check" on the installed copy of libguestfs.

The version of installed libguestfs being tested, and the version of the libguestfs source tree must
be the same.

Do:

./configure
make clean ||:
make
make installcheck

VALGRIND
When you do "make check-valgrind", it searches for any Makefile.am in the tree that has a
"check-valgrind:" target and runs it.

Writing the Makefile.am and tests correctly to use valgrind and working with automake parallel tests is
subtle.

If your tests are run via a shell script wrapper, then in the wrapper use:

$VG virt-foo

and in the Makefile.am use:

check-valgrind:
make VG="@VG@" check

However, if your binaries run directly from the "TESTS" rule, you have to modify the Makefile.am like
this:

LOG_COMPILER = $(VG)

check-valgrind:
make VG="@VG@" check

In either case, check that the right program is being tested by examining the tmp/valgrind* log files
carefully.

SUBMITTING PATCHES
Submit patches to the mailing list: https://lists.libguestfs.org and CC to rjones@redhat.com.

You do not need to subscribe to the mailing list if you don’t want to. There may be a short delay while
your message is moderated.

国際化 (i18n) サポート
We support i18n (gettext anyhow) in the library.

However many messages come from the daemon, and we don’t translate those at the moment. One reason is
that the appliance generally has all locale files removed from it, because they take up a lot of space.
So we'd have to readd some of those, as well as copying our PO files into the appliance.

Debugging messages are never translated, since they are intended for the programmers.

MISCELLANEOUS TOPICS

HOW OCAML PROGRAMS ARE COMPILED AND LINKED
Mostly this section is "how we make automake & ocamlopt work together" since OCaml programs themselves
are easy to compile.

Automake has no native support for OCaml programs, ocamlc nor ocamlopt. What we do instead is to treat
OCaml programs as C programs which happen to contain these "other objects" ("DEPENDENCIES" in automake-
speak) that happen to be the OCaml objects. This works because OCaml programs usually have C files for
native bindings etc.

So a typical program is described as just its C sources:

virt_customize_SOURCES = ... crypt-c.c perl_edit-c.c

For programs that have no explicit C sources, we create an empty dummy.c file, and list that instead:

virt_resize_SOURCES = dummy.c

The OCaml objects which contain most of the code are listed as automake dependencies (other dependencies
may also be listed):

virt_customize_DEPENDENCIES = ... customize_main.cmx

The only other special thing we need to do is to provide a custom link command. This is needed because
automake won't assemble the ocamlopt command, the list of objects and the "-cclib" libraries in the
correct order otherwise.

virt_customize_LINK = \
$(top_builddir)/ocaml-link.sh -cclib '-lutils' -- ...

The actual rules, which you can examine in customize/Makefile.am, are a little bit more complicated than
this because they have to handle:

• Compiling for byte code or native code.

• The pattern rules needed to compile the OCaml sources to objects.

These are now kept in subdir-rules.mk at the top level, which is included in every subdirectory
Makefile.am.

• Adding OCaml sources files to "EXTRA_DIST".

Automake isn't aware of the complete list of sources for a binary, so it will not add them all
automatically.

MAINTAINER TASKS

MAINTAINER MAKEFILE TARGETS
These "make" targets probably won’t work and aren't useful unless you are a libguestfs maintainer.

make maintainer-commit

This commits everything in the working directory with the commit message "Version $(VERSION).". You must
update configure.ac, clean and rebuild first.

make maintainer-tag

This tags the current HEAD commit with the tag "v$(VERSION)" and one of the messages:

Version $(VERSION) stable

Version $(VERSION) development

(See "LIBGUESTFS VERSION NUMBERS" in guestfs(3) for the difference between a stable and development
release.)

make maintainer-check-authors

Check that all authors (found in git commit messages) are included in the generator/authors.ml file.

make maintainer-check-extra-dist

This rule must be run after "make dist" (so there is a tarball in the working directory). It compares
the contents of the tarball with the contents of git to ensure that no files have been missed from
Makefile.am "EXTRA_DIST" rules.

make maintainer-upload-website

This is used by the software used to automate libguestfs releases to copy the libguestfs website to
another git repository before it is uploaded to the web server.

MAKING A STABLE RELEASE
When we make a stable release, there are several steps documented here. See "LIBGUESTFS VERSION NUMBERS"
in guestfs(3) for general information about the stable branch policy.

• Check "make && make check" works on at least:

Fedora (x86-64)
Debian (x86-64)
Ubuntu (x86-64)
Fedora (aarch64)
Fedora (ppc64)
Fedora (ppc64le)
• Check "./configure --without-libvirt" works.

• Finalize guestfs-release-notes.pod

• http://libguestfs.org/download の下に新しい安定版および開発版のディレクトリーを作成します。

• Edit website/index.html.in.

• Set the version (in configure.ac) to the new stable version, ie. 1.XX.0, and commit it:

./localconfigure
make distclean -k
./localconfigure
make && make dist
make maintainer-commit
make maintainer-tag

• Create the stable branch in git:

git branch stable-1.XX
git push origin stable-1.XX

• Do a full release of the stable branch.

• Set the version to the next development version and commit that. Optionally do a full release of the
development branch.

INTERNAL DOCUMENTATION

       This section documents internal functions inside libguestfs and various utilities.  It  is  intended  for
       libguestfs developers only.

       This section is autogenerated from "/**" comments in source files, which are marked up in POD format.

       These functions are not publicly exported, and may change or be removed at any time.

       __INTERNAL_DOCUMENTATION__

著者

       Richard W.M. Jones ("rjones at redhat dot com")

COPYRIGHT

       Copyright (C) 2009-2023 Red Hat Inc.

LICENSE

BUGS

       To      get      a      list      of      bugs      against      libguestfs,      use      this     link:
       https://bugzilla.redhat.com/buglist.cgi?component=libguestfs&product=Virtualization+Tools

       To      report      a      new       bug       against       libguestfs,       use       this       link:
       https://bugzilla.redhat.com/enter_bug.cgi?component=libguestfs&product=Virtualization+Tools

       When reporting a bug, please supply:

       •   The version of libguestfs.

       •   Where you got libguestfs (eg. which Linux distro, compiled from source, etc)

       •   Describe the bug accurately and give a way to reproduce it.

       •   Run libguestfs-test-tool(1) and paste the complete, unedited output into the bug report.

libguestfs-1.52.2                                  2024-07-08                                 guestfs-hacking(1)

名前

説明