Provided by: autopkgtest_5.49_all 
NAME
autopkgtest - test an installed binary package using the source package's tests
SYNOPSIS
autopkgtest [options...] [testbinary...] testsrc -- virt-server [virt-server-arg...]
DESCRIPTION
autopkgtest runs tests on binary Debian packages, as installed on a system (called "testbed"). The tests
are supplied in the source package.
autopkgtest runs each test supplied by a particular package and reports the results. It drives the
specified virtualisation regime as appropriate, parses the test description metadata, and arranges for
data to be copied to and from the testbed as required.
See /usr/share/doc/autopkgtest/README.running-tests.rst.gz for an introduction about how to use
autopkgtest.
TESTING A DEBIAN PACKAGE
Positional (non-option) arguments specify exactly one source package (containing the test code) and
optionally some binary packages to test.
testsrc can be one of:
.dsc file
Run tests from Debian .dsc source package. By default the package will also be built and the
resulting binaries will be used to satisfy test dependencies; to disable that, specify the
-B/--no-built-binaries option.
source package directory
Run tests from a Debian source tree directory. If that is an unbuilt tree, this is very similar to
specifying a .dsc. If that is a built tree, all test dependencies get satisfied by archive
packages, unless you explicitly specify locally built .debs as well.
Attention: If you just specify a bare directory name which is a legal Debian source package name,
it will be interpreted as the latter (see below). In this case, prefix the directory name with ./.
current directory
If no source package is specified on the command line and the current directory is a Debian source
package, this will be tested.
source package name
Downloads the given source package name with apt-get source in the testbed and run its tests. This
is similar to specifying a .dsc but avoids copying the source from the host to the testbed.
Possibly built binaries (if the test specifies build-needed) will not be used to satisfy
dependencies, as usually in this mode you want to test binaries from a real archive.
git URL or URL#branch
Git-clones the given URL (which must contain an unbuilt Debian source tree) and runs the tests
from that. If branch is given, this branch will be checked out instead of the default (usually
"master"). This can also be a more general refspec such as #refs/pull/123/head" for a GitHub pull
request.
This is very similar to cloning manually and specifying the checkout directory as test; i. e.
this is commonly used with --no-built-binaries. The git package will be installed if necessary.
.changes file
Run tests from the .dsc source package in the given .changes file. If the .changes contains .deb
packages, they will be used for the test. Acts as if you had specified the .debs and .dsc from
the .changes file as explicit arguments. Note that if the .changes contains only debs, the
corresponding .dsc still needs to be specified alongside, or the current directory must be the
source package.
All other positional arguments must be .deb binary packages. They will be used for both build and test
dependencies of the source package. If any binary package is given, then --no-built-binaries is implied.
TEST OPTIONS
-a ARCH | --test-architecture=ARCH
Make the testbed a multiarch system, set up the environment for cross-building (possibly
installing extra build dependencies), and run tests for the specified foreign architecture rather
than for the testbed native architecture. Assumes that ARCH binaries can be run on the testbed,
possibly via qemu-user-binfmt or qemu-user-static.
-B | --no-built-binaries
Binaries from unbuilt source packages (see above) will not be built or ignored, and dependencies
are satisfied with packages from the archive. Note that the source package still gets built if a
test requires build-needed.
--override-control=PATH
Read the test metadata from PATH instead of debian/tests/control.
--test-name=TEST
Run only the given test name (from test control file). If this option is used more than once, all
the named tests are run.
--skip-test=TEST
Skip the given test name (from test control file). If this option is used more than once, all the
named tests are skipped.
LOGGING OPTIONS
If you don't specify any option, autopkgtest only writes its output/results to stderr.
-o dir | --output-dir=dir
Specifies that test artifacts (stderr and stdout from the tests, the log file, built binary
packages etc.) should be placed in the given directory. dir must not exist yet or be empty,
otherwise autopkgtest will refuse to use it.
-l logfile | --log-file=logfile
Specifies that the trace log should be written to logfile instead of to output-dir.
--summary=summary
Specifies that a summary of the outcome should be written to summary. The events in the summary
are written to the log in any case.
-q | --quiet
Do not send a copy of autopkgtest's trace logstream to stderr. This option does not affect the
copy sent to logfile or output-dir. Note that without the trace logstream it can be very hard to
diagnose problems.
TEST BED SETUP OPTIONS
--setup-commands=commands
Run commands before running any test, but after adding and updating APT sources specified via
--add-apt-source, --add-apt-release, or --apt-pocket. This can be used to do anything that isn't
supported by an existing autopkgtest option. If commands is an existing file name, the commands
are read from that; otherwise it is a string with the actual commands that gets run as-is. File
names without directory will be searched in both the current directory and in
/usr/share/autopkgtest/setup-commands/ so you do not need to give the full path for setup scripts
shipped with autopkgtest.
Normally, if the setup commands fail, autopkgtest will consider this a transient testbed error
(exit code 16). However, if the setup commands exit with code 100, autopkgtest will consider this
an "erroneous package" (exit code 12) instead, so this can be used to e. g. detect upgrade errors
to a new version. Note that apt exits with exit code 100 in these cases.
This option can be specified multiple times.
If --user is given or the test bed provides a suggested-normal-user capability, the
$AUTOPKGTEST_NORMAL_USER environment variable will be set to that user.
If the setup commands affect anything in boot directories (like /boot or /lib/systemd/system) and
the testbed supports rebooting, the testbed will be rebooted after the setup commands. This can be
suppressed by creating a file /run/autopkgtest_no_reboot.stamp.
Warning: The way that autopkgtest uses this user means it will be completely trusted by root
inside the testbed.
--setup-commands-boot=commands
Run commands after the --setup-commands, and after every reboot. For example, these commands could
be used to add files in a tmpfs.
These commands never cause the testbed to be rebooted (because that could lead to an infinite
loop). Otherwise, they are just like the --setup-commands.
This option can be specified multiple times.
--add-apt-source='deb|deb-src [ options ] MIRROR SUITE COMPONENT...'
--add-apt-source=ppa:[USER:TOKEN@]OWNER/PPA[:FINGERPRINT]
--add-apt-source=FILENAME
Add the given apt source to the APT sources.list. Specifying this option triggers an apt-get
update. Multiple sources can be specified at once by separating them with a newline, but
specifying this option multiple times is to be preferred instead.
When the ppa: syntax is used, both binary ("deb") and source ("deb-src") entries are added. Also,
when USER and TOKEN are specified, then the PPA in assumed to be private and FINGERPRINT becomes a
mandatory field. Note that autopkgtest can leak TOKEN in many ways, notably when running in debug
mode. If credentials are private then autopkgtest should be run on a trusted platform.
When the FILENAME syntax is used, the contents of the specified file are configured as sources on
the testbed. FILENAME should end with .list or .sources.
--add-apt-release=RELEASE
Add the given RELEASE to the APT sources.list. Mirrors, components and options are taken from the
release appearing first in the existing APT sources.list; any "deb" sources.list entry for this
release will used as a template for adding RELEASE. Both binary ("deb") and source ("deb-src")
entries are added. Specifying this option triggers an apt-get update. This option can be
specified multiple times.
Note that autopkgtest ignores the "NotAutomatic: yes" setting that added releases may declare,
e.g. Debian "experimental", the Debian "backports" pocket, or the Ubuntu "proposed" pocket since
24.04 LTS (Noble Numbat).
--apt-pocket=POCKET[=PACKAGELIST]
Add the given POCKET to the APT sources.list. The pocket "base" release assumed to be the first
configured release in the APT sources.list. This is equivalent to specifying
--add-apt-release=BASE-POCKET where BASE is the pocket base release.
Specifying PACKAGELIST allows to pull only specific packages from the pocket. It is equivalent to
--pin-packages=BASE-POCKET=PACKAGELIST, where BASE is the pocket base release as defined above.
Refer to --pin-packages for more information on the PACKAGELIST syntax.
--pin-packages=RELEASE=PACKAGE[,PACKAGE2,...]
Set up apt pinning to use only specified packages from RELEASE. This can be used for minimizing
dependencies taken from RELEASE so that package updates in that release can be tested
independently from each other for better isolation. If PACKAGE is specified as src:SRCPACKAGE,
pinning will cover all binary packages built by source package SRCPACKAGE.
Attention: This does not currently resolve some situations where dependencies of the given
packages can only be resolved in RELEASE. In this case the apt pinning will be removed and
package installation will be retried with the entirety of RELEASE, unless --no-apt-fallback is
specified.
--no-apt-fallback
Disable the apt-get fallback which is used with --pin-packages or --apt-pocket in case
installation of dependencies fails due to strict pinning.
--apt-upgrade | -U
Run apt-get update, upgrade, autopurge in the testbed before running the tests. Any
--add-apt-source or --apt-pocket options take effect first, so this will upgrade packages from
those sources if appropriate.
--apt-default-release=SUITE
Select the release that should provide the test dependencies that are not explicitly pinned via
--apt-pocket or --pin-packages. Specifying this option is normally not necessary: suitable
pinning it configured automatically. This option is equivalent to setting APT::Default-Release.
--copy=HOSTPATH:TESTBEDPATH
Copy file or directory from host into testbed after opening. This happens before --setup-commands
thus you can use these files in the setup commands.
--env=VAR=value
Set arbitrary environment variable in the build and test. Can be specified multiple times.
--ignore-restrictions=RESTRICTION,RESTRICTION...
If a test would normally be skipped because it has Restrictions: RESTRICTION, run it anyway. Can
be specified multiple times.
For example, you might ignore the restriction isolation-machine when using the null virtualization
server if you know that autopkgtest itself is running on an expendable virtual machine. These
options also work for unknown restrictions, so they can be used when experimenting with new
restrictions.
USER/PRIVILEGE HANDLING OPTIONS
-u user | --user=user
Run builds and tests as user on the testbed. This needs root on the testbed; if root on the
testbed is not available then builds and tests run as whatever user is provided.
--gain-root=gain-root
Prefixes debian/rules binary with gain-root. The default is not to use anything, except that if
--user is supplied or root on the testbed is not available the default is fakeroot.
--insecure
Relaxes file permissions etc., to try to make things not fail with permission errors.
Passing this option might allow tests to corrupt the testbed (if it doesn't support resetting), or
to escape from it, or for untrusted processes on the host or the testbed to take over the account
running autopkgtest.
It is safe to pass --insecure on a dedicated throwaway host.
This partially restores the behaviour of autopkgtest 5.47 and earlier, which had several world-
writable directories.
DEBUGGING OPTIONS
--debug|-d
Include additional debugging information in the trace log. Each additional -d increases the
debugging level; the current maximum is -ddd. If you like to see what's going on, -d or -dd is
recommended.
--shell-fail|-s
Run an interactive shell in the testbed after a failed build, test, or dependency installation.
--shell
Run an interactive shell in the testbed after every test.
TIMEOUT OPTIONS
--timeout=seconds
Define the global timeout, limiting the time allowed to perform the build (if one is required) and
run the tests. If the global timeout is reached during the build, the run will be aborted in the
same way as if the build had failed. If the global timeout is exceeded while running a test, the
test is aborted and any remaining tests are skipped. (default: 0s, meaning a global timeout is not
in effect). The value must be specified as an integer number of seconds.
--timeout-which=seconds
Use a different timeout for operations on or with the testbed. There are five timeouts affected
by five values of which: short: supposedly short operations like setting up the testbed's apt and
checking the state (default: 100s); install: installation of packages including dependencies
(default: 3,000s); test: test runs (default: 10,000s); copy: copy files/directories between host
and testbed (default: 300s); and build: builds (default: 100,000s). The value must be specified
as an integer number of seconds.
--timeout-factor=double
Multiply all of the default timeouts by the specified factor (see --timeout-which above). Only
the defaults are affected; explicit timeout settings are used exactly as specified.
LOCALE OPTIONS
--set-lang=langval
When running commands on the testbed, sets the LANG environment variable to langval. The default
in autopkgtest is to set it to C.UTF-8.
OTHER OPTIONS
--no-auto-control
Disable automatic test generation with autodep8, even if it is installed. In that case, packages
without tests will exit with code 8 ("No tests in this package") just like without autodep8.
--build-parallel=N
Set parallel=N DEB_BUILD_OPTION for building packages. By default this is the number of available
processors. This is mostly useful in containers where you can restrict the available RAM, but not
restrict the number of CPUs.
--needs-internet=run|try|skip
Define how to handle the needs-internet restriction. With "try" tests with needs-internet
restrictions will be run, but if they fail they will be treated as flaky tests. With "skip" these
tests will skipped immediately and will not be run. With "run" the restriction is basically
ignored, this is the default.
-V|--validate
Validate the test control file and exit without running any tests.
-h|--help
Show command line help and exit.
VIRTUALIZATION SERVER
-- virt-server virt-server-arg...
Specifies the virtualisation regime server, as a command and arguments to invoke.
In the autopkgtest invocation, all the arguments and options after -- are passed to the
virtualisation server.
The autopkgtest package provides several virtualisation servers, named autopkgtest-virt-* (e.g.
autopkgtest-virt-qemu, autopkgtest-virt-unshare, ...), each coming with its own manpage. When one if
these virtualisation servers is to be used, the autopkgtest-virt- prefix in the command can be omitted.
The virtualisation server choice depends on several factors, such as the desired level of isolation of
the testbed and the user's preference towards specific tools. If unsure, the qemu virt server offers VM
isolation (thus allowing tests with the isolation-machine restriction) and only relies on QEMU being
available. See autopkgtest-virt-qemu(1) for more information.
OUTPUT FORMAT
During a normal test run, one line is printed for each test. This consists of a short string identifying
the test, some horizontal whitespace, and one of PASS, PASS details, FAIL reason, SKIP reason, or FLAKY
reason where the pass/fail indication is separated by any reason or details by some horizontal
whitespace.
The string to identify the test consists of a short alphanumeric string invented by autopkgtest to
distinguish different command-line arguments, the argid, followed by a hyphen and the test name.
SKIP indicates that a test was not run, or that the test code was started but detected that the test
could not complete, for instance because a required resource was not available.
FLAKY indicates that a test would ordinarily have failed, but because this particular test is known to be
unreliable, the failure was ignored.
Sometimes a SKIP will be reported when the name of the test is not known or not applicable: for example,
when there are no tests in the package, or a there is a test stanza which contains features not
understood by this version of autopkgtest. In this case * will appear where the name of the test should
be.
If autopkgtest detects that erroneous package(s) are involved, it will print the two lines blame:
blamed-thing... and badpkg: message. Here each whitespace-separated blamed-thing is one of arg:argument
(representing a pathname found in a command line argument), dsc:package (a source package name),
deb:package (a binary package name) or possibly other strings to be determined. This indicates which
arguments and/or packages might have contributed to the problem; the ones which were processed most
recently and which are therefore most likely to be the cause of a problem are listed last.
Other possible errors are: erroneous package: message, testbed failure: message, or quitting: message.
CONFIGURATION FILES
If you use lots of options or nontrivial virt server arguments, you can put any part of the command line
into a text file, with one line per option. E. g. you can create a file sid.cfg with contents like
-s
--output-dir=/tmp/testout
--apt-upgrade
--
qemu
/path/to/sid.img
and then run
autopkgtest foo_1_amd64.changes @sid.cfg
The contents of the configuration file will be expanded in-place as if you would have given its contents
on the command line. Please ensure that you don't place spaces between short options and their values,
they would become a part of the argument value.
EXIT STATUS
0 all tests passed
2 at least one test was skipped (or at least one flaky test failed)
4 at least one test failed
6 at least one test failed and at least one test skipped
8 no tests in this package, or all non-superficial tests were skipped
12 erroneous package
14 erroneous package and at least one test skipped
16 testbed failure
20 other unexpected failures including bad usage
SEE ALSO
/usr/share/doc/autopkgtest/README.running-tests.rst.gz
/usr/share/doc/autopkgtest/README.package-tests.rst.gz
AUTHORS AND COPYRIGHT
This manpage is part of autopkgtest, a tool for testing Debian binary packages. autopkgtest is Copyright
(C) 2006-2014 Canonical Ltd.
See /usr/share/doc/autopkgtest/CREDITS for the list of contributors and full copying conditions.
autopkgtest 2014 autopkgtest(1)