src_test

Function src_test
Purpose Run pre-install test scripts
Sandbox Enabled
Privilege user
Called for ebuild

Default src_test

The default test phase in EAPI 6 is equivalent to the following:

src_test() {
	if emake check -n &> /dev/null; then
		emake check
	elif emake test -n &> /dev/null; then
		emake test
	fi
}

Sample src_test

src_test() {
    cd "${S}"/src/testdir || die

    # Test 49 won't work inside a portage environment
    sed -i -e 's~test49.out~~g' Makefile || die

    # Try to run the non-gui tests only
    # pass -j1 if tests do not support being run in parallel
    emake -j1 test-nongui
}

Tests that require network or service access

Sometimes test suites (and other build-time programs) attempt to use remote or local network, or production servers running on the host. All of these are strictly forbidden. Developers should either fix such tests to work in an isolated environment, or disable them completely unless explicitly allowed by the user. At the bare minimum, the tests must not fail with FEATURES=network-sandbox being enabled.

Internet access within the build procedure is forbidden for the following reasons:

Fixing tests that require Internet access usually requires cooperation with upstream, and porting the tests to use test techniques such as mocking or using replay data. For this reason, developers report the issue upstream and skip tests that require network access. It is recommended to explicitly leave a note as to why the tests are skipped, so that other developers can re-enable them locally to run a more complete test suite.

Local server access within the build procedure is additionally forbidden for the following reasons:

Fixing tests that require access to local services is usually done via starting additional isolated instances of those services during the test phase. Those services must either be running on a UNIX socket or on the loopback interface, to reliably prevent remote access.

For all networked services exposed during the test phase (either by the ebuild or the tests themselves), UNIX sockets are strongly preferred over IP sockets as they provide better means for unique naming and access control mechanisms. IP sockets can be subject to port collisions with other local services and they can be accessed by local system users who may exploit a vulnerability through the tests.

Additional protection against those issues is provided through FEATURES=network-sandbox. However, this is only an optional Portage feature relying on specific Linux kernel namespace mechanisms and developers should not rely on it being enabled.

Tests that require X11

Some packages include tests (or other build-time applications) that attempt to use the user's X11 session and fail being unable to connect to it. Those tests need to be fixed to work independently of the X11 server that might or might not be running when packages are being built.

If the program in question does not strictly need X11 but merely attempts to take opportunity of the DISPLAY variable being set, the best solution is to simply unset this variable in the ebuild.

src_test() {
	# tests attempt to connect to X11 and fail when it is set
	# however, they work just fine without X11
	unset DISPLAY

	default
}

If the package actually requires a running X11 server to run the complete test suite, you can use the virtualx eclass to provide an isolated Xvfb environment for the tests to use. It provides a virtual X11 display that is not connected to any physical device and that programs can use reliably.

inherit virtualx

src_test() {
	virtx default
}

Common src_test Tasks

Often the default src_test is fine. Sometimes it is necessary to remove certain tests from the list if they cannot be used with a portage environment. Reasons for such a failure could include:

Usually, removing the relevant test from the Makefile using sed or skipping a particular make target is sufficient.

Try to ensure that tests work properly for your ebuild. A good test suite is extremely helpful for arch maintainers.

Skipping Tests

Sometimes it is necessary to skip tests entirely. This can be done by setting RESTRICT="test" in the ebuild.