Gentoo Development Guide

The Basics of Autotools

Todo

This is too long for General Concepts. It needs to be split up and moved somewhere, either to a top-level of its own or into Appendices.

An understanding of GNU autotools (automake, autoconf etc.) can be useful when working with ebuilds:

Major Autotools Components

Autotools is a collection of related packages which, when used together, remove much of the difficulty involved in creating portable software. These tools, together with a few relatively simple upstream-supplied input files, are used to create the build system for a package.

How autotools fits together

A basic overview of how the main autotools components fit together.

In a simple setup:

  • The autoconf program produces a configure script from either configure.in or configure.ac (see note below).
  • The automake program produces a Makefile.in from a Makefile.am.
  • The configure script is run to produce one or more Makefile files from Makefile.in files.
  • The make program uses the Makefile to compile the program.

Note

The configure.in name used to be standard. However, the GNU documentation now recommends configure.ac as it is more obvious which program should be used when processing it. The files perform the same purpose and have the same format — the only difference is the name.

You may see autotools being used in a variety of phase functions. Prior to EAPI2, the QA team preferred that the source code be manipulated in src_unpack — the rationale being that src_unpack handles "getting the package ready to be compiled."

EAPI2, however, introduced a new phase function: src_prepare. This is now the appropriate place to manipulate the source code prior to configuration and compilation. In particular, src_prepare is called before src_configure, which usually expects the configure script to exist.

The autoreconf tool supposedly runs autoconf (and automake, autoheader, aclocal, autopoint and libtoolize) as necessary. Sometimes it works. Some packages ship a shell script named autogen.sh which does the same thing (this is not related to autogen). The autotools.eclass contains helper functions for the stand-alone tools with their corresponding names e.g. eautoconf and eautomake.

Warning

You must not attempt to modify any of the generated files in between running configure and make. This can lead to autotools trying to be clever and recreate the files, in turn leading to your changes being removed. In some situations this will also result in ./configure arguments being silently dropped, which can give broken dependencies. The best way to proceed is usually to work with the .ac / .in files instead.

Simple Autotools Patching Example

The following snippet illustrates the correct way to proceed after patching either Makefile.am or configure.ac:

EAPI="5"

inherit autotools

src_prepare() {
    # Remove problematic LDFLAGS declaration
    sed -i -e '/^LDFLAGS/d' src/Makefile.am || die

    # Rerun autotools
    einfo "Regenerating autotools files..."
    WANT_AUTOCONF=2.5 eautoconf
    WANT_AUTOMAKE=1.9 eautomake
}

src_compile() {
    econf $(use_enable nls)
    emake
}

The einfo message before running autotools is not mandatory. However, these steps can sometimes take a while and may produce no output, so it may make sense to let the user know that something is still happening. See Messages.

The configure.ac File

The configure.ac file is used to create the ./configure script. It consists of a series of macros which are processed and expanded by autoconf. These macros can check for packages and libraries, handle --enable and --with switches, and generate various files.

Basic Format of configure.ac

The configure.ac file is a basic text file. Indenting and whitespace are largely irrelevant. Comments are indicated by the string dnl (dnl is actually a macro which discards the remainder of the input — it stands for "discard new line").

If the previous sentence made you uneasy, you should probably stop reading this page. It gets far worse.

A typical file might start with something like the following:

dnl Process this file with autoconf
AC_INIT(appname, 0.1)
AC_PREREQ(2.5)
AC_CONFIG_SRCDIR(src/main.c)
AC_CONFIG_AUX_DIR(config)
AM_INIT_AUTOMAKE(1.8)

The AC_PREREQ line, if present, tells you which autoconf version you need. This is useful, because autoconf is not compatible between versions. In the above example, we need =autoconf-2.5*. and we should export WANT_AUTOCONF="2.5" (or use the autoconf-2.59 script) when calling autoconf.

The AM_INIT_AUTOMAKE line tells us which automake version we need. Again, there is little chance that an automake-1.7 script will work properly with automake-1.8. Setting WANT_AUTOMAKE="1.8" in the environment can be used to make an unversioned automake call run the correct version.

Note

The WANT_ variables are a Gentoo feature that originally came from Mandrake. Other distributions may handle things differently.

Usually, some standard checks will follow:

dnl Check for toolchain and install components
AC_PROG_CC
AC_PROG_INSTALL
AC_PROG_LN_S
AC_PROG_RANLIB

For non-standard applications, you may also see manual checks:

dnl Check for sed
AC_CHECK_PROGS(regex_cmd, sed)
if test x$regex_cmd = "x" ; then
    AC_MSG_ERROR([sed is required to build the data files.])
fi

You may also see checks for compiler features:

dnl Check that our compiler can do const and inline
AC_C_CONST
AC_C_INLINE

Library and header checks:

dnl Check for standard headers:
AC_HEADER_STDC
AC_HEADER_DIRENT
AC_CHECK_HEADERS([stdlib.h stdio.h libintl.h locale.h])

dnl Check for libraries:
AC_CHECK_LIB(ssl, SSL_free)

And function checks:

dnl Check for functions:
AC_CHECK_FUNCS([setlocale strtol])

Often these will be all jumbled together without any useful comments. In some cases, many of the checks done won't even be necessary for the application in question — most autotools code is copy/pasted rather than written from scratch, and autoscan (a tool which helps write configure.ac) is sometimes over-eager to add in checks.

The file will finish with some output functions:

AM_CONFIG_HEADER(config.h)
AC_OUTPUT(Makefile src/Makefile nls/Makefile man/Makefile tests/Makefile)

These are used to make the ./configure script generate the relevant files.

Enable and Disable Checks

So far we've only seen 'hard' dependencies. Many packages have optional support for various extras (graphics toolkits, libraries which add functionality, interpreters, features, ...). This is (if we're lucky) handled via --enable-foo and --disable-foo switches to ./configure. which are generated from autoconf rules.

A simple --enable / --disable function might look something like the following:

AC_MSG_CHECKING(--enable-cscope argument)
AC_ARG_ENABLE(cscope,
    [  --enable-cscope         Include cscope interface.],
    [enable_cscope=$enableval],
    [enable_cscope="no"])
AC_MSG_RESULT($enable_cscope)
if test "$enable_cscope" = "yes"; then
  AC_DEFINE(FEAT_CSCOPE)
fi

Sometimes more complicated checks are included based upon whether or not an option is enabled. There are also some predefined macros which include AC_ARG_ENABLE. so grepping configure.ac for AC_ARG_ENABLE may not give a complete list. A better way is to use ./configure --help and check the output.

Important

The third argument is used when a --enable or --disable switch to ./configure is provided, and the fourth is used when such a switch is not passed. A common misconception is that the third is enable and the fourth is disable — this is not the case. You may encounter packages that get this wrong.

A simple way to check that a package is using this macro properly is to install the optional dependency, and then try all of ./configure. ./configure --enable-foo and ./configure --disable-foo. If the second and third runs give the same results, something is wrong. If the first run gives a different result to the second and third, there is a good chance that a misunderstanding of the AC_ARG_ENABLE arguments is to blame.

With and Without Checks

A simple --with / --without check might look like:

AC_MSG_CHECKING([whether to enable foo])
AC_ARG_WITH(foo,
    [  --with-foo           enable foo support],
    with_foo=$withval,
    with_foo=yes)
AC_MSG_RESULT($with_foo)

Again, the third argument is for 'specified' and the fourth for 'not specified', and there are standard macros which include with options.

Automatic Checks

It's possible to write autoconf rules which bypass the manual enable / disable convention (or which just ignore what the user asks for). If your package does this, it must be fixed to avoid dependency problems.

The most common form is packages which simply use AC_CHECK_LIB to decide whether or not to enable a feature. If you find a package which does this, you must change the behaviour.

Quoting Rules for configure.ac

Behind the scenes, autoconf makes heavy use of the m4 macro processor to do the work. The m4 quote characters are set by autoconf to be [ and ] for opening and closing quotes respectively. Using " or ' may produce unexpected results.

To include a literal left square bracket, the easiest thing to do is to use the special string @<:@. For a right bracket, use @:>@.

For example:

AC_MSG_RESULT(the first)
AC_MSG_RESULT([the second])
AC_MSG_RESULT("the third")
AC_MSG_RESULT(@<:@the fourth@:>@)
AC_MSG_RESULT([@<:@the fifth@:>@])

gives:

    the first
    the second
    "the third"
    [the fourth]
    [the fifth]

When in doubt, it is generally safest to quote macro arguments using [ ] rather than leaving things unquoted.

The Makefile.am Files

The Makefile.am file is processed by automake to create Makefile.in, which is in turn processed by configure to create Makefile, which is in turn used by make to build the software.

The basic format is like that of a Makefile. However, you will see various 'special' variables being set, rather than every rule being written manually.

A very simple example:

bin_PROGRAMS = myapp
myapp_SOURCES = myapp_commandline.c myapp.c

All the standard GNU rules will be generated, so make. make clean. make distclean. make dist and so on will all work here.

You may also see some standard Makefile constructs showing up whenever there isn't a standard automake way of handling a certain task. For example:

CLEANFILES = glep31check.1
man_MANS = glep31check.1
EXTRA_DIST = glep31check.in

%.1 : %.in
    @regex_cmd@ -e "s,\@VERSION\@,$(VERSION),g" $? > $@

Here, the @regex_cmd@ variable will be substituted with whatever configure detects (sed in this case) when creating the Makefile. This is handled via the macro AC_SUBST(VARNAME) in configure.ac.

Makefile Variables

Sometimes, badly behaved Makefile.am files will override user variables such as CFLAGS. This must not be allowed — see Not Filtering Variables. There are separate special variables which should be used in these situations — for setting CFLAGS. for example, a Makefile.am should use AM_CFLAGS so that user preferences are not ignored.

So, if a Makefile.am contains, say:

CFLAGS="-Wall"

You should sed or patch it to use:

AM_CFLAGS="-Wall"

Remember to manually run autoconf then automake if you do this.

The config.h.in File

The config.h.in file is generated by autoheader. You shouldn't have to worry about this usually, but occasionally you may need to run autoheader manually as part of the build process if upstream do not ship a pre-generated version.

aclocal and m4 Files

In the configure.ac or configure.in files you can call not only the default macros defined by autoconf and automake. but also other functions which can be defined by specific packages like libraries and programs to check for their features in the correct manner.

Those functions are (usually) defined in m4 files placed in the /usr/share/aclocal directory by packages. Problems can arise when you need to regenerate the autotools files, because the functions used by the configure.ac file can be defined in an m4 macro file which isn't installed on the user's system. This is the case for example for some optional features which require libraries and are disabled by USE flags. If the m4 files aren't installed in the user's system, the autoconf step will fail.

To resolve this, most packages just ship the m4 macro files needed for their configure.ac in an m4 subdirectory in the source package. Unfortunately, not all m4 directories are complete, even if they are present.

In those cases you need to find out the m4 file, usually installed by the dependency in /usr/share/aclocal. and make sure that the src_unpack stage both makes available those files to the autotools and calls aclocal to recreate the aclocal.m4 file which is used by autoconf when creating the configure script.

Usually it's more than one m4 file which is missing, so you probably want to package them in a tarball, and add it to SRC_URI. After making sure that the tarball is extracted somewhere in ${WORKDIR} (say, in a gentoo-m4 directory), you usually have two general ways to handle those macro files, one for packages which ships with an incomplete m4 directory and one for packages which ships without the m4 directory.

In the first case you usually want to do something like:

einfo "Regenerating autotools files..."
cp "${WORKDIR}/gentoo-m4" "${S}/m4" || die "m4 copy failed"
WANT_AUTOCONF="2.5" aclocal -I "${S}/m4" || die "aclocal failed"
WANT_AUTOCONF="2.5" autoconf || die "autoconf failed"

and so on. In the second case you can simplify it in this way:

einfo "Regenerating autotools files..."
WANT_AUTOCONF="2.5" aclocal -I "${WORKDIR}/gentoo-m4" || die "aclocal failed"
WANT_AUTOCONF="2.5" autoconf || die "autoconf failed"

without need to copy the files.

It's always better inform upstream when the needed m4 files are missing in the distribution, asking them to add all the needed files in the source tarball to avoid version conflicts if the macro changes.

Libtool

Todo

Stuff about libtool, libtoolize, elibtoolize, libtool.eclass. Which I know almost nothing about... Yay!

Further Autotools Reading

For more details on autotools: