Get Gentoo!
Development Guide
  1. Master index

Keywording and stabilization

Most ebuilds specify a KEYWORDS variable. This variable is used to indicate the suitability and stability of both the package and the ebuild on each given arch (sparc, ppc, x64-macos, ...).

A sample KEYWORDS entry might look like: 

KEYWORDS="-ia64 ~mips ~ppc sparc x86 ~ppc-macos"

The different levels of keyword are:

arch (x86, ppc-macos) ("stable")
Both the package version and the ebuild are widely tested, known to work and not have any serious issues on the indicated platform.
~arch (~x86, ~ppc-macos) ("testing")
The package version and the ebuild are believed to work and do not have any known serious bugs, but more testing is required before the package version is considered suitable for arch.
No keyword ("unkeyworded")
If a package has no keyword for a given arch, it means it is not known whether the package will work, or that insufficient testing has occurred for ~arch.
-arch (-x86, -ppc-macos)
The package version will not work on the arch. This could be caused by badly written code (for example, non-64-bit or endian clean code), relying upon particular hardware (for example, a BIOS querying tool would not work on non-BIOS architectures) or binary only packages.

The -* keyword is special. It is used to indicate package versions which are not worth trying to test on unlisted arches. For example, a binary-only package which is only supported upstream on ppc and x86 might use: 

KEYWORDS="-* ppc x86"

This is different in implication from "ppc x86" — the former implies that it will not work on other arches, whereas the latter implies that it has not been tested.

Do not use the * or ~* special keywords in ebuilds.

Equal visibility requirement

An ebuild must not depend upon any package that is of a lower keyword level than itself. For example, if foo-1.2 depends upon bar-1.2, and bar-1.2 is ~x86, then foo-1.2 must not be marked stable on x86 unless bar-1.2 is also stabilised.

You may assume that if a user accepts ~arch for a given arch then they also accept arch.

For optional dependencies, all possible dependencies must satisfy the above. Note that certain USE flags can be forcibly disabled on a per-profile basis — talk to the arch teams if you require this. For either-or dependencies, at least one of the options must be of equal or better visibility than the package in question.

Hard masks

The package.mask file can be used to 'hard mask' individual or groups of ebuilds. This should be used for testing ebuilds or beta releases of software, and may also be used if a package has serious compatibility problems. Packages which are not hard masked must not have a dependency upon hard masked packages.

It is good practice to file a bug for ebuilds listed in package.mask to allow feedback to be gathered in one location and to reduce the chance of forgetting about it. Mention the bug number in the mask message.

The only time it is acceptable for a user to see the Possibly a DEPEND problem error message is if they have manually changed visibility levels for a package (for example, through /etc/portage/) and have missed a dependency. You should never commit a change which could cause this error to appear on a user system.

Keywording new packages

Do not assume that your package works on all architectures. Do not assume that user submitted ebuilds will have correct KEYWORDS — chances are they just copied from somewhere else. Do not assume that upstream's 'supported architectures' list is correct. Do not assume that because your code is written in Perl / Python / Java / whatever that it will run on other arches (there is at least one case of a vim script which only worked on x86).

Note that most (non-amd64/x86) arches expect you to be on the arch team and bugzilla alias if you are committing packages with keywords for that arch, and may have additional requirements of which you should be aware (on mips, for example, there are multiple ABIs and byte orders to consider — a package working on your o32 box may not work on o64 or n32). Contact the individual arch teams for details.

It's important to note that alternative arches (like alpha, ia64, s390, sparc, hppa, ppc*) are mainly understaffed arches, some of them are slow, they have more basic problems and have a small userbase. Just file bugs for these architectures when a package is going to be a dependency of a package already keyworded.

Do not commit straight to arch ("stable").

Keywording on upgrades

When upgrading, drop all existing keywords from arch to ~arch ("testing"), and leave any existing ~arch keywords intact. It's easiest and safest to use ekeyword ~all my-new-version.ebuild from the app-portage/gentoolkit package. This must be done even if you think you're just making a trivial fix — there have been several examples of the stable tree getting broken this way.

This also applies to revision bumps, not just to upstream version changes.

If a new version introduces new dependencies which are not available on some architectures, then you should file a bug or ask on IRC before you upgrade the ebuild. If you really need to get the ebuild added in a hurry, for example, for a security fix, then you should drop any KEYWORDS which are causing problems and CC the relevant architectures on the bug — you must file a new bug to the architecture in question regarding this if no bug is already available.

Note that it is preferred to drop keywords on the package and request rekeywording of it together with its new dependencies within the same bug to allow the new code path(s) in your package to be tested. This won't happen if the new dependency is requested for keywording by itself and package.use.mask is used to mask the relevant new USE flag: only the new package dependency will be tested by arch testers. Also, the mask has to be manually removed during the testing process, which is cumbersome.

Moving from ~arch to arch

If a package has stable keywords, maintainers should regularly (subject to the rules below) file stabilization bugs for their packages, ideally approximately every 30 days after a new version is added. If a bug report for stabilization is filed by somebody else, the maintainer should respond with an acknowledgement ("ACK") if the ebuild is ready, and a negative acknowledgement ("NAK") if not.

Previous stable keywords should not be dropped without good cause and it is courteous to ping members of the relevant arch team first. Maintainers must not drop stable keywords simply because they don't have access to a platform: this is what Gentoo's arch teams are here for.

By convention, these bugs are assigned to package maintainers, but the only action expected from maintainers is to acknowledge or reject the stabilization rather than carry out additional testing on each required architecture themselves.

Stabilization, i.e., moving an ebuild from ~arch ("testing") to arch ("stable"), is done by the relevant architecture teams. If you have access to exotic hardware but are not on the arch teams, you may wish to make individual arrangements — the arch teams are happy for help, so long as they know what is going on.

In order to request stabilization of an ebuild, file a bug to the package's maintainer (which may be yourself) in the Stabilization component, and list all secondary maintainers in the bug's CC. When the maintainers consider the ebuild to be ready for stabilization, they will add the relevant architecture teams to the CC list. They can fill the package list field, add the CC-ARCHES keyword, and let NATTkA automatically add arch teams to CC (preferred), or do this manually. That way, teams can remove themselves from the list when they are done, giving a clear indication of which teams still remain to stabilize a package.

It is strongly recommended that NATTkA be used to file keywording or stabilization bugs to ensure that arches are not accidentally forgot about; the automation removes an often error-prone step in the arch-testing process. Ask in #gentoo-dev or on the gentoo-dev mailing list if you need help.

NATTkA will set the sanity-check field on Bugzilla to either - or + depending on whether the package list results in a consistent dependency graph. You may need to add more packages or adjust the arches listed for each package based on the output the bot posts as a comment on the bug. Please note that arch teams may not process or notice bugs with a blank or - sanity-check, so please fix this if it occurs — or ask for help to do so.

You may wish to read NATTkA's documentation for help on package list syntax or other information about the tool. The documentation details syntax which can be used, such as * to represent "all previously stable arches", or ^ to copy the arches from the line(s) above.

For an ebuild to move to stable, the following guidelines must be met (see GLEP 40 for further details):

  • The ebuild has spent a reasonable amount of time in ~arch first. Thirty days is the usual figure, although this is clearly only a guideline. For critical packages, a much longer duration is expected. For small packages that have only minor changes between versions, a shorter period is sometimes appropriate.
  • The ebuild must not have any non-arch dependencies.
  • The package version (and the ebuild) must not have any severe outstanding bugs or issues.
  • The package version must be widely tested.
  • If the package is a library, it should be known not to break any package which depends upon it.

For security fixes, the "reasonable amount of time" guideline may be relaxed. See the Vulnerability Treatment Policy.

Stabilization rules

These are rules for stabilizing packages by yourself on a particular architecture, not for filing bugs to request arch teams to handle it.

  • amd64, x86: If you are the maintainer of a package and own the respective amd64 or x86 hardware, you can do your own testing (stabilization and keywording) of your packages; as long as it is not a core system set dependency. Note that it is acceptable to test x86 using a specialized environment on amd64.
  • arm, arm64: These teams have a strict policy on passing testsuites. You must run test suites for packages with e.g. FEATURES=test for Portage. Keywording or stabilizing packages which fail these tests is avoided but acceptable if the test suite is fragile or failures are known and reported. It is preferred to have to keyword or stabilize more packages on these arches if needed for e.g. test dependencies rather than masking or disabling tests here.
  • sparc: You must have prior permission from the arch lead. Usually we expect you to be on the sparc alias for QA reasons, although other arrangements can be made if you will only be working with a small group of packages.

Exotic architectures (like hppa, ppc*, sparc) are short on help, so it's best if you avoid opening bugs for stabilization of new packages for them, unless it is absolutely necessary (e.g., a reverse dependency for your package).

Some architectures (alpha, ia64, mips, riscv, s390) do not maintain a stable keyword, so packages are not to be marked stable for one of these architectures.

Keeping track of pending stabilizations

Maintainers need some method or system to organize and start pending stabilizations once they become due.

There are several tools available that can help with this:

  • Use imlate from app-portage/gentoolkit
  • Use packages.gentoo.org's maintainer pages (which have a Stabilization tab)
  • Use pkgcheck's StableRequest check, e.g. grep -ri "larry@" */*/metadata.xml -l | cut -d'/' -f1-2 | xargs pkgcheck scan -k StableRequest

Simultaneous stabilization on all architectures

If you maintain an architecture-independent package (data files, icons, pure Python, ...), then you may request that your package be stabilized on all arches at once. To do this — when you are filing the stabilization bug — please add the keyword ALLARCHES and CC the arches that you would like to stabilize.

If your package is architecture-independent, you should add the <stabilize-allarches/> tag to metadata.xml. This allows consistency in future stabilizations and saves arch teams considerable work.

The arch teams, when encountering the ALLARCHES keyword, should perform their usual set of tests on a single convenient architecture. Then, if everything works, stabilize not only the arch that was used during testing, but also all of the other arches in CC on the bug. Afterwards, the CC field can be cleared and the bug closed if appropriate.

Note that first-time ALLARCHES stabilizations are done "as normal" — i.e. all arch teams test individually as if it was not set. This nuance in the procedure is part of why developers should not manually set ALLARCHES in bugs, but instead let NATTkA set it automatically based on metadata.xml.

Removing package versions

When removing ebuild, ensure that you do not remove the most recent version at any given keyword level on any profile. The aim here is:

  • Never to force a downgrade. (Exception: occasionally you really do want to force a downgrade, for example if the newly committed foo-1.3 turns out to be badly broken and that making everyone downgrade to foo-1.2 is the better option. This is rare.)
  • Do not break any existing dependencies.

If you would like a particular package version moved to stable on certain arches so that you can tidy up, file a bug.