Ubuntu logo

Packaging Guide

4. autopkgtest: pruebas automáticas para paquetes

The DEP 8 specification defines how automatic testing can very easily be integrated into packages. To integrate a test into a package, all you need to do is:

  • añadir un archivo de nombre debian/tests/control que especifique los requisitos para el banco de pruebas,
  • añadir las pruebas en debian/tests/.

4.1. Requisitos del banco de pruebas

In debian/tests/control you specify what to expect from the testbed. So for example you list all the required packages for the tests, if the testbed gets broken during the build or if root permissions are required. The DEP 8 specification lists all available options.

Más adelante vamos a ver el paquete fuente glib2.0. En un caso muy simple el archivo podría tener la siguiente forma:

Tests: build
Depends: libglib2.0-dev, build-essential

Para la prueba de debian/tests/build esto se aseguraría que los paquetes libglib2.0-dev y build-essential están instalados.

Nota

Puede usar @ en la línea Depends para indicar que desea que estén instalados todos los paquetes que son compilados por el paquete fuente en cuestión.

4.2. La prueba real

La prueba asociada al ejemplo anterior podría ser:

#!/bin/sh
# autopkgtest check: Build and run a program against glib, to verify that the
# headers and pkg-config file are installed correctly
# (C) 2012 Canonical Ltd.
# Author: Martin Pitt <martin.pitt@ubuntu.com>

set -e

WORKDIR=$(mktemp -d)
trap "rm -rf $WORKDIR" 0 INT QUIT ABRT PIPE TERM
cd $WORKDIR
cat <<EOF > glibtest.c
#include <glib.h>

int main()
{
    g_assert_cmpint (g_strcmp0 (NULL, "hello"), ==, -1);
    g_assert_cmpstr (g_find_program_in_path ("bash"), ==, "/bin/bash");
    return 0;
}
EOF

gcc -o glibtest glibtest.c `pkg-config --cflags --libs glib-2.0`
echo "build: OK"
[ -x glibtest ]
./glibtest
echo "run: OK"

Aquí una porción de código C muy simple se escribe en un directorio temporal. Luego es compilada con las bibliotecas del sistema (usando los marcadores y rutas de bibliotecas proporcionadas por pkg-config). Luego el binario compilado, que simplemente prueba algunas partes de la funcionalidad del núcleo de glib, se ejecuta.

While this test is very small and simple, it covers quite a lot: that your -dev package has all necessary dependencies, that your package installs working pkg-config files, headers and libraries are put into the right place, or that the compiler and linker work. This helps to uncover critical issues early on.

4.3. Ejecutando la prueba

While the test script can be easily executed on its own, it is strongly recommended to actually use autopkgtest from the autopkgtest package for verifying that your test works; otherwise, if it fails in the Ubuntu Continuous Integration (CI) system, it will not land in Ubuntu. This also avoids cluttering your workstation with test packages or test configuration if the test does something more intrusive than the simple example above.

The README.running-tests (online version) documentation explains all available testbeds (schroot, LXD, QEMU, etc.) and the most common scenarios how to run your tests with autopkgtest, e. g. with locally built binaries, locally modified tests, etc.

The Ubuntu CI system uses the QEMU runner and runs the tests from the packages in the archive, with -proposed enabled. To reproduce the exact same environment, first install the necessary packages:

sudo apt install autopkgtest qemu-system qemu-utils autodep8

Now build a testbed with:

autopkgtest-buildvm-ubuntu-cloud -v

(Please see its manpage and --help output for selecting different releases, architectures, output directory, or using proxies). This will build e. g. adt-trusty-amd64-cloud.img.

Then run the tests of a source package like libpng in that QEMU image:

autopkgtest libpng --- qemu adt-trusty-amd64-cloud.img

The Ubuntu CI system runs packages with only selected packages from -proposed available (the package which caused the test to be run); to enable that, run:

autopkgtest libpng -U --apt-pocket=proposed=src:foo --- qemu adt-release-amd64-cloud.img

or to run with all packages from -proposed:

autopkgtest libpng -U --apt-pocket=proposed --- qemu adt-release-amd64-cloud.img

The autopkgtest manpage has a lot more valuable information on other testing options.

4.4. Más ejemplos

Esta lista no es completa, pero podría ayudarle a hacer una mejor idea de cómo están implementadas y cómo se usan las pruebas automatizadas en Ubuntu.

  • The libxml2 tests are very similar. They also run a test-build of a simple piece of C code and execute it.
  • The gtk+3.0 tests also do a compile/link/run check in the «build» test. There is an additional «python3-gi» test which verifies that the GTK library can also be used through introspection.
  • In the ubiquity tests the upstream test-suite is executed.
  • The gvfs tests have comprehensive testing of their functionality and are very interesting because they emulate usage of CDs, Samba, DAV and other bits.

4.5. Infraestructura de Ubuntu

Packages which have autopkgtest enabled will have their tests run whenever they get uploaded or any of their dependencies change. The output of automatically run autopkgtest tests can be viewed on the web and is regularly updated.

Debian also uses autopkgtest to run package tests, although currently only in schroots, so results may vary a bit. Results and logs can be seen on http://ci.debian.net. So please submit any test fixes or new tests to Debian as well.

4.6. Llevando la prueba a Ubuntu

El proceso de enviar un autopkgtest para un paquete es muy parecido a fixing a bug in Ubuntu. En esencia debe simplemente:

  • ejecutar bzr branch ubuntu:<packagename>,
  • editar el archivo debian/control para activar las pruebas,
  • añadir el directorio debian/tests.
  • write the debian/tests/control based on the DEP 8 Specification,
  • añadir sus casos de prueba a debian/tests,
  • confirmar los cambios, empujarlos a Launchpad, proponer una integración y hacer que la revisen igual que cualquier otra mejora de un paquete fuente.

4.7. Lo que puede hacer

The Ubuntu Engineering team put together a list of required test-cases, where packages which need tests are put into different categories. Here you can find examples of these tests and easily assign them to yourself.

If you should run into any problems, you can join the #ubuntu-quality IRC channel to get in touch with developers who can help you.