runc/tests/integration
Kir Kolyshkin d5e68ceb0c tests/checkpoint.bats: fix test hang/failure
Commit a9e15e7e0 adds a check that stdin/out/err pipes
are restored correctly. Commit ec260653b7 copy/pastes
the same code to one more another test.

Problem is (as pointed out in commit 5369f9ade3) these tests
sometimes hang. I have also seen them fail.

Apparently, the code used to create pipes and open them to fds
is racy:

```shell
cat $fifo | cat $fifo &
pid=$!
exec 50</proc/$pid/fd/0
exec 51>/proc/$pid/fd/0
```

Since `cat | cat` is spawned asynchronously, by the time exec is used,
the second cat process (i.e. $pid) is already fork'ed but it might
not be exec'ed yet. As a result, we get this (`ls -l /proc/self/fd`):

```
lr-x------. 1 root root 64 Apr 20 02:39 50 -> /dev/pts/1
l-wx------. 1 root root 64 Apr 20 02:39 51 -> /dev/pts/1
```

or, in some cases:
```
lr-x------. 1 root root 64 Apr 20 02:45 50 -> /dev/pts/1
l-wx------. 1 root root 64 Apr 20 02:45 51 -> 'pipe:[215791]'
```

instead of expected set of pipes:

```
> lr-x------. 1 root root 64 Apr 20 02:45 50 -> 'pipe:[215791]'
> l-wx------. 1 root root 64 Apr 20 02:45 51 -> 'pipe:[215791]'
```

One possible workaround is to add `sleep 0.1` or so after cat|cat,
but it is outright ugly (besides, we already have one sleep in
the test code).

The solution is to not use any external processes to create pipes.
I admit this still looks not very comprehensible, but at least it
is easier than before, and it works.

While at it, remove code duplication, moving the setup and check
code into a pair of functions.

Finally, since the tests are working now, remove the skip.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-04-21 02:16:23 -07:00
..
testdata enable integration test on arm64 platform 2017-11-12 22:43:13 -08:00
README.md Fix the problem TESTFLAGS is not to be used in Makefile correctly 2018-07-11 17:50:47 +09:00
cgroups.bats tests/integration: rm kmem from upgrade tests 2020-04-13 16:34:45 -07:00
checkpoint.bats tests/checkpoint.bats: fix test hang/failure 2020-04-21 02:16:23 -07:00
config.json tests: add various !terminal tests 2017-10-25 00:12:21 +11:00
create.bats tests: fix all the things 2016-12-01 15:49:37 +11:00
debug.bats Write logs to stderr by default 2019-04-24 15:18:14 +03:00
delete.bats Ignore error when force deleting a non-existing container 2017-05-16 22:23:00 +02:00
events.bats Enable integration tests in cgroupv2 env 2020-04-13 18:40:08 -07:00
exec.bats Update busybox source and fix runc exec bug 2019-07-07 19:36:23 -07:00
help.bats tests: add rootless integration tests 2017-03-23 20:46:22 +11:00
helpers.bash tests/checkpoint.bats: simplify status checks 2020-04-21 01:55:54 -07:00
kill.bats tests: don't call wait_for_container after synchronous operations 2017-05-02 21:48:07 +03:00
list.bats tests: don't call wait_for_container after synchronous operations 2017-05-02 21:48:07 +03:00
mask.bats tests: don't call wait_for_container after synchronous operations 2017-05-02 21:48:07 +03:00
mounts.bats Fix file permissions for mounts.bats 2020-03-07 09:29:33 +01:00
multi-arch.bash Update busybox source and fix runc exec bug 2019-07-07 19:36:23 -07:00
pause.bats tests: don't call wait_for_container after synchronous operations 2017-05-02 21:48:07 +03:00
ps.bats tests: don't call wait_for_container after synchronous operations 2017-05-02 21:48:07 +03:00
root.bats tests: don't call wait_for_container after synchronous operations 2017-05-02 21:48:07 +03:00
spec.bats Update spec test to use go.mod 2020-03-07 09:29:32 +01:00
start.bats Only allow single container operation 2017-03-08 10:02:39 +08:00
start_detached.bats tests: add tests for rootless multi-mapping configurations 2017-09-09 12:45:33 +10:00
start_hello.bats tests: add tests for rootless multi-mapping configurations 2017-09-09 12:45:33 +10:00
state.bats tests: don't call wait_for_container after synchronous operations 2017-05-02 21:48:07 +03:00
tty.bats runc exec: don't enable terminal unless -t is set 2020-04-11 14:22:07 -07:00
update.bats tests/integration: disable swap tests for v2 2020-04-13 16:34:45 -07:00
version.bats bats: refactor run "$RUNC" -> runc 2016-05-26 19:17:39 +10:00

README.md

runc Integration Tests

Integration tests provide end-to-end testing of runc.

Note that integration tests do not replace unit tests.

As a rule of thumb, code should be tested thoroughly with unit tests. Integration tests on the other hand are meant to test a specific feature end to end.

Integration tests are written in bash using the bats framework.

Running integration tests

The easiest way to run integration tests is with Docker:

$ make integration

Alternatively, you can run integration tests directly on your host through make:

$ sudo make localintegration

Or you can just run them directly using bats

$ sudo bats tests/integration

To run a single test bucket:

$ make integration TESTPATH="/checkpoint.bats"

To run them on your host, you will need to setup a development environment plus bats For example:

$ cd ~/go/src/github.com
$ git clone https://github.com/sstephenson/bats.git
$ cd bats
$ ./install.sh /usr/local

Note: There are known issues running the integration tests using devicemapper as a storage driver, make sure that your docker daemon is using aufs if you want to successfully run the integration tests.

Writing integration tests

[helper functions] (https://github.com/opencontainers/runc/blob/master/test/integration/helpers.bash) are provided in order to facilitate writing tests.

#!/usr/bin/env bats

# This will load the helpers.
load helpers

# setup is called at the beginning of every test.
function setup() {
  # see functions teardown_hello and setup_hello in helpers.bash, used to
  # create a pristine environment for running your tests
  teardown_hello
  setup_hello
}

# teardown is called at the end of every test.
function teardown() {
  teardown_hello
}

@test "this is a simple test" {
  runc run containerid
  # "The runc macro" automatically populates $status, $output and $lines.
  # Please refer to bats documentation to find out more.
  [ "$status" -eq 0 ]

  # check expected output
  [[ "${output}" == *"Hello"* ]]
}