BATS assert_failure test not recognising exit 1?

Question

While trying to test a method that checks for 5 seconds whether a GitLab server is running, I am experiencing some difficulties in detecting the error message.

Function checking GitLab server status

check_for_n_seconds_if_gitlab_server_is_running() {
    duration=$1
    echo "duration=$duration"
    running="false"
    if [ "$running" == "false" ]; then
        echo "ERROR, did not find the GitLab server running within $duration seconds!"
        exit 1
    fi
}

Test Code

#!./test/libs/bats/bin/bats

load 'libs/bats-support/load'
load 'libs/bats-assert/load'
load 'libs/bats-file/load'

source src/helper.sh


@test "If error is thrown if the GitLab server is not running within 5 seconds after uninstallation." {
        
    # run the tested method
    check_for_n_seconds_if_gitlab_server_is_running 4
    
    assert_failure
    assert_output --partial "ERROR, did not find the GitLab server running within 4 seconds!"
}

Expected behaviour

I would expect the test to pass, because the exit 1 is reached and I think it throws a failure.

Observed behaviour

When the exit 1 is included, the test fails and the output of the test is:

✗ If error is thrown if the GitLab server is not running within 5 seconds after uninstallation.

When the exit 1 is commented out, the test fails and the output of the test is:

✗ If error is thrown if the GitLab server is not running within 5 seconds after uninstallation.
   (from function `assert_failure' in file test/libs/bats-assert/src/assert.bash, line 140,
    in test file test/long_test_helper.bats, line 17)
     `assert_failure' failed
   duration=4
   ERROR, did not find the GitLab server running within 4 seconds!
   
   -- command succeeded, but it was expected to fail --
   output : 
   --

Question

How can I ensure the test detects error that is thrown?/how should I throw the error/exit 1 command to ensure the assert_failure test passes?

Answer 1

The issue was that I was trying to run a function from the test function, instead of from a separate bash shell. I found out by reproducing another working example that used the run bash -c command, which did behave as expected on the same function. So in practice the following works:

Test code

#!./test/libs/bats/bin/bats

load 'libs/bats-support/load'
load 'libs/bats-assert/load'
load 'libs/bats-file/load'

@test "If error is thrown if the GitLab server is not running within 5 seconds after uninstallation." {
    
    run bash -c "source src/helper.sh && check_for_n_seconds_if_gitlab_server_is_running"
    assert_failure 
    assert_output --partial "ERROR, did not find the GitLab server running within 4 seconds!"
}

Tested function code

check_for_n_seconds_if_gitlab_server_is_running() {
    duration=$1
    echo "duration=$duration"
    running="false"
    if [ "$running" == "false" ]; then
        echo "ERROR, did not find the GitLab server running within $duration seconds!"
        exit 1
    fi
}

Expected behaviour

By calling the function via a the run bash -c command, the test passes:

✓ If error is thrown if the GitLab server is not running within 5 seconds after uninstallation.

1 test, 0 failures

Note

If someone is able to include a MWE that works without calling the function from a separate run bash -c command, but directly as a function from within the test function, please post it as a separate answer.