cabal test is inconsistent with its QuickCheck log

59 Views Asked by At

The following example fails (like it should) after 5 attempts:

prop_varName :: VarName -> Bool
prop_varName v = let s = content (getVarNameToken v) in
    length s <= 2 || isPrefixOf "a" s

main :: IO ()
main = quickCheck prop_varName

When I run cabal test I get an incorrect result (PASS)

$ cabal test # partially omitted for brevity ...
Build profile: -w ghc-9.8.1 -O1
Running 1 test suites...
Test suite dhscanner-ast-test: RUNNING...
Test suite dhscanner-ast-test: PASS
Test suite logged to:
dhscanner-ast-0.1.0.0-dhscanner-ast-test.log
1 of 1 test suites (1 of 1 test cases) passed.

But when I check the log, everything is fine (counterexample found)

$ sed -n "1,3p" dhscanner-ast-0.1.0.0-dhscanner-ast-test.log
Test suite dhscanner-ast-test: RUNNING...
*** Failed! Falsified (after 5 tests):
VarName (Named {content = "U\SOH4@", location = ...})

EDIT: Here is the complete cabal file, though in hindsight, it is not related to the actual problem.

cabal-version:      3.0
name:               dhscanner-ast
version:            0.1.0.0
-- synopsis:
-- description:
license:            GPL-3.0-only
license-file:       LICENSE
author:             ********
maintainer:         ***************@*****.com
-- copyright:
build-type:         Simple
extra-doc-files:    CHANGELOG.md
-- extra-source-files:

common warnings
    ghc-options: -Wall

library
    import:           warnings
    exposed-modules:
        Ast,
        Token,
        Location

    -- other-modules:
    -- other-extensions:
    build-depends:
        base ^>=4.19.0.0,
        containers,
        aeson

    hs-source-dirs: src

    default-language: Haskell2010

test-suite dhscanner-ast-test
    import:           warnings
    default-language: Haskell2010
    -- other-modules:
    -- other-extensions:
    type:             exitcode-stdio-1.0
    hs-source-dirs:   test
    main-is:          Main.hs
    build-depends:
        base ^>=4.19.0.0,
        dhscanner-ast,
        QuickCheck,
        random

Note: this is not a duplicate of this question

1

There are 1 best solutions below

0
OrenIshShalom On

Here is the actual code that I ended up using:

main :: IO ()
main = do
{
    result <- quickCheckResult prop_varName;
    case (isSuccess result) of
        True -> exitSuccess
        False -> exitFailure
}

Here is the output from cabal test:

Running 1 test suites...
Test suite dhscanner-ast-test: RUNNING...
*** Failed! Falsified (after 68 tests):
VarName (Named {content = "a\EOT{Z\\u3w+_", ... })
Test suite dhscanner-ast-test: FAIL
Test suite logged to: ...