Elixir: Integration test false positive, why is it failing and how to prevent further false positives?

Question

I'm just starting in Elixir, so I'm risking asking something that should be straight forward. I'm sorry if that's something that should be so obvious or easy.

I'll omit some code to keep the question concise, the code is actually for the exercise on the chapter 4 of Elixir in Action.

I wrote a struct with the following structure:

%TodoList{auto_id: Integer, entries: %{}}

I have CRUD functions implemented for that structure, the exercise is to actually import data from a csv file to create a structure with pre-defined data.

I've wrote the following test:

...
  describe "import/1" do
    test "import csv file" do
      result = %TodoList{
        auto_id: 4,
        entries: %{ 1 => %{date: ~D[2018-12-19], title: "Dentist"},
                    2 => %{date: ~D[2018-12-20], title: "Shopping"},
                    3 => %{date: ~D[2018-12-19], title: "Movies"}}}

      assert result = TodoList.CsvImporter.import("todos.csv")
    end
  end
...

Because of a mistake, the code isn't generating the correct structure as of yet, as you can see for the result I get if I run the import/1 function in the repl:

Interactive Elixir (1.12.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> TodoList.CsvImporter.import("todos.csv")
%TodoList{
  auto_id: 4,
  entries: %{
    1 => %{date: {:ok, ~D[2018-12-19]}, title: "Dentist"},
    2 => %{date: {:ok, ~D[2018-12-20]}, title: "Shopping"},
    3 => %{date: {:ok, ~D[2018-12-19]}, title: "Movies"}
  }
}
iex(2)> 

As you can see the error is on generating {:ok, DATE} instead of only DATE inside the map for the date field. I know where is the problem, and how to correct it, as it was a simple mistake in one of the steps to parse the file.

The problem is, when I run the tests, I got a false positive.

I'm running it like this:

elixir -r TodoList.CsvImporter.ex todo_tests.exs

I wanted to show this because the book suggested to create both modules (TodoList and TodoList.CsvImporter) in the same file, and I don't know if this can also be any source of error by ExUnit suit. So I'm running it calling a single file with two modules inside and a second file with the tests written.

What I got is multiple (false!) warnings, and all the tests passing when this one should clearly fail, right?

1. Why is the test not failing?
2. How to prevent false positives like this one?

(I guess that this last question is quite subjective and might not fit Stackoverflow, but any books or resources recommendations on testing would be appreciated?)

Answer 1

This is a typical mistake if you just started with Elixir so don't blame yourself to hard. Read this chapter to catch up, especially the part about the pin operator.

So here you're just assigning the test results to result:

assert result = TodoList.CsvImporter.import("todos.csv")

Try this instead:

assert ^result = TodoList.CsvImporter.import("todos.csv")

Now there should be a MatchError as you would expect.

You can also use the == operator to make the test fail:

assert result == TodoList.CsvImporter.import("todos.csv")

Both methods are useful, it depends on how you want to compare things. == is more strict because you have to specify exactly what the output should be. = can be more lax because sometimes because you're allowed to specify only part of the desired output, like when comparing maps or lists of maps.

Answer 2

As swippie answered, the pin operator or the equality operator should be used.

assert ^result = %{foo: "bar"}
assert result == %{foo: "bar"}

There is a way of preventing these false positives, using the --warnings-as-errors flag. This won't compile your code if you have warnings:

MIX_ENV=test mix do compile --warning-as-errors, test

Alternatively it can be added to the mix.exs file:

elixirc_options: [warnings_as_errors: true]