How to make do:block and macros context to have the right values and variables when creating a macro in Elixir?

Question

I’m trying to avoid redundant testing (that means, tests that are made in the same way all the time but with slightly different parameters). For example, I have this all the time when I want to test that the call to a controller was ok:

setup do
  count = 3
end

test "responds with right schema", %{conn: conn, swagger_schema: schema} do
    conn
    |> create_authenticated_conn()
    |> get(~p"/something",  %{a: 1})
    |> validate_resp_schema(schema, "Schema")
    |> json_response(200)
    |> assert_length(count)
end

The things that change are the method call (get, post…), the params, the schema to evaluate, and, optionally if I want to make more assertions (as checking the length of the result array).

So, I thought of creating a macro that simplifies all this work, parameterizing the values that need to be parameterized and also allowing a do: block element to add additional checks. I end up with this:

defmacro test_ok(schema_name, method, url, params \\ nil, do: block) do
  quote do
    test "responds with right schema", %{conn: conn, swagger_schema: schema} do
      response =
        conn
        |> create_authenticated_conn()
        |> unquote(method)(unquote(url), unquote(params))
        |> validate_resp_schema(schema, unquote(schema_name))
        |> json_response(200)

      evaluate_response(response, block)
    end

    defp evaluate_response(var!(response), block) do
      unquote(block)
    end
  end
end

If I ignore the response part and the block part (evaluate_response method) all works fine. As soon as I want to add to the macro the block part and pass the response to inside the body, I have two problems. First, let’s put an example of code:

test_ok("Schema", :get, ~p"/something") do
  assert_length(response, count)
end

Then, the first problem is about the response not being recognized inside the block code. And the other is count not being available neither, although in the way normal test macro, is recognized perfectly.

What is my mistake here?

Answer 1

1. Kernel.var!/2 does not magically make the argument visible globally. It just makes it not hygienized. That means if you assign a value to it inside a quote/2 block, it’d be visible from the caller. In your case, you explicitly pass response as a first argument to a call to evaluate_response/2, so var!/2 is a no-op.
2. In the call to evaluate_response/2 you pass an undefined local block variable as a second parameter, which is ignored in the implementation. The whole context from outside a block is not available inside, this is exactly what’s macro hygiene is. In the body of evaluate_response/2 you directly unquote block which has been passed an an argument to test_ok/5.
3. I am unsure about why count should be available anywhere, the setup/2 block is supposed to return map/keyword list, not define local variables.
4. You don’t need to wrap the whole test/2 call with macro (trust me, ExUnit.Case.test/3 does a damn lot of dark magic under the hood which you are going to lose wrapping it.) Instead, wrap the response evaluation as quote do: var!(response) = ... and then call additional stuff using this response variable (which would leak from inside the macro because of var!.
5. ^[bonus] Whenever you need to define a macro, do it using stub calls, specifically while you don’t feel comfortable understanding the difference between calling and callee contextes.