Elixir HTTPoisin.get avoid dedup in response data

Question

I am making call to HTTP Rest API in elixir

url = "http://localhost:8080/getScoreData"
   case HTTPoison.get(url) do
     {:ok, %{status_code: 200, body: body}} ->
       Logger.info("body is #{inspect(body)}")
       overall_score = Jason.decode!(body, as: [%OverallScore{}])
       {:ok, overall_score}
   end

which in the web browser returns

{
      "avgPass": 85.55,
      "avgFail": 14.45,
      "totalStudents": 80.0,
      "myScoreSchema": [
        {
          "average": 80.0,
          "count": 8.0,
          "percent": 80.0,
          "name": "John"
        },
        {
          "average": 0.0,
          "count": 0.0,
          "percent": 0.0,
          "name": "Cena"
        },
        {
          "average": 0.0,
          "count": 0.0,
          "percent": 0.0,
          "name": "Sunny"
        },
        {
          "average": 0.0,
          "count": 0.0,
          "percent": 0.0,
          "name": "Michael"
        }
      ]
    }

but the log as of the line Logger.info("body is #{inspect(body)}") from above code dedups the data and returns below data instead

{
      "avgPass": 85.55,
      "avgFail": 14.45,
      "totalStudents": 80.0,
      "myScoreSchema": [
        {
          "average": 80.0,
          "count": 8.0,
          "percent": 80.0,
          "name": "John"
        },
        {
          "average": 0.0,
          "count": 0.0,
          "percent": 0.0,
          "name": "Cena Sunny Michael"
        }
      ]
    }

Even though this is a smart feature but I don't want this dedup feature. How to avoid the dedup.

Answer 1

To eliminate whether or not HTTPoison is really deduping or not, you can use elixir to get the raw response back first:

defmodule Testmodule.HttpClient do
  def send_request(request, callback_host, callback_port) when is_integer(callback_port) do

    # Convert to charlist, as that's what the erlang function is expecting.
    some_host_in_net = callback_host |> String.to_charlist

    {:ok, socket} =
      :gen_tcp.connect(some_host_in_net, callback_port, [:binary, packet: :raw, active: false])

    :ok = :gen_tcp.send(socket, request)

    # Check response status
    case :gen_tcp.recv(socket, 0) do
      {:ok, response} -> :ok = :gen_tcp.close(socket)
        response
      {:error, :closed} -> IO.puts "Something went wrong."
    end

  end
end

Define your test request with:

def request_pool do
    """
    GET /getScoreData HTTP/1.1
    Host: localhost
    User-Agent: ExampleBrowser/1.0
    Accept: */*

    messagebody here
    """
  end

And finally, you can use Elixir/Erlang to call your endpoint:

Testmodule.HttpClient.send_request(request, "127.0.0.1", 8080)

This should get you what you're seeing in the browser, at which point you can parse it however your heart desires.

But as soon as you try to convert it using Poison.decode, keep in mind that it likely will dedupe your data, as it might treat duplicated keys as updates to the values, rather than allowing duplicate keys to exist in a map.