Conceptual questions regarding usage of pact

Question

I am the part of a team that is "pioneering" the usage of pact for our whole company. On this journey we faced many issues that were mostly due to misunderstanding the usage of pact for CDC-Testing. A lot of those questions could be solved over time, but there are still a few main questions I was not able to find any good solution or examples for. Since I think that answering these questions is quite fundamental, I thought trying to reach out to you directly would probably help us.

1. Question: When implementing provider tests, at what "layer" of the application should we implement our tests at?
   Background: When we first started adding CDC tests with pact to our applications, we did functional testing by starting an application context including an in memory database and we set up data in this database. This resulted in tests that were hard to maintain, additionally we were actually doing functional testing and pact is not meant for that. After overthinking the approach of how we want to implement tests several times, we ended up by just testing our boundry which includes the rest interface and (at best) input and output validation.
2. Question: What is the idea behind using multiple provider states?
   Background: Pact supports using multiple provider states to one interaction. We experimented with this feature, but a lot of our developers do not think that there is a big advantage in multiple provider states. Therefore, a lot of our cdc tests with pact have long and complex described provider states. So I think we did not understand the basic concept about this feature.
3. Question: What is the idea behind parameterized provider states?
   Background: Basically the same background as before. We experimented with this feature but a lot of our developers decided not to use it. Because this rejection of the feature is based in not understanding the feature completely, I would like to know what this feature is thought to be used for.
4. Question: How should we handling pact regarding our versioning strategy?
   Background: Currently, handling contracts between applications with semantic versioning is documented in the official pact documentation. We use SNAPSHOT versioning and changing the versioning strategy is currently not an option. Additionally, multiple other teams in our compoany exist that have different versioning strategies. Basically it is not possible to aggree on one strategy in general. What does this mean when we are talking about using cdc with pact in a across our whole company? What problems could that lead to? Will tagging contracts correctly (master, feature, development, production, ...) resolve any problems regarding the versioning?

Answer 1

1. Your provider tests shouldn't be affected by Pact. You should continue writing your unit tests and functional integration tests (by integration I mean tests that make sure the different components within the service work properly together).
   What Pact adds is a new step to your CI build. During your build, you should use pact-provider-verifier which takes the pact created by the consumer, replays it against your provider and compares the actual response to the expected response defined in the pact. This lets you verify that your provider can fulfill the expected interactions defined by the consumer.

2. I think the main idea behind multiple provider states is order and code reusability. Imagine your consumer defines 2 interactions with the following provider states:

   • "A user with ID 123 exists and it has one post"
   • "A user with ID 123 exists and it belongs to the admin group"

   
   Both states share some logic between them and therefore setting up provider state like this can get messy very quickly. Instead, mapping the following states to code that sets up each state separately is much nicer:

   • "A User with ID 123 exists"
   • "User 123 has one post"
   • "User 123 belongs to the admin group"

3. If you look back at the example above, you'll see that the provider is now very coupled with the user with ID 123. If the consumer decided to change that to ID 456, that will break the provider setup implementation. So the idea is to pass "123" as a parameter, rather than in the state description string. The pact would then look something like this:

"providerStates": [{
  "name": "The user exists",
  "params": {"id" : 123}
}]

4. The semantic versioning described in the docs is just a recommendation and a best-practice. I think whatever versioning strategy you choose will work and it doesn't even have to be consistent across services/teams.

Answer 2

In addition to what Simon answered:

1. If you look at the testing pyramid, pact tests sit on the line between unit and integrated tests. But if you can push them down into the unit space, they become easier to maintain. When I write my contact tests, I concentrate on the Contact to make sure it is not doing more.
2. Simon answered 2 and 3 well. The real reason for multiple provider states is someone asked for it. I don’t use them, one is always enough for me.
3. Again, as Simon answered. We introduced the parameters because we noticed there was test data coupling between the consumer and provider tests. The data the consumer was using had to match the data in the provider state so the response the provider generates would be relevant to the consumer. This makes for brittle tests. Before the parameters, the data had to be encoded into the description to break the coupling. I wrote lots of provider state methods that ran regexes over the description to extract the data.
4. The only requirement is to have a unique version for the consumer and provider. They don’t have to use the same scheme. Each published pact needs to be identifiable, as well as the published provider verification result. The versions also need to be sequential, so the broker can tell an older version from a newer one. Semantic versioning fits this well. A timestamp may also work here. Not as easy to read.

Answer 3

First, I'd like to say that it's great that you're taking the initiative to trial out Pact for your whole company :)

We are trying to improve how we communicate Pact as we understand that it's not a simple problem to solve or to relay to other developers. Any suggestions that you could make to improve the documentation/website would be greatly appreciated.

Now, onto the questions:

When implementing provider tests, at what "layer" of the application should we implement our tests at?

Pact essentially tries to replace/enhance integration tests, or what some would consider functional tests on the provider side. However, that nomenclature doesn't really translate well for some companies/teams since some would use 'functional' tests as meaning end to end tests through the browser.

Essentially, Pact is there to replace any tests that you had in the past that hit your provider in a particular way and then validate the output, because that is essentially what Pact does; the main advantage of this is that it's not based on what the developer of the provider thinks those inputs/outputs should, but instead puts the emphasis on how the consumer is actually using it.

Question: What is the idea behind using multiple provider states?

As Simon already stated, multiple provider states are just a way to promote data reuse and to prevent developers from redoing boilerplate code over and over. It's essentially just a way to setup what the data should like in a repeatable fashion instead of wasting time creating the data for every single state. That being said, sometimes your provider is simple enough that it simply won't need this feature.

Question: What is the idea behind parameterized provider states?

Parameters are a quick and easy way to inject some variable data into the state, like an ID, which the interaction might need to check exactly if the ID is exactly the same or you can also use it with multiple states to create a specific situation, like 'create user with id X', then 'disable user with id X'.

Question: How should we handling pact regarding our versioning strategy?

Pact mentions best practices for handling versioning and that is semantic versioning; it has consistently been a great way to understand if a user updates their code/dependencies, if it's a fix, an addition or something breaking.

However, Pact does not enforce this in the slightest and it's really up to you on how you want to do it. In the end, on the broker side, the contract is just 'tagged' with a string. That being said, you might want to consolidate your strategy as this doesn't just affect the provider, but the consumers as well, hence the need for a higher degree of collaboration.

I hope these answers all your questions. As you can see, Pact is fairly open to different use cases and strategies on purpose since we understand that not everyone works the same way, but at the same time, it puts more emphasis on the users to make sure they collaborate efficiently and set a standard for all to use or else it might become messy. Pact gives you enough rope to hang yourself with, so to speak.

Cheers,

M