Code structure for continuous testing

Question

I am building a CD pipeline. I am planning about the automated testing piece of it. I plan to do UI, WebService, Security, Perf testing. I have a question about the code structure. So what I plan is to have the tests in the same repo as the code and then have separate repos for the core test frameworks e.g.

Repo Product

• Product Code (Project)
• Integration Tests (Project)
• Functional/e2e Tests (Project)
  • UI Tests (Package)
  • WebSvc Tests (Package)
  • Perf Tests (Package)
  • Sec.Tests (Package)

Repo Test Core

• UI Test Framework Code (Project)
• WebSvc Test Framework Code (Project)
• Perf Test Framework Code (Project)
• Sec Test Framework Code (Project)

Does anyone see any problem with this structure? Any other ideas? Also I am little fuzzy about the what goes in the integration tests vs in the functional tests projects (e.g. WebSvc tests can be a part of both). And where does acceptance tests go (Functional or integration) ? It will be great if someone can point to some example repos or articles on this.

Thanks

Answer 1

I find this structure somewhat irritating.

From the proposed structure I deduct that you want to build your own test frameworks. That sounds fishy to me especially when you want to write 4 of them.

On the other hand you put them all in the same repository, so they seem to be closely related. Again: not necessarily bad/wrong, but really unexpected.

Since apart from the structure I can't find a hint in your question, that gives a good reason to have separate repositories, so I would recommend to use just one, assuming your "testframeworks" are just utilities to test your main project.

The basic rule is things that change together should go together (in one repository). Everything else makes development really cumbersome: change A, install, change B, run, debug, repeat instead of change, run, debug, repeat

Since you mentioned you are not completely clear, what will go where, I recommend the following:

Start with a single project. Write ALL tests in the test directory of that project. Observe if you experience problems. If so adapt. Things you might experience, that trigger extraction of projects:

• Tests run slow and you want to run them separately
• Tests need a deployed application, so they should run after building and installing everything else
• Tests in different modules need access to code that should not live in the main projects so it might end up in a module with test supporting code