How to effectively find bugs of particular pattern during CI in place of code review?

20 Views Asked by At

I have code:

void foo() {
    pageHead = GetPage(pageId)
    ret = do_work_1()
    if ret != ok:
        LeavePage(pageId)
        return
    ret = do_work_2()
+   if ret != ok:
        // LeavePage(pageId) is missing
+       return
    LeavePage(pageId)
}

Colleague Jim is committing code and adds two lines above. Let's say our code has several principles, one of them is that GetPage must be paired with LeavePage, including abnormal branches. But Jim is a newcomer and doesn't know some coding principles. We've set up CI for our repository, and want CI to fail Jim's merge request if he forgets to use LeavePage in above code.

Of course experienced teammates can code review for him and may or may not find this issue, or tell him to add cases to cover that abnormal branch. The second way is not always working, since many MRs are not shipped with enough cases or it's urgent to be merged. Anyway we want CI to help us detect this bug of particular pattern.

By the way, we've written many unit cases for this scenario. But his code is newly added, so it's not covered by present cases.

So by what mean we can stop this issue merging into mainline during CI? We may find this bug afterward, but anyway we want intercept it as early as possible.

We have several coding principles of different pattern, and need newcomers to abide by these principle.

I come up with two ideas:

  1. Use coverage ratio. Once new code is modified (especially newly added), coverage ratio will drop below the expected value. So in this way it will notify committer to add cases during CI. Suppose we've created a testing framework to check the condition of LeavePage is missing, once the committer use that framework to write cases, he'll automatically find that his cases cannot pass if LeavePage is not called. Thus he learns that LeavePage must be called in this situation.

  2. We try to invent a kind of tool, maybe concerning to machine learning. We teach the tool these principles and train it continuously. Finally it can help us stop "wrong code" accurately.

Way 1 may be feasible, but seems to hard to determine proper coverage ratio threshold; Way 2 cost may be too high.

Are there any better ideas?

0

There are 0 best solutions below