How do I test cross-thread queue?

Question

I am not 100% sure that this is SO-adequate question, but I guess it falls under "a specific programming problem". Tips to make it more SO-friendly are welcome.

A bit of context

In DLang there is no default data sharing between threads - instead we use message passing. As safe and clean that approach is, it makes it hard to scale our code horizontaly. Best example is multiple writer - multiple reader problem - it gets quite complicated when using std.concurrency.

Quite common way to solve that problem is to use an in-memory queue - writers push to that queue, readers pull from it, each thread runs on its own pace, and Bob's your uncle. So, I've decided to implement Queue for DLang myself.

The code

Queue has following API:

module javaesque.concurrency;

Queue!T queue(T)(){
    // constructor, out of struct itself for implementation sake
}

struct Queue(T){
    // internals not important for the sake of question

    void push(T val){
    // ...
    }

    T pull(){
    // ...
    }
}

And here's a sample app using that:

// import whatever's needed, stdio, concurrency, etc

void runnable(Queue!string q){
    try {
        while (true) {
            writeln(to!string(thisTid)~" "~q.pull());
        }
    } catch (OwnerTerminated ot) {
    }
}

void main(string[] args){
    Queue!string queue = queue!string();
    spawn(&runnable, queue);
    spawn(&runnable, queue);
    for (int i = 0; i< 20; ++i){
        queue.push(to!string(i));
    }
    readln();
}

Question

OK, so how do I test that? While prototyping I just tested it by running that sample app, but now that I've confirmed that the idea itself may work as expected, I want to write some unit tests. But how?

Please keep in mind that I didn't add dlang or related tags to this question. Even though I've supplied snippets in DLang and the background is highly D-related, I am looking for general help on testing this kind of structures, without constraining myself to this language. Obviously, general answer with DLang-specific addition is welcome, but the question itself should be treated as language-agnostic.

Answer 1

Well, the "generic" approach to testing is two-fold:

• you focus on the public contract of your constructs, and think up testcases that test each aspect of that contract
• you focus on the inner implementation and think of (additional) test cases to get you into specific corner cases

And beyond that: you obviously first test the whole construct in a single threaded manner. You could also look into similar things as a same thread service: you setup your environment to effectively use one thread only.

That might be sufficient for "most" of your code - then you might be fine with some few "integration" tests that actually test one expected end to end scenario (using multiple threads). There you could test for example that your multiple readers receive some expected result in the end.

Finally, from another angle: the key to good unit tests is to write code that can be unit tested easily. You need to be able actually look at your different units in isolation. But if you would provide that code here, that would rather turn into a codereview request (which wouldnt belong here).