Single producer, multiple parallel consumers implemented with boost fibers

Question

I'm trying to write a single producer, multiple consumer pipeline, where the consumers run in parallel threads. Or to find or share a simple example of that sort. With relatively easy code in Go, the output clearly shows the consumers work in parallel. I thought it might be similar with Boost 1.73 fibers, but I can't get beyond this code which (unsurprisingly) works sequentially:

#include <boost/fiber/buffered_channel.hpp>
#include <boost/fiber/fiber.hpp>

static void process(int item) {
    std::cout << "consumer processing " << item << std::endl;
    auto wasteOfTime = 0.;
    for (auto s = 0.; s < item; s += 1e-7) {
        wasteOfTime += sin(s);
    }
    if (wasteOfTime != 42) {
        std::cout << "consumer processed " << item << std::endl;
    }
}

static const std::uint32_t workers = 3;

int main() {
    boost::fibers::buffered_channel<int> channel { 2 };

    boost::fibers::fiber consumer[workers];
    for (int i = 0; i < workers; ++i) {
        consumer[i] = boost::fibers::fiber([&channel]() {
            for (auto item : channel) {
                process(item);
            }
        });
    }

    auto producer = boost::fibers::fiber([&channel]() {
        std::cout << "producer starting" << std::endl;
        channel.push(1);
        channel.push(2);
        channel.push(3);
        channel.close();
        std::cout << "producer ending" << std::endl;
    });

    producer.join();
    for (int i = 0; i < workers; ++i) {
        consumer[i].join();
    }
    return 0;
}

I tried to insert many variations of code fragments to get worker threads to schedule the fibers, but they always execute sequentially or not at all. The code from a question about the inverse problem seems a step in the right direction, though much more complicated than Go, but (when compiled with -DM_1_PI=3.14) that program also just sits idle for me.

Answer 1

Turns out I misinterpreted boost's code fragment for scheduling fibers. It does seem to work for me like this:

#include <boost/fiber/barrier.hpp>
#include <boost/fiber/buffered_channel.hpp>
#include <boost/fiber/fiber.hpp>
#include <boost/fiber/operations.hpp>
#include <boost/fiber/algo/work_stealing.hpp>
#include <optional>

static void process(int item) {
    std::cout << "processing " << item << std::endl;
    auto wasteOfTime = 0.;
    for (auto s = 0.; s < 1; s += 3e-9) {
        wasteOfTime += sin(s);
    }
    if (wasteOfTime != 42) {
        std::cout << "processed " << item << std::endl;
    }
}

static const std::uint32_t N_CONSUMING_FIBERS = 3;

void task() {
    boost::fibers::buffered_channel<int> channel{ 2 };

    boost::fibers::fiber fibers[1 + N_CONSUMING_FIBERS];
    fibers[0] = boost::fibers::fiber([&channel]() {
        std::cout << "producer starting" << std::endl;
        channel.push(1);
        channel.push(2);
        channel.push(3);
        channel.push(4);
        channel.push(5);
        channel.close();
        std::cout << "producer ending" << std::endl;
        });

    auto start = boost::fibers::barrier(N_CONSUMING_FIBERS);
    for (int i = 1; i <= N_CONSUMING_FIBERS; ++i) {
        fibers[i] = boost::fibers::fiber([&start, &channel]() {
            start.wait();
            for (auto item : channel) {
                process(item);
            }
            });
    }

    for (int i = 0; i <= N_CONSUMING_FIBERS; ++i) {
        fibers[i].join();
    }
}

class FiberExecutor {
    static std::vector<std::thread > extra_threads;
    static std::optional<boost::fibers::barrier> barrier;

public:
    static void init() {
        auto const N_THREADS = std::thread::hardware_concurrency();
        barrier.emplace(N_THREADS);
        extra_threads.reserve(N_THREADS - 1);
        for (auto i = decltype(N_THREADS){0}; i < N_THREADS - 1; ++i) {
            extra_threads.emplace_back([N_THREADS]() {
                boost::fibers::use_scheduling_algorithm<boost::fibers::algo::work_stealing>(N_THREADS);
                barrier->wait(); // start only after everyone has scheduled
                barrier->wait(); // end after main thread says to
                });
        }
        boost::fibers::use_scheduling_algorithm<boost::fibers::algo::work_stealing>(N_THREADS);
        barrier->wait(); // start only after everyone has scheduled
    }

    static void exit() {
        barrier->wait();
        for (auto& thread : extra_threads) {
            thread.join();
        }
    }
};

std::vector<std::thread> FiberExecutor::extra_threads;
std::optional<boost::fibers::barrier> FiberExecutor::barrier;

int Main() {
    FiberExecutor::init();

    std::cout << "Going once\n";
    task();
    std::cout << "Going twice\n";
    task();

    FiberExecutor::exit();
    return 0;
}

Beware that this might not be correct. It also does not schedule reliable onto all threads (at least on Windows).

It is in fact pretty much the same as the code in a question about the inverse problem, but that needed an update. Also, I find that using a condition variable instead of barriers isn't reliable, because the main thread may run ahead of the other threads.

I didn't model the initialization and cleanup as a class, to avoid the illusion you could instantiate it multiple times (in sequence or in parallel). work_stealing's constructor uses global variables. You cannot end the fiber-based task cleanly, and then start another fiber-based task, not even if coding everything in temporary threads and not tainting the main thread in any way.