Implementing a cold source with Mutiny

Question

I'd like to know, what is a proper way to implement my own cold source (publisher) using the Mutiny library.

Let's say there is huge file parser that should return lines as Multi<String> items according to the Subscriber's consumption rate. New lines should be read only after previous were processed to optimize memory usage, while buffering a couple of hundred items to eliminate consumer idling.

I know about the Multi.createFrom.emitter() factory method, but using it I can't see a convenient way to implement the backpressure.

Does Mutiny have a idiomatic way to create cold sources that produce next items only after requested by the downstream, or in this case I supposed to implement my own Publisher using the Java Reactive Streams API and then wrap it in Multi?

Answer 1

You can use Multi.createFrom().generator(...).

The function is called for every request. But you can pass a "state" to remember where you are, typically an Iterator.

This is the opposite of the emitter approach (which does not check for requests but has a backpressure strategy attached to it).

If you need more fine-grain back-pressure support, you would need to implement a Publisher.