What is limiting my libtorrent throughput?

Question

I am running 2 libtorrent peers on Digital Ocean (dedicated CPU, general purpose, 8GB, 2 cores). They are completely private and disconnected from mainline. DHT is disabled and I'm using my own trackers to test a handful of my own torrents. I'm using libtorrent's "SSL torrent" feature.

Only TCP is enabled. uTP in and out are disabled.

Given this setup, I'm able to get only 100 - 130 MB/s throughput between these nodes. If I try multiple torrents in parallel, they don't go anywhere near the 220 MB/s that I know is possible between these machines (verified with iperf3 and python's http.server). Even with multiple torrents, it seems limited to 130 MB/s and the individual torrents share that bandwidth.

Things I've tried:

• Setting the high_performance_seed settings
• Setting active_seeds, active_downloads, active_limits to -1 (unlimited)
• Setting max_out_request_queue and max_allowed_in_request_queue to crazy high levels, which removes the performance alerts posted by libtorrent (but doesn't actually help the transfers get any faster)
• Tried using dedicated CPU, CPU optimized, 8GB, 2 core but no signficant difference.

My questions:

• I've logged the session stats and run them through session_stats_parser.py to get the graphs, but I'm not quite sure how to interpret them. What am I looking for here that could point to a smoking gun?
• high_performance_seed covers everything in http://libtorrent.org/tuning.html and more. Is there something else I could try?
• What can I do to pinpoint exactly where the bottleneck is?
• Is my expectation that these two machines can do close to 200 MB/s over bittorrent unrealistic? Is my core count a limitation? How can I confirm this?

Answer 1

You haven't mentioned anything about the storage on those machines. Monitor your IO utilization with iostat -x 5 and check the specs of your disk(s).

Use ss -tie to get some information about TCP connections, whether it's sender-limited or congestion-limited. Selecting BBR as congestion controller may provide some additional stats there if you have a recent kernel and version of ss.

If neither provides enough insight you could try libtorrent's logging and see if it provides insight about its decisions, e.g. how many requests it queues, how much it's waiting for IO and other things.