I'm downloading files on a sd card mounted on a raspberry pi using rtorrent, at around 6MB/s (bandwidth cap). The user process doesn't flush the data itself, and the interval at which the disk cache is flushed depends on sysctl values.
Using iostat and nload I can see the network bandwidth and disk write profiles:
20M/s .....x.....x....xx... .....x....xx....xx... ....xx....xx....xx... 0M/s ....xx....xx....xx...
5M/s ..xxx....x.. x.....x. .xxxx...xx..xxx...xxx 0M/s xxxxx...xxx.xxx...xxx
Apparently the IO writer is always paused during flushing. It's a 4-core CPU with 3 cores permanently idle, and top says 100% iowait during flushing.
If I do
while true; do sync; sleep 1; done while downloading, the disk flush is smooth at 6MB every second. And the downloader process isn't choked so bandwidth is constant.
I can get the same effect with:
how often disk cache flushing kicks in – unit is 100's of a sec; default: 500
how old a page must be to be eligible for flushing in 100's of a sec; default: 3000
Setting these to 100 and 200 respectively achieves the same even disk flush profile and constant download bandwidth.
Is there a way to make this tuning automatic, i.e. flush as often as possible to avoid having to block the writer process ? For instance, if download speed is 1MB/s then flushing every 5sec would probably be enough.