MATLAB: Speed up the “for” loop

convertdecreasefastfor loopimproveloopsize;speed

Hi Everyone,
Could you please help to speed up this piece of code? What it does, it basically converts 50 Hz data to a 5 Hz data, averaging each 10 values of each field. Second and last field names are not under this conversion. And the last # of values of each field is less than ten, therefore , I use last two lines inside the loop to take care of it.
so, in total there are 11 fieldnames, each containing 12666529 values
tic
for n = [1, 3:length(names)-1]
for k = 0:count
data.(names{n})(k+1) = mean(data.(names{n})((1+10*k):(10*(k+1))));
end
data.(names{n})(count + 2) = mean(data.(names{n})(total - (total - (count+1)*10 + 1):total));
data.(names{n})(count+3:total) = [];
toc
end
The time I spent right now is 304 secs:
each loop takes around 25 secs
Elapsed time is 25.251421 seconds.
Elapsed time is 50.271446 seconds.
Elapsed time is 75.096751 seconds.
Elapsed time is 100.987307 seconds.
Elapsed time is 126.373710 seconds.
Elapsed time is 151.917404 seconds.
Elapsed time is 177.191153 seconds.
Elapsed time is 202.475424 seconds.
Elapsed time is 227.890811 seconds.
Elapsed time is 253.500255 seconds.
Elapsed time is 278.895494 seconds.
Elapsed time is 304.218428 seconds.
Elapsed time is 304.268410 seconds.
Thanks a lot! Nurlan

Best Answer

  • You forgot to pre-allocate the results. Letting the arrays grow iteratively requires a huge amount of memory allocations and copies. For 12666529 values the resulting vector grows 1266652 times and this requires sum(1:1266652) * 8 bytes to be allocated and copied: 802 GigaByte!
    So in a first step a pre-allocation is recommended:
    result = zeros(1, count + 1); % Pre-allocate !!!
    value = data.(names{n}); % Faster shortcut
    for k = 0:count
    result(k+1) = mean(value((1+10*k):(10*(k+1))));
    end
    For testing measure the timings with this method at first. Then in a next step the blockwise sum can be calculated in a vectorized form:
    result = sum(reshape(data.(names{n})(1:10*(count+1), 10, []), 1) / 10;