Ubuntu – Cat | dd inconsistent behavior

catcommand lineddpipe

From a given a file, I have a requirement to create a copy that is padded with zeros to a specific size.

If you create a file with the following.

echo test >testfile

The output of the following command is inconsistent.

cat testfile /dev/zero | dd bs=256k count=1 status=none | od -c

This is the output that I would expect.

0000000   t   e   s   t  \n  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
1000000

But you also randomly get either of the following.

0000000   t   e   s   t  \n
0000005
0000000   t   e   s   t  \n  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
0400000  \0  \0  \0  \0  \0
0400005

Why does this command have inconsistent behavior?

Even if dd is cutting the pipe off at the end of the first file, The 128k result is strange. I get the same inconsistent results under 16.04, 18.04 and 19.04 systems.

Best Answer

You need to specify full blocks. Try:

cat testfile /dev/zero | dd bs=256k iflag=fullblock count=1 status=none | od -c

Documentation

From man dd:

fullblock
        accumulate full blocks of input (iflag only)

Example

Observe that, without fullblock, the byte counts are inconsistent:

$ cat testfile /dev/zero | dd bs=256k count=1 status=none | wc -c
5
$ cat testfile /dev/zero | dd bs=256k count=1 status=none | wc -c
262144
$ cat testfile /dev/zero | dd bs=256k count=1 status=none | wc -c
262144
$ cat testfile /dev/zero | dd bs=256k count=1 status=none | wc -c
5

With iflag=fullbock, I see consistent full byte counts:

$ cat testfile /dev/zero | dd bs=256k iflag=fullblock count=1 status=none | wc -c
262144
$ cat testfile /dev/zero | dd bs=256k iflag=fullblock count=1 status=none | wc -c
262144
$ cat testfile /dev/zero | dd bs=256k iflag=fullblock count=1 status=none | wc -c
262144
$ cat testfile /dev/zero | dd bs=256k iflag=fullblock count=1 status=none | wc -c
262144
$ cat testfile /dev/zero | dd bs=256k iflag=fullblock count=1 status=none | wc -c
262144
$ cat testfile /dev/zero | dd bs=256k iflag=fullblock count=1 status=none | wc -c
262144
$ cat testfile /dev/zero | dd bs=256k iflag=fullblock count=1 status=none | wc -c
262144
$ cat testfile /dev/zero | dd bs=256k iflag=fullblock count=1 status=none | wc -c
262144