Linux – grep with colored output returns several empty lines


I have strange issue with grep (2.12-2, debian testing).

In some situations when I do something like this:
grep -rni '."spacer">.' . grep return me several empty lines:

enter image description here

If I add --color=none all lines become visible.

grep options in .bashrc:

export GREP_COLORS='fn=01;34:ms=01;33:ln=33'
alias grep='grep --color=auto'

Best Answer

A very useful tool for debugging this kind of thing is od. Passing the output of your grep command through od showed that your file contains DOS style line endings, carriage return (\r) followed by a new line (\n):

$ grep -i '."spacer">.' default.php | od -c
0000000  \t  \t  \t  \t  \t   <   d   i   v       c   l   a   s   s   =
0000020   "   s   p   a   c   e   r   "   >  \r  \n  \t  \t  \t   <   d
0000040   i   v       c   l   a   s   s   =   "   s   p   a   c   e   r
0000060   "   >  \r  \n

So, to test, I created this minimal test file:

$ echo -ne "<div class=\"spacer\">\r\n<div class=\"spacer\">\r\n" > foo.php
$ cat foo.php 
<div class="spacer">
<div class="spacer">

I confirmed that grep prints empty lines:

$ grep -i '."spacer">.' foo.php 


The reason it is printing empty lines is the carriage return (\r). You are asking grep to find the string spacer"> and the following character. In your file, the following character is \r. Printing \r in the terminal has the effect of clearing the last line printed so it results in displaying an empty line. You can test this with the following command:

$ echo -e "foo\rbar"

What actually happens is that first foo is printed, then deleted because of \r and replaced by bar. Check with od:

$ echo -e "foo\rbar" | od -c
0000000   f   o   o  \r   b   a   r  \n

Now, I don't understand why the colors option of grep is changing things. It must have something to do with how special characters are displayed. In any case, you can fix your problem by removing all \r:

$ sed 's/\r//g' default.php > bar.php

Then, remove the last . from your grep pattern (remember that by default . does not match newlines although it does match \r):

$ grep -ni '."spacer">' bar.php 
103:                <div class="spacer">
222:            <div class="spacer">