Linux – grep with colored output returns several empty lines

bashgreplinuxpipe

I have strange issue with grep (2.12-2, debian testing).

In some situations when I do something like this:
grep -rni '."spacer">.' . grep return me several empty lines:

enter image description here

If I add --color=none all lines become visible.

grep options in .bashrc:

export GREP_COLORS='fn=01;34:ms=01;33:ln=33'
alias grep='grep --color=auto'

Best Answer

A very useful tool for debugging this kind of thing is od. Passing the output of your grep command through od showed that your file contains DOS style line endings, carriage return (\r) followed by a new line (\n):

$ grep -i '."spacer">.' default.php | od -c
0000000  \t  \t  \t  \t  \t   <   d   i   v       c   l   a   s   s   =
0000020   "   s   p   a   c   e   r   "   >  \r  \n  \t  \t  \t   <   d
0000040   i   v       c   l   a   s   s   =   "   s   p   a   c   e   r
0000060   "   >  \r  \n
0000064

So, to test, I created this minimal test file:

$ echo -ne "<div class=\"spacer\">\r\n<div class=\"spacer\">\r\n" > foo.php
$ cat foo.php 
<div class="spacer">
<div class="spacer">

I confirmed that grep prints empty lines:

$ grep -i '."spacer">.' foo.php 


$

The reason it is printing empty lines is the carriage return (\r). You are asking grep to find the string spacer"> and the following character. In your file, the following character is \r. Printing \r in the terminal has the effect of clearing the last line printed so it results in displaying an empty line. You can test this with the following command:

$ echo -e "foo\rbar"
bar

What actually happens is that first foo is printed, then deleted because of \r and replaced by bar. Check with od:

$ echo -e "foo\rbar" | od -c
0000000   f   o   o  \r   b   a   r  \n
0000010

Now, I don't understand why the colors option of grep is changing things. It must have something to do with how special characters are displayed. In any case, you can fix your problem by removing all \r:

$ sed 's/\r//g' default.php > bar.php

Then, remove the last . from your grep pattern (remember that by default . does not match newlines although it does match \r):

$ grep -ni '."spacer">' bar.php 
103:                <div class="spacer">
222:            <div class="spacer">