Blank Line and Empty Line Removal

		GNU/Linux Desktop Survival Guide by Graham Williams

CLICK HERE TO VISIT THE UPDATED SURVIVAL GUIDE

Blank Line and Empty Line Removal

20190204 A task we sometimes find useful is to remove all of the blank lines from a file. Suppose we have empty lines separating sentences and want to bring them together into a single paragraph:

$ cat myfile.txt
Linux has many powerful command line tools dating from the 1970's.

A key philosophy of the Unix and the Linux operating systems is that
tools should focus on a specific task.

Tools should then freely collaborate together to achieve much more
than they could on their own.

We can remove blank lines using awk:

$ awk NF < myfile.txt
Linux has many powerful command line tools dating from the 1970's.
A key philosophy of the Unix and the Linux operating systems is that
tools should focus on a specific task.
Tools should then freely collaborate together to achieve much more
than they could on their own.

This very simple use of awk is powerful and elegant in its simplicity. The awk variable NF is the number of fields in the input record. For blank lines this is zero. The default awk action is to print lines for which the pattern is true. Here the pattern is NF. Zero is interpreted as false and so nothing is printed.

We can save the result into a new file:

$ awk NF < myfile.txt > noblanks.txt

To operate on the file in-place we can use sed. First test the command using:

$ sed '/^$/d' myfile.txt

The pattern here will match any empty line since ^ matches the beginning and $ matches the end of the line and allows for no other characters in between. For lines that match the pattern the d operator is utilised to delete the line.

Then we can operate on the file in-place using -i which is short for --in-place.

$ sed -i '/^$/d' myfile.txt

Here's some alternatives and interesting variations:

$ sed '/^ *$/d' myfile.txt           # Blank lines may have space.
$ sed '/^[[:space:]]*$/d' myfile.txt # Or they may include tabs.
$ sed '/^\s*$/d' myfile.txt          # Short version of above.
$ grep . myfile.txt                  # Grep non-empty lines.
$ grep '\S' myfile.txt               # Grep non-space only lines.