Go to TogaWare.com Home Page. GNU/Linux Desktop Survival Guide
by Graham Williams
Duck Duck Go

Blank Line and Empty Line Removal

20190204 A task we sometimes find useful is to remove all of the blank lines from a file. Suppose we have empty lines separating sentences and want to bring them together into a single paragraph:

$ cat myfile.txt
Linux has many powerful command line tools dating from the 1970's.

A key philosophy of the Unix and the Linux operating systems is that
tools should focus on a specific task.

Tools should then freely collaborate together to achieve much more
than they could on their own.

We can remove blank lines using awk:

$ awk NF < myfile.txt
Linux has many powerful command line tools dating from the 1970's.
A key philosophy of the Unix and the Linux operating systems is that
tools should focus on a specific task.
Tools should then freely collaborate together to achieve much more
than they could on their own.

This very simple use of awk is powerful and elegant in its simplicity. The awk variable NF is the number of fields in the input record. For blank lines this is zero. The default awk action is to print lines for which the pattern is true. Here the pattern is NF. Zero is interpreted as false and so nothing is printed.

We can save the result into a new file:

$ awk NF < myfile.txt > noblanks.txt

To operate on the file in-place we can use sed. First test the command using:

$ sed '/^$/d' myfile.txt

The pattern here will match any empty line since ^ matches the beginning and $ matches the end of the line and allows for no other characters in between. For lines that match the pattern the d operator is utilised to delete the line.

Then we can operate on the file in-place using -i which is short for -in-place.

$ sed -i '/^$/d' myfile.txt

Here's some alternatives and interesting variations:

$ sed '/^ *$/d' myfile.txt           # Blank lines may have space.
$ sed '/^[[:space:]]*$/d' myfile.txt # Or they may include tabs.
$ sed '/^\s*$/d' myfile.txt          # Short version of above.
$ grep . myfile.txt                  # Grep non-empty lines.
$ grep '\S' myfile.txt               # Grep non-space only lines.

See also StackOverflow.

Copyright © 1995-2019 Togaware Pty Ltd
Support further development through the purchase of the PDF version of the book.
Brought to you by Togaware and the author of open source software including Rattle and wajig.
Also the author of Data Mining with Rattle and Essentials of Data Science.