Sed is a simple UNIX program. It does not create and edit files. It only modifies the data that passes through its input and presents the modified data on its output. Despite that, sed is considered to be one of the most powerful UNIX utilities that you will encounter in Linux, especially when it is combined with other programs in your system.
This article aims to teach the basics of how to use this tiny program. Further, it will also give a brief overview of the UNIX philosophy and how that idea is important in understanding how your computer works.
Also read:5 Best Linux-Libre Distributions for Better Security
What Is the UNIX Philosophy?
The UNIX philosophy is the idea that a program should only do a single task and do it well. It encourages you to write simple programs that you can reuse for different functions instead of creating complex programs that you cannot reuse for other tasks.
A good example of this is the program cat. This is a simple utility that only takes inputs and prints it as a stream of text.
The UNIX philosophy allows you to use the program in conjunction with other utilities in the system. In turn, this approach ultimately creates a single interface for your system that you can learn once and reuse for different contexts.
For example, you can feed a cat stream to a program such as grep. This is a simple program that only does a single thing. It looks for a string of text in an input and it prints the lines where that text appears.
On the other hand, you can instead feed the same cat stream to a screen pager such as less. This, in turn, will allow you to scroll through the stream as if it is a single text file.
Also read:6 of the Best Linux Text Editors
How Does Sed Work?
With that being said, sed is a program that read a stream of text and modify it. However, sed only modifies the text stream, which means that it does not directly change the files in your computer. This can be confusing for a beginner since any change that you make in sed will not appear in the disk.
Despite that, sed is still a powerful utility. For example, you can run the following line to read the output of the printf command:
printf "hello\nmaketecheasier\nworld\n" | sed n
Also read:Using du to Free Up Disk Space in Linux
The Basics of Using Sed
Knowing that, it is incredibly easy to use sed for basic tasks. Consider the following example:
sed -n p hello.txt
In this, I used the n command to print the contents of the file “hello”.
1. Selecting and Trimming Text Streams
Further, you can also tell sed to modify this stream of text. For example, you can use the 2q command to tell it to only print the first two lines of the “hello” file:
sed 2q hello.txt
This can be incredibly helpful if you only want to get the first few lines of a text file. However, you can also use the p command to tell sed to only print a range of lines in a file.
sed -n 3,4p hello.txt
In the example above, I used sed to print the third and fourth line of the “hello” file. You can also use the same p command to print non-adjacent lines in your text file. Consider the following example:
sed -n 1~4p hello.txt
With that, I am telling sed to only print the first and fourth line of the “hello” file. This can come in handy if you want to combine multiple lines from different streams together to a single file.
For example, you can use sed with the > operator to write the modified stream to a file.
sed -n 1~4p hello.txt > welcome.txt
Also read:How to Use the dd Command in Linux
2. Removing Text
Another great function of sed is with deleting text in a stream. Unlike printing, this allows you to select the lines of text that you want sed to omit from the stream.
For example, I used the d command to delete the second and fifth line of the hello file.
sed 2~5d hello.txt
One advantage of using this instead of the p command is that this allows you to only remove the line that you do not want. This is useful if you want to either clean up a large file or trim an output to only show the information that you need.
3. Adding New Text
With that, you can also use sed with itself to include new data into your text stream. For example, you can use the a command to append a line of text to the current stream:
sed -n 2p hello.txt | sed a\ "this is a new line"
Further, it is also possible to include entire files into a text stream. In order to do this, you can use the r command which will allow you to read a single file into your current stream. To do that, you can run the following command to include the welcome file after the second line of hello.
sed -n 2p hello.txt | sed rwelcome.txt
Also read:How to Use Emacs As a USENET Reader With Gnus
4. Text Substitution and Stream Manipulation
One brilliant use of sed is with modifying existing streams. This is when you take a text input stream and use sed’s built-in functions to change the content of the stream.
This is arguably the most powerful function of sed. As this gives you the ability to dynamically edit data as it passes through pipes. For example, you can easily edit the output of a program as if it is a text file:
printf "hello\nmaketecheasier\nworld\n" | sed s/world/website/
In this example, I used the s command to substitute the word “world” with “website”. Unlike the previous commands, however, this command uses a specific syntax to work properly.
Knowing that, the general form of the s command looks something like this:
s/[regex]/(string)/{options}
- The first column indicates that this is a substitute command. In most cases, you will only need to use the
scommand to modify a text stream. - The second column indicates the word that you want to replace using structural regular expressions. This is a way of describing words and grammar using general rules. For example, you can describe the words “prints”, “prince” and “primer” by writing
pri[n,m]... - The third column describes the word that you want the old string to be. Unlike the previous column, this column does not use regular expressions. Instead, you can use plain text to describe your new word.
- The last column tells sed to apply any additional options that you need for a substitution. In that, the most common flag that you might want to use is
g. This indicates that the command will replace every instance of the regex with the new string.
Also read:The Beginner’s Guide to Regular Expressions
5. Using Text Substitution with Files
You can apply the s command to manipulate any kind of text stream that you want. This is useful not only because it allows you to modify data on the fly but also because you can use this with almost anything in the Linux command line.
For example, you can use the s command with a text file by adding the file name at the end of the command:
sed s/hello/welcome/ hello.txt
In this, I am telling sed to read the “hello” file and replace the first instance of the word “hello” with “welcome”. I can then do this for every instance of the word “hello” in the file by adding the g option:
sed s/hello/welcome/g hello.txt
Also read:5 Better USENET Readers for Linux
Frequently Asked Questions
Is it possible to write modifications to a file using sed?
Yes! Aside from using file operators, it is possible to use sed to directly write any modified text stream to a file. To do this, you can either use sed with the -i flag or use the w command to write the stream to a file.
One important thing to note, however, is that these commands do not comply with the POSIX standard. This means that these will not work if you are using a non-GNU version of sed.
I am getting an "unterminated" error whenever I use sed. Is my installation broken?
Not at all! This is an issue that happens when you do not write a sed command properly. This is common with substitution commands where you are trying to either match or replace a string with spaces.
For example, running the following command will result in an “unterminated” error: sed s/hello/welcome back/g hello.txt.
You can fix this by adding the “\” escape character before your spaces. This is a special character that tells sed to include the space when either searching or replacing text.
With that, the correct version of the command above looks something like this: sed s/hello/welcome\ back/g hello.txt.
Aside from "g", are there other flags that I can use with the "s" command?
Yes! Aside from g, you can also use the p flag to tell sed to display any modified lines.
Another flag that you can use with s is e. This allows you to run shell commands every time sed replaces a word. For example, you can run the following command to run touch every time it finds the word “hello” in the file: sed s/hello\ /touch\ /e hello.txt.
Lastly, you can also use the i flag to tell sed to be case insensitive when looking for matches. This is useful if you want to match the same word but with different case styles. For example, the following command will replace the word “hello” with “welcome” regardless of its case: sed s/HELLO/welcome/i hello.txt.
If all this talk made you curious with the Linux command line. You can read our earlier article where we talk about some of the most interesting Bash prompts that you can use today.
Image credit: Unsplash
Our latest tutorials delivered straight to your inbox
