pavement

Sed

From FreeBSDwiki
(Difference between revisions)
Jump to: navigation, search
m (example on global sed replacement)
m (Reverted edits by Judy123 (Talk) to last revision by Jimbo)
 
(6 intermediate revisions by 4 users not shown)
Line 1: Line 1:
Short for "Streamline Editor", sed allows you to run a file through it and either match or change data without actually editing the file itself. This can be particularly handy if you need to make a lot of similar changes to a file -- e.g., you find out you consistently put in a wrong hostname in a configuration file, and you need to change your conf file to point to the right server....except the same change needs to be made 100 times. [[sed]] to the rescue.
+
Short for "Stream Editor", '''sed''' allows you to run a stream (for example, the contents of a file, or the output of a program) through it and either match or change data. This can be particularly handy if you need to make a lot of similar changes to a file -- e.g., you find out you consistently put in a wrong hostname in a configuration file, and you need to change your conf file to point to the right server....except the same change needs to be made 100 times. [[sed]] to the rescue.
  
 
Note that the change will only happen on the first instance of the phrase in each line.
 
Note that the change will only happen on the first instance of the phrase in each line.
Line 5: Line 5:
 
The most common usage is to change all instances of a phrase to another; to wit:
 
The most common usage is to change all instances of a phrase to another; to wit:
  
  dave@samizdata:~% cat sed_testfile
+
  dave@samizdata:~% '''cat sed_testfile'''
 
   dave
 
   dave
 
   dave
 
   dave
Line 12: Line 12:
 
   dave
 
   dave
 
   freebsd
 
   freebsd
  dave@samizdata:~% sed s/dave/david/ sed_testfile
+
  dave@samizdata:~% '''sed s/dave/david/ sed_testfile'''
 
   david
 
   david
 
   david
 
   david
Line 19: Line 19:
 
   david
 
   david
 
   freebsd
 
   freebsd
  dave@samizdata:~% more sed_testfile
+
  dave@samizdata:~% '''more sed_testfile'''
 
   dave
 
   dave
 
   dave
 
   dave
Line 28: Line 28:
 
  dave@samizdata:~%
 
  dave@samizdata:~%
  
As you can see, the changes were not actually made to the file. I didn't tell it to put the changes anywhere! Easily fixed with the help of [[redirection]]:
+
As you can see, no changes were made to the file itself - '''sed''' merely wrote the edited stream to standard output.  You can easily put the changed data in a new file with the help of [[redirection]]:
  
  dave@samizdata:~% sed s/dave/david/ sed_testfile > sed_testfile_new
+
  dave@samizdata:~% '''sed s/dave/david/ sed_testfile > sed_testfile_new'''
  dave@samizdata:~% more sed_testfile_new
+
  dave@samizdata:~% '''more sed_testfile_new'''
 
   david
 
   david
 
   david
 
   david
Line 46: Line 46:
 
To give you another example of how useful [[sed]] can be, let's say that you have a DHCP server and want to change a whole segment's range:
 
To give you another example of how useful [[sed]] can be, let's say that you have a DHCP server and want to change a whole segment's range:
  
  samizdata# sed s/10.1.0./10.2.0./ dhcpd.conf > new_dhcpd.conf
+
  samizdata# '''sed s/10.1.0./10.2.0./ dhcpd.conf > new_dhcpd.conf'''
  
 
Will give you a new_dhcpd.conf file with the changes made. You'll want to check the file by hand to make sure that the changes are correct, but you just saved yourself a lot of typing :)
 
Will give you a new_dhcpd.conf file with the changes made. You'll want to check the file by hand to make sure that the changes are correct, but you just saved yourself a lot of typing :)
  
For more info, check out this page: http://www.student.northpark.edu/pemente/sed/
+
'''sed''' can also be used to make changes to a file or files directly, while backing up the original version of the file(s).  This is done using the -i argument, along with the desired suffix for backed up files.  For example:
 +
 
 +
samizdata# '''sed -i .bak s/10.1.0./10.2.0./ dhcpd.conf'''
 +
samizdata# '''ls | grep dhcpd'''
 +
dhcpd.conf
 +
dhcpd.conf.bak
 +
 
 +
Note that the standard delimiter is "/" but it can be just about anything; %, _, or even commas. So changing a file with lots of forward-slashes (/) can be as ugly as
 +
> sed s/\/dev\/am0/\/dev\/gm0/g example.fstab > example.fstab.new
 +
or as simple as
 +
> sed s%/dev/am0%/dev/gm0%g example.fstab > example.fstab.new
 +
 
 +
If you want to use [[regular expression]]s with '''sed''', you should install the [[gsed]] [[port]] - FreeBSD's native '''sed''' does not handle anything more complicated than a wildcard the way you'd expect it to.  For example, say you want to use the '''\b''' (word boundary) regex code to change instances of "david" to "dave", ''without'' breaking any instances of "davidson":
 +
 
 +
samizdata# '''echo david davidson | sed s/david/dave/g'''
 +
  dave daveson
 +
 
 +
Not what we want!  So we'll use the standard '''\b''' (word boundary) regex code to make sure we don't change words that begin with "david":
 +
 
 +
samizdata# '''echo david davidson | sed "s/david\b/dave/g"'''
 +
  david davidson
 +
 
 +
Not what we expected!  But if we install [[gsed]] from [[ports]] and use it instead:
 +
 
 +
samizdata# '''cd /usr/ports/textproc/gsed && make install clean && [[rehash]]'''
 +
samizdata# '''echo david davidson | gsed "s/david\b/dave/g"'''
 +
  dave davidson
 +
 
 +
Much better!  You may be tempted to simply overwrite the base sed with [[gsed]], but carefully consider it before you do so: if you have any utilities installed which depend on FreeBSD's native implementations differences in regex (or flag) handling, you could break them.  This generally should not be a problem, but BACK UP THE ORIGINAL VERSION of FreeBSD's '''sed''' before overwriting it with [[gsed]] if you decide to do so.
 +
 
 +
For more info, check out these pages:  
 +
 
 +
http://www.student.northpark.edu/pemente/sed/
 +
 
 +
http://www.faqs.org/faqs/editor-faq/sed/
 +
 
 +
http://www.grymoire.com/Unix/Sed.html
  
 
[[Category:System Commands]]
 
[[Category:System Commands]]

Latest revision as of 09:14, 28 June 2010

Short for "Stream Editor", sed allows you to run a stream (for example, the contents of a file, or the output of a program) through it and either match or change data. This can be particularly handy if you need to make a lot of similar changes to a file -- e.g., you find out you consistently put in a wrong hostname in a configuration file, and you need to change your conf file to point to the right server....except the same change needs to be made 100 times. sed to the rescue.

Note that the change will only happen on the first instance of the phrase in each line.

The most common usage is to change all instances of a phrase to another; to wit:

dave@samizdata:~% cat sed_testfile
 dave
 dave
 davedave
 jimbo
 dave
 freebsd
dave@samizdata:~% sed s/dave/david/ sed_testfile
 david
 david
 daviddave
 jimbo
 david
 freebsd
dave@samizdata:~% more sed_testfile
 dave
 dave
 davedave
 jimbo
 dave
 freebsd
dave@samizdata:~%

As you can see, no changes were made to the file itself - sed merely wrote the edited stream to standard output. You can easily put the changed data in a new file with the help of redirection:

dave@samizdata:~% sed s/dave/david/ sed_testfile > sed_testfile_new
dave@samizdata:~% more sed_testfile_new
 david
 david
 daviddave
 jimbo
 david
 freebsd
dave@samizdata:~%

You'll also note that it only changed the first instance of the word per line; to change all instances on each line, you'll want to use the "g" modifier (to make global switches) like this:

dave@samizdata:~% sed s/dave/david/g sed_testfile > sed_testfile_new

See the man page for more details.

To give you another example of how useful sed can be, let's say that you have a DHCP server and want to change a whole segment's range:

samizdata# sed s/10.1.0./10.2.0./ dhcpd.conf > new_dhcpd.conf

Will give you a new_dhcpd.conf file with the changes made. You'll want to check the file by hand to make sure that the changes are correct, but you just saved yourself a lot of typing :)

sed can also be used to make changes to a file or files directly, while backing up the original version of the file(s). This is done using the -i argument, along with the desired suffix for backed up files. For example:

samizdata# sed -i .bak s/10.1.0./10.2.0./ dhcpd.conf
samizdata# ls | grep dhcpd
dhcpd.conf
dhcpd.conf.bak

Note that the standard delimiter is "/" but it can be just about anything; %, _, or even commas. So changing a file with lots of forward-slashes (/) can be as ugly as

> sed s/\/dev\/am0/\/dev\/gm0/g example.fstab > example.fstab.new

or as simple as

> sed s%/dev/am0%/dev/gm0%g example.fstab > example.fstab.new

If you want to use regular expressions with sed, you should install the gsed port - FreeBSD's native sed does not handle anything more complicated than a wildcard the way you'd expect it to. For example, say you want to use the \b (word boundary) regex code to change instances of "david" to "dave", without breaking any instances of "davidson":

samizdata# echo david davidson | sed s/david/dave/g
 dave daveson

Not what we want! So we'll use the standard \b (word boundary) regex code to make sure we don't change words that begin with "david":

samizdata# echo david davidson | sed "s/david\b/dave/g"
 david davidson

Not what we expected! But if we install gsed from ports and use it instead:

samizdata# cd /usr/ports/textproc/gsed && make install clean && rehash
samizdata# echo david davidson | gsed "s/david\b/dave/g"
 dave davidson

Much better! You may be tempted to simply overwrite the base sed with gsed, but carefully consider it before you do so: if you have any utilities installed which depend on FreeBSD's native implementations differences in regex (or flag) handling, you could break them. This generally should not be a problem, but BACK UP THE ORIGINAL VERSION of FreeBSD's sed before overwriting it with gsed if you decide to do so.

For more info, check out these pages:

http://www.student.northpark.edu/pemente/sed/

http://www.faqs.org/faqs/editor-faq/sed/

http://www.grymoire.com/Unix/Sed.html

Personal tools