Sed replace word in file

For a posix compliant alternative, consider replacing word boundary matches (b) by an expanded equivalent ([^a-zA-Z0-9]), also taking into account occurrences at start of line (^) and end of line ($).

However, this quickly becomes impractical if you want to support repeated occurrences of the word to replace (e.g. oldtext oldtext). sed --posix doesn’t recognize expressions such as (^|[^a-zA-Z0-9]), and you can’t make use of lookarounds.

It seems we have to explictly match all possible cases. Here’s a solution to replace mint with minty:

echo 'mint 0mint mint mint0 mint__mint mint__ mint_ -mint mint mint mint_ mint -mint- mint mint mintmint mint' 
  | sed --posix '   
s/^mint$/minty/g;
s/^mint([^a-zA-Z0-9])/minty1/g;
s/([^a-zA-Z0-9])mint$/1minty/g;
s/([^a-zA-Z0-9])mint([^a-zA-Z0-9])mint([^a-zA-Z0-9])mint([^a-zA-Z0-9])/1minty2minty3minty4/g;
s/([^a-zA-Z0-9])mint([^a-zA-Z0-9])mint([^a-zA-Z0-9])/1minty2minty3/g;
s/([^a-zA-Z0-9])mint([^a-zA-Z0-9])/1minty2/g;
'
# minty 0mint minty mint0 minty__minty minty__ minty_ -minty minty minty minty_ minty -minty- minty minty mintmint minty

Introduction

The sed (stream editor) utility is a line-oriented text parsing and transformation tool. The sed command uses a simple programming language and regular expressions to process text streams and files. The most used feature of the sed command is string substitution.

This guide shows how to use sed to find and replace strings through examples.

Prerequisites

• Access to the command line/terminal.
• A text file (this guide provides an example.txt file).
• Basic terminal commands (grab our free cheat sheet).

The syntax to find and replace text using the sed command is:

sed -i 's/<search regex>/<replacement>/g' <input file>

The command consists of the following:

• -i tells the sed command to write the results to a file instead of standard output.
• s indicates the substitute command.
• / is the most common delimiter character. The command also accepts other characters as delimiters, which is useful when the string contains forward slashes.
• <search regex> is the string or regular expression search parameter.
• <replacement> is the replacement text.
• g is the global replacement flag, which replaces all occurrences of a string instead of just the first.
• <input file> is the file where the search and replace happens.

The single quotes help avoid meta-character expansion in the shell.

The BDS version of sed (which includes macOS) does not support case-insensitive matching or file replacement. The command for file replacement looks like this:

sed 's/<search regex>/<replacement>/g' <input file> > <output file>

Alternatively, install the GNU version of sed on macOS with homebrew:

brew install gnu-sed

Run the GNU sed command as follows:

gsed -i 's/<search regex>/<replacement>/g' <input file>

Replace the sed command with gsed to follow the examples below.

sed Replace Examples

The examples from this guide use a sample file to replace strings.

1. Create a sample text file:

nano example.txt

2. Add the following contents:

foobarbazfoobarbaz
foo bar baz foo bar baz
Foo Bar Baz Foo Bar Baz
FOO BAR BAZ FOO BAR BAZ
/foo/bar/baz /foo/bar/baz

Use the file as input to test the examples below.

Replace First Matched String

1. To replace the first found instance of the word bar with linux in every line of a file, run:

sed -i 's/bar/linux/' example.txt

2. The -i tag inserts the changes to the example.txt file. Check the file contents with the cat command:

cat example.txt

The command replaces the first instance of bar with linux in every line, including substrings. The match is exact, ignoring capitalization variations.

Global Replace

To replace every string match in a file, add the g flag to the script. For example:

sed -i 's/bar/linux/g' example.txt

The command globally replaces every instance of bar with linux in the file.

Match and Replace All Cases

To find and replace all instances of a word and ignore capitalization, use the I parameter:

sed -i 's/bar/linux/gI' example.txt

The command replaces all instances of the word bar in the text, ignoring capitalization.

Ignore Substrings

Add word boundaries (b) to the sed command to ignore substrings when replacing strings in a file. For example:

sed -i 's/bbarb/linux/gI' example.txt

Alternatively, change the delimiter to make the command easier to read:

sed -i 's:bbarb:linux:gI' example.txt

The command ignores substrings, matching only the whole word.

Find and Replace Strings With Slashes

Escape the forward slash character to find and replace a string with slashes. For example, to replace /foo/bar/baz with /home/kb, use the following syntax:

sed -i 's//foo/bar/baz//home/kb/g' example.txt

Alternatively, change the delimiter to avoid escaping characters:

sed -i 's:/foo/bar/baz:/home/kb:g' example.txt

Use this syntax to replace paths and other strings with slashes.

Find and Replace with Regular Expressions

The search pattern for the sed command accepts regular expressions, similar to grep regex. For example, to match all capital letters and replace them with 5, use:

sed -i 's/[A-Z]/5/g' example.txt

The regex pattern helps find all capital letters and replaces them with the number in the file.

Reference Found String

Use the ampersand character (&) to reference the found string. For example, to add a forward slash (/) before every instance of foo in a file, use:

sed -i 's/foo//&/gI'example.txt

Instead of retyping the search parameter, the & sign references the found string.

Create a Backup

To create a backup file before overwriting the existing one, add the .bak parameter to the -i tag.

sed -i.bak 's/foo/FOO/g' example.txt

The command creates a backup (example.txt.bak) before overwriting the original. Use this method to keep a copy in the original format and avoid overwriting.

Recursive Find and Replace

Use the find command to search for files and combine it with sed to replace strings in files recursively. For example:

find -name 'example*' -exec sed -i 's/[a-z]/5/gI' {} +

The command finds all files starting with example and executes the sed command on the files. The executed command replaces all letters with 5, ignoring capitalization.

Conclusion

After going through the examples in this guide, you know how to use sed to replace strings in files. The sed command is a powerful text manipulation utility with many advanced features.

Next, check out the awk or gawk command to learn about other text manipulation tools.

1. Replacing all occurrences of one string with another in all files in the current directory:

These are for cases where you know that the directory contains only regular files and that you want to process all non-hidden files. If that is not the case, use the approaches in 2.

All sed solutions in this answer assume GNU sed. If using FreeBSD or macOS, replace -i with -i ''. Also note that the use of the -i switch with any version of sed has certain filesystem security implications and is inadvisable in any script which you plan to distribute in any way.

• Non recursive, files in this directory only:

   sed -i -- 's/foo/bar/g' *
   perl -i -pe 's/foo/bar/g' ./* 
  

(the perl one will fail for file names ending in | or space)).

• Recursive, regular files (including hidden ones) in this and all subdirectories

   find . -type f -exec sed -i 's/foo/bar/g' {} +
  

  If you are using zsh:

   sed -i -- 's/foo/bar/g' **/*(D.)
  

  (may fail if the list is too big, see zargs to work around).

  Bash can’t check directly for regular files, a loop is needed (braces avoid setting the options globally):

   ( shopt -s globstar dotglob;
       for file in **; do
           if [[ -f $file ]] && [[ -w $file ]]; then
               sed -i -- 's/foo/bar/g' "$file"
           fi
       done
   )
  

  The files are selected when they are actual files (-f) and they are writable (-w).

2. Replace only if the file name matches another string / has a specific extension / is of a certain type etc:

• Non-recursive, files in this directory only:

  sed -i -- 's/foo/bar/g' *baz*    ## all files whose name contains baz
  sed -i -- 's/foo/bar/g' *.baz    ## files ending in .baz
  

• Recursive, regular files in this and all subdirectories

  find . -type f -name "*baz*" -exec sed -i 's/foo/bar/g' {} +
  

  If you are using bash (braces avoid setting the options globally):

  ( shopt -s globstar dotglob
      sed -i -- 's/foo/bar/g' **baz*
      sed -i -- 's/foo/bar/g' **.baz
  )
  

  If you are using zsh:

  sed -i -- 's/foo/bar/g' **/*baz*(D.)
  sed -i -- 's/foo/bar/g' **/*.baz(D.)
  

The -- serves to tell sed that no more flags will be given in the command line. This is useful to protect against file names starting with -.

• If a file is of a certain type, for example, executable (see man find for more options):

  find . -type f -executable -exec sed -i 's/foo/bar/g' {} +
  

zsh:

    sed -i -- 's/foo/bar/g' **/*(D*)

3. Replace only if the string is found in a certain context

• Replace foo with bar only if there is a baz later on the same line:

   sed -i 's/foo(.*baz)/bar1/' file
  

In sed, using ( ) saves whatever is in the parentheses and you can then access it with 1. There are many variations of this theme, to learn more about such regular expressions, see here.

• Replace foo with bar only if foo is found on the 3d column (field) of the input file (assuming whitespace-separated fields):

   gawk -i inplace '{gsub(/foo/,"baz",$3); print}' file
  

(needs gawk 4.1.0 or newer).

• For a different field just use $N where N is the number of the field of interest. For a different field separator (: in this example) use:

   gawk -i inplace -F':' '{gsub(/foo/,"baz",$3);print}' file
  

Another solution using perl:

    perl -i -ane '$F[2]=~s/foo/baz/g; $" = " "; print "@Fn"' foo 

NOTE: both the awk and perl solutions will affect spacing in the file (remove the leading and trailing blanks, and convert sequences of blanks to one space character in those lines that match). For a different field, use $F[N-1] where N is the field number you want and for a different field separator use (the $"=":" sets the output field separator to :):

    perl -i -F':' -ane '$F[2]=~s/foo/baz/g; $"=":";print "@F"' foo 

• Replace foo with bar only on the 4th line:

   sed -i '4s/foo/bar/g' file
   gawk -i inplace 'NR==4{gsub(/foo/,"baz")};1' file
   perl -i -pe 's/foo/bar/g if $.==4' file
  

4. Multiple replace operations: replace with different strings

• You can combine sed commands:

   sed -i 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
  

Be aware that order matters (sed 's/foo/bar/g; s/bar/baz/g' will substitute foo with baz).

• or Perl commands

   perl -i -pe 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
  

• If you have a large number of patterns, it is easier to save your patterns and their replacements in a sed script file:

   #! /usr/bin/sed -f
   s/foo/bar/g
   s/baz/zab/g
  

• Or, if you have too many pattern pairs for the above to be feasible, you can read pattern pairs from a file (two space separated patterns, $pattern and $replacement, per line):

   while read -r pattern replacement; do   
       sed -i "s/$pattern/$replacement/" file
   done < patterns.txt
  

• That will be quite slow for long lists of patterns and large data files so you might want to read the patterns and create a sed script from them instead. The following assumes a <<!>space<!>> delimiter separates a list of MATCH<<!>space<!>>REPLACE pairs occurring one-per-line in the file patterns.txt :

   sed 's| *([^ ]*) *([^ ]*).*|s/1/2/g|' <patterns.txt |
   sed -f- ./editfile >outfile
  

The above format is largely arbitrary and, for example, doesn’t allow for a <<!>space<!>> in either of MATCH or REPLACE. The method is very general though: basically, if you can create an output stream which looks like a sed script, then you can source that stream as a sed script by specifying sed‘s script file as -stdin.

• You can combine and concatenate multiple scripts in similar fashion:

   SOME_PIPELINE |
   sed -e'#some expression script'  
       -f./script_file -f-          
       -e'#more inline expressions' 
   ./actual_edit_file >./outfile
  

A POSIX sed will concatenate all scripts into one in the order they appear on the command-line. None of these need end in a newline.

• grep can work the same way:

   sed -e'#generate a pattern list' <in |
   grep -f- ./grepped_file
  

• When working with fixed-strings as patterns, it is good practice to escape regular expression metacharacters. You can do this rather easily:

   sed 's/[]$&^*./[]/\&/g
        s| *([^ ]*) *([^ ]*).*|s/1/2/g|
   ' <patterns.txt |
   sed -f- ./editfile >outfile
  

5. Multiple replace operations: replace multiple patterns with the same string

• Replace any of foo, bar or baz with foobar

   sed -Ei 's/foo|bar|baz/foobar/g' file
  

• or

   perl -i -pe 's/foo|bar|baz/foobar/g' file
  

6. Replace File paths in multiple files

Another use case of using different delimiter:

sed -i 's|path/to/foo|path/to/bar|g' *

Introduction

Before we begin talking about how to use sed commands to find and replace strings in the files. Let’s briefly understand — What is sed?

Sometimes when working with text files there is a requirement for users to find and replace strings of text in one or more files. sed is a stream editor. It helps in the manipulation of files and input streams like pipelines. Using sed user can search, find and replace, insert, and delete words and lines. It supports both basic and extended regular expressions that allow users to match certain complex patterns.

In this tutorial, you will learn how to find and replace strings in the files with the sed command. We will also address some of the FAQs related to the sed command.

Step 1 — Find and Replace String with sed

1) sed has different versions with some functionality differences. macOS uses the BSD version and most of the Linux distributions come with the pre-installed GNU. We will be using GNU in this article.

Following is the format for finding and replacing strings with sed command:

sed -i 's/SEARCH_REGEX/REPLACEMENT/g' INPUTFILE

• -i — sed writes its output to standard output. This option indicates sed editing files in place. If an extension is supplied (ex -i.bak) then, a backup of the original file will be created.
• s — It is the substitute command.
• / / / —  The Delimiter character. It can be any character but usually the slash (/) character is useful.
• SEARCH_REGEX — A normal string, or regular expression to search for.
• REPLACEMENT — It is the replacement string.
• g — The Global replacement flag. By default, sed reads file line by line. and only changes the first occurrence of SEARCH_REGEX but if the replacement flag is given, all occurrences get replaced.
• INPUTFILE — This is the name of the file on which you want to run the command.

2) It is a good practice to put quotes around the argument so that the shell meta-characters won’t expand.

Now, let’s check how to use sed command to search and replace text in files with some of it’s most commonly used options and flags.

123 Foo foo foo 
foo /bin/bash Ubuntu foobar 456

3) If you will omit the g flag only first instance of the search string will be replaced in each line:

sed -i 's/foo/linux/' file.txt

Output

123 Foo linux foo 
linux /bin/bash Ubuntu foobar 456

4) With global replacement flag sed replaces all occurrences of search pattern:

sed -i 's/foo/linux/g' file.txt

Output

123 Foo linux linux
linux /bin/bash Ubuntu linuxbar 456

5) As you will notice, in the previous example, the substring foo inside foobar string also gets replaced. If it is not the required behavior, use word-boundary expression (b) at both ends of the search string. It will ensure that the partial words are not matched:

sed -i 's/bfoob/linux/g' file.txt

Output

123 Foo linux linux
linux /bin/bash Ubuntu foobar 456

6) To make pattern match case insensitive. You will use the I flag. Like below you can use both g as well as I flags:

sed -i 's/foo/linux/gI' file.txt

Output

123 linux linux linux 
linux /bin/bash Ubuntu linuxbar 456

7) If you want to find and replace a string containing delimiter character (/) then, you will need to use the backslash (). It will escape the slash. Like to replace /bin/bash with /usr/bin/zsh you will use the following:

sed -i 's//bin/bash//usr/bin/zsh/g' file.txt

The easier and much readable option is to use another delimiter character. Many people use vertical bar (|) or colon (:) . But you can use any other character as well:

sed -i 's|/bin/bash|/usr/bin/zsh|g' file.txt

Output

123 Foo foo foo 
foo /usr/bin/zsh Ubuntu foobar 456

9) You can also use regular expressions, say you want to search for all 3 digit numbers and replace them with the string number you can use the following command:

sed -i 's/b[0-9]{3}b/number/g' file.txt

Output

number Foo foo foo 
foo /bin/bash demo foobar number

10) Other useful feature of sed is using the ampersand character & . It corresponds to the matched pattern. The character is useful multiple times. If you want to add curly braces {} around each 3 digit number. Then you will need to type:

sed -i 's/b[0-9]{3}b/{&}/g' file.txt

Output

{123} Foo foo foo 
foo /bin/bash demo foobar {456}

11) It is always recommended to make a backup of the file on which you are using sed command. For example if you are working on file.txt, you can save the original file as file.txt.bak using the following command:

sed -i.bak 's/foo/linux/g' file.txt

12) If you want to make sure that backup is created, list the files with ls command:

ls

Output

file.txt file.txt.bak

Step 2 — Recursive Find and Replace

1) Sometimes you will want to recursively search directories for a file that contains a string and replace that particular string in all the files. This is done by using commands like find or grep . It is to recursively find files in the directory and piping the file names to sed.

The below command will recursively search for files in the current working directory and pass the file names to sed:

find . -type f -exec sed -i 's/foo/bar/g' {} +

2) Now to avoid issues with files having space in their names you can use the -print0 option. It tells find to print the file name along with a null character and pipe the output to sed using xargs -0:

find . -type f -print0 | xargs -0 sed -i 's/foo/bar/g'

3) If you want to skip a directory, you should use -not -path option. For example, if you want to replace a string in your local git repository to exclude all the files starting with dot (.), use the following command:

find . -type f -not -path '*/.*' -print0 | xargs -0 sed -i 's/foo/bar/g'

4) Now, if you want to search and replace text only on files with specific extension you will use the following command:

find . -type f -name "*.md" -print0 | xargs -0 sed -i 's/foo/bar/g'

5) Other option is to use grep command. It will recursively find all files having search pattern. Further, to pipe the filenames to sed:

grep -rlZ 'foo' . | xargs -0 sed -i.bak 's/foo/bar/g'

Conclusion

We hope this detailed guide helped you in understanding how to use sed to find and replace string in the files.

If you have any queries, please leave them in the comment below. We’ll be happy to answer them.