Linux replace word file

There’s multitude of ways to achieve it. Depending on the complexity of what one tries to achieve with string replacement, and depending on tools with which user is familiar, some methods may be preferred more than others.

In this answer I am using simple input.txt file, which you can use to test all examples provided here. The file contents:

roses are red , violets are blue
This is an input.txt and this doesn't rhyme

BASH

Bash isn’t really meant for text processing, but simple substitutions can be done via parameter expansion , in particular here we can use simple structure ${parameter/old_string/new_string}.

#!/bin/bash
while IFS= read -r line
do
    case "$line" in
       *blue*) printf "%sn" "${line/blue/azure}" ;;
       *) printf "%sn" "$line" ;;
    esac
done < input.txt

This small script doesn’t do in-place replacement, meaning that you would have to save new text to new file, and get rid of the old file, or mv new.txt old.txt

Side note: if you’re curious about why while IFS= read -r ; do ... done < input.txt is used, it’s basically shell’s way of reading file line by line. See this for reference.

AWK

AWK, being a text processing utility, is quite appropriate for such task. It can do simple replacements and much more advanced ones based on regular expressions. It provides two functions: sub() and gsub(). The first one only replaces only the first occurrence, while the second — replaces occurrences in whole string. For instance, if we have string one potato two potato , this would be the result:

$ echo "one potato two potato" | awk '{gsub(/potato/,"banana")}1'
one banana two banana

$ echo "one potato two potato" | awk '{sub(/potato/,"banana")}1'                                      
one banana two potato 

AWK can take an input file as argument, so doing same things with input.txt , would be easy:

awk '{sub(/blue/,"azure")}1' input.txt

Depending on the version of AWK you have, it may or may not have in-place editing, hence the usual practice is save and replace new text. For instance something like this:

awk '{sub(/blue/,"azure")}1' input.txt > temp.txt && mv temp.txt input.txt

SED

Sed is a line editor. It also uses regular expressions, but for simple substitutions it’s sufficient to do:

sed 's/blue/azure/' input.txt

What’s good about this tool is that it has in-place editing, which you can enable with -i flag.

Perl

Perl is another tool which is often used for text processing, but it’s a general purpose language, and is used in networking, system administration, desktop apps, and many other places. It borrowed a lot of concepts/features from other languages such as C,sed,awk, and others. Simple substitution can be done as so:

perl -pe 's/blue/azure/' input.txt

Like sed, perl also has the -i flag.

Python

This language is very versatile and is also used in a wide variety of applications. It has a lot of functions for working with strings, among which is replace(), so if you have variable like var="Hello World" , you could do var.replace("Hello","Good Morning")

Simple way to read file and replace string in it would be as so:

python -c "import sys;lines=sys.stdin.read();print lines.replace('blue','azure')" < input.txt

With Python, however, you also need to output to new file , which you can also do from within the script itself. For instance, here’s a simple one:

#!/usr/bin/env python
import sys
import os
import tempfile

tmp=tempfile.mkstemp()

with open(sys.argv[1]) as fd1, open(tmp[1],'w') as fd2:
    for line in fd1:
        line = line.replace('blue','azure')
        fd2.write(line)

os.rename(tmp[1],sys.argv[1])

This script is to be called with input.txt as command-line argument. The exact command to run python script with command-line argument would be

 $ ./myscript.py input.txt

or

$ python ./myscript.py input.txt

Of course, make sure that ./myscript.py is in your current working directory and for the first way, ensure it is set executable with chmod +x ./myscript.py

Python can also have regular expressions , in particular, there’s re module, which has re.sub() function, which can be used for more advanced replacements.

1. Replacing all occurrences of one string with another in all files in the current directory:

These are for cases where you know that the directory contains only regular files and that you want to process all non-hidden files. If that is not the case, use the approaches in 2.

All sed solutions in this answer assume GNU sed. If using FreeBSD or macOS, replace -i with -i ''. Also note that the use of the -i switch with any version of sed has certain filesystem security implications and is inadvisable in any script which you plan to distribute in any way.

• Non recursive, files in this directory only:

   sed -i -- 's/foo/bar/g' *
   perl -i -pe 's/foo/bar/g' ./* 
  

(the perl one will fail for file names ending in | or space)).

• Recursive, regular files (including hidden ones) in this and all subdirectories

   find . -type f -exec sed -i 's/foo/bar/g' {} +
  

  If you are using zsh:

   sed -i -- 's/foo/bar/g' **/*(D.)
  

  (may fail if the list is too big, see zargs to work around).

  Bash can’t check directly for regular files, a loop is needed (braces avoid setting the options globally):

   ( shopt -s globstar dotglob;
       for file in **; do
           if [[ -f $file ]] && [[ -w $file ]]; then
               sed -i -- 's/foo/bar/g' "$file"
           fi
       done
   )
  

  The files are selected when they are actual files (-f) and they are writable (-w).

2. Replace only if the file name matches another string / has a specific extension / is of a certain type etc:

• Non-recursive, files in this directory only:

  sed -i -- 's/foo/bar/g' *baz*    ## all files whose name contains baz
  sed -i -- 's/foo/bar/g' *.baz    ## files ending in .baz
  

• Recursive, regular files in this and all subdirectories

  find . -type f -name "*baz*" -exec sed -i 's/foo/bar/g' {} +
  

  If you are using bash (braces avoid setting the options globally):

  ( shopt -s globstar dotglob
      sed -i -- 's/foo/bar/g' **baz*
      sed -i -- 's/foo/bar/g' **.baz
  )
  

  If you are using zsh:

  sed -i -- 's/foo/bar/g' **/*baz*(D.)
  sed -i -- 's/foo/bar/g' **/*.baz(D.)
  

The -- serves to tell sed that no more flags will be given in the command line. This is useful to protect against file names starting with -.

• If a file is of a certain type, for example, executable (see man find for more options):

  find . -type f -executable -exec sed -i 's/foo/bar/g' {} +
  

zsh:

    sed -i -- 's/foo/bar/g' **/*(D*)

3. Replace only if the string is found in a certain context

• Replace foo with bar only if there is a baz later on the same line:

   sed -i 's/foo(.*baz)/bar1/' file
  

In sed, using ( ) saves whatever is in the parentheses and you can then access it with 1. There are many variations of this theme, to learn more about such regular expressions, see here.

• Replace foo with bar only if foo is found on the 3d column (field) of the input file (assuming whitespace-separated fields):

   gawk -i inplace '{gsub(/foo/,"baz",$3); print}' file
  

(needs gawk 4.1.0 or newer).

• For a different field just use $N where N is the number of the field of interest. For a different field separator (: in this example) use:

   gawk -i inplace -F':' '{gsub(/foo/,"baz",$3);print}' file
  

Another solution using perl:

    perl -i -ane '$F[2]=~s/foo/baz/g; $" = " "; print "@Fn"' foo 

NOTE: both the awk and perl solutions will affect spacing in the file (remove the leading and trailing blanks, and convert sequences of blanks to one space character in those lines that match). For a different field, use $F[N-1] where N is the field number you want and for a different field separator use (the $"=":" sets the output field separator to :):

    perl -i -F':' -ane '$F[2]=~s/foo/baz/g; $"=":";print "@F"' foo 

• Replace foo with bar only on the 4th line:

   sed -i '4s/foo/bar/g' file
   gawk -i inplace 'NR==4{gsub(/foo/,"baz")};1' file
   perl -i -pe 's/foo/bar/g if $.==4' file
  

4. Multiple replace operations: replace with different strings

• You can combine sed commands:

   sed -i 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
  

Be aware that order matters (sed 's/foo/bar/g; s/bar/baz/g' will substitute foo with baz).

• or Perl commands

   perl -i -pe 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
  

• If you have a large number of patterns, it is easier to save your patterns and their replacements in a sed script file:

   #! /usr/bin/sed -f
   s/foo/bar/g
   s/baz/zab/g
  

• Or, if you have too many pattern pairs for the above to be feasible, you can read pattern pairs from a file (two space separated patterns, $pattern and $replacement, per line):

   while read -r pattern replacement; do   
       sed -i "s/$pattern/$replacement/" file
   done < patterns.txt
  

• That will be quite slow for long lists of patterns and large data files so you might want to read the patterns and create a sed script from them instead. The following assumes a <<!>space<!>> delimiter separates a list of MATCH<<!>space<!>>REPLACE pairs occurring one-per-line in the file patterns.txt :

   sed 's| *([^ ]*) *([^ ]*).*|s/1/2/g|' <patterns.txt |
   sed -f- ./editfile >outfile
  

The above format is largely arbitrary and, for example, doesn’t allow for a <<!>space<!>> in either of MATCH or REPLACE. The method is very general though: basically, if you can create an output stream which looks like a sed script, then you can source that stream as a sed script by specifying sed‘s script file as -stdin.

• You can combine and concatenate multiple scripts in similar fashion:

   SOME_PIPELINE |
   sed -e'#some expression script'  
       -f./script_file -f-          
       -e'#more inline expressions' 
   ./actual_edit_file >./outfile
  

A POSIX sed will concatenate all scripts into one in the order they appear on the command-line. None of these need end in a newline.

• grep can work the same way:

   sed -e'#generate a pattern list' <in |
   grep -f- ./grepped_file
  

• When working with fixed-strings as patterns, it is good practice to escape regular expression metacharacters. You can do this rather easily:

   sed 's/[]$&^*./[]/\&/g
        s| *([^ ]*) *([^ ]*).*|s/1/2/g|
   ' <patterns.txt |
   sed -f- ./editfile >outfile
  

5. Multiple replace operations: replace multiple patterns with the same string

• Replace any of foo, bar or baz with foobar

   sed -Ei 's/foo|bar|baz/foobar/g' file
  

• or

   perl -i -pe 's/foo|bar|baz/foobar/g' file
  

6. Replace File paths in multiple files

Another use case of using different delimiter:

sed -i 's|path/to/foo|path/to/bar|g' *

If you manage to take a deeper glimpse inside the ecosystem of the Linux operating system environment, you will discover that its built-in commands are sufficient enough to solve most of our computing problems.

One such problem is the need to find and replace text, word, or string in a file especially when you are in a server environment. A solution to this problem lets you handle nagging issues like updating the “/etc/apt/sources.list” file after a successful Linux system upgrade.

Creating the Test Text File in Linux

Create a text file with a name like “test.txt” and populate it with some text, word, or string characters. Let this file be on the same path as your terminal instance.

$ nano test.txt
OR
$ vi test.txt 

We will be using the cat command to flexible note the changes on our created test file.

$ cat test.txt

Using Sed to Find and Replace Text, Word, or String in a File

The power of this stream editor is in its easiness in accomplishing basic input streams transformation. The sed command offers a quick, easy, and timeless approach in finding and replacing targeted text, words, or string patterns in a file.

Find and Replace the First Occurrences of Text, Word, or String in File

From our created test file, we are going to update all the instances of the word “LinuxShellTips” with the alternative “this site”. The syntax of the sed command we will be using to accomplish this simple task is as follows:

$ sed -i 's/[THE_OLD_TERM]/[THE_NEW_TERM]/' [TARGETED_FILE] 

With reference to the above sed command usage and syntax, we can replace our test file’s “LinuxShellTips” term with “this site” term as follows:

$ sed -i 's/LinuxShellTips/this site/' test.txt

We can now use the cat command to preview the above-proposed changes to our test file to note if any substantial edits took place.

$ cat test.txt

As you can see, the first two occurrences of the word “LinuxShellTips” have been replaced with the phrase “this site”.

Find and Replace All Occurrences of Text, Word, or String in File

Despite the above command having replaced all the targeted word patterns in our test file, its usage tends to be selective in most circumstances. The command works best for small files because, in big files, only the first few occurrences of a targeted word pattern might benefit from its ‘find and replace’ prowess.

To find and replace all occurrences of a word pattern in any editable file, you should adhere to the following sed command syntax.

$ sed -i 's/[THE_OLD_TERM]/[THE_NEW_TERM]/g' [TARGETED_FILE]    

As you have noted, the 'g' in the above find & replace command syntax acts as a global variable so that all the global occurrences of a term in a targeted file are considered.

$ sed -i 's/this site/LinuxShellTips/g' test.txt

The above command finds all the global occurrences of the term “this site” and replaces it with the term “LinuxShellTips”.

Using Awk to Find and Replace Text, Word, or String in File

If you are familiar with the awk command-line utility, then you know that its powerful scripting language metrics make it a pro at text processing. Linux system administrators and professional users find this tool effective in data extraction and reporting.

The awk command syntax for a simple find-and-replace operation looks like the following snippet:

$ awk '{gsub("[THE_OLD_TERM]","[THE_NEW_TERM]"); print}' [TARGETED_FILE]

In the above syntax, awk will substitute THE_OLD_TERM from THE_NEW_TERM in the TARGETED_FILE and print the resulting file content on the system terminal.

Let us take a practical approach:

$ awk '{gsub("Linux","ubuntu"); print}' test.txt

With the dynamic functionalities of both sed and awk command tools, you should now find, replace, and overwrite text, word, or string patterns in a targeted file. These tools give a system user the needed flexibility while on a command-line interface.