How to do word count in python

Strings are essential data types in any programming language, including python. We need to perform many different operations, also known as string preprocessing like removing the unnecessary spaces, counting the words in a string, making the string in the same cases (uppercase or lowercase). In this article, we will learn how to count words in a string in python. 

We will learn how to count the number of words in a string. For example- We have a string-” Hello, this is a string.” It has five words. Also, we will learn how to count the frequency of a particular word in a string. 

• Count Words Using For loop- 
• Using split() to count words in a string
• Count frequency of words in a string using a dictionary
• Count frequency of words in string Using Count() 

1. Count Words Using For loop- 

Using for loop is the naïve approach to solve this problem. We count the number of spaces between the two characters. 

<pre class="wp-block-syntaxhighlighter-code">def count_words(string):
    # Removing the spaces from start and end
    string1=string.strip()
    # Initializing the count from 1 because we there is no space at the last 
    count=1
    # <a href="https://www.pythonpool.com/python-iterate-through-list/" target="_blank" rel="noreferrer noopener">Iterating</a> through the string
    for i in string1:
    # If we encounter space, increment the count with 1.
        if i==" ":
            count+=1
     
    return count
string="Python is an interpreted, high-level, general-purpose programming language"
print("'{}'".format(string),"has total words:",count_words(string))
string2=" Hi. My name is Ashwini "
print("'{}'".format(string2),"has total words:",count_words(string2))</pre>

Output- 

''Python is an interpreted, high-level, general-purpose programming language' has total words: 8'' 
Hi. My name is Ashwini ' has total words: 5 

2. Using split() to count words in a string

We can use split() function to count words in string. 

def word_count(string):
    # Here we are removing the spaces from start and end,
    # and breaking every word whenever we encounter a space
    # and storing them in a list. The len of the list is the
    # total count of words.
    return(len(string.strip().split(" ")))

string="Python is an interpreted, high-level, general-purpose programming language"
print("'{}'".format(string),"has total words:",count_words(string))
string2=" Hi. My name is Ashwini "
print("'{}'".format(string2),"has total words:",word_count(string2))

Output- 

''Python is an interpreted, high-level, general-purpose programming language' has total words: 8'' 
Hi. My name is Ashwini ' has total words: 5 

3. Count the frequency of words in a String in Python using Dictionary

<pre class="wp-block-syntaxhighlighter-code">def wordFrequency(string):
    # converting the string into lowercase
    string=string.lower()
    # Whenever we encounter a space, break the string
    string=string.split(" ")
    # Initializing a dictionary to store the frequency of words
    word_frequency={}
    # <a href="https://www.pythonpool.com/python-iterate-through-list/" target="_blank" rel="noreferrer noopener">Iterating</a> through the string
    for i in string:
    
    # If the word is already in the keys, increment its frequency
        if i in word_frequency:
            word_frequency[i]+=1
            
    # It means that this is the first occurence of the word
        else:
            word_frequency[i]=1
    return(word_frequency)
string="Woodchuck How much wood would a woodchuck chuck if a woodchuck could chuck wood ?" 
print(wordFrequency(string)) </pre>

Output- 

{'woodchuck': 3, 'how': 1, 'much': 1, 'wood': 2, 'would': 1, 'a': 2, 'chuck': 2, 'if': 1, 'could': 1, '?': 1} 

4. Count frequency of words in string in Python Using Count() 

Count() can be used to count the number of times a word occurs in a string or in other words it is used to tell the frequency of a word in a string. We just need to pass the word in the argument.  

def return_count(string,word):
    string=string.lower() 
    # In string, what is the count that word occurs
    return string.count(word)

string2="Peter Piper picked a peck of pickled peppers. How many pickled peppers did Peter Piper pick?"
return_count(string2,'piper')

Output- 

2 

If we want to know the number of times every word occurred, we can make a function for that. 

set1=set()
string="Woodchuck How much wood would a woodchuck chuck if a woodchuck could chuck wood ?"
string=string.lower()
# splitting the string  whenever we encounter a space
string=string.split(" ")
# iterate through list-string
for i in string:
    # Storing the word and its frequency in the form of tuple in a set
    # Set is used to avoid repetition
    set1.add((i,string.count(i)))
print(set1)

Output- 

{('how', 1), ('would', 1), ('woodchuck', 3), ('a', 2), ('chuck', 2), ('could', 1), ('if', 1), ('?', 1), ('wood', 2), ('much', 1)} 

If we want to know how many times a particular word occur in a string in an interval, we can use start and end parameters of count(). 

For example- 

string="Can you can a can as a canner can can a can?"
# if you want to take cases into account remove this line
string=string.lower()
# between index=8 and 17, how many times the word 'can' occurs
print(string.count("can",8,17))

Output- 

2 

Must Read

• How to Convert String to Lowercase in
• How to Calculate Square Root
• User Input | Input () Function | Keyboard Input
• Best Book to Learn Python

Conclusion 

In the current era, data is very important. And as the world of Data Science is growing rapidly, and that too using python, data preprocessing is very important. We need to count words in a string in python to preprocess textual data and for that, the above-discussed methods are very important. 

Try to run the programs on your side and let us know if you have any queries.

Happy Coding!

In this tutorial, you’ll learn how to use Python to count the number of words and word frequencies in both a string and a text file. Being able to count words and word frequencies is a useful skill. For example, knowing how to do this can be important in text classification machine learning algorithms.

By the end of this tutorial, you’ll have learned:

• How to count the number of words in a string
• How to count the number of words in a text file
• How to calculate word frequencies using Python

Reading a Text File in Python

The processes to count words and calculate word frequencies shown below are the same for whether you’re considering a string or an entire text file. Because of this, this section will briefly describe how to read a text file in Python.

If you want a more in-depth guide on how to read a text file in Python, check out this tutorial here. Here is a quick piece of code that you can use to load the contents of a text file into a Python string:

# Reading a Text File in Python
file_path = '/Users/datagy/Desktop/sample_text.txt'

with open(file_path) as file:
    text = file.read()

I encourage you to check out the tutorial to learn why and how this approach works. However, if you’re in a hurry, just know that the process opens the file, reads its contents, and then closes the file again.

Count Number of Words In Python Using split()

One of the simplest ways to count the number of words in a Python string is by using the split() function. The split function looks like this:

# Understanding the split() function
str.split(
   sep=None     # The delimiter to split on
   maxsplit=-1  # The number of times to split
)

By default, Python will consider runs of consecutive whitespace to be a single separator. This means that if our string had multiple spaces, they’d only be considered a single delimiter. Let’s see what this method returns:

# Splitting a string with .split()
text = 'Welcome to datagy! Here you will learn Python and data science.'
print(text.split())

# Returns: ['Welcome', 'to', 'datagy!', 'Here', 'you', 'will', 'learn', 'Python', 'and', 'data', 'science.']

We can see that the method now returns a list of items. Because we can use the len() function to count the number of items in a list, we’re able to generate a word count. Let’s see what this looks like:

# Counting words with .split()
text = 'Welcome to datagy! Here you will learn Python and data science.'
print(len(text.split()))

# Returns: 11

Count Number of Words In Python Using Regex

Another simple way to count the number of words in a Python string is to use the regular expressions library, re. The library comes with a function, findall(), which lets you search for different patterns of strings.

Because we can use regular expression to search for patterns, we must first define our pattern. In this case, we want patterns of alphanumeric characters that are separated by whitespace.

For this, we can use the pattern w+, where w represents any alphanumeric character and the + denotes one or more occurrences. Once the pattern encounters whitespace, such as a space, it will stop the pattern there.

Let’s see how we can use this method to generate a word count using the regular expressions library, re:

# Counting words with regular expressions
import re
text = 'Welcome to datagy! Here you will learn Python and data science.'
print(len(re.findall(r'w+', text)))

# Returns: 11

Calculating Word Frequencies in Python

In order to calculate word frequencies, we can use either the defaultdict class or the Counter class. Word frequencies represent how often a given word appears in a piece of text.

Using defaultdict To Calculate Word Frequencies in Python

Let’s see how we can use defaultdict to calculate word frequencies in Python. The defaultdict extend on the regular Python dictionary by providing helpful functions to initialize missing keys.

Because of this, we can loop over a piece of text and count the occurrences of each word. Let’s see how we can use it to create word frequencies for a given string:

# Creating word frequencies with defaultdict
from collections import defaultdict
import re

text = 'welcome to datagy! datagy will teach data. data is fun. data data data!'

counts = defaultdict(int)
for word in re.findall('w+', text):
    counts[word] += 1

print(counts)

# Returns:
# defaultdict(<class 'int'>, {'welcome': 1, 'to': 1, 'datagy': 2, 'will': 1, 'teach': 1, 'data': 5, 'is': 1, 'fun': 1})

Let’s break down what we did here:

1. We imported both the defaultdict function and the re library
2. We loaded some text and instantiated a defaultdict using the int factory function
3. We then looped over each word in the word list and added one for each time it occurred

Using Counter to Create Word Frequencies in Python

Another way to do this is to use the Counter class. The benefit of this approach is that we can even easily identify the most frequent word. Let’s see how we can use this approach:

# Creating word frequencies with Counter
from collections import Counter
import re

text = 'welcome to datagy! datagy will teach data. data is fun. data data data!'
counts =  Counter(re.findall('w+', text))
print(counts)

# Returns:
# Counter({'data': 5, 'datagy': 2, 'welcome': 1, 'to': 1, 'will': 1, 'teach': 1, 'is': 1, 'fun': 1})

Let’s break down what we did here:

1. We imported our required libraries and classes
2. We passed the resulting list from the findall() function into the Counter class
3. We printed the result of this class

One of the perks of this is that we can easily find the most common word by using the .most_common() function. The function returns a sorted list of tuples, ordering the items from most common to least common. Because of this, we can simply access the 0th index to find the most common word:

# Finding the Most Common Word
from collections import Counter
import re

text = 'welcome to datagy! datagy will teach data. data is fun. data data data!'
counts =  Counter(re.findall('w+', text))
print(counts.most_common()[0])

# Returns:
# ('data', 5)

Conclusion

In this tutorial, you learned how to generate word counts and word frequencies using Python. You learned a number of different ways to count words including using the .split() method and the re library. Then, you learned different ways to generate word frequencies using defaultdict and Counter. Using the Counter method, you were able to find the most frequent word in a string.

Additional Resources

To learn more about related topics, check out the tutorials below:

• Python str.split() – Official Documentation
• Python Defaultdict: Overview and Examples
• Python: Count Number of Occurrences in List (6 Ways)
• Python: Count Number of Occurrences in a String (4 Ways!)

Data preprocessing is an important task in text classification. With the emergence of Python in the field of data science, it is essential to have certain shorthands to have the upper hand among others. This article discusses ways to count words in a sentence, it starts with space-separated words but also includes ways to in presence of special characters as well. Let’s discuss certain ways to perform this.

Quick Ninja Methods: One line Code to find count words in a sentence with Static and Dynamic Inputs.

Output:

Method #1: Using split() split function is quite useful and usually quite generic method to get words out of the list, but this approach fails once we introduce special characters in the list. 

The original string is : Geeksforgeeks is best Computer Science Portal
The number of words in string are : 6

Method #2 : Using regex(findall()) Regular expressions have to be used in case we require to handle the cases of punctuation marks or special characters in the string. This is the most elegant way in which this task can be performed. 

Example

The original string is : Geeksforgeeks, is best @# Computer Science Portal.!!!
The number of words in string are : 6

Method #3 : Using sum() + strip() + split() This method performs this particular task without using regex. In this method we first check all the words consisting of all the alphabets, if so they are added to sum and then returned. 
 

The original string is : Geeksforgeeks, is best @# Computer Science Portal.!!!
The number of words in string are : 6

Method #4: Using count() method

The original string is : Geeksforgeeks is best Computer Science Portal
The number of words in string are : 6

Method #5 : Using the shlex module:

Here is a new approach using the split() method in shlex module:

The shlex module provides a lexical analyzer for simple shell-like syntaxes. It can be used to split a string into a list of words while taking into account quotes, escapes, and other special characters. This makes it a good choice for counting words in a sentence that may contain such characters.

Note: The shlex.split function returns a list of words, so you can use the len function to count the number of words in the list. The count method can also be used on the list to achieve the same result.

Method #6: Using operator.countOf() method

The original string is : Geeksforgeeks is best Computer Science Portal
The number of words in string are : 6

The time complexity of this approach is O(n), where n is the length of the input string. 
The Auxiliary space is also O(n), as the shlex.split function creates a new list of words from the input string. This approach is efficient for small to medium-sized inputs, but may not be suitable for very large inputs due to the use of additional memory.

Method #7:Using reduce()

1. Initialize a variable res to 1 to account for the first word in the string.
2. For each character ch in the string, do the following:
   a. If ch is a space, increment res by 1.
3. Return the value of res as the result.

The original string is : Geeksforgeeks is best Computer Science Portal
The number of words in string are : 6

The time complexity of the algorithm for counting the number of words in a string using the count method or reduce function is O(n), where n is the length of the string. This is because we iterate over each character in the string once to count the number of spaces.

The auxiliary space of the algorithm is O(1), since we only need to store a few variables (res and ch) at any given time during the execution of the algorithm. The space required is independent of the length of the input string.

Method #8: Using  numpy:

Algorithm:

1. Initialize the input string ‘test_string’
2. Print the original string
3. Use the numpy ‘char.count()’ method to count the number of spaces in the string and add 1 to it to get the count of words.
4. Print the count of words.

Output:
The original string is : Geeksforgeeks is best Computer Science Portal
The number of words in string are : 6

Time complexity: O(n)
The time complexity of the ‘char.count()’ method is O(n), where n is the length of the input string. The addition operation takes constant time. Therefore, the time complexity of the code is O(n).

Auxiliary Space: O(1)
The space complexity of the code is constant, as we are not using any additional data structures or variables that are dependent on the input size. Therefore, the space complexity of the code is O(1).