Finding the duplicates in excel

Sometimes duplicate data is useful, sometimes it just makes it harder to understand your data. Use conditional formatting to find and highlight duplicate data. That way you can review the duplicates and decide if you want to remove them.

1. Select the cells you want to check for duplicates.

   Note: Excel can’t highlight duplicates in the Values area of a PivotTable report.

2. Click Home > Conditional Formatting > Highlight Cells Rules > Duplicate Values.

   

3. In the box next to values with, pick the formatting you want to apply to the duplicate values, and then click OK.

   ​​​​​​​

Remove duplicate values

When you use the Remove Duplicates feature, the duplicate data will be permanently deleted. Before you delete the duplicates, it’s a good idea to copy the original data to another worksheet so you don’t accidentally lose any information.

1. Select the range of cells that has duplicate values you want to remove.

2. Click Data > Remove Duplicates, and then Under Columns, check or uncheck the columns where you want to remove the duplicates.

   

   For example, in this worksheet, the January column has price information I want to keep.

   

   So, I unchecked January in the Remove Duplicates box.

   

3. Click OK.

Note: The counts of duplicate and unique values given after removal may include empty cells, spaces, etc. 

Need more help?

​​​​​​​

Duplicate Values | Triplicates | Duplicate Rows

This example teaches you how to find duplicate values (or triplicates) and how to find duplicate rows in Excel.

Duplicate Values

To find and highlight duplicate values in Excel, execute the following steps.

1. Select the range A1:C10.

2. On the Home tab, in the Styles group, click Conditional Formatting.

3. Click Highlight Cells Rules, Duplicate Values.

4. Select a formatting style and click OK.

Result. Excel highlights the duplicate names.

Note: select Unique from the first drop-down list to highlight the unique names.

Triplicates

By default, Excel highlights duplicates (Juliet, Delta), triplicates (Sierra), etc. (see previous image). Execute the following steps to highlight triplicates only.

1. First, clear the previous conditional formatting rule.

2. Select the range A1:C10.

3. On the Home tab, in the Styles group, click Conditional Formatting.

4. Click New Rule.

5. Select ‘Use a formula to determine which cells to format’.

6. Enter the formula =COUNTIF($A$1:$C$10,A1)=3

7. Select a formatting style and click OK.

Result. Excel highlights the triplicate names.

Explanation: =COUNTIF($A$1:$C$10,A1) counts the number of names in the range A1:C10 that are equal to the name in cell A1. If COUNTIF($A$1:$C$10,A1) = 3, Excel formats cell A1. Always write the formula for the upper-left cell in the selected range (A1:C10). Excel automatically copies the formula to the other cells. Thus, cell A2 contains the formula =COUNTIF($A$1:$C$10,A2)=3, cell A3 =COUNTIF($A$1:$C$10,A3)=3, etc. Notice how we created an absolute reference ($A$1:$C$10) to fix this reference.

Note: you can use any formula you like. For example, use this formula =COUNTIF($A$1:$C$10,A1)>3 to highlight names that occur more than 3 times.

Duplicate Rows

To find and highlight duplicate rows in Excel, use COUNTIFS (with the letter S at the end) instead of COUNTIF.

1. Select the range A1:C10.

2. On the Home tab, in the Styles group, click Conditional Formatting.

3. Click New Rule.

4. Select ‘Use a formula to determine which cells to format’.

5. Enter the formula =COUNTIFS(Animals,$A1,Continents,$B1,Countries,$C1)>1

6. Select a formatting style and click OK.

Note: the named range Animals refers to the range A1:A10, the named range Continents refers to the range B1:B10 and the named range Countries refers to the range C1:C10. =COUNTIFS(Animals,$A1,Continents,$B1,Countries,$C1) counts the number of rows based on multiple criteria (Leopard, Africa, Zambia).

Result. Excel highlights the duplicate rows.

Explanation: if COUNTIFS(Animals,$A1,Continents,$B1,Countries,$C1) > 1, in other words, if there are multiple (Leopard, Africa, Zambia) rows, Excel formats cell A1. Always write the formula for the upper-left cell in the selected range (A1:C10). Excel automatically copies the formula to the other cells. We fixed the reference to each column by placing a $ symbol in front of the column letter ($A1, $B1 and $C1). As a result, cell A1, B1 and C1 contain the same formula, cell A2, B2 and C2 contain the formula =COUNTIFS(Animals,$A2,Continents,$B2,Countries,$C2)>1, etc.

7. Finally, you can use the Remove Duplicates tool in Excel to quickly remove duplicate rows. On the Data tab, in the Data Tools group, click Remove Duplicates.

In the example below, Excel removes all identical rows (blue) except for the first identical row found (yellow).

Note: visit our page about removing duplicates to learn more about this great Excel tool.

It is very easy to find duplicates in Excel. We can use built in tools (Conditional Formats, Filters) or formula (COUNTIF or VLOOKUP)  to find duplicates in Excel Columns or Rows. Let us see the best and easy methods to find duplicate values in Excel.

Find duplicates Using Conditional Formatting

We can easily identify the duplicates using Conditional Formatting Tools in Excel. We can select the range and highlight the duplicate values with color.  Following are the different methods to find duplicate entries in excel and strikethrough the duplicates using Conditional Formatting.

Highlighting duplicate values in Excel

We can highlight the duplicate values in the Excel with specific color. We can change the fill color and font color for the duplicate values. Please follow the below steps to identify the duplicates.

• Select the required range of cell to find duplicates
• Go to ‘Home’ Tab in the Ribbon Menu
• Click on the ‘Conditional Formatting’ command
• Go to ‘Highlight Cells Rules’ and Click on ‘Duplicate Values…’
• And Choose the formatting Options from the drop down list and Click on ‘OK’

Now you can see that all the duplicate values are highlighted as shown in the screen-shot.

Strike through duplicate values in Excel

We can strike through the duplicate values in the Excel. It is very helpful to strike out all duplicate entries in a range of cells. We can use the conditional formatting command in Excel to strike through the duplicate Cells.

• Select the required range of cell to strike through the duplicates
• Go to ‘Home’ Tab in the Ribbon
• Click on the ‘Conditional Formatting’ command
• Go to ‘Highlight Cells Rules’ and Click on ‘Duplicate Values…’
• And Choose the Custom Format from formatting Options from the drop down list
• Select the Strikethrough option from Font effects and Click on ‘OK’

Now you can see that all the duplicate values are stroked out as shown in the screen-shot.

Formula for checking duplicates

We can create formula using COUNTIF or VLOOKUP Functions to check the duplicates in Excel. Following is the step by explanation for finding duplicate values using Excel Formula.

Finding Duplicates using COUNTIF Formula

We can use COUNTIF Formula in Excel Cells to identify duplicates in Excel. COUNTIF formula helps to find duplicates in One Column, returns the number of occurrences of a given value. We can identify the duplicates if the Count is greater than 1. We can create new column to mark the duplicates using Excel COUNTIF Formula.

You can see that we have enter the COUNTIF formula in Range B2 to check the total occurrences of A1 value in the entire Column A:A.This will return the frequency of each Cell value of Column A in Column B.

We can identify the duplicate Cells based on the value in the Column B. If the values is greater than 1, we confirm that the values contains duplicate entries in the specified range or column.

Mark Duplicates and Unique Entries using Formula

We can enhance the above formula to mark all duplicates in a Column. The following formula will return ‘Duplicate’ if the cell value contains duplicate values, otherwise returns ‘Unique’.

=IF(COUNTIF(A:A,A2)>1,"Duplicate","Unique")

Formula will return ‘Duplicate’ in the Column B if the value repeats (count >1).

It will return ‘Unique’ in the Column B if the count is 1.

Find duplicate values in excel using VLookup

We can use VLOOKUP formula to compare two columns (or lists) and find the duplicate values. Vlookup helps to find duplicates in Two Column and duplicate rows based on Multiple Columns. Following are steps to find duplicates using VLOOKUP function.

We have two lists in the Excel List 1 in Column A and List 2 in Column B. Let us find the duplicate values in the List 2 which are part of List 1.

=VLOOKUP(B2,$A$2:$A$15,1,FALSE)

VLOOKUP Formula will check for the Cell B2 value in the specified Range (A2:A5) and Returns the value if found, it will return #NA if it is not found in List 1. We can confirm if the the value is duplicated in List 1 and 2 if it returns the Value, it will be unique if it returns #NA.

How to find duplicate values in Excel using VlookUp?

You can use Excel formula to find the duplicate values in Excel by using vlookup formula as described below. You can use the Excel VlookUp to identify the duplicates in Excel.

=IF(ISNA(VLOOKUP(B2,$A$2:$A$15,1,FALSE)),"Unique","Duplicate")

We can enhance this formula to identify the duplicates and unique values in the Column 2. We can use the Excel IF and ISNA Formulas along with VLOOKUP to return the required Labels.

You can use this method to compare with other worksheets and find duplicate values in Different Excel Sheets.

How to find duplicate values in two columns in excel using VlookUp?

Follow the below steps to find the duplicate values in two columns in excel using VlookUp. We generally need this to compare two columns and check if a value is existing in two columns.

=IF(IF(ISNA(VLOOKUP(C2,$A$2:$A$15,1,FALSE)),0,1)
+IF(ISNA(VLOOKUP(C2,$B$2:$B$15,1,FALSE)),0,1)=2,"Duplicated","Unique")

Here, we have Two Target Columns A and B , we check if the items in Column C are duplicated or not.

If you wants to compare two columns and check for values are repeated in both the columns or not, you can use this formula.

Share This Story, Choose Your Platform!

2 Comments

1. Guru
   August 12, 2019 at 7:45 pm — Reply

   Thanks for very useful instructions to check duplicates in Excel.

2. Tom
   August 12, 2019 at 7:48 pm — Reply

   This is very clear and simple! Thanks a lot for providing clear help to identify the duplicates in Excel.

Note: This tutorial on how to find duplicates in Excel is suitable for Excel 2007, Excel 2010, Excel 2013, Excel 2013, Excel 2019 and Office 365 users. 

Duplicate rows of data in a spreadsheet are every Excel user’s cause for a headache. They are annoying to deal with and eat a lot of time while cleaning up. 

You might have come across some guides bombarding you with complicated formulas to deal with duplicate rows. 

Related: 

Excel Goal Seek—the Easiest Guide (3 Examples)

Create A Pivot Table In Excel—the Easiest Guide

Excel Conditional Formatting -the Best Guide (Bonus Video)

Don’t fret. I have created this easy guide on how to find duplicates in Excel, making it a walk in the park for you. 

By the end of this guide, you’ll be able to find, highlight, count, filter, and remove duplicates in Excel instantly. 

In this article we’ll cover:

• Find Duplicates in Excel Using Conditional Formatting
  • How to Highlight Duplicates in Excel?
  • How to Find Duplicate Cells with the Exact Number of Occurrences?
  • How to Find Duplicate Rows in Excel?
• How to Remove Duplicate Rows in Excel?
• How to Use COUNTIF / COUNTIFS to Find Duplicates in Excel?
  • How to Find Duplicates in Excel Using COUNTIF?
  • How to Find Case Sensitive Duplicates?
  • How to Find Duplicate Rows in Excel Using COUNTIFS?
  • How to Count Duplicates in Excel Using COUNTIF?
  • How to Count the Number of Duplicate Rows in Excel Using COUNTIFS?
  • How to Filter Duplicates in Excel?

Find Duplicates in Excel Using Conditional Formatting

Conditional formatting is one of the quickest ways to find duplicates in Excel. It is a straightforward tool for highlighting duplicate values or duplicate rows in a sheet.

But, it’ll only work properly when you keep some important things in mind. We’ll cover these with the help of the following examples.  

How to Highlight Duplicates in Excel?

Let’s look at how to accomplish this with the help of these simple steps.

1. Get your Data ready as show below. I recommend you create named ranges for your columns, but it is not necessary. 
2. Select your Data and click on the Conditional Formatting button under the Home Ribbon
3. Under Highlight Cell Rules, click on Duplicate Values. 

1. Choose your preferred Format option or create a custom format. 

1. Click OK

1. You have successfully highlighted duplicates in Excel. 

Notice that doing this will highlight all instances of recurrence in the data range.

Also Read: 

The Best Excel Project Management Template In 2021

How To Use Excel Countifs: The Best Guide

Excel Sumifs & Sumif Functions – The No.1 Complete Guide

How to Find Duplicate Cells with the Exact Number of Occurrences?

Sometimes, you may want to find and highlight cells that repeat only a certain number of times.

For such cases, follow these steps. 

1. Clear any existing conditional formatting in your data. 

1. Under conditional formatting, locate and click on the New Rule option.

1. In the next window, under the rule type, select the Use a formula to determine which cells to format option.

1. In this example, Under “Format Values where this formula is true” I enter =COUNTIF($A$1:$C$13,A1)=2, since my data spans from cell A1 to cells C13.

In our case, we want to find the number of cells that repeat only two times. Hence, the “=2” at the end of the formula. 

You can use any number or logical operator (<,>, etc) here that you prefer. In this case, the COUNTIF formula returns the number of occurrences where a cell repeats in the data range A1:C13. If the value is equal to two, conditional formatting is applied. 

1. Choose your preferred Formatting style and Click OK.

How to Find Duplicate Rows in Excel?

Sometimes, you may want to find and highlight duplicate rows of data instead of just cells. I’ll show you a foolproof method to implement this in Excel. Follow these steps. 

1. Select the Data Range that contains the duplicate rows. 
2. Clear any existing conditional formatting in your data. 
3. Under Conditional Formatting, locate and click on the New Rule option.

1. In the next window, under the Rule Type select the Use a formula to determine which cells to format option.
2. Under “Format Values where this formula is true” I enter =COUNTIFS($A$1:$A$13,$A1,$B$1:$B$13,$B1,$C$1:$C$13,$C1)>1

The COUNTIFS formula checks every column simultaneously for cases where all these duplicate cells repeat in the same row. That is it checks for duplicate rows. Change A1:A13, B1:B13, C1:C13 suitably as per your data range.

1. Choose your preferred Formatting Style and Click OK. 

How to Remove Duplicate Rows in Excel?

In some cases, just highlighting duplicate rows is not enough. You also need to delete them. 

Follow these steps to easily remove duplicate rows. 

1. Get your data ready
2. Locate the Data Tools section under the Data Tab of your Excel Sheet

1. Under Data Tools, click on the Remove Duplicates button. 

4. Select all the columns for deleting duplicate rows and Click OK.  Voila! Excel removes all duplicate rows in just a click.

How to Use COUNTIF / COUNTIFS to Find Duplicates in Excel?

When it comes to handling duplicates, there cannot be a more robust and suitable formula than COUNTIFS. It can be used for granular level actions like finding duplicates excluding the first instance, finding case sensitive duplicates, counting the number of duplicates until each row, etc. I’ll show you how to do all of this with the help of examples.

How to Find Duplicates in Excel Using COUNTIF?

To find duplicates using COUNTIF, follow these steps. 

Let’s assume I have the column “A” full of repetitive data and I need to identify these duplicates using the COUNTIF formula. 

How to Find Duplicates Including the First Instance?

1. Get your Data ready
2. In the ‘B2’ cell, i.e right next to column “A”, type in 

= IF(COUNTIF($A$2:$A$13,A2)>1,”Yes”,””)

Here, the COUNTIF function compares each cell in the range A2:A13 for duplicates and returns the total number of repetitions for each case. The “IF” function then checks if the number of repetitions is greater than one. If it is greater than one, it is a duplicate, else, not a duplicate. 

1. Drag this formula to the end of the data range using the fill handle. 
2. All your duplicate cells are now marked with “YES”

Did you notice that it is marking even the first instances of data as duplicates?

Sometimes this is not required, especially if you are planning to delete or filter the data later. To avoid this from happening, follow the steps below. 

How to Find Duplicates Excluding the First Instance?

I am working with the same set of Data in Column “A”

1. Get your Data ready.

1. In the ‘B2’ cell, i.e right next to column “A”, type in =IF(COUNTIF($A$2:$A2,A2)>1,”Yes”,””)

Here, the COUNTIF function searches each Cell in the Data Range of Column “A” for duplicates. It then returns the number of repetitions until that particular row. The IF function then compares if the number of repetitions is greater than one, in order to mark as duplicates. Hence, the first instance of any duplicate will return a value of 1 and the first instance will not be marked as a duplicate. 

1. Drag the formula to the end of the Data range using the fill handle.

1. All your duplicates except the first one are marked with “YES”. 

How to Find Case Sensitive Duplicates?

All the above methods consider data with different text-cases as the same and count them as duplicates. 

But, sometimes, you may want to find an exactly matching duplicate. To accomplish this, follow these simple steps. 

1. Get your data ready
2. In the ‘B2’ cell, i.e right next to column “A”, type in =IF(SUM((–EXACT($A$2:$A$13,A2)))<=1,””,”Yes”)

Here, the “EXACT” function searches for exact matches to return either 1 or 0. The SUM function returns the total number of exact duplicates. If it is greater than 1, the cell is marked as a duplicate. 

1. Drag the formula to the end of the Data range using the fill handle.
2. All your case sensitive duplicates are marked with “Yes”.

How to Find Duplicate Rows in Excel Using COUNTIFS?

Now, let’s see how to easily find duplicate rows in your sheet using the COUNTIFS functions. 

If you want to include the first instance, just follow these simple steps. 

1. Get your Data ready. I am using a sheet that contains three columns of data. 
2. In ‘D2’  cell, i.e right next to the Data enter =IF(COUNTIFS($A$2:$A$13,A2,$B$2:$B$13,B2,$C$2:$C$13,C2)>1, “Yes”, “”)

This is exactly the same as the COUNTIF function, except here it checks all the columns simultaneously for duplicates. Hence, it can find all duplicate rows easily. 

1. Drag the formula to the end of the data range using the fill handle.
2. All your duplicate rows including the first instances are marked with “Yes”

If you want to exclude the first instance, just follow these simple steps

1. Get your Data ready, I am using the same Sheet here. 

1. In ‘D2’  cell, i.e right next to the Data enter =IF(COUNTIFS($A$2:$A2,$A2,$B$2:$B2,$B2,$C$2:$C2,$C2)>1, “Yes”, “”)

Here, the COUNTIFS function compares each row in the data range for duplicates and returns the number of repetitions until that particular row.

The IF function then compares if the number of repetitions is greater than one to mark as duplicates. The first instance of any duplicate will return a value of 1. Hence, the first instance will not be marked as a duplicate

1. Drag the formula to the end of the Data range using the fill handle.

1. All your duplicate rows excluding the first instances are marked with “Yes”

How to Count Duplicates in Excel Using COUNTIF?

There may be situations where you need to find the number of duplicates in your Data. 

Let’s see how to tackle it easily.

How to Count Duplicates for Each Cell?

To count the number of duplicates for each cell just follow these steps. 

1. Get your Data ready.

1. In ‘B2’ Cell, enter =COUNTIF($A$2:$A$13,A2)

The COUNTIF function simply counts and returns the number of times a single data value gets repeated in the Range A2:A13. 

1. Drag the formula to the end of the data range using the fill handle.
2. All your duplicate data have a corresponding cell that displays their number of occurrences. 

How to Count Duplicates for Each Cell Until Each Row?

To count the number of repeats for each cell until each row, follow these steps.

1. Get your Data ready
2. In ‘B2’ Cell, enter =COUNTIF($A$2:$A$13,A2)

Here, the COUNTIF function searches each Cell in the Data Range of Column “A” for duplicates. It then returns the number of repetitions until that particular row. Hence, we get the number of instances until each row. 

1. Drag the formula to the end of the data range using the fill handle.
2. Each duplicate data has a corresponding cell that displays their number of instances until that row. 

How to Count the Number of Duplicate Rows in Excel Using COUNTIFS?

You can use the same methods to find the number of occurrences for duplicate rows. Just use the COUNTIFS functions in the place of the COUNTIF function. 

For example, in the following case I enter =COUNTIFS($A$2:$A$13,A2,$B$2:$B$13,B2,$C$2:$C$13,C2) in the “D2” cell to get the desired result

How to Filter Duplicates in Excel?

Finally, you may want to filter all duplicates found in your Sheet. This is a very useful feature that will come in handy if you want to further delete, move or copy these duplicates. 

Let’s see how to do this easily. Just follow these steps. 

1. Select the Data where you already have found the duplicates using the COUNTIFS formula.
2. Under the Data Tab, locate and click on the Filter button. 
3. Now, click on the arrow on the “Repeat” column header and check only “Yes” to show only the duplicates.
4. You can also uncheck “Yes” to hide all duplicates. 

Do anything as you please with these filtered duplicate rows of data. You can remove, copy to another sheet, or simply hide them. 

Suggested Reads:

How To Protect Cells In Excel Workbooks-the Easiest Way

Create An Excel Dashboard In 5 Minutes – The Best Guide

Dynamic Dropdown Lists In Excel – Top Data Validation Guide

FAQs

Closing Thoughts

We are at the end of this guide on how to find duplicates in Excel. I have covered almost all important user-friendly ways to find duplicates in Excel. All of these methods will come in handy and will suffice for most users. 

If you need more high-quality guides on Excel functions and features check out our free Excel resources section. 

If you are interested in gaining in-depth knowledge and practice in Excel try our Excel Courses here. 

Watch Video – How to Find and Remove Duplicates in Excel

With a lot of data…comes a lot of duplicate data. 

Duplicates in Excel can cause a lot of troubles. Whether you import data from a database, get it from a colleague, or collate it yourself, duplicates data can always creep in. And if the data you are working with is huge, then it becomes really difficult to find and remove these duplicates in Excel.

In this tutorial, I’ll show you how to find and remove duplicates in Excel.

Find and Highlight Duplicates in Excel

Duplicates in Excel can come in many forms. You can have it in a single column or multiple columns. There may also be a duplication of an entire row.

Finding and Highlight Duplicates in a Single Column in Excel

Conditional Formatting makes it simple to highlight duplicates in Excel.

Here is how to do it:

• Select the data in which you want to highlight the duplicates.

• Go to Home –> Conditional Formatting –> Highlight Cell Rules –> Duplicate Values.

• In the Duplicate Values dialog box, select Duplicate in the drop down on the left, and specify the format in which you want to highlight the duplicate values. You can choose from the ready-made format options (in the drop down on the right), or specify your own format.

• This will highlight all the values that have duplicates.

Quick Tip: Remember to check for leading or trailing spaces. For example, “John” and “John ” are considered different as the latter has an extra space character in it. A good idea would be to use the TRIM function to clean your data.

Finding and Highlight Duplicates in Multiple Columns in Excel

If you have data that spans multiple columns and you need to look for duplicates in it, the process is exactly the same as above.

Here is how to do it:

• Select the data.
• Go to Home –> Conditional Formatting –> Highlight Cell Rules –> Duplicate Values.
• In the Duplicate Values dialog box, select Duplicate in the drop down on the left, and specify the format in which you want to highlight the duplicate values.
• This will highlight all the cells that have duplicates value in the selected data set.

Finding and Highlighting Duplicate Rows in Excel

Finding duplicate data and finding duplicate rows of data are 2 different things. Have a look:

Finding duplicate rows is a bit more complex than finding duplicate cells.

Here are the steps:

• In an adjacent column, use the following formula:
  =A2&B2&C2&D2
  Drag this down for all the rows. This formula combines all the cell values as a single string. (You can also use the CONCATENATE function to combine text strings)

By doing this, we have created a single string for each row. If there are duplicate rows in this dataset, then these strings would be exactly the same for it.

Now that we have the combined strings for each row, we can use conditional formatting to highlight duplicate strings. A highlighted string implies that the row has a duplicate.

Here are the steps to highlight duplicate strings:

• Select the range that has the combined strings (E2:E16 in this example).
• Go to Home –> Conditional Formatting –> Highlight Cell Rules –> Duplicate Values.
• In the Duplicate Values dialog box, make sure Duplicate is selected and then specify the color in which you want to highlight the duplicate values.

This would highlight the duplicate values in column E.

In the above approach, we have highlighted only the strings that we created.

But what if you want to highlight all the duplicate rows (instead of highlighting cells in one single column)?

Here are the steps to highlight duplicate rows:

• In an adjacent column, use the following formula:
  =A2&B2&C2&D2
  Drag this down for all the rows. This formula combines all the cell values as a single string.

• Select the data A2:D16.
• With the data selected, go to Home –> Conditional Formatting –> New Rule.

• In the ‘New Formatting Rule’ dialog box, click on ‘Use a formula to determine which cells to format’.
• In the field below, use the following COUNTIF function:
  =COUNTIF($E$2:$E$16,$E2)>1

• Select the format and click OK.

This formula would highlight all the rows that have a duplicate.

Remove Duplicates in Excel

In the above section, we learned how to find and highlight duplicates in excel. In this section, I will show you how to get rid of these duplicates.

Remove Duplicates from a Single Column in Excel

If you have the data in a single column and you want to remove all the duplicates, here are the steps:

• Select the data.
• Go to Data –> Data Tools –> Remove Duplicates.
• In the Remove Duplicates dialog box:
  • If your data has headers, make sure the ‘My data has headers’ option is checked.
  • Make sure the column is selected (in this case there is only one column).

• Click OK.

This would remove all the duplicate values from the column, and you would have only the unique values.

CAUTION: This alters your data set by removing duplicates. Make sure you have a back-up of the original data set. If you want to extract the unique values at some other location, copy this dataset to that location and then use the above-mentioned steps. Alternatively, you can also use Advanced Filter to extract unique values to some other location.

Remove Duplicates from Multiple Columns in Excel

Suppose you have the data as shown below:

In the above data, row #2 and #16 have the exact same data for Sales Rep, Region, and Amount, but different dates (same is the case with row #10 and #13). This could be an entry error where the same entry has been recorded twice with different dates.

To delete the duplicate row in this case:

• Select the data.
• Go to Data –> Data Tools –> Remove Duplicates.
• In the Remove Duplicates dialog box:
  • If your data has headers, make sure the ‘My data has headers’ option is checked.
  • Select all the columns except the Date column.

• Click OK.

This would remove the 2 duplicate entries.

NOTE: This keeps the first occurrence and removes all the remaining duplicate occurrences.

Remove Duplicate Rows in Excel

To delete duplicate rows, here are the steps:

• Select the entire data.
• Go to Data –> Data Tools –> Remove Duplicates.
• In the Remove Duplicates dialog box:
  • If your data has headers, make sure the ‘My data has headers’ option is checked.
  • Select all the columns.

• Click OK.

Use the above-mentioned techniques to clean your data and get rid of duplicates.

You May Also Like the Following Excel Tutorials:

• 10 Ways to Clean Data in Excel Spreadsheets.
• Remove Leading and Trailing Spaces in Excel.
• 24 Daily Excel Issues and their Quick Fixes.
• How to Find Merged Cells in Excel.

Duplicate values in your data can be a big problem! It can lead to substantial errors and over estimate your results.

But finding and removing them from your data is actually quite easy in Excel.

In this tutorial, we are going to look at 7 different methods to locate and remove duplicate values from your data.

Video Tutorial

What Is A Duplicate Value?

Duplicate values happen when the same value or set of values appear in your data.

For a given set of data you can define duplicates in many different ways.

In the above example, there is a simple set of data with 3 columns for the Make, Model and Year for a list of cars.

1. The first image highlights all the duplicates based only on the Make of the car.
2. The second image highlights all the duplicates based on the Make and Model of the car. This results in one less duplicate.
3. The second image highlights all the duplicates based on all columns in the table. This results in even less values being considered duplicates.

The results from duplicates based on a single column vs the entire table can be very different. You should always be aware which version you want and what Excel is doing.

Find And Remove Duplicate Values With The Remove Duplicates Command

Removing duplicate values in data is a very common task. It’s so common, there’s a dedicated command to do it in the ribbon.

Select a cell inside the data which you want to remove duplicates from and go to the Data tab and click on the Remove Duplicates command.

Excel will then select the entire set of data and open up the Remove Duplicates window.

1. You then need to tell Excel if the data contains column headers in the first row. If this is checked, then the first row of data will be excluded when finding and removing duplicate values.
2. You can then select which columns to use to determine duplicates. There are also handy Select All and Unselect All buttons above you can use if you’ve got a long list of columns in your data.

When you press OK, Excel will then remove all the duplicate values it finds and give you a summary count of how many values were removed and how many values remain.

This command will alter your data so it’s best to perform the command on a copy of your data to retain the original data intact.

Find And Remove Duplicate Values With Advanced Filters

There is also another way to get rid of any duplicate values in your data from the ribbon. This is possible from the advanced filters.

Select a cell inside the data and go to the Data tab and click on the Advanced filter command.

This will open up the Advanced Filter window.

1. You can choose to either to Filter the list in place or Copy to another location. Filtering the list in place will hide rows containing any duplicates while copying to another location will create a copy of the data.
2. Excel will guess the range of data, but you can adjust it in the List range. The Criteria range can be left blank and the Copy to field will need to be filled if the Copy to another location option was chosen.
3. Check the box for Unique records only.

Press OK and you will eliminate the duplicate values.

Advanced filters can be a handy option for getting rid of your duplicate values and creating a copy of your data at the same time. But advanced filters will only be able to perform this on the entire table.

Find And Remove Duplicate Values With A Pivot Table

Pivot tables are just for analyzing your data, right?

You can actually use them to remove duplicate data as well!

You won’t actually be removing duplicate values from your data with this method, you will be using a pivot table to display only the unique values from the data set.

First, create a pivot table based on your data. Select a cell inside your data or the entire range of data ➜ go to the Insert tab ➜ select PivotTable ➜ press OK in the Create PivotTable dialog box.

With the new blank pivot table add all fields into the Rows area of the pivot table.

You will then need to change the layout of the resulting pivot table so it’s in a tabular format. With the pivot table selected, go to the Design tab and select Report Layout. There are two options you will need to change here.

1. Select the Show in Tabular Form option.
2. Select the Repeat All Item Labels option.

You will also need to remove any subtotals from the pivot table. Go to the Design tab ➜ select Subtotals ➜ select Do Not Show Subtotals.

You now have a pivot table that mimics a tabular set of data!

Pivot tables only list unique values for items in the Rows area, so this pivot table will automatically remove any duplicates in your data.

Find And Remove Duplicate Values With Power Query

Power Query is all about data transformation, so you can be sure it has the ability to find and remove duplicate values.

Select the table of values which you want to remove duplicates from ➜ go to the Data tab ➜ choose a From Table/Range query.

Remove Duplicates Based On One Or More Columns

With Power Query, you can remove duplicates based on one or more columns in the table.

You need to select which columns to remove duplicates based on. You can hold Ctrl to select multiple columns.

Right click on the selected column heading and choose Remove Duplicates from the menu.

You can also access this command from the Home tab ➜ Remove Rows ➜ Remove Duplicates.

= Table.Distinct(#"Previous Step", {"Make", "Model"})

If you look at the formula that’s created, it is using the Table.Distinct function with the second parameter referencing which columns to use.

Remove Duplicates Based On The Entire Table

To remove duplicates based on the entire table, you could select all the columns in the table then remove duplicates. But there is a faster method that doesn’t require selecting all the columns.

There is a button in the top left corner of the data preview with a selection of commands that can be applied to the entire table.

Click on the table button in the top left corner ➜ then choose Remove Duplicates.

= Table.Distinct(#"Previous Step")

If you look at the formula that’s created, it uses the same Table.Distinct function with no second parameter. Without the second parameter, the function will act on the whole table.

Keep Duplicates Based On A Single Column Or On The Entire Table

In Power Query, there are also commands for keeping duplicates for selected columns or for the entire table.

Follow the same steps as removing duplicates, but use the Keep Rows ➜ Keep Duplicates command instead. This will show you all the data that has a duplicate value.

Find And Remove Duplicate Values Using A Formula

You can use a formula to help you find duplicate values in your data.

First you will need to add a helper column that combines the data from any columns which you want to base your duplicate definition on.

= [@Make] & [@Model] & [@Year]

The above formula will concatenate all three columns into a single column. It uses the ampersand operator to join each column.

= TEXTJOIN("", FALSE , CarList[@[Make]:[Year]])

If you have a long list of columns to combine, you can use the above formula instead. This way you can simply reference all the columns as a single range.

You will then need to add another column to count the duplicate values. This will be used later to filter out rows of data that appear more than once.

= COUNTIFS($E$3:E3, E3)

Copy the above formula down the column and it will count the number of times the current value appears in the list of values above.

If the count is 1 then it’s the first time the value is appearing in the data and you will keep this in your set of unique values. If the count is 2 or more then the value has already appeared in the data and it is a duplicate value which can be removed.

Add filters to your data list.

• Go to the Data tab and select the Filter command.
• Use the keyboard shortcut Ctrl + Shift + L.

Now you can filter on the Count column. Filtering on 1 will produce all the unique values and remove any duplicates.

You can then select the visible cells from the resulting filter to copy and paste elsewhere. Use the keyboard shortcut Alt + ; to select only the visible cells.

Find And Remove Duplicate Values With Conditional Formatting

With conditional formatting, there’s a way to highlight duplicate values in your data.

Just like the formula method, you need to add a helper column that combines the data from columns. The conditional formatting doesn’t work with data across rows, so you’ll need this combined column if you want to detect duplicates based on more than one column.

Then you need to select the column of combined data.

To create the conditional formatting, go to the Home tab ➜ select Conditional Formatting ➜ Highlight Cells Rules ➜ Duplicate Values.

This will open up the conditional formatting Duplicate Values window.

1. You can select to either highlight Duplicate or Unique values.
2. You can also choose from a selection of predefined cell formats to highlight the values or create your own custom format.

Warning: The previous methods to find and remove duplicates considers the first occurrence of a value as a duplicate and will leave it intact. However, this method will highlight the first occurrence and will not make any distinction.

With the values highlighted, you can now filter on either the duplicate or unique values with the filter by color option. Make sure to add filters to your data. Go to the Data tab and select the Filter command or use the keyboard shortcut Ctrl + Shift + L.

1. Click on the filter toggle.
2. Select Filter by Color in the menu.
3. Filter on the color used in the conditional formatting to select duplicate values or filter on No Fill to select unique values.

You can then select just the visible cells with the keyboard shortcut Alt + ;.

Find And Remove Duplicate Values Using VBA

There is a built in command in VBA for removing duplicates within list objects.

Sub RemoveDuplicates() Dim DuplicateValues As Range Set DuplicateValues = ActiveSheet.ListObjects("CarList").Range DuplicateValues.RemoveDuplicates Columns:=Array(1, 2, 3), Header:=xlYes End Sub

The above procedure will remove duplicates from an Excel table named CarList.

Columns:=Array(1, 2, 3)

The above part of the procedure will set which columns to base duplicate detection on. In this case it will be on the entire table since all three columns are listed.

Header:=xlYes

The above part of the procedure tells Excel the first row in our list contains column headings.

You will want to create a copy of your data before running this VBA code, as it can’t be undone after the code runs.

Conclusions

Duplicate values in your data can be a big obstacle to a clean data set.

Thankfully, there are many options in Excel to easily remove those pesky duplicate values.

So, what’s your go to method to remove duplicates?

When you are working on a large number of data then you will encounter a few duplicate entries.

Sometimes these duplicate values are very useful but many times you want to remove the duplicate data. If you have 1000 data then it becomes quite difficult to find and remove the duplicate data.

Here we have provided the step-by-step process on how to find duplicates in excel.

Take a look and follow the process.

There are many ways to find duplicate items and values in excel. You might be thinking as to why should I apply any formula or method to find duplicate values as it is easy.

But it’s not about few data, you can apply a formula or method when you have lots of data.

The method or formula to find and remove the duplicate items make the process easier and saves your time. Here you can check three different processes.

1. Find Duplicates in Excel using Conditional Formatting

To find duplicate values in Excel, you can use conditional formatting excel formula, Vlookup, and Countif formula. After finding out the duplicate values, you can remove them if you want by using different methods that are described below.

Remove duplicate values

• Select all the duplicate cells or highlighted cells
• Delete the values by pressing the Delete button.
• After deleting the values, go to the conditional formatting.
• Choose Clear rules.
• And then choose clear rules from an entire sheet.

Another way to remove the duplicate value

This method will delete all the duplicate data permanently. If you want the duplicate data again, then copy the data on another worksheet.

• Select the cells on which you want to find and remove the duplicate values.
• Go to the ribbon and find the data option.
• In the data tab, you will find the remove duplicates option.
• Now check or uncheck the column on which you want to apply this constraint.
• Then click OK.

2. Find Duplicates in One Column using COUNTIF

Another way to detect duplicates is via the COUNTIF function. If you want to apply the COUNTIF function then check here how to find duplicates in Excel using the Excel COUNT function.

• If you have the list of items from where you want to find duplicates, then you need to apply the COUNTIF formula to it.
• The syntax for COUNTIF: “=COUNTIF(B:B, B2)>1”.
• In column A, you have the Buyer’s name and in column B, you have the fruits name that he or she likes. Now you want to find the duplicate items.
• Put this COUNTIF formula in column C.
• For all duplicate fields, it shows TRUE whereas, for non-duplicate fields, it shows FALSE.
• Now by following the above step-by-step process, you can delete all the duplicate items that you do not need.

Find Duplicates in Two Columns in Excel

Above we have seen how to find duplicate values in one column, now we will see here how to find duplicates in two columns in excel.

In this example, we have taken a table where the candidate name is in column A and Fruits is in column B. Now we want to find duplicate values having the same name and fruits.

The formula to find duplicate values in two columns is

=If (Countifs($A$2:$A$8, $A2, $B$2:$B$8, $B2)>1, "Duplicate Values"," ")

How to Count Duplicates in Excel

If you want to know the total number of duplicate values then you need to use the Count function. For counting duplicate values you need to use the countif formula.

=Countif($A$2:$A$8, $A2)

3. Filter Duplicates in Excel

Another method to find duplicates in excel is to filter data to get duplicate values.

Remove Duplicates Online

To remove duplicates, you need to follow the below-mentioned process

• Select the table by dragging the mouse.
• Right-click on the table.
• Delete the row by choosing the Delete Row option.

Highlight Duplicates in Excel

In order to highlight the duplicate values, you can either use a conditional formula to find duplicates in excel or follows the below-mentioned process.

• Select the duplicate values.
• Go to the Fill color option and select the color.

• This will highlight the duplicate values in excel.

Apart from these, there are other processes available by which you can find the duplicate items available in your data set. After finding out the duplicate values, you can remove them easily. Stay connected with our website to know other Excel features.

Related Queries

Here we have listed a few queries that are asked by Excel learners. If you are also facing this kind of problem then you can check it here.

Q1. How to delete duplicate e-mail addresses if you have 10,000 emails in your data set?

Ans. If you have all email addresses in a single column then select the column and go to the “Data” ribbon. You will find the ‘Remove Duplicates’ tab in data tools.

Click on Remove Duplicates and you will find another dialog box where you need to make a selection. Press Ok and you receive the unique email addresses.

Q2. How to find the repeated teacher’s name in Column A? The data is

• B1= Maths/English Jasmine, Eon
• B2= Physics Johana
• B3= History Poole
• B4= English/Physics Sam, Jasmine
• B5= Chemistry Romeo

Ans. To solve this problem, create three separate columns. In the first column, mention the class name. In the second column, put the teacher’s name, and in the third mention another teacher’s name. Now in the fourth column, put the formula =COUNTIF(B:B,B1)>1

Using this formula, you will find “TRUE” if the value is duplicate.

Q3. How to sort data before applying a filter?

Ans. To sort data you need to choose the Sort tab in excel. Sorting a data set is quite an easy process. Follow the process mentioned below and sort your data completely.

• Select the data set.
• Go to the Data ribbon.
• In the Sort & Filter section, you will find the Sort icon. Click on the icon and sort your data.

Q4. How to compare two columns in excel?

This question was mainly asked by the interviewer at the time of the interview. There are many ways to compare the two columns:

1. =A2=B2 This is the simple formula used to compare two columns.
2. Conditional Formatting: In this, you need to follow the above process, you can check the conditional formatting excel link to get detailed information.
3. Vlookup: You can also use the Vlookup formula to compare two columns using Vlookup for research, ISNA for performance, and If to customize the result.
4. Excel If Statement: This is a quite useful function in excel and it can be used in many ways. To get the detailed information Click Here.

Q5. How to filter data in excel?

Filtering in Excel is an easy process. In the sort & filter section, you will find the Filter icon. Click on the icon and filter your data.

In this tutorial, we have covered all the excel topics that are best for beginners and experts. We always try to solve all the problems that people face while working on excel. If you have any other query or want to ask something then you can mention in the comment box below. One of our team members will be back with the answer.

You may also like the following Excel tutorials:

• How to Remove Formulas in Excel (and keep the data)
• How to Fill Blank Cells with Value above in Excel
• How to Generate Random Numbers in Excel (Without Duplicates)
• How to Find Merged Cells in Excel
• Highlight Cell If Value Exists in Another Column in Excel
• Using Conditional Formatting with OR Criteria in Excel
• How to Remove Duplicate Rows based on one Column in Excel?
• Get Unique Values from a Column in Excel
• How to Count Unique Values in Excel (Formulas)
• How to Use Greater Than or Equal to Operator in Excel Formula?