5 concise ways to remove duplicates from an array in JavaScript?
There are several ways to remove duplicate elements from a Javascript array. In this article, I show you the most common ways.
If what you are looking for is to remove elements from an array, even if they are not duplicate elements, see our article How to remove an element from an array in JavaScript?
Table of Contents
Remove duplicate elements from an array using Set
The simplest and most concise way to eliminate repeated values from an array is by using the Set
structure.
Set
was introduced in ES6. Set
objects are collections of unique values. The Set
object allows you to store unique values of any type, whether primitive values or object references.
The Set
constructor creates Set
objects and will return a new Set
object.
const numbers = [1, 2, 3, 4, 1, 2, 5, 6];
// Returns a new Set
const uniqueValuesSet = new Set(numbers);
// Convert the new Set to an Array
const uniqueValuesArray = [...uniqueValuesSet];
// If you want to create the Set and convert it to an Array
// in only one line
const mostConcise = [...new Set(numbers)];
Remove duplicate elements from an array using filter
In addition to using filter
, we will have to use the indexOf
method.
The Array indexOf
method returns the first index where it finds the specified value, it’ll return -1
if the value is not found.
const numbers = [7, 8, 9, 1, 2, 9];
// Note there are two values of 9
// indexOf returns the index of the first 9
numbers.indexOf(9); // ๐ 2
The filter
method of Array creates a shallow copy of a given array, filtered down to the elements that passed the test implemented by the callback function.
const numbers = [7, 8, 9, 1, 2, 8, 9];
// filter returns a new array with numbers greater than 5
const higherThanFive = numbers.filter((n) => {
return n > 5;
});
console.log(higherThanFive); // ๐ [7, 8, 9, 8, 9]
The second parameter of the callback function is the index of the current element being processed in the array.
So how can we use filter
and indexOf
to delete duplicate numbers?
const numbers = [7, 8, 9, 1, 2, 8, 9];
const unique = numbers.filter((n, index) => numbers.indexOf(n) === index);
console.log(unique); // ๐ [7, 8, 9, 1, 2]
Solution explained: When the index is not equal to the result of indexOf
, the current element is a duplicate. Remember, indexOf
returns the index of the first occurrence it finds.
To visualize the solution, let’s look at the following table. We identify the repeated value when the index of the element does not correspond to the return value of indexOf
. Note the rows where the condition is false
.
Element | Index | indexOf returns | Condition |
7 | 0 | 0 | true |
8 | 1 | 1 | true |
9 | 2 | 2 | true |
1 | 3 | 3 | true |
2 | 4 | 4 | true |
8 | 5 | 1 | false |
9 | 6 | 2 | false |
false
when the value in the column “Index” and the value in the column “indexOf
returns” are different. Remove duplicate elements from an array using reduce
The reduce
method executes a function on each element of an array and returns a single value. For an explanation of how reduce
works, see the article How to use reduce in Javascript? [5 EXAMPLES].
The includes
method returns true
if it finds the specified value in the array, otherwise, it returns false
. Here is a simple example of using includes
:
[1, 2, 3, 4].includes(3) // ๐ true
[1, 2, 3, 4].includes(6) // ๐ false
Use both reduce
and includes
to detect repeated values in an array.
const numbers = [7, 8, 9, 1, 2, 8, 9];
const unique = numbers.reduce((accumulator, item) => {
return accumulator.includes(item) ? accumulator : [...accumulator, item];
}, []);
If you don’t like using ternary expressions, here is an alternative version.
const numbers = [7, 8, 9, 1, 2, 8, 9];
const unique = numbers.reduce((accumulator, item) => {
if (!accumulator.includes(item)) {
accumulator.push(item);
}
return accumulator;
}, []);
Remove duplicate elements from a JavaScript array using forEach or other loops
For junior developers, perhaps this is the easiest one to understand. You can come up with a variation of this solution using the classic for
loop.
const numbers = [7, 8, 9, 1, 2, 8, 9];
const unique = [];
numbers.forEach((item) => {
if (!unique.includes(item)) {
unique.push(item);
}
})
console.log(unique); // ๐ [7, 8, 9, 1, 2];
Remove duplicate values from an array using Lodash’s _.uniq
An easy way to eliminate repeated elements from an array is by using a library such as Lodash. It is a “modern JavaScript utility library delivering modularity, performance & extras”.
After downloading and installing the library in your project, it would be as simple as this:
const _ = require('lodash/core');
const numbers = [7, 8, 9, 1, 2, 8, 9];
const unique = _.uniq(numbers);
Of course, using a library like Lodash has a cost. I don’t mean that it costs money, using it is free. What I mean is that we have to make a trade-off, we will receive the benefit of using the _.un
iq function in exchange for having a dependency on our project.
If other Lodash functions will be useful to you in the same project, then you will have to weigh whether it is worth having the library as a dependency. If the only function you plan to use is _.uniq
, my recommendation is that you not install Lodash and program your own function instead.
Code your own function to remove duplicate elements
You could take any of the solutions above and create your very own utility function. You will be able to reuse the function throughout the project.
function removeDuplicates(arr) {
ย ย const set = new Set(arr);
ย ย
ย ย return [...set];
}
const numbers = [7, 8, 9, 1, 2, 8, 9];
const unique = removeDuplicates(numbers);
When I program reusable functions, I usually place them in a file called utils.js
.
Of course, if you have several of these types of reusable functions and want to be more specific, you can have more appropriately named files, like array.utils.js
, string.utils.js
, and so on.