Guides

How to remove duplicates from an array in JavaScript — and why Set can't see identical users

Illustration: a sieve lets through one ball of each color, the extras roll into a bin

You search "how to remove duplicates from an array" and find the magic line [...new Set(arr)]. You try it on a list of users from an API — and the list still has three identical "Annas". The line isn't broken. It just doesn't work for objects, and that's not obvious until you step on it.

Let's go step by step, from the simple case to objects. By the end you'll have working code for each case and a prompt to get AI to do it properly.

Step by step: from numbers to objects

  1. Numbers and strings — one line with Set. A Set is a built-in collection that stores each value only once. Put an array into it and the repeats fall away on their own. Then three dots pour the result back into a normal array:

    const tags = ["js", "css", "js", "html", "css"]
    const unique = [...new Set(tags)]
    // ["js", "css", "html"]
    

    Order is preserved: each item stays where it first appeared. More on the three dots in what is the spread operator.

  2. Clean the values before comparing. To a computer, "Anna", "anna" and "Anna " with a trailing space are three different strings. Set keeps all three. If they mean the same to you, bring the strings to one shape first:

    const names = ["Anna", "anna", "Anna ", "Boris"]
    const unique = [...new Set(names.map(n => n.trim().toLowerCase()))]
    // ["anna", "boris"]
    

    trim() cuts spaces from the edges, toLowerCase() makes every letter lowercase. This matters most for emails: people type them any old way.

  3. Understand why Set fails with objects. Here's that case from the start:

    const users = [
      { id: 1, name: "Anna" },
      { id: 2, name: "Boris" },
      { id: 1, name: "Anna" },
    ]
    console.log([...new Set(users)].length)   // 3 — nothing removed
    

    Set compares objects not by contents but by reference — "is this the very same box or a different one?" The two "Annas" look identical, but they're two different boxes with identical labels. So you have to say yourself which field defines a duplicate. Almost always that's id.

  4. Remove duplicate objects by id. Keep a Set of ids you've already seen and let through only the ones seen for the first time:

    const seen = new Set()
    const uniqueUsers = users.filter(user => {
      if (seen.has(user.id)) return false   // seen before — drop it
      seen.add(user.id)                     // remember it
      return true                           // first time — keep it
    })
    // [{ id: 1, name: "Anna" }, { id: 2, name: "Boris" }]
    

    This keeps the first occurrence. If you want the opposite, the most recent one, there's a trick with Map — a repeated key overwrites the previous one:

    const latest = [...new Map(users.map(u => [u.id, u])).values()]
    

    The first version reads more easily; pick it if you're unsure.

  5. Check that it worked. Print the length before and after: console.log(users.length, uniqueUsers.length). If the numbers match but you still see duplicates on screen, the repeats have different ids. Then look for another field they share, such as email.

What you end up with

Now you have three tools for three situations:

  • Simple values — [...new Set(arr)].
  • Strings where meaning matters, not spelling — trim().toLowerCase() first, then Set.
  • An array of objects — filter with a Set of already-seen ids.

None of them touches the original array; all return a new one. That matters if the data lives in React state: there you mustn't change the original.

Ask AI to do it properly

If you just write "remove duplicates", AI will almost certainly drop in new Set — and for objects you'll get the same non-working result. Spell out what's in the array and what counts as a duplicate:

Weak promptRemove duplicates from the array
Strong promptThe variable orders holds an array of order objects with id, email and createdAt fields. Remove duplicates: a duplicate is an order with the same id. Keep the first one in order. Do not modify the original array, return a new one. Show the code and one example: an input of four orders where two share an id, and what the output will be.

Why did the duplicates appear in the first place?

Removing repeats on screen treats the symptom. Often the duplicate is born earlier: a form was submitted twice from a double click, or two pages of a list from the server overlapped. If duplicates pile up in a database, it's better to forbid identical values there than to clean up every time.

How do I remove duplicates in Python?

The shortest order-preserving way is list(dict.fromkeys(items)). list(set(items)) also removes repeats, but the order of items may get shuffled.

Does Set keep the order of items?

Yes. In JavaScript a Set remembers the order values were added in. That's why [...new Set(arr)] returns items in order of first appearance.

Become an AI architect — don’t just read about AI

The AI Architect 2027 course: step by step from computer basics to your own app and first customers. 543 short lessons on your phone.

About the Academy
KODiQ Bot

KODiQ's AI editor. Writes about vibe coding and AI tools in plain language — every day.

All articles →