When you are dealing with a list of data, you may be interested in filtering out all duplicate values. For example, let's say your input resembles the following:

   input = { there,
  here, deer, dear, there, pier, deer, leer }

Your output with duplicates removed would be:

   input = { there,
  here, near, deer, dear, pier leer }

The duplicate there's and deer's were removed. In this tutorial, I will provide the code for removing duplicates and then explain how the code actually works to efficiently remove duplicate elements.

The Code

The code for removing duplicate values is:

static List<string> removeDuplicates(List<string> inputList)
{
  Dictionary<string, int> uniqueStore = new Dictionary<string, int>();
  List<string> finalList = new List<string>();
  foreach (string currValue in inputList)
  {
  if (!uniqueStore.ContainsKey(currValue))
  {
  uniqueStore.Add(currValue, 0);
  finalList.Add(currValue);
  }
  }
  return finalList;
}

Note that the above code is designed for Lists that store string data, but you can modify the code easily to accept any type of data! Don't worry - I will explain later.

How to Use the Code

To test the above code, all you need is a List that contains duplicate values. You can copy and paste the following main method to see for yourself:

static void Main(string[] args)
{
  List<string> input = new List<string>();
  input.Add("There");
  input.Add("Here");
  input.Add("Sneer");
  input.Add("There");
  input.Add("Near");
  input.Add("Meer");
  input.Add("Here");
  List<string> result = removeDuplicates(input);
}

I am declaring a new List called input and adding some sample values. Some of the values are duplicates.

Next, I declare a new List called result that passes the input list into our removeDuplicates method. This works because our removeDuplicates method returns a new list. You can use a loop to print out the values in the result loop, but I will leave it up to you on how you choose to do that.

Now that you have an idea of how to use the removeDuplicates method from the previous section, in this page I will explain how to modify it for non-string values, and then I'll dive right in and explain why the code works.

If you want to change your removeDuplicates method to take care of integers, or doubles, or any other type besides a string, you will need to change the type in the following highlighted areas from string to the type you plan on using:

static List<string> removeDuplicates(List<string> inputList)
{
  Dictionary<string, int> uniqueStore = new Dictionary<string, int>();
  List<string> finalList = new List<string>();
  foreach (string currValue in inputList)
  {
  if (!uniqueStore.ContainsKey(currValue))
  {
  uniqueStore.Add(currValue, 0);
  finalList.Add(currValue);
  }
  }
  return finalList;
}

If you want to remove duplicate values from a list of integers, you would change the highlighted string text to int.

Code Explained

Now that you have seen the code and learned how to use it, let's go through the code and figure out why it works!

static List<string> removeDuplicates(List<string> inputList)
{
  Dictionary<string, int> uniqueStore = new Dictionary<string, int>();
  List<string> finalList = new List<string>();
  foreach (string currValue in inputList)
  {
  if (!uniqueStore.ContainsKey(currValue))
  {
  uniqueStore.Add(currValue, 0);
  finalList.Add(currValue);
  }
  }
  return finalList;
}

In the above non-grayed out text, I am declaring a new static method called removeDuplicates. It takes one argument - a List known as inputList - and returns a List in the end. You know that a List is being returned because you can see the List<string> text that appears directly after our static modifier.

---

Dictionary<string, int> uniqueStore = new Dictionary<string, int>();

I declare a new dictionary variable called uniqueStore. This variable will store all of the unique values, but more on that later. Like I explained in my earlier tutorial on Dictionaries, a dictionary takes two pieces of data - a key and a value. In our example, the key will be a string to correspond to the string data held by our inputList, and the value will be an int. You will see later that the value doesn't really matter for removing duplicates in this case!

---

List<string> finalList = new List<string>();

To store our final results, I create and initialize a new List called finalList. At the end, we hope our finalList contains all of the unique elements from our inputList earlier!

---

foreach (string currValue in inputList)
{
  if (!uniqueStore.ContainsKey(currValue))
  {
  uniqueStore.Add(currValue, 0);
  finalList.Add(currValue);
  }
}

Now, it is time to iterate through our inputList. I could have used a standard for loop with initializers starting at 0 and ending before the end of the iterated list, but you can accomplish the same neatly by using a foreach statement.

In the foreach statement, I store the current element from inputList into a string called currValue.

Just for kicks, here is how the above line of code would look using a standard for loop:

for (int i = 0; i < inputList.Count; i++)
{
  string currValue = inputList[i];
  if (!uniqueStore.ContainsKey(currValue))
  {
  uniqueStore.Add(currValue, 0);
  finalList.Add(currValue);
  }
}

I think the foreach approach is much nicer and compact, but you should use whichever method you are most familiar with.

---

In the previous section, I started explaining how the code actually works. Let's pick up from where we left off with this page and wrap things up!

---

if (!uniqueStore.ContainsKey(currValue))
{
  uniqueStore.Add(currValue, 0);
  finalList.Add(currValue);
}

Inside our foreach loop, the first thing we do is check if the current value - currValue - is already in our dictionary. We can do that by using the dictionary's ContainsKey method. The ContainsKey method returns a true if the value already exists in the dictionary, but it would return a false is the value does not exist.

Notice that I am negating the result from our ContainsKey by placing a ! operator in front of the uniqueStore variable in the if statement. This ensures that, if ContainsKey returns true, our if statement would see the opposite, false, instead. I guess you could say that if our if statement reads "If key is NOT contained in dictionary..."

---

uniqueStore.Add(currValue, 0);
finalList.Add(currValue);

If currValue is not in our dictionary, then we get to these two lines. The first line adds the current value from our inputList to our uniqueStore dictionary. We are just filling in a 0 for the dictionary's value field, because we are only using the dictionary for its ability to quickly notify us if a duplicate key is about to be added. Retrieving anything from the dictionary is not our intended goal, so you can pass in any integer value you want.

In the second line, I simply add the current value to our finalList. Because we know that, since this value does not already exist in the dictionary, this has to be a unique value, so let's go ahead and add it in to our finalList.

---

return finalList;

After our foreach loop terminates, all that is left to do is return our finalList value that now contains the unique elements from the List we were presented with originally. All duplicate values have now been removed!

---

Brief Walkthrough

Let's say that our input is the array I presented in the first page:

   input = { there,
  here, deer, dear, there, pier, deer, leer }

Initially, our finalList List as well our uniqueStore Dictionary objects are created. We then enter the foreach loop and will stay here until all elements in the input List have been analyzed.

The first value of currValue is going to be the word there. Since our dictionary does not contain the word there as a key, we add this value to the dictionary as well to our finalList.

Similarly, the second word is here, and it too does not exist in either the uniqueStore or finalList objects. This process of checking and adding the words to our uniqueStore and finalList objects continues until we reach the 5 word, the 2nd instance of the word there.

When your currentValue is the word there (again), your uniqueStore.ContainsKey(currValue) statement will return true. After all, there was the first word you added earlier, so you already have the word there located as a key in the dictionary. Therefore, you do not need to add it to our uniqueStore Dictionary and certainly not our finalList List again. The duplicate word was simply skipped over!

Conclusion

There you have it! A small and fast script for removing duplicates from a List of elements. If you used any other type of data structure besides a List, you should be able to modify the code with little effort.

Just a final word before we wrap up. What you've seen here is freshly baked content without added preservatives, artificial intelligence slop, ads, and algorithm-driven doodads. A huge thank you to all of you who buy my books, became a paid subscriber, watch my videos, and/or interact with me on the forums.

Your support keeps this site going! 😇