Remove duplicate elements from std::vector but starting from the front?

Question

How can I delete duplicate elements from a vector but starting from the front?

So

2 3 4 5 2 5 would become 3 4 2 5

1 5 3 1 would become 5 3 1

I would like the solution to be easy to read, and it would be nice if it had good performance as well.

Answer 1

If a container supports bidirectional iterators then it is unimportant from which side of the container you are trying to remove duplicate elements because you can use reverse iterators.

Here is a demonstrative program.

#include <iostream> 
#include <vector> 
#include <iterator> 
#include <algorithm> 
  
template <typename ForwardIterator> 
ForwardIterator remove_duplicates( ForwardIterator first, ForwardIterator last ) 
{ 
    for ( ; first != last; ++first ) 
    { 
        last = std::remove( std::next( first ), last, *first ); 
    } 
      
    return last; 
} 
  
int main() 
{ 
    std::vector<int> v = { 1, 2, 3, 4, 5, 4, 3, 2, 1 }; 
      
    for ( const auto &item : v ) std::cout << item << ' '; 
    std::cout << '\n'; 
  
    v.erase( remove_duplicates( std::begin( v ), std::end( v ) ), std::end( v ) ); 
      
    for ( const auto &item : v ) std::cout << item << ' '; 
    std::cout << '\n'; 
  
    std::cout << '\n'; 
      
    v.assign( { 1, 2, 3, 4, 5, 4, 3, 2, 1 } ); 
  
    for ( const auto &item : v ) std::cout << item << ' '; 
    std::cout << '\n'; 
      
    v.erase( std::begin( v ), remove_duplicates( std::rbegin( v ), std::rend( v ) ).base() ); 
     
    for ( const auto &item : v ) std::cout << item << ' '; 
    std::cout << '\n'; 
} 

The program output is

1 2 3 4 5 4 3 2 1 
1 2 3 4 5 

1 2 3 4 5 4 3 2 1 
5 4 3 2 1 

Answer 2

An asymptotically efficient algorithm (N log N):

• Create a range of iterators each pointing to the original container
• Sort the iterators with a stable sort. Use a comparison function that indirects through the iterator.
• Remove consecutive duplicates (std::unique) using reverse iterator and similar custom comparison function.
• std::remove_if from the vector with a predicate function that removes elements whose iterator isn't in the secondary container.

It's a bit more complex than the O(N*N) solution. Here is an example implementation. I cannot guarantee that it is correct:

template<class It, class Sentinel>
auto remove_duplicates(It first, Sentinel last)
{
    std::vector<It> iterators;

    auto iterator_generator = [it = first]() mutable {
        return it++;
    };
    std::generate_n(
        std::back_inserter(iterators),
        last - first,
        iterator_generator);

    auto iterator_compare = [](const auto& l, const auto& r) {
        return *l < *r;
    };
    
    std::stable_sort(
        iterators.begin(),
        iterators.end(),
        iterator_compare);

    auto iterator_eq = [](const auto& l, const auto& r) {
        return *l == *r;
    };
    auto last_unique = std::unique(
        iterators.begin(),
        iterators.end(),
        iterator_eq);
    iterators.erase(last_unique, iterators.end());

    auto keep_generator = [it = first]() mutable {
        return it++;
    };
    std::vector<bool> remove(last - first, true);
    for(auto it : iterators) {
        auto index = it - first;
        remove[index] = false;
    }
    
    auto remove_predicate = [index = 0, remove = std::move(remove)](const auto& el) mutable {
        return remove[index++];
    };
    return std::remove_if(first, last, std::move(remove_predicate));
}

// usage with reverse iterators
v.erase(
    v.rend().base(),
    remove_duplicates(v.rbegin(), v.rend()).base());