Collect pairs from a stream

Question

I have a stream of objects like this:

"0", "1", "2", "3", "4", "5",

How can I transform it to stream of pairs :

{ new Pair("0", "1"), new Pair("2", "3"), new Pair("4", "5")}.

The stream size is unknown. I am reading data from a file that might be big. I have only iterator to collection and I transform this iterator to stream using spliterator. I know that here is a answer for processing adjacent pairs with StreamEx : Collect successive pairs from a stream Can this be done in java or StreamEx ? Thanks

Answer 1

If you don't want to collect the elements

The title of the question says collect pairs from a stream, so I'd assume that you want to actually collect these, but you commented:

Your solution works, the problem is that it loads the data from file to PairList and then I may use stream from this collection to process pairs. I can't do it because the data might be too big to store in the memory.

so here's a way to do this without collecting the elements.

It's relatively straightforward to transform an Iterator<T> into an Iterator<List<T>>, and from that to transform a stream into a stream of pairs.

  /**
   * Returns an iterator over pairs of elements returned by the iterator.
   * 
   * @param iterator the base iterator
   * @return the paired iterator
   */
  public static <T> Iterator<List<T>> paired(Iterator<T> iterator) {
    return new Iterator<List<T>>() {
      @Override
      public boolean hasNext() {
        return iterator.hasNext();
      }

      @Override
      public List<T> next() {
        T first = iterator.next();
        if (iterator.hasNext()) {
          return Arrays.asList(first, iterator.next());
        } else {
          return Arrays.asList(first);
        }
      }
    };
  }

  /**
   * Returns an stream of pairs of elements from a stream.
   * 
   * @param stream the base stream
   * @return the pair stream
   */
  public static <T> Stream<List<T>> paired(Stream<T> stream) {
    return StreamSupport.stream(Spliterators.spliteratorUnknownSize(paired(stream.iterator()), Spliterator.ORDERED),
        false);
  }

  @Test
  public void iteratorAndStreamsExample() {
    List<String> strings = Arrays.asList("a", "b", "c", "d", "e", "f");
    Iterator<List<String>> pairs = paired(strings.iterator());
    while (pairs.hasNext()) {
      System.out.println(pairs.next());
      // [a, b]
      // [c, d]
      // [e, f]
    }

    paired(Stream.of(1, 2, 3, 4, 5, 6, 7, 8)).forEach(System.out::println);
    // [1, 2]
    // [3, 4]
    // [5, 6]
    // [7, 8]
  }

If you want to collect the elements...

I'd do this by collecting into a list, and using an AbstractList to provide a view of the elements as pairs.

First, the PairList. This is a simple AbstractList wrapper around any list that has an even number of elements. (This could easily be adapted to handle odd length lists, once the desired behavior is specified.)

  /**
   * A view on a list of its elements as pairs.
   * 
   * @param <T> the element type
   */
  static class PairList<T> extends AbstractList<List<T>> {
    private final List<T> elements;

    /**
     * Creates a new pair list.
     * 
     * @param elements the elements
     * 
     * @throws NullPointerException if elements is null
     * @throws IllegalArgumentException if the length of elements is not even
     */
    public PairList(List<T> elements) {
      Objects.requireNonNull(elements, "elements must not be null");
      this.elements = new ArrayList<>(elements);
      if (this.elements.size() % 2 != 0) {
        throw new IllegalArgumentException("number of elements must have even size");
      }
    }

    @Override
    public List<T> get(int index) {
      return Arrays.asList(elements.get(index), elements.get(index + 1));
    }

    @Override
    public int size() {
      return elements.size() / 2;
    }
  }

Then we can define the collector that we need. This is essentially shorthand for collectingAndThen(toList(), PairList::new):

  /**
   * Returns a collector that collects to a pair list.
   * 
   * @return the collector
   */
  public static <E> Collector<E, ?, PairList<E>> toPairList() {
    return Collectors.collectingAndThen(Collectors.toList(), PairList::new);
  }

Note that it could be worthwhile defining a PairList constructor that doesn't defensively copy the list, for the use case that we know the backing list is freshly generated (as in this case). That's not really essential right now, though. But once we did that, this method would be collectingAndThen(toCollection(ArrayList::new), PairList::newNonDefensivelyCopiedPairList).

And now we can use it:

  /**
   * Creates a pair list with collectingAndThen, toList(), and PairList::new
   */
  @Test
  public void example() {
    List<List<Integer>> intPairs = Stream.of(1, 2, 3, 4, 5, 6)
        .collect(toPairList());
    System.out.println(intPairs); // [[1, 2], [2, 3], [3, 4]]

    List<List<String>> stringPairs = Stream.of("a", "b", "c", "d")
        .collect(toPairList());
    System.out.println(stringPairs); // [[a, b], [b, c]]
  }

Here's a complete source file with a runnable example (as a JUnit test):

package ex;

import java.util.AbstractList;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Objects;
import java.util.stream.Collector;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.junit.Test;

public class PairCollectors {

  /**
   * A view on a list of its elements as pairs.
   * 
   * @param <T> the element type
   */
  static class PairList<T> extends AbstractList<List<T>> {
    private final List<T> elements;

    /**
     * Creates a new pair list.
     * 
     * @param elements the elements
     * 
     * @throws NullPointerException if elements is null
     * @throws IllegalArgumentException if the length of elements is not even
     */
    public PairList(List<T> elements) {
      Objects.requireNonNull(elements, "elements must not be null");
      this.elements = new ArrayList<>(elements);
      if (this.elements.size() % 2 != 0) {
        throw new IllegalArgumentException("number of elements must have even size");
      }
    }

    @Override
    public List<T> get(int index) {
      return Arrays.asList(elements.get(index), elements.get(index + 1));
    }

    @Override
    public int size() {
      return elements.size() / 2;
    }
  }

  /**
   * Returns a collector that collects to a pair list.
   * 
   * @return the collector
   */
  public static <E> Collector<E, ?, PairList<E>> toPairList() {
    return Collectors.collectingAndThen(Collectors.toList(), PairList::new);
  }

  /**
   * Creates a pair list with collectingAndThen, toList(), and PairList::new
   */
  @Test
  public void example() {
    List<List<Integer>> intPairs = Stream.of(1, 2, 3, 4, 5, 6)
        .collect(toPairList());
    System.out.println(intPairs); // [[1, 2], [2, 3], [3, 4]]

    List<List<String>> stringPairs = Stream.of("a", "b", "c", "d")
        .collect(toPairList());
    System.out.println(stringPairs); // [[a, b], [b, c]]
  }    
}

Answer 2

I know I'm late to the party, but all of the answers seem to be really complicated or have a lot of GC overhead/short-lived objects (which is not a big deal with modern JVMs), but why not do it simply like this?

public class PairCollaterTest extends TestCase {
    static class PairCollater<T> implements Function<T, Stream<Pair<T, T>>> {
        T prev;

        @Override
        public Stream<Pair<T, T>> apply(T curr) {
            if (prev == null) {
                prev = curr;
                return Stream.empty();
            }
            try {
                return Stream.of(Pair.of(prev, curr));
            } finally {
                prev = null;
            }
        }
    }

    public void testPairCollater() {
        Stream.of("0", "1", "2", "3", "4", "5").sequential().flatMap(new PairCollater<>()).forEach(System.out::println);
    }
}

Prints:

(0,1)
(2,3)
(4,5)

Answer 3

Assuming there is a Pair with left, right and getters and a constructor:

 static class Paired<T> extends AbstractSpliterator<Pair<T>> {

    private List<T> list = new ArrayList<>(2);

    private final Iterator<T> iter;

    public Paired(Iterator<T> iter) {
        super(Long.MAX_VALUE, 0);
        this.iter = iter;
    }

    @Override
    public boolean tryAdvance(Consumer<? super Pair<T>> consumer) {
        getBothIfPossible(iter);
        if (list.size() == 2) {
            consumer.accept(new Pair<>(list.remove(0), list.remove(0)));
            return true;
        }
        return false;
    }

    private void getBothIfPossible(Iterator<T> iter) {
        while (iter.hasNext() && list.size() < 2) {
            list.add(iter.next());
        }
    }

}

Usage would be:

 Iterator<Integer> iterator = List.of(1, 2, 3, 4, 5).iterator();
 Paired<Integer> p = new Paired<>(iterator);
 StreamSupport.stream(p, false)
            .forEach(pair -> System.out.println(pair.getLeft() + "  " + pair.getRight()));

Answer 4

It's not a natural fit but you can do

List input = ...
List<Pair> pairs = IntStream.range(0, input.size() / 2)
                            .map(i -> i * 2)
                            .mapToObj(i -> new Pair(input.get(i), input.get(i + 1)))
                            .collect(Collectors.toList());

To create Pairs as you go in a stream you need a stateful lambdas which should be generally avoided but can be done. Note: this will only works if the stream is single threaded. i.e. not parallel.

Stream<?> stream = 
assert !stream.isParallel();
Object[] last = { null };
List<Pair> pairs = stream.map(a -> {
        if (last[0] == null) {
            last[0] = a;
            return null;
        } else {
            Object t = last[0];
            last[0] = null;
            return new Pair(t, a);
        }
     }).filter(p -> p != null)
       .collect(Collectors.toList());
assert last[0] == null; // to check for an even number input.

Answer 5

Just replace IntStream.range(1, 101) with your stream (you don't need to know your stream's size) -

import java.util.ArrayList;
import java.util.List;
import java.util.stream.IntStream;

public class TestClass {

    public static void main(String[] args) {

        final Pair pair = new Pair();
        final List<Pair> pairList = new ArrayList<>();

        IntStream.range(1, 101)
                .map(i -> {
                    if (pair.a == null) {
                        pair.a = i;
                        return 0;
                    } else {
                        pair.b = i;
                        return 1;
                    }
                })
                .filter(i -> i == 1)
                .forEach(i -> {
                    pairList.add(new Pair(pair));
                    pair.reset();
                });

        pairList.stream().forEach(p -> System.out.print(p + " "));
    }

    static class Pair {
        public Object a;
        public Object b;

        public Pair() {
        }

        public Pair(Pair orig) {
            this.a = orig.a;
            this.b = orig.b;
        }

        void reset() {
            a = null;
            b = null;
        }

        @Override
        public String toString() {
            return "{" + a + "," + b + '}';
        }
    }

}