Streams & Collectors
New APIs for map / filter / reduce
José Paumard
blog.paumard.org
@JosePaumard
Module Outline
Introduction: map / filter / reduce
What is a « Stream »?
Patterns to build a Stream
Operations on a Stream
Map / Filter / Reduce
Example:
Let’s take a list a Person
List<Person> list = new ArrayList<>() ;
Map / Filter / Reduce
Example:
Let’s take a list a Person
List<Person> list = new ArrayList<>() ;
Suppose we want to compute the
« average of the age of the people older than 20 »
Map / Filter / Reduce
1st step: mapping
Map / Filter / Reduce
1st step: mapping
The mapping step takes a List<Person> and returns a List<Integer>
The size of both lists is the same
Map / Filter / Reduce
2nd step: filtering
The filtering step takes a List<Integer> and returns a List<Integer>
But there some elements have been filtered out in the process
Map / Filter / Reduce
3rd step: average
This is the reduction step, equivalent to the SQL aggregation
What Is a Stream?
Technical answer: a typed interface
public interface Stream<T> extends BaseStream<T, Stream<T>> {
// ...
}
What Is a Stream?
Technical answer: a typed interface
public interface Stream<T> extends BaseStream<T, Stream<T>> {
// ...
}
And a new concept!
What Is a Stream?
What does it do?
What Is a Stream?
What does it do?
It gives ways to efficiently process large amounts of data… and also
smaller ones
What Is a Stream?
What does efficiently mean?
What Is a Stream?
What does efficiently mean?
Two things:
What Is a Stream?
What does efficiently mean?
Two things:
In parallel, to leverage the computing power of multicore CPUs
Pipelined, to avoid unnecessary intermediary computations
What Is a Stream?
Why can’t a Collection be a Stream?
What Is a Stream?
Why can’t a Collection be a Stream?
Because Stream is a new concept, and we dont want to change the
way the Collection API works
What Is a Stream?
So what is a Stream?
What Is a Stream?
So what is a Stream?
An object on which one can define operations
What Is a Stream?
So what is a Stream?
An object on which one can define operations
An object that does not hold any data
What Is a Stream?
So what is a Stream?
An object on which one can define operations
An object that does not hold any data
An object that should not change the data it processes
What Is a Stream?
So what is a Stream?
An object on which one can define operations
An object that does not hold any data
An object that should not change the data it processes
An object able to process data in « one pass »
What Is a Stream?
So what is a Stream?
An object on which one can define operations
An object that does not hold any data
An object that should not change the data it processes
An object able to process data in « one pass »
An object optimized from the algorithm point of view, and able to
process data in parallel
How Can We Build a Stream?
Many patterns!
How Can We Build a Stream?
Many patterns!
List<Person> persons = ... ;
Stream<Person> stream = persons.stream();
A First Operation
First operation: forEach()
List<Person> persons = ... ;
Stream<Person> stream = persons.stream();
stream.forEach(p ‐> System.out.println(p));
A First Operation
First operation: forEach()
List<Person> persons = ... ;
Stream<Person> stream = persons.stream();
stream.forEach(p ‐> System.out.println(p));
Prints all the elements of the list
A First Operation
First operation: forEach()
List<Person> persons = ... ;
Stream<Person> stream = persons.stream();
stream.forEach(p ‐> System.out.println(p));
Prints all the elements of the list
It takes an instance of Consumer as an argument
A First Operation
Interface Consumer<T>
@FunctionalInterface
public interface Consumer<T> {
void accept(T t);
}
A First Operation
Interface Consumer<T>
@FunctionalInterface
public interface Consumer<T> {
void accept(T t);
}
Consumer<T> is a functional interface
Can be implemented by a lambda expression
Consumer<T> c = p ‐> System.out.println(p);
A First Operation
Interface Consumer<T>
@FunctionalInterface
public interface Consumer<T> {
void accept(T t);
}
Consumer<T> is a functional interface
Can be implemented by a lambda expression
Consumer<T> c = p ‐> System.out.println(p);
Consumer<T> c = System.out::println; // Method reference
A First Operation
In fact Consumer<T> is a bit more complex
@FunctionalInterface
public interface Consumer<T> {
void accept(T t);
default Consumer<T> andThen(Consumer<? super T> after) {
Objects.requireNonNull(after);
return (T t) ‐> { accept(t); after.accept(t); };
}
}
A First Operation
In fact Consumer<T> is a bit more complex
@FunctionalInterface
public interface Consumer<T> {
void accept(T t);
default Consumer<T> andThen(Consumer<? super T> after) {
Objects.requireNonNull(after);
return (T t) ‐> { accept(t); after.accept(t); };
}
}
One can chain consumers!
A First Operation
Let’s chain consumers
List<String> list = new ArrayList<>();
Consumer<String> c1 = s ‐> list.add(s);
Consumer<String> c2 = s ‐> System.out.println(s);
A First Operation
Let’s chain consumers
List<String> list = new ArrayList<>();
Consumer<String> c1 = list::add;
Consumer<String> c2 = System.out::println;
A First Operation
Let’s chain consumers
List<String> list = new ArrayList<>();
Consumer<String> c1 = list::add;
Consumer<String> c2 = System.out::println;
Consumer<String> c3 = c1.andThen(c2);
A First Operation
Only way to have several consumers on a single stream
List<String> result = new ArrayList<>();
List<Person> persons = ...;
Consumer<String> c1 = result::add;
Consumer<String> c2 = System.out::println;
persons.stream()
.forEach(c1.andThen(c2));
Because forEach() does not return anything
A Second Operation: Filter
Example:
List<Person> list = ...;
Stream<Person> stream = list.stream();
Stream<Person> filtered =
stream.filter(person ‐> person.getAge() > 20);
A Second Operation: Filter
Example:
List<Person> list = ...;
Stream<Person> stream = list.stream();
Stream<Person> filtered =
stream.filter(person ‐> person.getAge() > 20);
Takes a predicate as a parameter:
Predicate<Person> p = person ‐> person.getAge() > 20;
A Second Operation: Filter
Predicate interface:
@FunctionalInterface
public interface Predicate<T> {
boolean test(T t);
}
A Second Operation: Filter
Predicate interface, with default methods:
@FunctionalInterface
public interface Predicate<T> {
boolean test(T t);
default Predicate<T> and(Predicate<? super T> other) { ... }
default Predicate<T> or(Predicate<? super T> other) { ... }
default Predicate<T> negate() { ... }
}
A Second Operation: Filter
Predicates combinations examples:
Predicate<Integer> p1 = i ‐> i > 20;
Predicate<Integer> p2 = i ‐> i < 30;
Predicate<Integer> p3 = i ‐> i == 0;
Predicate<Integer> p = p1.and(p2).or(p3); // (p1 AND p2) OR p3
Predicate<Integer> p = p3.or(p1).and(p2); // (p3 OR p1) AND p2
A Second Operation: Filter
Predicates combinations examples:
Predicate<Integer> p1 = i ‐> i > 20;
Predicate<Integer> p2 = i ‐> i < 30;
Predicate<Integer> p3 = i ‐> i == 0;
Predicate<Integer> p = p1.and(p2).or(p3); // (p1 AND p2) OR p3
Predicate<Integer> p = p3.or(p1).and(p2); // (p3 OR p1) AND p2
Warning: method calls do not handle priorities
A Second Operation: Filter
Predicate interface, with static method:
@FunctionalInterface
public interface Predicate<T> {
boolean test(T t);
// default methods
static <T> Predicate<T> isEqual(Object o) { ... }
}
A Second Operation: Filter
Predicate interface, with static method:
@FunctionalInterface
public interface Predicate<T> {
boolean test(T t);
// default methods
static <T> Predicate<T> isEqual(Object o) { ... }
}
Example:
Predicate<String> p = Predicate.isEqual("two") ;
A Second Operation: Filter
Use case:
Predicate<String> p = Predicate.isEqual("two") ;
Stream<String> stream1 = Stream.of("one", "two", "three") ;
Stream<String> stream2 = stream1.filter(p) ;
The filter method returns a Stream
A Second Operation: Filter
Use case:
Predicate<String> p = Predicate.isEqual("two") ;
Stream<String> stream1 = Stream.of("one", "two", "three") ;
Stream<String> stream2 = stream1.filter(p) ;
The filter method returns a Stream
This Stream is a new instance
A Second Operation: Filter
Question: what do I have in this new Stream?
A Second Operation: Filter
Question: what do I have in this new Stream?
Simple answer: the filtered data
A Second Operation: Filter
Question: what do I have in this new Stream?
Simple answer: the filtered data
Really?
A Second Operation: Filter
Question: what do I have in this new Stream?
Simple answer: the filtered data
Really?
We just said: « a stream does not hold any data »
A Second Operation: Filter
Question: what do I have in this new Stream?
Simple answer: the filtered data WRONG!
A Second Operation: Filter
Question: what do I have in this new Stream?
Simple answer: the filtered data WRONG!
The right answer is: nothing, since a Stream does not hold any data
A Second Operation: Filter
Question: what do I have in this new Stream?
Simple answer: the filtered data WRONG!
The right answer is: nothing, since a Stream does not hold any data
So, what does this code do?
List<Person> list = ... ;
Stream<Person> stream = list.stream();
Stream<Person> filtered =
stream.filter(person ‐> person.getAge() > 20);
A Second Operation: Filter
Question: what do I have in this new Stream?
Simple answer: the filtered data WRONG!
The right answer is: nothing, since a Stream does not hold any data
So, what does this code do?
List<Person> list = ... ;
Stream<Person> stream = list.stream();
Stream<Person> filtered =
stream.filter(person ‐> person.getAge() > 20);
Answer is: nothing
This call is only a declaration, no data is processed
A Second Operation: Filter
The call to the filter method is lazy
A Second Operation: Filter
The call to the filter method is lazy
And all the methods of Stream that return another Stream are lazy
A Second Operation: Filter
The call to the filter method is lazy
And all the methods of Stream that return another Stream are lazy
Another way of saying it:
an operation on a Stream that returns a Stream
is called an intermediary operation
Back to the Consumer
What does this code do?
List<String> result = new ArrayList<>();
List<Person> persons = ... ;
persons.stream()
.peek(System.out::println)
.filter(person ‐> person.getAge() > 20)
.peek(result::add);
Back to the Consumer
What does this code do?
List<String> result = new ArrayList<>();
List<Person> persons = ... ;
persons.stream()
.peek(System.out::println)
.filter(person ‐> person.getAge() > 20)
.peek(result::add);
Hint: the peek() method returns a Stream
Back to the Consumer
What does this code do?
List<String> result = new ArrayList<>();
List<Person> persons = ... ;
persons.stream()
.peek(System.out::println)
.filter(person ‐> person.getAge() > 20)
.peek(result::add);
Answer: nothing!
This code does not print anything
The list « result » is empty
Summary
The Stream API defines intermediary operations
We saw 3 operations:
forEach(Consumer)
peek(Consumer)
filter(Predicate)
Summary
The Stream API defines intermediary operations
We saw 3 operations:
forEach(Consumer) (not lazy)
peek(Consumer) (lazy)
filter(Predicate) (lazy)
Mapping Operation
Example:
List<Person> list = ... ;
Stream<Person> stream = list.stream();
Stream<String> names =
stream.map(person ‐> person.getName());
Mapping Operation
Example:
List<Person> list = ... ;
Stream<Person> stream = list.stream();
Stream<String> names =
stream.map(person ‐> person.getName());
map() returns a Stream, so it is an intermediary operation
Mapping Operation
A mapper is modeled by the Function interface
@FunctionalInterface
public interface Function<T, R> {
R apply(T t);
}
Mapping Operation
… with default methods to chain and compose mappings
@FunctionalInterface
public interface Function<T, R> {
R apply(T t);
default <V> Function<V, R> compose(Function<V, T> before);
default <V> Function<T, V> andThen(Function<R, V> after);
}
Mapping Operation
… with default methods to chain and compose mappings
@FunctionalInterface
public interface Function<T, R> {
R apply(T t);
default <V> Function<V, R> compose(Function<V, T> before);
default <V> Function<T, V> andThen(Function<R, V> after);
}
In fact this is the simplified version, beware the generics!
Mapping Operation
compose() and andThen() methods with their exact signatures
@FunctionalInterface
public interface Function<T, R> {
R apply(T t);
default <V> Function<V, R> compose(
Function<? super V, ? extends T> before);
default <V> Function<T, V> andThen(
Function<? super R, ? extends V> after);
}
Mapping Operation
One static method: identity
@FunctionalInterface
public interface Function<T, R> {
R apply(T t);
// default methods
static <T> Function<T, T> identity() {
return t ‐> t;
}
}
Flatmapping Operation
Method flatMap()
Signature:
<R> Stream<R> flatMap(Function<T, Stream<R>> flatMapper);
<R> Stream<R> map(Function<T, R> mapper);
Flatmapping Operation
Method flatMap()
Signature:
<R> Stream<R> flatMap(Function<T, Stream<R>> flatMapper);
<R> Stream<R> map(Function<T, R> mapper);
The flatMapper takes an element of type T, and returns an element of
type Stream<R>
Flatmapping Operation
Method flatMap()
Signature:
<R> Stream<R> flatMap(Function<T, Stream<R>> flatMapper);
<R> Stream<R> map(Function<T, R> mapper);
If the flatMap was a regular map, it would return a
Stream<Stream<R>>
Flatmapping Operation
Method flatMap()
Signature:
<R> Stream<R> flatMap(Function<T, Stream<R>> flatMapper);
<R> Stream<R> map(Function<T, R> mapper);
If the flatMap was a regular map, it would return a
Stream<Stream<R>>
Thus a « stream of streams »
Flatmapping Operation
Method flatMap()
Signature:
<R> Stream<R> flatMap(Function<T, Stream<R>> flatMapper);
<R> Stream<R> map(Function<T, R> mapper);
If the flatMap was a regular map, it would return a
Stream<Stream<R>>
But it is a flatMap!
Flatmapping Operation
Method flatMap()
Signature:
<R> Stream<R> flatMap(Function<T, Stream<R>> flatMapper);
<R> Stream<R> map(Function<T, R> mapper);
If the flatMap was a regular map, it would return a
Stream<Stream<R>>
But it is a flatMap!
Thus the « stream of streams » is flattened, and becomes a stream
Summary
3 categories of operations:
Summary
3 categories of operations:
forEach() and peek()
Summary
3 categories of operations:
forEach() and peek()
filter()
Summary
3 categories of operations:
forEach() and peek()
filter()
map() and flatMap()
Reduction
And what about the reduction step?
Reduction
And what about the reduction step?
Two kinds of reduction in the Stream API
Reduction
And what about the reduction step?
Two kinds of reduction in the Stream API
1st: aggregation = min, max, sum, etc…
Reduction
How does it work?
List<Integer> ages = ... ;
Stream<Integer> stream = ages.stream();
Integer sum =
stream.reduce(0, (age1, age2) ‐> age1 + age2);
Reduction
How does it work?
List<Integer> ages = ... ;
Stream<Integer> stream = ages.stream();
Integer sum =
stream.reduce(0, (age1, age2) ‐> age1 + age2);
1st argument: identity element of the reduction operation
Reduction
How does it work?
List<Integer> ages = ... ;
Stream<Integer> stream = ages.stream();
Integer sum =
stream.reduce(0, (age1, age2) ‐> age1 + age2);
1st argument: identity element of the reduction operation
2nd argument: reduction operation, of type BinaryOperator<T>
BinaryOperator
A BinaryOperator is a special case of BiFunction
@FunctionalInterface
public interface BiFunction<T, U, R> {
R apply(T t, U u);
// plus default methods
}
BinaryOperator
A BinaryOperator is a special case of BiFunction
@FunctionalInterface
public interface BiFunction<T, U, R> {
R apply(T t, U u);
// plus default methods
}
@FunctionalInterface
public interface BinaryOperator<T>
extends BiFunction<T, T, T> {
// T apply(T t1, T t2);
// plus static methods
}
Identity Element
The bifunction takes two arguments, so…
Identity Element
The bifunction takes two arguments, so…
What happens if the Stream is empty?
Identity Element
The bifunction takes two arguments, so…
What happens if the Stream is empty?
What happens if the Stream has only one element?
Identity Element
The bifunction takes two arguments, so…
What happens if the Stream is empty?
What happens if the Stream has only one element?
The reduction of an empty Stream is the identity element
Identity Element
The bifunction takes two arguments, so…
What happens if the Stream is empty?
What happens if the Stream has only one element?
The reduction of an empty Stream is the identity element
If the Stream has only one element, then the reduction is that element
Aggregations
Examples:
Stream<Integer> stream = ...;
BinaryOperation<Integer> sum = (i1, i2) ‐> i1 + i2;
Integer id = 0; // identity element for the sum
int red = stream.reduce(id, sum);
Stream<Integer> stream = Stream.empty();
int red = stream.reduce(id, sum);
System.out.println(red);
Will print:
> 0
Aggregations
Examples:
Stream<Integer> stream = ...;
BinaryOperation<Integer> sum = (i1, i2) ‐> i1 + i2;
Integer id = 0; // identity element for the sum
int red = stream.reduce(id, sum);
Stream<Integer> stream = Stream.of(1);
int red = stream.reduce(id, sum);
System.out.println(red);
Will print:
> 1
Aggregations
Examples:
Stream<Integer> stream = ...;
BinaryOperation<Integer> sum = (i1, i2) ‐> i1 + i2;
Integer id = 0; // identity element for the sum
int red = stream.reduce(id, sum);
Stream<Integer> stream = Stream.of(1, 2, 3, 4);
int red = stream.reduce(id, sum);
System.out.println(red);
Will print:
> 10
Aggregations: Corner Case
Suppose the reduction is the max
BinaryOperation<Integer> max =
(i1, i2) ‐>
i1 > i2 ? i1 : i2;
Aggregations: Corner Case
Suppose the reduction is the max
BinaryOperation<Integer> max =
(i1, i2) ‐>
i1 > i2 ? i1 : i2;
The problem is, there is no identity element for the max reduction
Aggregations: Corner Case
Suppose the reduction is the max
BinaryOperation<Integer> max =
(i1, i2) ‐>
i1 > i2 ? i1 : i2;
The problem is, there is no identity element for the max reduction
So the max of an empty Stream is undefined…
Aggregations: Corner Case
Then what is the return type of this call?
List<Integer> ages = ... ;
Stream<Integer> stream = ages.stream();
... max =
stream.max(Comparator.naturalOrder());
Aggregations: Corner Case
Then what is the return type of the this call?
List<Integer> ages = ... ;
Stream<Integer> stream = ages.stream();
... max =
stream.max(Comparator.naturalOrder());
If it is an int, then the default value is 0…
Aggregations: Corner Case
Then what is the return type of the this call?
List<Integer> ages = ... ;
Stream<Integer> stream = ages.stream();
... max =
stream.max(Comparator.naturalOrder());
If it is an Integer, then the default value is null…
Optionals
Then what is the return type of the this call?
List<Integer> ages = ... ;
Stream<Integer> stream = ages.stream();
Optional<Integer> max =
stream.max(Comparator.naturalOrder());
Optional means « there might be no result »
Optionals
How to use an Optional?
Optional<String> opt = ... ;
if (opt.isPresent()) {
String s = opt.get() ;
} else {
...
}
The method isPresent() returns true if there is something in the
optional
Optionals
How to use an Optional?
Optional<String> opt = ... ;
if (opt.isPresent()) {
String s = opt.get() ;
} else {
...
}
The method isPresent() returns true if there is something in the
optional
The method get() returns the value held by this optional
Optionals
How to use an Optional?
Optional<String> opt = ... ;
if (opt.isPresent()) {
String s = opt.get() ;
} else {
...
}
The method orElse() encapsulates both calls
String s = opt.orElse("") ; // defines a default value
Optionals
How to use an Optional?
Optional<String> opt = ... ;
if (opt.isPresent()) {
String s = opt.get() ;
} else {
...
}
The method orElseThrow() defines a thrown exception
String s = opt.orElseThrow(MyException::new) ; // lazy construct.
Reductions
Available reductions:
max(), min()
count()
Reductions
Available reductions:
max(), min()
count()
Boolean reductions
allMatch(), noneMatch(), anyMatch()
Reductions
Available reductions:
max(), min()
count()
Boolean reductions
allMatch(), noneMatch(), anyMatch()
Reductions that return an optional
findFirst(), findAny()
Reductions
Reductions are terminal operations
They trigger the processing of the data
Terminal Operation
Example:
List<Person> persons = ...;
Optional<Integer> minAge =
persons.map(person ‐> person.getAge()) // Stream<Integer>
.filter(age ‐> age > 20) // Stream<Integer>
.min(Comparator.naturalOrder()); // terminal operation
Terminal Operation
Example, optimization:
List<Person> persons = ... ;
persons.map(person ‐> person.getLastName())
.allMatch(length < 20); // terminal op.
Terminal Operation
Example, optimization:
List<Person> persons = ... ;
persons.map(person ‐> person.getLastName())
.allMatch(length < 20); // terminal op.
The map / filter / reduce operations are evaluated in one pass over the
data
Summary
Reduction seen as an aggregation
Intermediary / terminal operation
Optional: needed because default values cant be always defined
Collectors
There is another type of reduction
Collectors
There is another type of reduction
Called « mutable » reduction
Collectors
There is another type of reduction
Called « mutable » reduction
Instead of aggregating elements, this reduction put them in a
« container »
Collecting in a String
Example:
List<Person> persons = ... ;
String result =
persons.stream()
.filter(person ‐> person.getAge() > 20)
.map(Person::getLastName)
.collect(
Collectors.joining(", ")
);
Result is a String with all the names of the people in persons, older
than 20, separated by a comma
Collecting in a List
Example:
List<Person> persons = ... ;
List<String> result =
persons.stream()
.filter(person ‐> person.getAge() > 20)
.map(Person::getLastName)
.collect(
Collectors.toList()
);
Result is a List of String with all the names of the people in persons,
older than 20
Collecting in a Map
Example:
List<Person> persons = ... ;
Map<Integer, List<Person>> result =
persons.stream()
.filter(person ‐> person.getAge() > 20)
.collect(
Collectors.groupingBy(Person::getAge)
);
Result is a Map containing the people of persons, older than 20
The keys are the ages of the people
The values are the lists of the people of that age
Collecting in a Map
Example:
List<Person> persons = ... ;
Map<Integer, List<Person>> result =
persons.stream()
.filter(person ‐> person.getAge() > 20)
.collect(
Collectors.groupingBy(Person::getAge)
);
It is possible to « post-process » the values,
with a downstream collector
Collecting in a Map
Example:
List<Person> persons = ... ;
Map<Integer, Long> result =
persons.stream()
.filter(person ‐> person.getAge() > 20)
.collect(
Collectors.groupingBy(Person::getAge),
Collectors.counting() // the downstream collector
);
Collectors.counting() just counts the number of people of each age
So What Is a Stream?
An object that allows one to define processings on data
There is no limit on the amount of data that can be processed
Those processings are typically map / filter / reduce operations
So What Is a Stream?
An object that allows one to define processings on data
There is no limit on the amount of data that can be processed
Those processings are typically map / filter / reduce operations
Those processings are optimized :
First, we define all the operations
Then, the operations are triggered
So What Is a Stream?
Last remark:
A Stream cannot be « reused »
Once it has been used ot process a set of data, it cannot be used again
to process another set
Summary
Quick explanation of the map / filter / reduce
What is a Stream
The difference between intermediary and final operations
The « consuming » operations: forEach() and peek()
The « mapping » operations: map() and flatMap()
The « filter » operation: filter()
The « reduction » operations:
Aggregations: reduce(), max(), min(), …
Mutable reductions: collect, Collectors
