Await 2.
0
Stackless Resumable
Functions
M O S T S C AL ABLE , M O S T E F F I C IEN T, M O S T O P E N
C O R O U TI N E S O F AN Y P R O G R AM MIN G L AN G U AGE I N
E X I S TEN C E
CppCon 2014 • Gor Nishanov ([email protected]) • Microsoft
What this talk is about
• Evolution of N3858 and N3977
• Stackless Resumable Functions (D4134)
• Lightweight, customizable coroutines
• Proposed for C++17
• Experimental implementation “to be” released in Visual Studio “14”
• What are they?
• How they work?
• How to use them?
• How to customize them?
CppCon 2014 • Stackless Resumable Functions 2
56 years
Coroutines ago
• Introduced in 1958 by Melvin Conway
• Donald Knuth, 1968: “generalization of subroutine”
subroutines coroutines
call Allocate frame, pass parameters Allocate frame, pass parameters
return Free frame, return result Free frame, return eventual result
suspend x yes
resume x yes
CppCon 2014 • Stackless Resumable Functions 3
User Mode Threads / Fibers
Coroutine classification Stackless Resumable Functions
• Symmetric / Asymmetric
• Modula-2 / Win32 Fibers / Boost::context are symmetric (SwitchToFiber)
• C# asymmetric (distinct suspend and resume operations)
• First-class / Constrained
• Can coroutine be passed as a parameter, returned from a function, stored in a
data structure?
• Stackful / Stackless
• How much state coroutine has? Just the locals of the coroutine or entire stack?
• Can coroutine be suspended from nested stack frames
CppCon 2014 • Stackless Resumable Functions 4
Stackful vs. Stackless
Coroutine State Coroutine State
Captured Coroutine State:
(chained stack)
Parameters
4k stacklet
Locals &
Temporaries
4k stacklet
4k stacklet
1 meg of stack 1 meg
4k stacklet of stack
4k stacklet
CppCon 2014 • Stackless Resumable Functions 5
Design Goals
• Highly scalable (to hundred millions of concurrent coroutines)
• Highly efficient (resume and suspend operations comparable
in cost to a function call overhead)
• Seamless interaction with existing facilities with no overhead
• Open ended coroutine machinery allowing library designers to
develop coroutine libraries exposing various high-level
semantics, such as generators, goroutines, tasks and more.
• Usable in environments where exception are forbidden or not
available
CppCon 2014 • Stackless Resumable Functions 6
Anatomy of a Function
std::future<ptrdiff_t> tcp_reader(int total)
{
char buf[64 * 1024];
ptrdiff_t result = 0;
auto conn =
CppCon 2014 • Stackless Resumable Functions 7
Anatomy of a Resumable Function
std::future<ptrdiff_t> tcp_reader(int total)
{
char buf[64 * 1024];
ptrdiff_t result = 0;
auto conn = await Tcp::Connect("127.0.0.1", 1337);
do
{
auto bytesRead = await conn.Read(buf, sizeof(buf));
total -= bytesRead;
result += std::count(buf, buf + bytesRead, 'c');
}
while (total > 0);
return result;
CppCon 2014 • Stackless Resumable Functions 8
Satisfies
Coroutine Promise Requirements
Anatomy of a Stackless Resumable Function
Coroutine Coroutine Frame
Return Object
Coroutine Promise
std::future<ptrdiff_t> tcp_reader(int total) Platform Context*
{
char buf[64 * 1024];
Formals (Copy)
ptrdiff_t result = 0; Locals / Temporaries
auto conn = await Tcp::Connect("127.0.0.1", 1337);
do
Suspend {
Points auto bytesRead = await conn.Read(buf, sizeof(buf));
total -= bytesRead;
result += std::count(buf, buf + bytesRead, 'c');
}
while (total > 0);
return result; Satisfies Awaitable
Requirements
}
Coroutine await <initial-suspend>
Eventual Result
CppCon 2014 • Stackless Resumable Functions await <final-suspend> 9
Compiler vs Coroutine Promise
return <expr> <Promise>.set_value(<expr>);
goto <end>
<unhandled-exception> <Promise>.set_exception (
std::current_exception())
<get-return-object> <Promise>.get_return_object()
yield <expr> await <Promise>.yield_value(<expr>)
await() await <Promise>.read_value()
<after-first-curly> await <Promise>.initial_suspend()
<before-last-curly> await <Promise>.final_suspend()
<cancel-check> if(<Promise>.cancellation_requested())
<goto end>
CppCon 2014 • Stackless Resumable Functions 10
2x2x2
• Two new keywords
• await
• yield
• Two new concepts
• Awaitable
• Coroutine Promise
• Two new types
• resumable_handle
• resumable_traits
CppCon 2014 • Stackless Resumable Functions 11
Examples
CppCon 2014 • Stackless Resumable Functions 12
Coroutine Promise
current_value
Active / Cancelling /
Generator coroutines Closed
exception
generator<int> fib(int n)
{ generator<int>
int a = 0;
int b = 1;
while (n-- > 0) generator<int>::iterator
{
yield a;
auto next = a + b; {
a = b; auto && __range = fib(35);
b = next; for (auto __begin = __range.begin(),
} __end = __range.end()
} ;
__begin != __end
;
int main() {
++__begin)
for (auto v : fib(35))
{
{
if (v > 10) auto v = *__begin;
break; {
cout << v << ' '; if (v > 10) break;
} cout << v << ' ';
} }
}
CppCon 2014 • Stackless Resumable Functions } 13
Recursive Generators
recursive_generator<int> range(int a, int b)
{
auto n = b - a;
if (n <= 0)
return; int main()
{
if (n == 1) auto r = range(0, 100);
{ copy(begin(r), end(r),
yield a; ostream_iterator<int>(cout, " "));
}
return;
}
auto mid = a + n / 2;
yield range(a, mid);
yield range(mid, b);
}
CppCon 2014 • Stackless Resumable Functions 14
Parent-stealing scheduling
spawnable<int> fib(int n) {
if (n < 2) return n;
return await(fib(n - 1) + fib(n - 2));
}
int main() { std::cout << fib(5).get() << std::endl; }
1,4 billion recursive invocations to compute fib(43), uses less than 16k of space
Not using parent-stealing, runs out of memory at fib(35)
n
1 1 1 x
1 0 1
= y
CppCon 2014 • Stackless Resumable Functions 15
Goroutines?
goroutine pusher(channel<int>& left, channel<int>& right) {
for (;;) {
auto val = await left.pull();
await right.push(val + 1);
}
}
CppCon 2014 • Stackless Resumable Functions 16
Goroutines? Sure. 100,000,000 of them
goroutine pusher(channel<int>& left, channel<int>& right) {
for (;;) {
auto val = await left.pull();
await right.push(val + 1);
}
}
int main() {
const int N = 100 * 1000 * 1000; c0-g0-c1
vector<channel<int>> c(N + 1); c1-g1-c2
…
for (int i = 0; i < N; ++i)
goroutine::go(pusher(c[i], c[i + 1])); cn-gn-cn+1
c.front().sync_push(0);
cout << c.back().sync_pull() << endl;
}
CppCon 2014 • Stackless Resumable Functions 17
Reminder: Just Core Language Evolution
BE-DEVs
FE-DEVs
Library Designer Paradise
• Lib devs can design new coroutines types
• generator<T>
• goroutine
• spawnable<T>
• task<T>
• …
• Or adapt to existing async facilities
• std::future<T>
• concurrency::task<T>
• IAsyncAction, IAsyncOperation<T>
• …
CppCon 2014 • Stackless Resumable Functions 18
Awaitable
CppCon 2014 • Stackless Resumable Functions 19
Reminder: Range-Based For
{
auto && __range = fib(35);
for (auto __begin = __range.begin(),
__end = __range.end()
;
__begin != __end
;
++__begin)
{
int main() { auto v = *__begin;
for (auto v : fib(35)) cout << v << endl;
cout << v << endl; }
} }
CppCon 2014 • Stackless Resumable Functions 20
If <expr> is a class type and
await <expr> unqualified ids await_ready,
await_suspend or await_resume
Expands into an expression equivalent of are found in the scope of a class
{
auto && __tmp = <expr>;
if (!__tmp.await_ready()) {
__tmp.await_suspend(<resumption-function-object>);
suspend
resume
}
<cancel-check>
return __tmp.await_resume();
}
CppCon 2014 • Stackless Resumable Functions 21
Otherwise
await <expr> (see rules for range-based-for
lookup)
Expands into an expression equivalent of
{
auto && __tmp = <expr>;
if (! await_ready(__tmp)) {
await_suspend(__tmp, <resumption-function-object>);
suspend
resume
}
<cancel-check>
return await_resume(__tmp);
}
CppCon 2014 • Stackless Resumable Functions 22
Trivial Awaitable #1
struct _____blank____ {
bool await_ready(){ return false; }
template <typename F>
void await_suspend(F const&){}
void await_resume(){}
};
CppCon 2014 • Stackless Resumable Functions 23
Trivial Awaitable #1
struct suspend_always {
bool await_ready(){ return false; }
template <typename F>
void await_suspend(F const&){}
void await_resume(){}
};
await suspend_always {};
CppCon 2014 • Stackless Resumable Functions 24
Trivial Awaitable #2
struct suspend_never {
bool await_ready(){ return true; }
template <typename F>
void await_suspend(F const&){}
void await_resume(){}
};
CppCon 2014 • Stackless Resumable Functions 25
Simple Awaitable #1
std::future<void> DoSomething(mutex& m) {
unique_lock<mutex> lock = await lock_or_suspend{m};
// ...
}
struct lock_or_suspend {
std::unique_lock<std::mutex> lock;
lock_or_suspend(std::mutex & mut) : lock(mut, std::try_to_lock) {}
bool await_ready() { return lock.owns_lock(); }
template <typename F>
void await_suspend(F cb)
{
std::thread t([this, cb]{ lock.lock(); cb(); });
t.detach();
}
auto await_resume() { return std::move(lock);}
};
CppCon 2014 • Stackless Resumable Functions 26
Simple Awaiter #2: Making Boost.Future awaitable
#include <boost/thread/future.hpp>
namespace boost {
template <class T>
bool await_ready(unique_future<T> & t) {
return t.is_ready();
}
template <class T, class F>
void await_suspend(unique_future<T> & t,
F resume_callback)
{
t.then([=](auto&){resume_callback();});
}
template <class T>
auto await_resume(unique_future<T> & t) {
return t.get(); }
}
}
CppCon 2014 • Stackless Resumable Functions 27
Awaitable
Interacting with C APIs
CppCon 2014 • Stackless Resumable Functions 28
2x2x2
• Two new keywords
• await
• yield
• Two new concepts
• Awaitable
• Coroutine Promise
• Two new types
• resumable_handle
• resumable_traits
CppCon 2014 • Stackless Resumable Functions 29
resumable_handle
template <typename Promise = void> struct resumable_handle;
== != < > <= >=
template <> struct resumable_handle<void> {
void operator() ();
void * to_address();
static resumable_handle<void> from_address(void*);
…
};
template <typename Promise>
struct resumable_handle: public resumable_handle<> {
Promise & promise();
static resumable_handle<Promise> from_promise(Promise*);
…
};
CppCon 2014 • Stackless Resumable Functions 30
Simple Awaitable #2: Raw OS APIs
await sleep_for(10ms);
class sleep_for {
static void TimerCallback(PTP_CALLBACK_INSTANCE, void* Context, PTP_TIMER) {
std::resumable_handle<>::from_address(Context)();
}
PTP_TIMER timer = nullptr;
std::chrono::system_clock::duration duration;
public:
sleep_for(std::chrono::system_clock::duration d) : duration(d){}
bool await_ready() const { return duration.count() <= 0; }
void await_suspend(std::resumable_handle<> resume_cb) {
int64_t relative_count = -duration.count();
timer = CreateThreadpoolTimer(TimerCallback, resume_cb.to_address(), 0);
SetThreadpoolTimer(timer, (PFILETIME)&relative_count, 0, 0);
}
void await_resume() {}
~sleep_for() { if (timer) CloseThreadpoolTimer(timer); }
};
CppCon 2014 • Stackless Resumable Functions 31
2x2x2
• Two new keywords
• await
• yield
• Two new concepts
• Awaitable
• Coroutine Promise
• Two new types
• resumable_handle
• resumable_traits
CppCon 2014 • Stackless Resumable Functions 32
resumable_traits
generator<int> fib(int n)
std::resumable_traits<generator<int>, int>
template <typename R, typename... Ts>
struct resumable_traits {
using allocator_type = std::allocator<char>;
using promise_type = typename R::promise_type;
};
CppCon 2014 • Stackless Resumable Functions 33
Defining Coroutine Promise for boost::future
namespace std {
template <typename T, typename… anything>
struct resumable_traits<boost::unique_future<T>, anything…> {
struct promise_type {
boost::promise<T> promise;
auto get_return_object() { return promise.get_future(); }
template <class U> void set_value(U && value) {
promise.set_value(std::forward<U>(value));
}
void set_exception(std::exception_ptr e) {
promise.set_exception(std::move(e));
}
suspend_never initial_suspend() { return{}; }
suspend_never final_suspend() { return{}; }
bool cancellation_requested() { return false; }
};
};
CppCon 2014 • Stackless Resumable Functions 34
Awaitable
and Exceptions
CppCon 2014 • Stackless Resumable Functions 35
Exceptionless Error Propagation (Await Part)
#include <boost/thread/future.hpp>
namespace boost {
template <class T>
bool await_ready(unique_future<T> & t) { return t.is_ready();}
template <class T, class F>
void await_suspend(
unique_future<T> & t, F rh)
{
t.then([=](auto& result){
rh();
});
}
template <class T>
auto await_resume(unique_future<T> & t) { return t.get(); }
}
CppCon 2014 • Stackless Resumable Functions 36
Exceptionless Error Propagation (Await Part)
#include <boost/thread/future.hpp>
namespace boost {
template <class T>
bool await_ready(unique_future<T> & t) { return t.is_ready();}
template <class T, class Promise>
void await_suspend(
unique_future<T> & t, std::resumable_handle<Promise> rh)
{
t.then([=](auto& result){
if(result.has_exception())
rh.promise().set_exception(result.get_exception_ptr());
rh();
});
}
template <class T>
auto await_resume(unique_future<T> & t) { return t.get(); }
}
CppCon 2014 • Stackless Resumable Functions 37
Exceptionless Error Propagation (Promise Part)
namespace std {
template <typename T, typename… anything>
struct resumable_traits<boost::unique_future<T>, anything…> {
struct promise_type {
boost::promise<T> promise;
auto get_return_object() { return promise.get_future(); }
suspend_never initial_suspend() { return{}; }
suspend_never final_suspend() { return{}; }
template <class U> void set_value(U && value) {
promise.set_value(std::forward<U>(value));
}
void set_exception(std::exception_ptr e) {
promise.set_exception(std::move(e));
}
bool cancellation_requested() { return false; }
};
}; CppCon 2014 • Stackless Resumable Functions 38
Exceptionless Error Propagation (Promise Part)
namespace std {
template <typename T, typename… anything>
struct resumable_traits<boost::unique_future<T>, anything…> {
struct promise_type {
boost::promise<T> promise;
bool cancelling = false;
auto get_return_object() { return promise.get_future(); }
suspend_never initial_suspend() { return{}; }
suspend_never final_suspend() { return{}; }
template <class U> void set_value(U && value) {
promise.set_value(std::forward<U>(value));
}
void set_exception(std::exception_ptr e) {
promise.set_exception(std::move(e)); cancelling = true;
}
bool cancellation_requested() { return cancelling; }
};
}; CppCon 2014 • Stackless Resumable Functions 39
Simple Happy path and reasonable error
propagation
std::future<ptrdiff_t> tcp_reader(int total)
{
char buf[64 * 1024];
ptrdiff_t result = 0;
auto conn = await Tcp::Connect("127.0.0.1", 1337);
do
{
auto bytesRead = await conn.Read(buf, sizeof(buf));
total -= bytesRead;
result += std::count(buf, buf + bytesRead, 'c');
}
while (total > 0);
return result;
CppCon 2014 • Stackless Resumable Functions 40
Reminder: await <expr>
Expands into an expression equivalent of
{
auto && __tmp = <expr>;
if (! await_ready(__tmp)) {
await_suspend(__tmp, <resumption-function-object>);
suspend
resume
}
if (<promise>.cancellation_requested()) goto <end-label>;
return await_resume(__tmp);
}
CppCon 2014 • Stackless Resumable Functions 41
Done!
CppCon 2014 • Stackless Resumable Functions 42
What this talk was about
• Stackless Resumable Functions (D4134)
• Lightweight, customizable coroutines
• Proposed for C++17
• Experimental implementation “to be” released in Visual Studio “14”
• What are they?
• How they work?
• How to use them?
• How to customize them?
CppCon 2014 • Stackless Resumable Functions 43
To learn more:
• https://github.com/GorNishanov/await/
• Draft snapshot: D4134 Resumable Functions v2.pdf
• In October 2014 look for
• N4134 at http://isocpp.org
• http://open-std.org/JTC1/SC22/WG21/
CppCon 2014 • Stackless Resumable Functions 44
Backup
CppCon 2014 • Stackless Resumable Functions 45
Introduction
CppCon 2014 • Stackless Resumable Functions 46
How does it work?
CppCon 2014 • Stackless Resumable Functions 47
Generator coroutines
generator<int> fib(int n)
{
int a = 0;
int b = 1;
while (n-- > 0) {
{ auto && __range = fib(35);
yield a; for (auto __begin = __range.begin(),
auto next = a + b; __end = __range.end()
a = b; ;
b = next; __begin != __end
} ;
} ++__begin)
{
int main() { auto v = *__begin;
for (auto v : fib(35)) cout << v << endl;
cout << v << endl; }
} }
CppCon 2014 • Stackless Resumable Functions 48
x86_x64 Windows ABI
Execution generator<int> fib(int n)
RSP auto && __range = fib(35)
Stack
Suspend!!!!
ret-addr __range RCX = &__range
RDX = 35
slot1
&__range slot2
savedRDI
Heap
slot3
savedRSI slot4 X
savedRB
RDI = n
Coroutine Frame
ret-main savedRBP RSI = a Coroutine
RBX = b Promise
slot1 slot2
RBP = $fp saved
RDI slot
RDI
slot3 slot4
saved
RSI slot
RSI
saved RBX
RBX slot
RAX = &__range
saved
RIP slot
RIP
CppCon 2014 • Stackless Resumable Functions 49
x86_x64 Windows ABI
Resume generator<int>::iterator::operator ++()
struct iterator {
for(…;…; ++__begin) iterator& operator ++() {
RSP Stack resume_cb(); return *this; }
ret-addr __range RCX = $fp …
slot1 slot2
savedRDI
resumable_handle<Promise> resume_cb;
};
slot3
savedRSI slot4
savedRBX Heap
RDI = n
Coroutine Frame
ret-main savedRBP RSI = a Coroutine
RBX = b Promise
slot1 slot2
slot3 slot4 RBP = $fp saved
RDI slot
RDI
saved
RSI slot
RSI
RBX slot
saved RBX
saved
RIP slot
RIP
CppCon 2014 • Stackless Resumable Functions 50
If await_suspend
returns bool
await <expr>
Expands into an expression equivalent of
{
auto && __tmp = <expr>;
if (! await_ready(__tmp) &&
await_suspend(__tmp, <resumption-function-object>) {
suspend
resume
}
if (<promise>.cancellation_requested()) goto <end-label>;
return await_resume(__tmp);
}
CppCon 2014 • Stackless Resumable Functions 55
Yield implementation
compiler: yield <expr> await <Promise>.yield_value(<expr>)
library:
suspend_now
generator<T>::promise_type::yield_value(T const& expr) {
this->current_value = &expr;
return{};
}
CppCon 2014 • Stackless Resumable Functions 56
awaitable_overlapped_base
struct awaitable_overlapped_base : public OVERLAPPED
{
ULONG IoResult;
ULONG_PTR NumberOfBytesTransferred;
std::resumable_handle<> resume;
static void __stdcall io_complete_callback( PTP_CALLBACK_INSTANCE,
PVOID, PVOID Overlapped, ULONG IoResult,
ULONG_PTR NumberOfBytesTransferred,
PTP_IO)
{
auto o = reinterpret_cast<OVERLAPPED*>(Overlapped);
auto me = static_cast<awaitable_overlapped_base*>(o);
me->IoResult = IoResult;
me->NumberOfBytesTransferred = NumberOfBytesTransferred;
me->resume();
}
};
CppCon 2014 • Stackless Resumable Functions 57
Dial awaitable
class Dial : public awaitable_overlapped_base {
ports::endpoint remote;
Connection conn;
public:
Dial(string_view str, unsigned short port) : remote(str, port) {}
bool await_ready() const { return false; }
void await_suspend(std::resumable_handle<> cb) {
resume = cb;
conn.handle = detail::TcpSocket::Create();
detail::TcpSocket::Bind(conn.handle, ports::endpoint("0.0.0.0"));
conn.io = CreateThreadpoolIo(conn.handle, &io_complete_callback, 0,0);
if (conn.io == nullptr) throw_error(GetLastError());
StartThreadpoolIo(conn.io);
auto error = detail::TcpSocket::Connect(conn.handle, remote, this);
if (error) { CancelThreadpoolIo(conn.io); throw_error(GetLastError());
}
Connection await_resume() {
if (conn.error) throw_error(error);
return std::move(conn);
}
};
CppCon 2014 • Stackless Resumable Functions 58
Connection::Read
auto Connection::read(void* buf, size_t bytes) {
class awaiter : public awaitable_overlapped_base {
void* buf; size_t size;
Connection * conn;
public:
awaiter(void* b, size_t n, Connection * c): buf(b), size(n), conn(c) {}
bool await_ready() const { return false; }
void await_suspend(std::resumable_handle<> cb) {
resume = cb;
StartThreadpoolIo(conn->io);
auto error = TcpSocket::Read(conn->handle, buf, (uint32_t)size, this);
if (error)
{ CancelThreadpoolIo(conn->io); throw_error(error); }
}
int await_resume() {
if (IoResult)
{ throw_error(IoResult); }
return (int)this->NumberOfBytesTransferred;
}
};
return awaiter{ buf, bytes, this };
}
CppCon 2014 • Stackless Resumable Functions 59
asynchronous iterator helper: await for
goroutine foo(channel<int> & input) {
await for(auto && i : input) {
cout << “got: “ << i << endl;
}
}
await for expands into: {
auto && __range = range-init;
for ( auto __begin = await (begin-expr),
__end = end-expr;
__begin != __end;
await ++__begin )
{
for-range-declaration = *__begin;
statement
}
}
Recursive Tree Walk (Stackful) from N3985
void traverse ( node_t * n, std::push_coroutine<std::string> & yield) {
if(n-> left ) traverse (n->left, yield);
yield (n-> value);
if(n-> right ) traverse (n->right, yield);
}
node * root1 = create_tree();
node * root2 = create_tree();
std::pull_coroutine<std::string> reader1( [&](auto & yield ){ traverse (root1, yield);});
std::pull_coroutine<std::string> reader2( [&](auto & yield ){ traverse (root2, yield);});
std :: cout << “equal = “ << std::equal (begin (reader1), end( reader1), begin(reader2))
<< std :: endl ;
Recursive Tree Walk (Stackless)
generator<std::string> traverse(node_t* n)
{
if (p->left) yield traverse(p->left);
yield p->name;
if (p->right) yield traverse(p->right);
}
node * root1 = create_tree();
node * root2 = create_tree();
auto reader1 = traverse (root1);
auto reader2 = traverse (root2);
std :: cout << “equal = “ << std::equal(begin(reader1), end(reader1),
begin(reader2) )
<< std :: endl ;
