datastore: async/await support

[c4rlo]

It would be awesome if the datastore library supported async/await style operations, e.g.:

entity = await client.get_async(key)

Something similar was previously suggested in googleapis/google-cloud-python#40. That was folded into googleapis/google-cloud-python#557, which isn't really the same thing, especially since ndb does not support async/await with no plans to add this support (see googleapis/python-ndb#289).

[busunkim96]

CC @software-dov

[lidizheng]

@c4rlo I'm currently working on adding AsyncIO support to gRPC (doc). Suggestions and comments are welcomed. Can you provide more details about your usage? E.g. Are you building for new services, or would like to integrate AsyncIO into existing ones? What's your Python version?

[c4rlo]

Cool!

Not quite what I was asking for here though; ideally I'd like to use the datastore library rather than gRPC directly.

Once gRPC supports asyncio, is it likely that the datastore library will make await support available to users?

Meanwhile it seems that the gRPC Python API already has a .future() method on the stub methods (that's how ndb works its tasklet-based magic). I wonder whether this could in principle also be used to support await in the datastore library? Maybe not worth it though if improved async support in gRPC is not far off...

Edit: As for my usage, it's for a hobby project I've got on Google App Engine (https://vimhelp.org) which I'm looking to migrate from Python 2.7 to Python 3.7/3.8. It's currently using ndb, but it's not much code and I'd like to migrate it to whatever is the nicest way of doing things :)

[dannymilsom]

Big +1! I recently commented on this on googleapis/google-cloud-python#3329 (comment)

I'm also using the google cloud datastore library on Google App Engine (Python 3).

[lidizheng]

@dannymilsom Reviews and comments are welcomed. Also, can you provide more details? What is the most important value of providing AsyncIO API for you? E.g. improving performance, compatible with existing AsynciO application, or better programming practice...

[paulking86]

+1 too.

https://docs.djangoproject.com/en/3.0/topics/async/

Django is planning to support fully async views and middleware soon, and I would very much like to use datastore on app engine!

[dannymilsom]

@lidizheng Hi! We build lots of web apps on GCP - particularly Google App Engine with Datastore. Historically this has centred around Django which has always been WSGI only, but there are now more and more Python web frameworks moving to support ASGI (we are actively using FastAPI and Django even has it on the 3.0 roadmap.

Leveraging the async API in the cloud datastore library would almost definitely enable better performance and lower running costs on cloud services.

[dannymilsom]

Just to also add that this seems to be a popular feature request from the community based on older issues - see googleapis/google-cloud-python#3103

[paulking86]

django/django@fc0fa72

Django async support has now been implemented. Any update on this? @crwilcox

[nikita-davydov]

BUMP for this issue

[faynburd]

+1