Why are allowed scopes limited for credentials_gce?

[samterfa]

Thank you so much for your work on this package! I had struggled to pass in credentials to my publicly available Plumber API application running on Google Cloud Run without a bunch of gcloud commands, yaml files, or just baking the credentials into the application (not a good idea, I know). credentials_gce() allows me to grab secrets from Google Secrets Manager super easily with a quick API call during the build.

I'm wondering if there's a compelling reason that credentials_gce() limits the scopes for which a token can be fetched to Google Cloud and Big Query. The Google Secrets API call I use works because the scope needed is just "https://www.googleapis.com/auth/cloud-platform" which is on the blessed list of scopes.

# We add a special case for the cloud-platform -> bigquery scope implication.
  if ("https://www.googleapis.com/auth/cloud-platform" %in% instance_scopes) {
    instance_scopes <- c(
      "https://www.googleapis.com/auth/bigquery",
      instance_scopes
    )
  }
  if (!all(scopes %in% instance_scopes)) {
    return(NULL)
  }

By adding "https://www.googleapis.com/auth/pubsub" to the instance_scopes allowed in credentials_gce(), and giving my default compute service account in Cloud Run the Pub/Sub Publisher role, I was able to run PubSub API calls without passing in credentials to the application. As I understand it, something like this is Google's preferred method of passing in Google Cloud API credentials to an application based on this page: "If your application runs inside a Google Cloud environment that has a default service account, your application can retrieve the service account credentials to call Google Cloud APIs. Such environments include Compute Engine, Google Kubernetes Engine, App Engine, Cloud Run, and Cloud Functions. We recommend using this strategy because it is more convenient and secure than manually passing credentials."

I do understand that I could generate a json file credential, upload it to Google Secrets Manager, then load it via the same API calls I'm making, but it's less convenient. Maybe it's better practice though.

[samterfa]

I admit I am a GCP novice, but I think there's a misunderstanding about what an instance (or "access") scope of "https://www.googleapis.com/auth/cloud-platform" means. According to this documentation, having an instance scope of https://www.googleapis.com/auth/cloud-platform means you "allow full access to all Google Cloud APIs".

"Generally, you can just set the cloud-platform access scope to allow full access to all Cloud APIs, then grant the service account only relevant IAM roles. The combination of access scopes granted to the virtual machine instance and the IAM roles granted to the service account determines the amount of access the service account has for that instance. The service account can execute API methods only if they are allowed by both the access scope and its IAM roles."

When I set up a Cloud Run service and run print(gargle:::get_instance_scopes('default')), I get a list of instance (or "access") scopes which includes https://www.googleapis.com/auth/cloud-platform as well as an assortment of non-Google Cloud API scopes. Therefore, the expected behavior for credentials_gce() to me would be to allow any Google Cloud API scope in my requests (including pubsub, cloud scheduler, etc) as well as those non-Google Cloud API scopes listed. This is not what credentials_gce() does though. It interprets the access scope of https://www.googleapis.com/auth/cloud-platform as an API scope, adds Big Query to it, and returns NULL if your requested API scopes aren't a subset of c("https://www.googleapis.com/auth/cloud-platform", "https://www.googleapis.com/auth/bigquery", {various non-Google Cloud API scopes}).

The easiest fix to me would be to not filter the requested scopes for credentials_gce() at all. Google will still require your service account to have the proper IAM roles necessary to perform the API calls.

Thank you for considering. This function is SUPER helpful when deploying to Cloud Run!

[samterfa]

I'm wondering how to elicit a response on this. Would you like a PR? More clarification? credentials_gce() is potentially super useful, but right now it's unnecessarily limited. A single commented out line would "fix" the issue.

Thank you!

[jennybc]

Hi @samterfa, I'm simply dug in on other things and this is also a usage mode that I don't personally have, so I can't quickly dip in w/ a change or feel very confident re: forming an opinion re: a change. A PR is probably a good idea. It sounds like it would be easy to make and makes things one click more concrete.

Just in the abstract, it seems like instead of removing all restrictions on scopes, we might move towards a more refined check, yes? I assume there must have been some reason for the original restrictions, i.e. it wasn't some random or capricious choice (makes me think of Chesterton's Fence). This is the sort of thing that we have to consider as we arrive at the "real" solution here.

[craigcitro]

@samterfa can you say more about the trouble you're having?

In particular, IIUC the situation you're in is the following:

• you want to talk to the pubsub API
• you're on a GCE instance that has the cloud-platform scope
• when you call credentials_gce(scopes = 'https://www.googleapis.com/auth/pubsub'), you get an error.

If that's right, two thoughts:

• you can immediately sidestep the problem by passing cloud-platform into the credentials_gce call.
• we should consider switching the if (!all(...)) { return(NULL) } block. It's there as a safeguard; reasonable options include (1) dropping it, (2) making it a warning, or (3) only doing anything if cloud-platform isn't in instance_scopes.

[samterfa]

@jennybc Thank you for the intro to Chesterton's Fence. It's a great way to think about these issues for sure.
@craigcitro Thank you for the easy bypass using the cloud-platform scope; that works great. I really, really thought I had tried that so I'm confused but happy it's working. Google's page explains how to use this default service account but that it's not best-practice, so to me that seems like a warning kind of situation.
I really appreciate all the work done on this package!

[jennybc]

Seems relevant to this conversation (and a similar one I'm having with myself re: workload identity federation and scopes #176)

https://cloud.google.com/iam/docs/service-accounts#access_scopes

Access scopes are a legacy method of specifying permissions for a Compute Engine virtual machine (VM) instance. They define the default OAuth scopes used in requests from the gcloud tool and client libraries.

Google Cloud now uses IAM, not access scopes, to specify permissions for Compute Engine instances. However, you are still required to set an access scope when you configure an instance to impersonate a service account.

Also see https://cloud.google.com/compute/docs/access/service-accounts#accesscopesiam

[craigcitro]

I sent #185 which implements the "drop the overzealous safeguard" approach I mentioned above. I think that should have you unblocked.