`user-agent` header control

[annevk]

Should fetch() set user-agent by default? Allow appending bytes? Allow replacing it? Allow it to be omitted?

See w3c/ServiceWorker#348 (comment) for context.

And why is it on the forbidden header list? It has been since forever, but is there strong rationale?

[hallvors]

I think we 'inherited' User-Agent from a list of headers Flash disallowed changing in its implementation. There was some speculation that some services might exist that only allowed specific customized browsers to access them, using UA string as a sort of auth token, but I don't think anybody ever found a real-life case of such a system. (Obviously it would be astonishingly poor design).

Personally, I think we can and should drop User-Agent from the list of headers you're not allowed to set.

[annevk]

If we allow setting it to any value and effectively replace anything the user agent would set we should probably make it a parameter of RequestInit as the supplied headers argument combines with existing headers (and never replaces). So e.g.

fetch(url, {userAgent: "Hi"})

[hallvors]

(It might be worth noting that GitHub itself recommends User-Agent set to your GH user name in API requests - this is of course trivial if you write Python/Node/whatever clients, but not currently possible from an API consumer running in a browser. So there's a big and important use case right here.)

[domenic]

If we allow setting it to any value and effectively replace anything the user agent would set we should probably make it a parameter of RequestInit as the supplied headers argument combines with existing headers (and never replaces). So e.g.

Hmm, I'm not sure I necessarily agree. It seems a straightforward extension to say that headers provided by the RequestInit overwrite default headers. What do you think would be confusing about allowing User-Agent to be set through the normal mechanisms?

[dgraham]

GitHub itself recommends User-Agent set to your GH user name in API requests

https://developer.github.com/v3/#user-agent-required

This helps us contact application owners when there are problems, like a rogue script making lots of API calls. However, many API resources are only available to authenticated sessions, so the user account is already known. And when accessing the API through a browser, the CORS request includes the Origin header.

The User-Agent header guidance in the API is helpful, but it's not necessarily an example for or against this capability being added to fetch.

[dgraham]

If User-Agent is allowed to be assigned, I think the expected usage would be to pass it in with the rest of the headers for the request.

fetch('/users', {
  headers: {
    'User-Agent': 'Web/2.0',
    'Accept': 'application/json'
  }
})

[annevk]

I guess that might be okay as well. We could have a step in https://fetch.spec.whatwg.org/#http-network-fetch that appends it if it is not already present in HTTPRequest's header list.

[annevk]

So actually, the reason you want it to be an argument rather than part of headers is so you can force it to be omitted.

[annevk]

Probable rationale for why it is currently forbidden is in this email by @sicking: https://lists.w3.org/Archives/Public/public-webapi/2008May/0456.html

[domenic]

So actually, the reason you want it to be an argument rather than part of headers is so you can force it to be omitted.

Can you expand on this? I don't quite understand why one location or the other would matter for this, or why you would force it to be omitted.

[annevk]

If we offer customization, I think it would make sense to also offer omitting it altogether e.g. to reduce the size of the request or debug a server. The location matters since by default it is included and the Headers class is not an instruction set but rather a list of headers included in the request.

[domenic]

Oh, I see!

Would it be conceivable not to set the header by default at all? Hmm, probably not very good, I'm being too Node-influenced here...

So how would you omit it? omitUserAgent: true? userAgent: null?

[annevk]

null was my idea, since undefined already means doing the default, which is to include it and seems highly unlikely to change.

[domenic]

Why would userAgent: null be good, but 'User-Agent': null not be good?

Related question: when you do const h = new Headers(), does h.get("User-Agent") return something? Why or why not?