Repository navigation
Accept hyperlink.URLs #212
Description
You can't pass treq a URL because of how twisted.web.client.Agent works:
import hyperlink, treq, twisted.internet.task
@twisted.internet.task.react
def run(reactor):
return treq.get(hyperlink.URL.from_text(u"https://twistedmatrix.com/")) Traceback (most recent call last):
File "/tmp/example.py", line 3, in <module>
@twisted.internet.task.react
File "/home/mrw/.virtualenvs/tmp-963c1ecd0f4cfd34/lib/python2.7/site-packages/twisted/internet/task.py", line 908, in react
finished = main(_reactor, *argv)
File "/tmp/example.py", line 5, in run
return treq.get(hyperlink.URL.from_text(u"https://twistedmatrix.com/"))
File "/home/mrw/.virtualenvs/tmp-963c1ecd0f4cfd34/lib/python2.7/site-packages/treq/api.py", line 24, in get
return _client(**kwargs).get(url, headers=headers, **kwargs)
File "/home/mrw/.virtualenvs/tmp-963c1ecd0f4cfd34/lib/python2.7/site-packages/treq/client.py", line 109, in get
return self.request('GET', url, **kwargs)
File "/home/mrw/.virtualenvs/tmp-963c1ecd0f4cfd34/lib/python2.7/site-packages/treq/client.py", line 218, in request
bodyProducer=bodyProducer)
File "/home/mrw/.virtualenvs/tmp-963c1ecd0f4cfd34/lib/python2.7/site-packages/twisted/web/client.py", line 1973, in request
deferred = self._agent.request(method, uri, headers, bodyProducer)
File "/home/mrw/.virtualenvs/tmp-963c1ecd0f4cfd34/lib/python2.7/site-packages/twisted/web/client.py", line 2041, in request
deferred = self._agent.request(method, uri, headers, bodyProducer)
File "/home/mrw/.virtualenvs/tmp-963c1ecd0f4cfd34/lib/python2.7/site-packages/twisted/web/client.py", line 1838, in request
lastRequest = _FakeUrllib2Request(uri)
File "/home/mrw/.virtualenvs/tmp-963c1ecd0f4cfd34/lib/python2.7/site-packages/twisted/web/client.py", line 1720, in __init__
self.uri = nativeString(uri)
File "/home/mrw/.virtualenvs/tmp-963c1ecd0f4cfd34/lib/python2.7/site-packages/twisted/python/compat.py", line 412, in nativeString
raise TypeError("%r is neither bytes nor unicode" % s)
TypeError: URL.from_text(u'https://twistedmatrix.com/') is neither bytes nor unicode
python-hyper/hyperlink#55 means we can call bytes on URLs to get something we can pass Agent.request.
On the one hand, calling bytes on any old object seems bad, but on the other, we already use URL to convert unicode URLs to bytes.
If there's any API change, I favor adding an else that does url = bytes(url), but we'd also need to fix _combine_query_params.
However, I think I prefer that treq only accept bytes and unicode, and that we tell people to do bytes(theirURL) themselves.
Reacted by Glyph and Wilfredo Sánchez Vega
Metadata
Metadata
Assignees
Labels
No labels
Activity
twm commentedon Mar 18, 2020
I definitely want to accept
hyperlink.URLdirectly. When I use treq I often usehyperlink.URLto build the URLs I pass to treq. Since treq uses the class internally to manage IDN (and, once #265 is merged, params), I see no real downside to acceptinghyperlink.URL— it's strictly fewer conversions. Plus, accepting the class allows us to highlighthyperlinkas the useful and correct library it is. I think that users will often be better-served withhyperlinkthan any of the functions inurllib.parse.