hive parse_url 函数的使用

原创已于 2022-03-03 17:53:09 修改 · 9.6k 阅读

6 ·

CC 4.0 BY-SA版权

文章标签：

#hive parse_url

于 2018-03-21 13:42:42 首次发布

hive 专栏收录该内容

18 篇文章

订阅专栏

parse_url(url, partToExtract[, key]) - extracts a part from a URL

解析URL字符串，partToExtract的选项包含[HOST,PATH,QUERY,REF,PROTOCOL,FILE,AUTHORITY,USERINFO]。

【host,path,query,ref,protocol,file,authority,userinfo】

select parse_url('http://facebook.com/path/p1.php?query=1', 'HOST') from dw.d_city limit 1; ---- facebook.com

select parse_url('http://facebook.com/path/p1.php?query=1', 'PATH') from dw.d_city limit 1; ---- /path/p1.php

select parse_url('http://facebook.com/path/p1.php?query=1', 'QUERY') from dw.d_city limit 1;----query=1

select parse_url('http://facebook.com/path/p1.php?query=1&name=3', 'QUERY','query')

from dw.d_city limit 1;------ 1

select parse_url('http://facebook.com/path/p1.php?query=1&name=3', 'QUERY','name')

from dw.d_city limit 1;---- 3

select parse_url('http://facebook.com/path/p1.php?query=1', 'REF') from dw.d_city limit 1; ---- 空

select parse_url('http://facebook.com/path/p1.php?query=1', 'PROTOCOL') from dw.d_city limit 1;---- http

select parse_url('http://facebook.com/path/p1.php?query=1', 'FILE') from dw.d_city limit 1;----/path/p1.php?query=1

select parse_url('http://facebook.com/path/p1.php?query=1', 'AUTHORITY') from dw.d_city limit 1;----facebook.com

select parse_url('http://facebook.com/path/p1.php?query=1', 'USERINFO') from dw.d_city limit 1;----空

hive url 解码

select reflect('java.net.URLDecoder', 'decode', 'https%3A%2F%2F2.zoppoz.workers.dev%3A443%2Fhttps%2Fwww.baidu.com%2Froute%2Fj%2Fhh%2Fload%2FpsgHtml%23pages%2Fclassification%3Fid%3D1', 'UTF-8')

解码后 https://www.baidu.com/route/j/hh/load/psgHtml#pages/classification?id=1