[4]Carbondata integration-presto查询carbondata

1、编译carbondata获得presto connector相关jar.

参考:CarbonData编译与可能的依赖错误
在presto(建议0.210+版本,否则spi接口不一致presto无法识别carbondata)安装目录的plugin目录下新建carbondata目录,将carbondata编译生成的相关jar拷贝到该新建目录:

cd plugin
mkdir carbondata
cp <carbon-data-installation-directory>/integration/presto/target/carbondata-presto-1.5.1-SNAPSHOT/* <presto-installation-directory>/plugin/carbondata

2、presto相关配置

这里演示单机presto
在etc/catalog下新建carbondata.properties

connector.name=carbondata
hive.metastore.uri=thrift://localhost:9083 

Carbondata becomes one of the supported format of presto hive plugin, so the configurations and setup is similar to hive connector of presto. Please refer https://prestodb.io/docs/current/connector/hive.html for more details.

Note: Since carbon can work only with hive metastore, it is necessary that spark also connects to same metastore db for creating tables and updating tables. All the operations done on spark will be reflected in presto immediately. It is mandatory to create Carbon tables from spark using CarbonData 1.5.2 or greater version since input/output formats are updated in carbon table properly from this version.

其他配置文件参考

(1)config.properties

coordinator=true
datasources=mysql,hive,carbondata
node-scheduler.include-coordinator=true
http-server.http.port=8080
query.max-memory=5GB
query.max-memory-per-node=1GB
discovery-server.enabled=true
discovery.uri=http://localhost:8080

(2) jvm.config

-server
-Xmx16G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOv
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值