1、编译carbondata获得presto connector相关jar.
参考:CarbonData编译与可能的依赖错误
在presto(建议0.210+版本,否则spi接口不一致presto无法识别carbondata)安装目录的plugin目录下新建carbondata目录,将carbondata编译生成的相关jar拷贝到该新建目录:
cd plugin
mkdir carbondata
cp <carbon-data-installation-directory>/integration/presto/target/carbondata-presto-1.5.1-SNAPSHOT/* <presto-installation-directory>/plugin/carbondata
2、presto相关配置
这里演示单机presto
在etc/catalog下新建carbondata.properties
connector.name=carbondata
hive.metastore.uri=thrift://localhost:9083
Carbondata becomes one of the supported format of presto hive plugin, so the configurations and setup is similar to hive connector of presto. Please refer https://prestodb.io/docs/current/connector/hive.html for more details.
Note: Since carbon can work only with hive metastore, it is necessary that spark also connects to same metastore db for creating tables and updating tables. All the operations done on spark will be reflected in presto immediately. It is mandatory to create Carbon tables from spark using CarbonData 1.5.2 or greater version since input/output formats are updated in carbon table properly from this version.
其他配置文件参考
(1)config.properties
coordinator=true
datasources=mysql,hive,carbondata
node-scheduler.include-coordinator=true
http-server.http.port=8080
query.max-memory=5GB
query.max-memory-per-node=1GB
discovery-server.enabled=true
discovery.uri=http://localhost:8080
(2) jvm.config
-server
-Xmx16G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOv