准备
已经成功安装单机presto(没有安装也可以),配置过hive.catalog文件
git clone master分支
gihub:https://github.com/prestodb/presto
1-Building Presto
在根目录执行,会下载依赖至 (~/.m2/repository
),这会一并执行测试部分,时间长些
./mvnw clean install
若想跳过其中的test 测试代码,也可选择执行
./mvnw clean install -DskipTests
或者./
mvn clean install -DskipTests -Dcheckstyle.skip=true (当IDEA启动presto server 有问题时,重新执行一遍)
2-Running Presto in your IDE
推荐使用IntelliJ IDEA,通过Open from the File menu and select the root pom.xml
file 打开工程,确定Java SDK 配置正确:
- Open the File menu and select Project Structure
- In the SDKs section, ensure that a 1.8 JDK is selected (create one if none exist)
- In the Project section, ensure the Project language level is set to 8.0 as Presto makes use of several Java 8 language features
配置运行环境:
- Main Class:
com.facebook.presto.server.PrestoServer
- VM Options:
-ea -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+UseGCOverheadLimit -XX:+ExplicitGCInvokesConcurrent -Xmx2G -Dconfig=etc/config.properties -Dlog.levels-file=etc/log.properties
-Dhive.metastore.uri=thrift://localhost:9083
- Working directory:
$MODULE_DIR$
- Use classpath of module:
presto-main
The working directory should be the presto-main
subdirectory. In IntelliJ, using $MODULE_DIR$
accomplishes this automatically.
备注:对于hive plugin必须配置Hive metastore Thrift service的location,在VM options的列表中添加下面的参数 把 localhost:9083
改成你的实际参数(如果你没有hive metastore 直接使用下面的参数):
-Dhive.metastore.uri=thrift://localhost:9083
3-启动hive-meta和hive-server2
hive --service hiveserver2 &
hive --service metastore &
如果没有安装presto 跳过4到7,直接按8中说明IDEA中完全执行
4-启动Presto server
/Documents/software/Presto/presto-server-0.191$ bin/launcher start
Started as 56523
5-Running the CLI
IDEA中Start the CLI to connect to the server and run SQL queries:
presto-cli/target/presto-cli-*-executable.jar
6-IDEA查询hive
presto> SELECT * FROM hive.hive_test.student;
SELECT * FROM hive.hive_test.student;
id | name
------+----------
1001 | zhangsan
1002 | lisi
(2 rows)
Query 20171220_031213_00003_8z8an, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:00 [2 rows, 23B] [26 rows/s, 300B/s]
presto>
7-UI
localhost:8080
以上是利用IDEA中源码中的presto-cli代码部分提交statement(sql),可以debug看presto-cli部分代码的执行流程。
然后,通过restful接口调用本地的单机presto server (“4-启动Presto server”)执行查询,查询本地单机hive中的数据
其中主入口为:
Console::runConsole方法中等待Console中用户输入的statement,之后
LineReader接收到输入,转换成sql
process(queryRunner, split.statement(), outputFormat, tableNameCompleter::populateCache, true);
然后,通过restful接口调用本地的单机presto server (“4-启动Presto server”)执行查询,查询本地单机hive中的数据
8-Debug 模式学习源码
要debug执行代码, 则需要本地IDEA中运行persto Server ,步骤主要是:
关闭 “4-启动Presto server”中启动的本地的单机presto server (否则已有一个sever idea中再启用一个会出错),命令行jps 查询pid kill掉
(1)如“ 2-Running Presto in your IDE” 中配置一个run application 如命名为persto
然后run 或 debug presto,同时run CLI 如 “5-Running the CLI”所示,IDEA中启动了persto Server
这时,就可从IEDA中的cli中提交sql,同时debug 在IDEA 中看server中具体执行,server的具体代码解析后续学习后 再整理
备注debug 过程中CLI 中提示timeout ,这时可以将配置参数调的大些