shell中spark-sql语句调试、执行方式

1.命令方式执行sparksql查询

SQL="use mydatatable;;select count(1) from tab_videousr_onlne 
where p_regiion=101 and p_date='2017-04-05' and p_hour=21;"

/home/mr/spark/bin/beeline -u jdbc:hive2://localhost:18000 
-n mr -p mr --maxwidth=3000 -e "$SQL" >> /home/mr/query_result.txt

注：在执行语句后重定向（>>）生成文件，可以收集查询结果。

p.s. 也可以通过shell命令手动输入sql语句：

echo -n input the query SQL: ; read SQL;

2.脚本方式执行sparksql查询

su - mr -c "beeline -u jdbc:hive2://localhost:18000/mydatatable 
-n mr -f /home/1209sparktableupgrade/spark_add_table.sql"

原文链接：https://www.cnblogs.com/sayhihi/p/7402814.html

beeline命令使用帮助：

cd $SPARK_HOME/bin 
beeline --help

-u <database url>               the JDBC URL to connect to
-n <username>                   the username to connect as
-p <password>                   the password to connect as
-d <driver class>               the driver class to use
-i <init file>                  script file for initialization
-e <query>                      query that should be executed
-f <exec file>                  script file that should be executed
-w (or) --password-file <password file>  the password file to read password from
--hiveconf property=value       Use value for given property
--hivevar name=value            hive variable name and value
                                This is Hive specific settings in which variables
                                can be set at session level and referenced in Hive
                                commands or queries.
--color=[true/false]            control whether color is used for display
--showHeader=[true/false]       show column names in query results
--headerInterval=ROWS;          the interval between which heades are displayed
--fastConnect=[true/false]      skip building table/column list for tab-completion
--autoCommit=[true/false]       enable/disable automatic transaction commit
--verbose=[true/false]          show verbose error messages and debug info
--showWarnings=[true/false]     display connection warnings
--showNestedErrs=[true/false]   display nested errors
--numberFormat=[pattern]        format numbers using DecimalFormat pattern
--entireLineAsCommand=[true/false] control whether entire line treats as one command
--force=[true/false]            continue running script even after errors
--maxWidth=MAXWIDTH             the maximum width of the terminal
--maxColumnWidth=MAXCOLWIDTH    the maximum width to use when displaying columns
--silent=[true/false]           be more silent
--autosave=[true/false]         automatically save preferences
--outputformat=[table/vertical/csv2/tsv2/dsv/csv/tsv]  format mode for result display
                                Note that csv, and tsv are deprecated - use csv2, tsv2 instead
--truncateTable=[true/false]    truncate table column when it exceeds length
--delimiterForDSV=DELIMITER     specify the delimiter for delimiter-separated values output format (default: |)
--isolation=LEVEL               set the transaction isolation level
--nullemptystring=[true/false]  set to true to get historic behavior of printing null as empty string
--addlocaldriverjar=DRIVERJARNAME Add driver jar file in the beeline client side
--addlocaldrivername=DRIVERNAME Add drvier name needs to be supported in the beeline client side
--incremental=[true/false]      Print output incrementally
--help                          display this message