共計(jì) 6864 個(gè)字符,預(yù)計(jì)需要花費(fèi) 18 分鐘才能閱讀完成。
Elasticsearch SQL 的用法是什么,針對(duì)這個(gè)問題,這篇文章詳細(xì)介紹了相對(duì)應(yīng)的分析和解答,希望可以幫助更多想解決這個(gè)問題的小伙伴找到更簡(jiǎn)單易行的方法。
丸趣 TV 小編將介紹不同版本中 Elasticsearch SQL 的使用方法,總結(jié)了實(shí)際中常用的方法和操作,并給出了幾個(gè)具體例子。
一、5.x 中 ES-SQL 用法
Elasticsearch 5.x 版本中,SQL 功能還沒有集成到 Elasticsearch 源碼中,需要下載第三方插件后才能使用,配置過程如下:
1. 安裝 ES-SQL 依賴 node npm
ES-SQL 5.x 版本以后,安裝需要依賴 node 和 npm,先安裝 node 和 npm,安裝后在檢查 node 及 npm 的安裝,命令如下:
yum -y install nodejs npm
node -v
npm -v
2. 下載 ES-SQL 并安裝
然后切換到 ES 的根目錄下,執(zhí)行如下命令,下載并安 ES-SQL 插件:
./bin/elasticsearch-plugin install https://github.com/NLPchina/elasticsearch-sql/releases/download/5.6.30/.elasticsearch-sql-5.6.3.0.zip
離線包安裝可以執(zhí)行:
./bin/elasticsearch-plugin install file:/elasticsearch-sql-5.6.3.0.zip
3. 重啟 ES 服務(wù)
執(zhí)行完上述三步,你就可以使用 SQL 探索數(shù)據(jù)了,以 kibana 中的使用為例:
二、6.4 Elasticsearch SQL 用法
1 Elasticsearch SQL 支持的數(shù)據(jù)類型
首先我們看下 Elasticsearch SQL 和標(biāo)準(zhǔn) SQL 中數(shù)據(jù)類型的對(duì)應(yīng)關(guān)系:
2 Elasticsearch SQL 的使用方式
Elasticsearch SQL 支持三種 client: REST Interface,command-line,JDBC
2.1 REST Interface
建議先在 kibana 中測(cè)試 (可以一次執(zhí)行多個(gè) SQL),查詢通過之后把查詢 copy 到項(xiàng)目中進(jìn)行測(cè)試。
6.3+ Elasticsearch SQL 有個(gè)非常實(shí)用的功能,就是可以用 translate api 把 SQL 語句翻譯成 ES DSL 語句,對(duì)于學(xué)習(xí) DSL 感到頭痛的同學(xué)有福啦。
2.2 command-line
命令行界面的進(jìn)入方式:
./elasticsearch-sql-cli IP:PORT(本機(jī) ip 和 es 的端口)
進(jìn)入后的界面如下:
命令行一般作為 SQL 測(cè)試時(shí)使用。
2.3 JDBC
該組件為 X -Pack 中的收費(fèi)組件,感興趣的同學(xué)可以參考官方文檔:https://www.elastic.co/guide/en/elasticsearch/reference/current/sql-jdbc.html
3 常用 SQL 語句
注意:查詢單個(gè)索引名一定要用 引上,否則會(huì)報(bào)錯(cuò)
* 查看當(dāng)前用戶所有的索引:“SHOW TABLES;”
精準(zhǔn)查詢某個(gè)索引:“SHOW TABLES LIKE‘indexname’;”
通配符查詢某些索引:“SHOW TABLES LIKE‘’;”
* 查看某個(gè)索引結(jié)構(gòu):“DESCRIBE table;”或者“DESC table;”
上面兩個(gè)命令都是“SHOW COLUMNS [FROM | IN] ? table”命令的別名
* 查看函數(shù):“SHOW FUNCTIONS [LIKE? pattern?]?”
精準(zhǔn)查詢某個(gè)函數(shù):
通配符查詢某些函數(shù):
查看所有函數(shù):
常用的聚合函數(shù):
SELECT MIN(value_1) min, MAX(value_1) max, AVG(value_1) avg,SUM(value_1) sum,COUNT(*) count,COUNT(DISTINCT value_1) dictinct_count FROM micloud_es_sink_zhouyongbo_test-2018.10.19”;
SELECT 語句的語法排序如下:
SELECT select_expr [, ...]
[ FROM table_name ]
[ WHERE condition ]
[ GROUP BY grouping_element [, ...] ]
[ HAVING condition]
[ ORDER BY expression [ ASC | DESC ] [, ...] ]
[ LIMIT [ count ] ]
* 限定返回?cái)?shù)據(jù)的條數(shù):“l(fā)imit”
SELECT * FROM micloud_es_sink_zhouyongbo_test-2018.10.19” limit 10 ;
注意 SQL 中的 limit 比 fetch_size 中的優(yōu)先級(jí)高, 例如下面的例子返回的是 5 條 :
{
query : SELECT * FROM micloud_es_sink_zhouyongbo_test-2018.10.19” limit 5 ,
fetch_size :10
}
* 排序:“order by + 字段名字 + asc/desc”
SELECT * FROM micloud_es_sink_zhouyongbo_test-2018.10.19” ORDER BY value_1 asc/desc;
根據(jù)多個(gè)字段排序:
select city c,value_1 + 1 vp from micloud_es_sink_zhouyongbo_test-2018.10.19 group by c,vp order by c desc,vp asc;
*WHERE 根據(jù)條件查詢:
WHERE 后面跟 ES 復(fù)雜數(shù)據(jù)類型:
SELECT first_name FROM index WHERE first_name.raw = ‘John’ ;
WHERE 后面跟多個(gè)查詢條件:
SELECT * FROM micloud_es_sink_zhouyongbo_test* where city=‘北京 and value_1=8 ORDER BY value_1 desc ;
*group by 分組查詢:
根據(jù)單個(gè)字段分組查詢:
select city,count(city) as count_city,sum(value_1) as count_value_1 from micloud_es_sink_zhouyongbo_test-2018.10.19 group by city;
根據(jù)多個(gè)字段分組查詢:
select city,count(city) count_city,sum(value_1) count_value_1 from micloud_es_sink_zhouyongbo_test-2018.10.19 group by city,value_1;
對(duì)于比較長(zhǎng)的字段,也可以對(duì)該字段聲明別名,并對(duì)別名進(jìn)行分組查詢,聲明字段別名的“as”可省略:
select city c,count(city) count_city,sum(value_1) count_value_1 from micloud_es_sink_zhouyongbo_test-2018.10.19 group by c,value_1;
還可以對(duì)某字段進(jìn)行計(jì)算,然后按照計(jì)算結(jié)果分組查詢:
select city c,value_1 + 1 vp from micloud_es_sink_zhouyongbo_test-2018.10.19 group by c,vp;
*HAVING 過濾分組結(jié)果(ES-SQL 引擎同樣會(huì)在分組之后計(jì)算 HAVING 語句):
Select city c,count(*) count from micloud_es_sink_zhouyongbo_test-2018.10.19 group by c having count 53834;
* 查詢嵌套類型:
select * from zhouyongbo_test04 where love.kaishu=‘魯公’;
* 用通配符查詢多個(gè)索引:
注意被查詢索引必須有相同的 mapping,否則會(huì)有如下報(bào)錯(cuò):
常用的方法和操作匯總:
* 比較操作:
Equality (=)
select * from micloud_es_sink_zhouyongbo_test-2018.10.19 where value_1 = 6 limit 5;
Inequality ( or != or
select * from micloud_es_sink_zhouyongbo_test-2018.10.19 where value_1 6 limit 5;
Comparison (,
select * from micloud_es_sink_zhouyongbo_test-2018.10.19 where value_1 = 6 limit 5;
BETWEEN
select * from micloud_es_sink_zhouyongbo_test-2018.10.19 where value_1 between 6 and 8 limit 5;
IS NULL/IS NOT NULL
select * from micloud_es_sink_zhouyongbo_test-2018.10.19 where value_1 is not NULL limit 5;
* 邏輯操作:
AND
select * from micloud_es_sink_zhouyongbo_test-2018.10.19 where value_1 5 and value_1 7 limit 5;
OR
select * from micloud_es_sink_zhouyongbo_test-2018.10.19 where value_1 = 5 or value_1 = 7 limit 5;
NOT
select * from micloud_es_sink_zhouyongbo_test-2018.10.19 where not value_1 5 limit 5;
* 數(shù)學(xué)運(yùn)算操作:
Add (+)
select 1 + 1 as x;
Subtract (infix -)
select 1 - 1 as x;
Negate (unary -)
select - 1 as x;
Multiply (*)
select 6 * 6 as x;
Divide (/)
select 30 / 5 as x;
Modulo or Reminder(%)
select 30 % 7 as x;
* 數(shù)學(xué)函數(shù):(分為通用函數(shù)和三角函數(shù)兩部分):
通用函數(shù):
ABS:求數(shù)字的絕對(duì)值
select ABS(value_1) from micloud_es_sink_zhouyongbo_test-2018.10.19 limit 5;
CBRT:求數(shù)字的立方根,返回 double
select value_1 v,CBRT(value_1) cbrt from micloud_es_sink_zhouyongbo_test-2018.10.19 limit 5;
CEIL:返回大于或者等于指定表達(dá)式最小整數(shù)(double)
select value_1 v,CEIL(value_1) from micloud_es_sink_zhouyongbo_test-2018.10.19 limit 5;
CEILING:等同于 CEIL
select value_1 v,CEILING(value_1) from micloud_es_sink_zhouyongbo_test-2018.10.19 limit 5;
E:返回自然常數(shù) e(2.718281828459045)
select value_1,E(value_1) from micloud_es_sink_zhouyongbo_test-2018.10.19 limit 5;
ROUND:四舍五入精確到個(gè)位
select ROUND(-3.14);
FLOOR:向下取整
select FLOOR(3.14);
LOG:計(jì)算以 2 為底的自然對(duì)數(shù)
select LOG(4);
LOG10:計(jì)算以 10 為底的自然對(duì)數(shù)
select LOG10(100);
SQRT:求一個(gè)非負(fù)實(shí)數(shù)的平方根
select SQRT(9);
EXP:此函數(shù)返回 e(自然對(duì)數(shù)的底) 的 X 次方的值
select EXP(3);
EXPM1:返回 e x -1
select EXPM1(3);
三角函數(shù):
DEGREES:返回 X 從弧度轉(zhuǎn)換為度值
select DEGREES(x);
RADIANS:返回 X 從度轉(zhuǎn)換成弧度的值
select RADIANS(x);
SIN:返回 X 的正弦
select SIN(x);
COS:返回 X,X 值是以弧度給出的余弦值
select COS(角度);
TAN:返回參數(shù) X,表示以弧度的切線值
select TAN(角度);
ASIN:返回 X 的反正弦,X 的值必須在 - 1 至 1 范圍內(nèi),返回 NULL
select ASIN(x);
ACOS:返回 X 的反正弦,X 值必須 - 1 到 1 之間范圍否則將返回 NULL
select ACOS(x);
ATAN:返回 X 的反正切
select ATAN(x);
SINH:返回 X 的雙曲正弦值
select SINH(x);
COSH:返回 X 的雙曲余弦值
select COSH(x);
* 日期和時(shí)間處理相關(guān)方法:
YEAR:
SELECT YEAR(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS year;
MONTH_OF_YEAR() or MONTH():
SELECT MONTH(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS month;
WEEK_OF_YEAR() or WEEK():
SELECT WEEK(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS week;
DAY_OF_YEAR() or DOY(),效果等同于 EXTRACT(
SELECT DOY(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS day;
DAY_OF_MONTH(), DOM(), or DAY():
SELECT DAY(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS day;
DAY_OF_WEEK() or DOW():
SELECT DOW(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS day;
HOUR_OF_DAY() or HOUR():
SELECT HOUR(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS hour;
MINUTE_OF_DAY():
SELECT MINUTE_OF_DAY(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS minute;
MINUTE_OF_HOUR() or MINUTE():
SELECT MINUTE(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS minute;
SECOND_OF_MINUTE() or SECOND():
SELECT SECOND(CAST( 2018-10-23T16:59:27Z AS TIMESTAMP)) AS second;
如上就是 6.4 Elasticsearch SQL 支持的主要用法了,如果在優(yōu)化 SQL 語句之后還不滿足查詢需求,可以拿 SQL 和 DSL 混用,ES 會(huì)先根據(jù) SQL 進(jìn)行查詢,然后根據(jù) DSL 語句對(duì) SQL 的執(zhí)行結(jié)果進(jìn)行二次查詢,下面是個(gè)小例子:
POST /_xpack/sql?format=txt
{
query : SELECT * FROM library ORDER BY page_count DESC ,
filter : {
range : {
page_count : {
gte : 100,
lte : 200
}
}
},
fetch_size : 5
}
這個(gè)查詢就會(huì)先根據(jù)“query”后面的 SQL 進(jìn)行查詢,然后用執(zhí)行“filter”和“fetch_size”DSL 語法對(duì)查詢結(jié)果進(jìn)行過濾,進(jìn)而返回最終結(jié)果。
關(guān)于 Elasticsearch SQL 的用法是什么問題的解答就分享到這里了,希望以上內(nèi)容可以對(duì)大家有一定的幫助,如果你還有很多疑惑沒有解開,可以關(guān)注丸趣 TV 行業(yè)資訊頻道了解更多相關(guān)知識(shí)。