sqoop中query \$CONDITIONS HUE執行失敗怎么辦

188次閱讀

共計 2728 個字符，預計需要花費 7 分鐘才能閱讀完成。

這篇文章給大家介紹 sqoop 中 query \$CONDITIONS HUE 執行失敗怎么辦，內容非常詳細，感興趣的小伙伴們可以參考借鑒，希望對大家能有所幫助。

1. HUE sqoop 命令框中直接輸入：

sqoop-import –connect jdbc:mysql://ip 地址: 端口 / 數據庫 –username 用戶名 –password 密碼 –query select * from 表名 where 1=1 and \$CONDITIONS –split-by id –hive-table hive 表 –target-dir /apps/hive/warehouse/mydb.db/product/dt=${date} –hive-partition-key dt –hive-partition-value ${date} –hive-import –driver com.mysql.jdbc.Driver

運行會提示報錯，是因為 SQL 語句被解析成 sqoop 參數

2784 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Error parsing arguments for import:
2016-09-11 15:21:46,485 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(304)) – Error parsing arguments for import:
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: *
2016-09-11 15:21:46,486 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(307)) – Unrecognized argument: *
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: from
2016-09-11 15:21:46,486 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(307)) – Unrecognized argument: from
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: product
2016-09-11 15:21:46,486 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(307)) – Unrecognized argument: p_product
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: where
2016-09-11 15:21:46,486 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(307)) – Unrecognized argument: where
2785 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool – Unrecognized argument: $CONDITIONS

2. 解決方案：最好采用參數化的方式

workflow-app name= test xmlns= uri:oozie:workflow:0.5
start to= sqoop-fb61 /
kill name= Kill
message Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}] /message
/kill
action name= sqoop-fb61
sqoop xmlns= uri:oozie:sqoop-action:0.2
job-tracker ${jobTracker} /job-tracker
name-node ${nameNode} /name-node
arg import /arg
arg –connect /arg
arg quot;jdbc:mysql://IP 地址: 端口 / 數據庫 quot; /arg
arg –username /arg
arg quot; 用戶名 quot; /arg
arg –password /arg
arg quot; 密碼 quot; /arg
arg –query /arg
arg select * from product where $CONDITIONS /arg ## 注意：這個地方沒有反斜杠（\）
arg –split-by /arg
arg quot;id quot; /arg
arg –hive-table /arg
arg quot;HIVE 表名 quot; /arg
arg –target-dir /arg
arg quot;/apps/hive/warehouse/mydb.db/product/dt=${date} quot; /arg
arg –hive-partition-key /arg
arg quot;dt quot; /arg
arg –hive-partition-value /arg
arg quot;${date} quot; /arg
arg –hive-import /arg
arg –driver /arg
arg quot;com.mysql.jdbc.Driver quot; /arg
/sqoop
ok to= End /
error to= Kill /
/action
end name= End /
/workflow-app