zoukankan      html  css  js  c++  java
  • troubleshooting-When importing query results in parallel, you must specify --split-by.

    原因分析

    -m 4 导数命令中map task number=4,当-m 设置的值大于1时,split-by必须设置字段(需要是 int 类型的字段),如果不是 int类型的字段,则需要加上参数
    -Dorg.apache.sqoop.splitter.allow_text_splitter=true
    例子:
    sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true
    --connect ${conn_str}
    --username ${db_username}
    --password ${db_password}

    解决办法

    并行导入数据需要加上 --split-by columnName 属性。

    异常日志

    18/09/17 14:23:58 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-cdh6.0.0
    18/09/17 14:23:58 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
    When importing query results in parallel, you must specify --split-by.
    Try --help for usage instructions.
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Table not found user_base_delta

  • 相关阅读:
    Dynamic Shortest Path CodeForces
    New task CodeForces
    New Roads CodeForces
    Music in Car CodeForces
    互联网思维体系--史上最全的互联网思维精髓总结
    OpenGL Shader Key Points (3)
    Netfilter的使用和实现
    数据包接收系列 — IP协议处理流程(二)
    数据包接收系列 — IP协议处理流程(一)
    数据包接收系列 — NAPI的原理和实现
  • 原文地址:https://www.cnblogs.com/chwilliam85/p/9693268.html
Copyright © 2011-2022 走看看