QBoard » Big Data » Big Data - Hadoop Eco-System » Hadoop “Unable to load native-hadoop library for your platform” warning
User Dashboard

Hadoop “Unable to load native-hadoop library for your platform” warning

Back To Topics
Tags : hadoop java Linux hadoop2 java.library.path
  • 3
    I'm currently configuring hadoop on a server running CentOs. When I run start-dfs.sh or stop-dfs.sh, I get the following error:

    WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

    I'm running Hadoop 2.2.0.

    Doing a search online brought up this link: http://balanceandbreath.blogspot.ca/2013/01/utilnativecodeloader-unable-to-load.html

    However    , the contents of /native/ directory on hadoop 2.x appear to be different so I am not sure what to do.

    I've also added these two environment variables in hadoop-env.sh:

    export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=/usr/local/hadoop/lib/"

    export HADOOP_COMMON_LIB_NATIVE_DIR="/usr/local/hadoop/lib/native/"

    Any ideas?
      August 6, 2020 4:17 PM IST
    0
  • 129 8
    I assume you're running Hadoop on 64bit CentOS. The reason you saw that warning is the native Hadoop library $HADOOP_HOME/lib/native/libhadoop.so.1.0.0 was actually compiled on 32 bit.

    Anyway, it's just a warning, and won't impact Hadoop's functionalities.

    Here is the way if you do want to eliminate this warning, download the source code of Hadoop and recompile libhadoop.so.1.0.0 on 64bit system, then replace the 32bit one.

    Steps on how to recompile source code are included here for Ubuntu:

    http://www.ercoppa.org/Linux-Compile-Hadoop-220-fix-Unable-to-load-native-hadoop-library.htm
    Good     luck.
      August 6, 2020 4:20 PM IST
    0
  • 53

    Just append word native to your HADOOP_OPTS like this:

    export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native"
    This post was edited by Nitara Bobal at October 15, 2020 5:36 PM IST
      October 15, 2020 5:34 PM IST
    0
  • 90 6

    I had the same issue. It's solved by adding following lines in .bashrc:

    export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
      October 15, 2020 5:35 PM IST
    0
  • 461

    This line right here:

    export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native:$JAVA_LIBRARY_PATH

     

    From KunBetter's answer is where the money is

     
      November 12, 2021 1:44 PM IST
    0
  • 84

    There may be two possibilities:

    1. You have installed the wrong Java JDK8 package. Please ensure to download the 64-bit JDK8 and remove your current 32-bit JDK8.
    2. It's a warning due to the Hadoop libraries being compiled for 32bits. You are probably running on a 64bit OS.

    It's safe to ignore this warning.

      December 16, 2020 12:53 PM IST
    0