QBoard » Big Data » Big Data - Hadoop Eco-System » Hadoop: start-dfs.sh permission denied

Hadoop: start-dfs.sh permission denied

  • I am installing Hadoop on my laptop. SSH works fine, but I cannot start hadoop.

    munichong@GrindPad:~$ ssh localhost
    Welcome to Ubuntu 12.10 (GNU/Linux 3.5.0-25-generic x86_64)
    
     * Documentation:  https://help.ubuntu.com/
    
    0 packages can be updated.
    0 updates are security updates.
    
    Last login: Mon Mar  4 00:01:36 2013 from localhost
    
    munichong@GrindPad:~$ /usr/sbin/start-dfs.sh
    chown: changing ownership of `/var/log/hadoop/root': Operation not permitted
    starting namenode, logging to /var/log/hadoop/root/hadoop-munichong-namenode-GrindPad.out
    /usr/sbin/hadoop-daemon.sh: line 136: /var/run/hadoop/hadoop-munichong-namenode.pid: Permission denied
    usr/sbin/hadoop-daemon.sh: line 135: /var/log/hadoop/root/hadoop-munichong-namenode-GrindPad.out: Permission denied
    head: cannot open `/var/log/hadoop/root/hadoop-munichong-namenode-GrindPad.out' for reading: No such file or directory
    localhost: chown: changing ownership of `/var/log/hadoop/root': Operation not permitted
    localhost: starting datanode, logging to /var/log/hadoop/root/hadoop-munichong-datanode-GrindPad.out
    localhost: /usr/sbin/hadoop-daemon.sh: line 135: /var/log/hadoop/root/hadoop-munichong-datanode-GrindPad.out: Permission denied
    localhost: /usr/sbin/hadoop-daemon.sh: line 136: /var/run/hadoop/hadoop-munichong-datanode.pid: Permission denied
    localhost: head: cannot open `/var/log/hadoop/root/hadoop-munichong-datanode-GrindPad.out' for reading: No such file or directory
    localhost: chown: changing ownership of `/var/log/hadoop/root': Operation not permitted
    localhost: starting secondarynamenode, logging to /var/log/hadoop/root/hadoop-munichong-secondarynamenode-GrindPad.out
    localhost: /usr/sbin/hadoop-daemon.sh: line 136: /var/run/hadoop/hadoop-munichong-secondarynamenode.pid: Permission denied
    localhost: /usr/sbin/hadoop-daemon.sh: line 135: /var/log/hadoop/root/hadoop-munichong-secondarynamenode-GrindPad.out: Permission denied
    localhost: head: cannot open `/var/log/hadoop/root/hadoop-munichong-secondarynamenode-GrindPad.out' for reading: No such file or directory
    
    munichong@GrindPad:~$ sudo /usr/sbin/start-dfs.sh
    [sudo] password for munichong: 
    starting namenode, logging to /var/log/hadoop/root/hadoop-root-namenode-GrindPad.out
    localhost: Permission denied (publickey,password).
    localhost: Permission denied (publickey,password).
    

     

    I used "sudo". But the permission is still denied.

    Is there anyone can help me?

    Thanks in advance!

      January 1, 2022 2:17 PM IST
    0
  • I was stuck at the same issue for last couple of hours but finally solved it. I had the hadoop installation extracted by same user as one I am using to run hadoop. So user privilege is not issue.
    My cofiguration is like this: Ubuntu linux machine on Google Cloud.

    Hadoop installation /home/ Hadoop data directory /var/lib/hadoop and the directory access bits are 777 so anybody can access. I did ssh into the remote machine made changes to the config files and executed start-dfs.sh, then it gave me "Permission denied (Public key)" So here is the solution: In the same ssh terminal:

    1. ssh-keygen

    2.It will ask for folder location where it will copy the keys, I entered /home/hadoop/.ssh/id_rsa

    3.it will ask for pass phrase, keep it empty for simplicity.

    4.cat /home/hadoop/.ssh/id_rsa.pub >> .ssh/authorized_keys (To copy the newly generated public key to auth file in your users home/.ssh directory)

    1. ssh localhost

    2. start-dfs.sh (Now it should work!)

      January 10, 2022 12:21 PM IST
    0
  • I think the problem is that root and user's ssh connection. here is my copy https://askubuntu.com/questions/497895/permission-denied-for-rootlocalhost-for-ssh-connection Solved my case.

    By default, the SSH server denies password-based login for root. In /etc/ssh/sshd_config,

    change: PermitRootLogin without-password to PermitRootLogin yes

    And restart SSH: sudo service ssh restart

    Or, you can use SSH keys. If you don't have one, create one using ssh-keygen (stick to the default for the key, and skip the password if you feel like it). Then do sudo -s (or whatever your preferred method of becoming root is), and add an SSH key to /root/.ssh/authorized_keys:

    cat /home/user/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys

      January 17, 2022 1:52 PM IST
    0