QBoard » Big Data » Big Data - Hadoop Eco-System » find file in hadoop filesystem

find file in hadoop filesystem

  • Is there a way to locate a specific file in hadoop?

    I know, that I can use this: hadoop fs -find /some_directory

    But, is there a command like this: hadoop locate some_file_name?

      June 12, 2019 11:41 AM IST
    0
  • hadoop fs -ls defaults to /user/userName, so you can leave the path blank to view the contents of your home directory.

    hadoop fs -ls
    

     

      September 6, 2021 1:46 PM IST
    0
  • To find a file in the Hadoop Distributed file system:

    hdfs dfs -ls -R / | grep [search_term]
    


    In the above command,

    -ls is for listing files

    -R is for recursive(iterate through sub directories)

    / means from the root directory

    | to pipe the output of first command to the second

    grep command to extract matching strings

    [search_term] file name to be searched for in the list of all files in the hadoop file system.

    Alternatively the below command can also be used find and also apply some expressions:

    hadoop fs -find / -name test -print
    


    Finds all files that match the specified expression and applies selected actions to them. If no path is specified then defaults to the current working directory. If no expression is specified then defaults to -print.

      August 11, 2021 1:32 PM IST
    0
  • The hadoop fs -ls command allows you to view the files and directories in your HDFS filesystem, much as the ls command works on Linux / OS X / *nix. A user's home directory in HDFS is located at /user/userName. For example, my home directory is /user/akbar.
      October 26, 2021 12:52 PM IST
    0
  • If you are looking for equivalent of locate Linux command than such option does not exist in Hadoop. But if you are looking for the way of how to find specific file you can use name parameter of fs -findcommand for this:

    hadoop fs -find /some_directory -name some_file_name

    If you are looking for the actual location of hdfs file in your local file system you can use fsckcommand for this:

    hdfs fsck /some_directory/some_file_name -files -blocks -locations
      June 12, 2019 11:45 AM IST
    0