How to convert List to a JavaRDD in Spark?
December 23, 2020How to purge or delete messages in a Kafka topic?
December 28, 2020If you are trying to access HDFS, you might have come across the below three types of commands. In this post we will see the difference between the 3 of them.
- hadoop fs {args}
- hadoop dfs {args}
- hdfs dfs {args}
hadoop fs {args}
In the above command, fs refers to a generic file system and can point to your local file system, HDFS and other file systems like S3, SFTP etc.
hadoop dfs {args}
hadoop dfs commands are used when you are working with HDFS and not with other file systems. Note that this command is deprecated and when you use this command it is delegated to hdfs dfs command.
hdfs dfs {args}
As the command itself suggests, the command is specific to HDFS and use this when you are working with hdfs.
In summary, hadoop fs is versatile and can work with many file systems including HDFS. hdfs dfs works with HDFS and use it if your only need is to access HDFS.
