What is the difference between get_json_object and json_tuple functions in Hive?
February 2, 2022How to view the contents of a GZiped file in HDFS?
February 16, 2022Simple problem with a simple solution.
Solution
Use hdfs dfs -count to get the count of files and directories inside the directory.
[hirw@wk1 ~]$ hdfs dfs -count /user/hirw/course_data 1 0 0 /user/hirw/course_data
You will see 4 columns in the output – DIR_COUNT, FILE_COUNT, CONTENT_SIZE, PATHNAME
In this directory, we can see that the FILE_COUNT is 0 indicating the number of files is 0 in the directory.
Big Data In Real World
We are a group of Big Data engineers who are passionate about Big Data and related Big Data technologies. We have designed, developed, deployed and maintained Big Data applications ranging from batch to real time streaming big data platforms. We have seen a wide range of real world big data problems, implemented some innovative and complex (or simple, depending on how you look at it) solutions.
