How much memory your Namenode need?

This is going to be a very short post. When you are building a cluster from scratch, Hadoop developers and admins often debate the amount of memory that is needed to be allocated to the Namenode.

Here is a rule of thumb – allocate 1,000 MB to the Namenode per million blocks stored in HDFS.

This means, if the block size of your cluster is 128 MB then million blocks would equate to

128 MB * 1,000,000 blocks = 128, 000, 000 MB = 128 TB

So 1,000 MB allocated to Namenode is what is required to manage a cluster with 128 TB of raw disk space. Please note the 1,000 MB is to be used just by the Namenode process for holding the block metadata in memory . The node itself will need additional memory to cater OS and other services running on the node.

For more details refer this Hadoop JIRA.