What does the method getLocalCacheArchives do?

Question

I am not sure what this method does. It is a part of the class DistributedCache.

I am actually using it in the code for Hadoop Blast. I have a file RunnerMap.java, which has a method setup(Context context) which gets the path for localDB and localBlastProgram as follows:

Configuration conf = context.getConfiguration();
Path[] local = DistributedCache.getLocalCacheArchives(conf);
localDB = local[0].toUri().getPath() + File.separator + conf.get(DataAnalysis.DB_ARCHIVE) + File.separator + conf.get(DataAnalysis.DB_NAME);
localBlastProgram = local[0].toUri().getPath();

I am relatively new to Java, so I cannot make much sense out of these four lines.

I also cannot find the description of getLocalCacheArchives on MapReduce documentations.

Answer 1

The DistributedCache is used, so that the nodes of the cluster can share some files/archives. Archives are typically zip, tar and tgz/tar.gz files.

So, at the main method, you should set those files or archives that you want your nodes to share (read-only access) and then, at the setup method, you can get those files, as you do in the posted lines, using the getLocalCacheArchives() method.

Perhaps, you will find some useful info and examples on this old documentation, as DistributedCache is now deprecated.