Biginsights 3.0 and Yarn

Question

I installed Biginsights 3.0, but there isn't Yarn. How can I install Yarn or may be I can turn on it? I searched any information on the IBM site, but there isn't any information about it.

Answer 1

IBM does not ship YARN - as they promote their own "adaptive map/reduce" (or something like that). I didn't use BigInsight 3.0 but from my experience with previous versions, if you want to get anything running there you'd need to compile it yourself using IBM's hadoop jars and java otherwise you'd get security exceptions. So while it is probably technically possible, I wouldn't recommend that

Answer 2

If you are still looking for the answer, IBM Biginsights supports YARN from version 4.0. As Arnon suggested, IBM BIginsights 3.0 comes with Adaptive MapReduce.

Please go through the below link for more information,
http://www-01.ibm.com/support/knowledgecenter/SSPT3X_4.0.0/com.ibm.swg.im.infosphere.biginsights.product.doc/doc/yarn.html

Answer 3

One can install YARN. Install Hadoop under specially created account, specify fs.defaultFS as in biginsights, then disable profile biginsights-env.sh in order to avoid all environment variables specific to biginsights.

$chmod 770 /opt/ibm/biginsights/biginsights-env.sh 

Login to your specially created account and check that your HADOOP_HOME does not point to biginsights and run start-yarn.sh

I was able to run YARN and hive with tez on biginsights this way.

One do not even need to stop mapreduce 1 framework.