AWS EMR Master node is missing avro files

53 Views Asked by At

I am following the AWS workshop to create AWS EMR Cluster and play with HUDI - https://catalog.us-east-1.prod.workshops.aws/workshops/c86bd131-f6bf-4e8f-b798-58fd450d3c44/en-US

  1. I created only the Master node
  2. I am able to copy the /usr/lib/hudi/hudi-spark-bundle.jar and /usr/lib/spark/jars/httpclient-4.5.9.jar
  3. However, I am getting errors while trying to complete the following step ( hdfs dfs -copyFromLocal /usr/lib/spark/external/lib/spark-avro.jar hdfs:///user/hadoop/) - https://catalog.us-east-1.prod.workshops.aws/workshops/c86bd131-f6bf-4e8f-b798-58fd450d3c44/en-US/hudi-workshop/prereqs#step-3:-connect-to-the-master-node-of-emr-cluster-using-ssh
  4. I would like to mention that I don't see the directory at all on the master node.

Really appreciate if any one could help !

Thanks

0

There are 0 best solutions below