Hadoop not listening to intended port (Setting up a single node cluster)

25 Views Asked by At

Firstly, I'm extremely new to Hadoop, and I'm trying this tutorial.

This is my mapred-site.xml file:

<configuration>
<property>
  <name>mapred.job.tracker</name>
  <value>localhost:54311</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If "local", then jobs are run in-process as a single map
  and reduce task.
  </description>
</property>
</configuration>

And this is my core-site.xml file:

<configuration>
 <property>
  <name>hadoop.tmp.dir</name>
  <value>usr/local/hadoop-datastore/hadoop/tmp/dir/hadoop-xin</value>
  <description>A base for other temporary directories.</description>
 </property>

 <property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:54310</value>
  <description>The name of the default file system.  A URI whose
  scheme and authority determine the FileSystem implementation.  The
  uri's scheme determines the config property (fs.SCHEME.impl) naming
  the FileSystem implementation class.  The uri's authority is used to
  determine the host, port, etc. for a filesystem.</description>
 </property>
</configuration>

The steps for it is quite outdated(locations changed with different versions, deprecated instructions...), but I've managed to setup the whole thing slowly- until the step where I run:

sudo netstat -plten | grep java

From the steps of the tutorial, Hadoop should be listening to port 54310 and 54311 at this step (according to the mapred-site and core-site xml files), as below:

hadoop@ubuntu:~$ sudo netstat -plten | grep java
tcp   0  0 0.0.0.0:50070   0.0.0.0:*  LISTEN  1001  9236  2471/java
tcp   0  0 0.0.0.0:50010   0.0.0.0:*  LISTEN  1001  9998  2628/java
tcp   0  0 0.0.0.0:48159   0.0.0.0:*  LISTEN  1001  8496  2628/java
tcp   0  0 0.0.0.0:53121   0.0.0.0:*  LISTEN  1001  9228  2857/java
tcp   0  0 127.0.0.1:54310 0.0.0.0:*  LISTEN  1001  8143  2471/java #this
tcp   0  0 127.0.0.1:54311 0.0.0.0:*  LISTEN  1001  9230  2857/java #this
tcp   0  0 0.0.0.0:59305   0.0.0.0:*  LISTEN  1001  8141  2471/java
tcp   0  0 0.0.0.0:50060   0.0.0.0:*  LISTEN  1001  9857  3005/java
tcp   0  0 0.0.0.0:49900   0.0.0.0:*  LISTEN  1001  9037  2785/java
tcp   0  0 0.0.0.0:50030   0.0.0.0:*  LISTEN  1001  9773  2857/java
hadoop@ubuntu:~$

But this is what I get-it's listening to 42157 and 54310 instead of 54310 and 54311.

xin@ubuntu:/usr/local/hadoop/hadoop-3.3.6/etc/hadoop$ sudo netstat -plten | grep java
tcp   0      0 0.0.0.0:9864            0.0.0.0:*  LISTEN      0          214237     17223/java          
tcp   0      0 0.0.0.0:9866            0.0.0.0:*  LISTEN      0          212927     17223/java          
tcp   0      0 0.0.0.0:9867            0.0.0.0:*  LISTEN      0          215069     17223/java          
tcp   0      0 0.0.0.0:9868            0.0.0.0:*  LISTEN      0          215164     17463/java          
tcp   0      0 0.0.0.0:9870            0.0.0.0:*  LISTEN      0          212262     17067/java          
tcp   0      0 0.0.0.0:8088            0.0.0.0:*  LISTEN      0          213645     17708/java          
tcp   0      0 0.0.0.0:8030            0.0.0.0:*  LISTEN      0          222243     17708/java          
tcp   0      0 0.0.0.0:8031            0.0.0.0:*  LISTEN      0          222237     17708/java          
tcp   0      0 0.0.0.0:8032            0.0.0.0:*  LISTEN      0          222249     17708/java          
tcp   0      0 0.0.0.0:8033            0.0.0.0:*  LISTEN      0          213680     17708/java          
tcp   0      0 0.0.0.0:8040            0.0.0.0:*  LISTEN      0          220244     17852/java          
tcp   0      0 0.0.0.0:8042            0.0.0.0:*  LISTEN      0          219131     17852/java          
tcp   0      0 127.0.0.1:42157         0.0.0.0:*  LISTEN      0          212940     17223/java #this          
tcp   0      0 127.0.0.1:54310         0.0.0.0:*  LISTEN      0          210773     17067/java #this         
tcp   0      0 0.0.0.0:45277           0.0.0.0:*  LISTEN      0          220236     17852/java      

I've searched really long but to no avail-there's no similar questions asked. Thanks in advance for any assistance!

Edit:tried netstat -anlp | grep 54311 nothing happens

    xin@ubuntu:/usr/local/hadoop/hadoop-3.3.6/etc/hadoop$ sudo netstat -anlp | grep 54311 
xin@ubuntu:/usr/local/hadoop/hadoop-3.3.6/etc/hadoop$ sudo netstat -anlp | grep 54310 
tcp 0 0 127.0.0.1:54310 0.0.0.0:* LISTEN 17067/java 
tcp 0 0 127.0.0.1:58024 127.0.0.1:54310 TIME_WAIT - 
tcp 0 0 127.0.0.1:39792 127.0.0.1:54310 ESTABLISHED 17223/java 
tcp 0 0 127.0.0.1:54310 127.0.0.1:39792 ESTABLISHED 17067/java 
xin@ubuntu:/usr/local/hadoop/hadoop-3.3.6/etc/hadoop$ sudo netstat -anlp | grep 42157 
tcp 0 0 127.0.0.1:42157 0.0.0.0:*LISTEN 17223/java
0

There are 0 best solutions below