starcluster / SGE launch EC2 (node / RAM management issue)

Question

I launched several EC2 instances using starcluster and ran the same pipeline on each node.

1. I used qsub to submit job without specifying the nodes. Why starcluster /SGE assign the job to node that already be occupied instead of to empty node?
2. Can these nodes borrow threads and RAM from each other?

Can someone share some idea on these?

Answer 1

I'm not familiar with starcluster, but I assume it has similar node allocation policies to the Maui scheduler:

http://docs.adaptivecomputing.com/maui/5.2nodeallocation.php

Specifically, see MINRESOURCE policy:

http://docs.adaptivecomputing.com/maui/5.2nodeallocation.php#MINRESOURCE

5.2.2.5 MINRESOURCE

This algorithm priorities nodes according to the configured resources on each node. Those nodes with the fewest configured resources which still meet the job's resource constraints are selected.

This is the default for Maui, at least on the install I have done, which makes it submit jobs in the manner you describe.

I would assume that starcluster is behaving similarly.