Changing ulimit on ubuntu 12.04 Never works

Question

I am working with hadoop and need to change number of open files ulimit -n. I have seen similar questions on stackoverflow and elsewhere and have tried everything in those answers, but it still does not work. I am working with ubuntu 12.04 LTS. Here is what i have done:

change limits in /etc/security/limits.conf, i have put in settings for * and root. I have also changed limits to some number like 10000 and unlimited.

* soft nofile 1513687
* hard nofile 1513687
root soft nofile 1513687
root hard nofile 1513687

I have also tried above setting with - instead of soft and hard. After these changes, i have made changes to /etc/pam.d/ files such as:

common-session 
common-session-noninterative
login
cron
sshd
su
sudo

i have added session required pam_limits.so to the beginning of each file. I made a restart of the box in question and the settings did not take effect.

I have also found that there were files inside /etc/security/limits.d/ directory for the users hbase mapred and hdfs. I have tried changing limits in these individual files as well to no avail.

I have tried putting ulimit -S -n unlimited inside /etc/profile as well. It did not work.

Finally, i have tried putting limit nofile unlimited unlimited inside /etc/init.d/hadoop* files as first line. Did not work.

One interesting thing though is, i do not have hbase installed on the box but i do have a hbase.conf file inside /etc/security/limits.d/ directory. The settings in this file are reflected with ulimit -n. But settings from hdfs.conf and mapred.conf are not reflected. Which suggests that something is overwriting settings for hdfs and mapred.

I guess i have tried everything people suggested on several forums, is there anything else that i may have missed or done incorrectly?

I am using CDH 4.4.0 as my hadoop distribution.

Answer 1

How are you checking ulimit?

I was experiencing a similar issue, where I would run sudo ulimit -n, and still see 1024. This is because ulimit is a Bash built-in. In order to see the changes reflected in /etc/security/limits.conf, I had to run ulimit as the actual user.