Macos – OS X Yosemite – Too many files open

homebrewmacososx-yosemite

I recently upgraded from Mavericks to Yosemite….headaches ensued

I'm pretty sure my problem lies w/ the number of files I can openbut I'm not sure how to solve it. I'm also pretty sure I've read every article/tip about creating the /etc/sysctl.conf file and /etc/launchd.conf file

For clarity's sake this is what both of them currently look like n.b. (I'm not even sure if these are the right commands to put in them anymore – I've tried just about everything and every combination. ie: higher values, lower values, removing commands, adding commands)

/etc/launchd.conf

limit maxfiles 16384 32768
limit maxproc 1000 2000

/etc/sysctl.conf

# Turn up maxproc
kern.maxproc=2048
# Turn up the maxproc per user
kern.maxprocperuid=512

I've also tried increasing my ulimit values – both globally and locally for my current session…no bueno

ulimit -a

-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-v: address space (kbytes)          unlimited
-l: locked-in-memory size (kbytes)  unlimited
-u: processes                       709
-n: file descriptors                4096

Ok – this is where my headache initially started…My DB (percona-server 5.6.21-69.0 installed via homebrew) started choking and dying, and in the mysql-error.log it was filled w/ the error too many files open.

2014-10-21 15:19:50 5190 [ERROR] /usr/local/Cellar/percona-server/5.6.21-69.0/bin/mysqld: Can't open file: './ie/tokenorder_products.frm' (errno: 24 - Too many open files)
2014-10-21 15:19:50 5190 [ERROR] /usr/local/Cellar/percona-server/5.6.21-69.0/bin/mysqld: Can't open file: './ie/tokenorders.frm' (errno: 24 - Too many open files)
2014-10-21 15:19:50 5190 [ERROR] /usr/local/Cellar/percona-server/5.6.21-69.0/bin/mysqld: Can't open file: './ie/tsdevices.frm' (errno: 24 - Too many open files)
2014-10-21 15:19:50 5190 [ERROR] /usr/local/Cellar/percona-server/5.6.21-69.0/bin/mysqld: Can't open file: './ie/tsracks.frm' (errno: 24 - Too many open files)
2014-10-21 15:19:50 5190 [ERROR] /usr/local/Cellar/percona-server/5.6.21-69.0/bin/mysqld: Can't open file: './ie/v_classunity_classlist.frm' (errno: 24 - Too many open files)
2014-10-21 15:20:48 5190 [ERROR] Error in accept: Too many open files

This is where I initially started to try and solve that problem by "up-ing" my ulimit, maxfiles, maxproc,etc…

Eventually – frustrated, I moved on and would come back to that issue later. So then I was trying to sudo gem install nokogiri and over and over it would fail and spit out this same error (a lot of repeats about the builder.rb failing to build the gem native extension – followed by a bunch of repeated stacktraces Logs Gist

nokogiri Install Errors
nokogiri install errors

I've tried/googled a bunch of different approaches to solve this issue (ie: additional flags, etc..). Whats amazing – and when I started thinking this issue was tied to the # of files/processes open is when I checked top while the gem was installing….I was pretty surprised to see what I found

top during gem install nokogiri
sudo gem install nokogiri

It looks like my process keeps forking, which then made sense for this ONE line in my previous picture (see "nokogiri install errors picture")

sh: fork: Resource temporarily unavailable

So I'm kinda out of ideas, but I'm not really sure how to debug the # of files anymore?

UPDATE

Well, I managed to get nokogiri installed. Unfortunately, I'm not exactly sure what fixed it b/c I've tried so many things. But I think it had to deal w/ reinstalling ruby. However, I'm still getting the same issue w/ my DB complaining about too many files open when I use any sort of database that is not trivially small.

Best Answer

Modifying the /etc/launchd.conf per a lot of google queries and SO suggestions didn't seem to work for me in Yosemite (10.10). What did end up working, after numerous change/reboot/test combinations, was modifying (or creating if it doesn't exist) the /etc/sysctl.conf file.

This is what I had to put in to make it work

kern.maxfiles=65536
kern.maxfilesperproc=65536

I'm not sure if kern.maxfiles needs to be in there, but when I had it in there by itself I still had the same issue, when I added the kern.maxfilesperproc everything started working.