Debugging a dying Eclipse instance
The story starts with our decision to go to Devoxx. With a booth. By a caravan. Driving 2,200 kilometers. But this is not going to be a travel story, do not worry.
During the travel we taught our marketing guys to be prepared for doing demos in our booth. After all, what’s the benefit of dragging all the droids to Antwerp if they are not even good for a demo or two.
As we had ~30 hours of driving time ahead of us, we thought we have enough time to get the droids up to speed.
First stop – verifying the required infrastructure. One of the droids had managed to install a JDK and Eclipse all by himself. So far so good. But when firing up the Eclipse and adding our demo project resulted in CPU usage jumping to 100% and Eclipse becoming completely unresponsive.
So what is going on here? After all, we are hardcore Java troubleshooters, so this should be a task suitable just for us. No problems, lets just attach jconsole to Eclipse and find out what is going on. First setback – jconsole fails to attach to Eclipse and displays the following error message instead:
Well, apparently the process we are trying to connect to is so exhausted it cannot even create the JMX connection anymore. Next tool in our belt to the rescue is jstat.
Turning to the jstat –gcutil PID we see that something bad is indeed going on – the permanent generation is filled up to it’s maximum capacity and indeed it might be tough for Eclipse to load additional classes.
my-macbook:demo$ jstat -gcutil 21742 1s S0 S1 E O P YGC YGCT FGC FGCT GCT 0.00 0.00 3.24 59.20 100.00 45 0.346 499 169.383 169.729 0.00 0.00 3.24 59.20 100.00 45 0.346 499 169.383 169.729 0.00 0.00 3.24 59.20 100.00 45 0.346 499 169.383 169.729 0.00 0.00 3.24 59.20 100.00 45 0.346 499 169.383 169.729 0.00 0.00 3.24 59.20 100.00 45 0.346 499 169.383 169.729 0.00 0.00 3.27 59.20 100.00 45 0.346 499 169.383 169.729 0.00 0.00 3.27 59.20 100.00 45 0.346 499 169.383 169.729 0.00 0.00 3.27 59.20 100.00 45 0.346 499 169.383 169.729 0.00 0.00 3.27 59.20 100.00 45 0.346 499 169.383 169.729
Next step – how much memory has the unlucky droid given to his Eclipse. Our friend jstat comes to help us out again here:
my-macbook:demo $ jstat -gccapacity 21742 NGCMN NGCMX ... PGCMN PGCMX PGC PC YGC FGC 16384.0 16384.0 ... 12288.0 65536.0 65536.0 65536.0 39 122
For some reason, the droid is running Eclipse with just 64MB available for permanent generation (PGCMX). And all of this available space is already being used (PGC). Now it becomes clear why the CPU is 100% stuck – JVM makes its last desperate efforts to free up some memory.
Now when searched for the configuration which should be located in eclipse.ini file we are left with bedazzling looks in our faces. Apparently its not where it is supposed to be:
my-macbook:juno$ pwd /Applications/eclipse/juno my-macbook:juno$ find . -name eclipse.ini my-macbook:juno$
On whatever reason he had managed to download an Eclipse bundle without the eclipse.ini file in it. I guess it must require some skill. Or Eclipse has been distributed without the configuration. But at least we have found our culprit – without the settings.ini in place Eclipse fails back to JVM’s defaults for setting up heap and permgen sizes. Which are nowhere near to what this IDE needs for a normal run.
Fast forward three minutes and we have a working eclipse.ini copy in place and the IDE resumes it’s normal behaviour. So we can continue educating our droids.
Lessons learned. In order of importance:
- Do not let marketroids near technical infrastructure. They mess up even with Eclipse installation. Which requires the exclusive knowledge about downloading and unzipping a ZIP archive.
- Know thy tools. A bunch of them – when the first one fails on you, you need to have an alternative under your belt.