When you compile and launch the following Java code snippet on Linux (I used the latest stable Ubuntu version):

package eu.plumbr.demo;

public class OOM {

public static void main(String[] args){
	java.util.List<int[]> l = new java.util.ArrayList();
	for (int i = 10000; i < 100000; i++) {
			try {
				l.add(new int[100_000_000]);
			} catch (Throwable t) {
				t.printStackTrace();
			}
		}
	}
}

then you will face an error similar to the following in the system logs ( /var/log/kern.log in our example):

Jun  4 07:41:59 plumbr kernel: [70667120.897649] Out of memory: Kill process 29957 (java) score 366 or sacrifice child
Jun  4 07:41:59 plumbr kernel: [70667120.897701] Killed process 29957 (java) total-vm:2532680kB, anon-rss:1416508kB, file-rss:0kB

Note that you might need to tweak the swapfile and heap sizes, in our testcase we used a 2g heap specified by -Xmx2g and had the following swap configuration:

swapoff -a 
dd if=/dev/zero of=swapfile bs=1024 count=655360
mkswap swapfile
swapon swapfile