Max memory = [-Xmx] + [-XX:MaxPermSize] + number_of_threads * [-Xss]
|init||represents the initial amount of memory (in bytes) that the Java virtual machine requests from the operating system for memory management during startup. The Java virtual machine may request additional memory from the operating system and may also release memory to the system over time. The value of init may be undefined.|
|used||represents the amount of memory currently used (in bytes).|
|committed||represents the amount of memory (in bytes) that is guaranteed to be available for use by the Java virtual machine. The amount of committed memory may change over time (increase or decrease). The Java virtual machine may release memory to the system and committed could be less than init. committed will always be greater than or equal to used.|
|max||represents the maximum amount of memory (in bytes) that can be used for memory management. Its value may be undefined. The maximum amount of memory may change over time if defined. The amount of used and committed memory will always be less than or equal to max if max is defined. A memory allocation may fail if it attempts to increase the used memory such that used > committed even if used <= max would still be true (for example, when the system is low on virtual memory).|
reserved memory 是指JVM 通过mmaped PROT_NONE 申请的虚拟地址空间，在页表中已经存在了记录（entries），保证了其他进程不会被占用，会page faults, committed memory 是JVM向操做系统实际分配的内存（malloc/mmap）,mmaped PROT_READ | PROT_WRITE,仍然会page faults 但是跟 reserved 不同，完全内核处理像什么也没发生一样。 used memory 是JVM实际存储了数据（Java对象）的大小，当used~=committed的时候，heap就会grow up，-Xmx设置了上限。
- reserved 但是没有 committed pages 不算 rss.
- page out 的算committed，但是不算 rss.
- 已经 committed 的也不一定在rss内(committed > rss): malloc/mmap is lazy unless told otherwise. Pages are only backed by physical memory once they're accessed.
In thinking about virtual memory, there are two concepts that every programmer should understand: resident set size and commit charge. The second is easiest to explain: it's the total amount of memory that your program might be able to modify (ie, it excludes read-only memory-mapped files and program code). The potential commit charge for an entire system is the sum of RAM and swap space, and no program can exceed this. It doesn't matter how big your virtual address space is: if you have 2G of RAM, and 2G of swap, you can never work with more than 4G of in-memory data; there's no place to store it.
One final concept: pages in the resident set can be “dirty,” meaning that the program has changed their content. A dirty page must be written to swap space before its physical memory can be used by another page. By comparison, a clean (unmodified) page may simply be discarded; it will be reloaded from disk when needed. If you can guarantee that a page will never be modified, it doesn't count against a program's commit charge — we'll return to this topic when discussing memory-mapped files.
Below is a picture showing an example of a memory pool:
+----------------------------------------------+ +//////////////// | + +//////////////// | + +----------------------------------------------+ |--------| init |---------------| used |---------------------------| committed |----------------------------------------------|
JVM非堆的内存可能会有哪些？GC,JIT,Threads,Classes and Classloaders(PermGen),NIO(direct buffer)
But besides the memory consumed by your application, the JVM itself also needs some elbow room. The need for it derives from several different reasons:
Garbage collection. As you might recall, Java is a garbage-collected language. In order for the garbage collector to know which objects are eligible for collection, it needs to keep track of the object graphs. So this is one part of the memory lost to this internal bookkeeping. G1 is especially known for its excessive appetite for additional memory, so be aware of this.
JIT optimization. Java Virtual Machine optimizes the code during runtime. Again, to know which parts to optimize it needs to keep track of the execution of certain code parts. So again, you are going to lose memory.
Off-heap allocations. If you happen to use off-heap memory, for example while using direct or mapped ByteBuffers yourself or via some clever 3rd party API then voila – you are extending your heap to something you actually cannot control via JVM configuration.
JNI code. When you are using native code, for example in the format of Type 2database drivers, then again you are loading code in the native memory.
Metaspace. If you are an early adopter of Java 8, you are using metaspace instead of the good old permgen to store class declarations. This is unlimited and in a native part of the JVM.
But RSS is also misleading, especially on a lightly loaded machine. The operating system doesn't expend a lot of effort to reclaiming the pages used be a process. There's little benefit to be gained by doing so, and the potential for an expensive page fault if the process touches the page in the future. As a result, the RSS statistic may include lots of pages that aren't in active use.
Examining a heap dump for java.nio.DirectByteBuffer instances should provide further insight.
jconsole non-heap memory:
Non-heap memory includes a method area shared among all threads and memory required for the internal processing or optimization for the Java VM. It stores per-class structures such as a runtime constant pool, field and method data, and the code for methods and constructors. The method area is logically part of the heap but, depending on the implementation, a Java VM may not garbage collect or compact it. Like the heap memory, the method area may be of a fixed or variable size. The memory for the method area does not need to be contiguous.
Garbage Collection of Direct/Mapped Buffers
That brings up another topic: how does the non-heap memory for direct buffers and mapped files get released? After all, there's no method to explicitly close or release them. The answer is that they get garbage collected like any other object, but with one twist: if you don't have enough virtual memory space or commit charge to allocate a direct buffer, that will trigger a full collection even if there's plenty of heap memory available. Normally, this won't be an issue: you probably won't be allocating and releasing direct buffers more often than heap-resident objects. If, however, you see full GC's appearing when you don't think they should, take a look at your program's use of buffers.
In fact, the only reason that I can see for using direct buffers in a pure Java program is that they won't be moved during garbage collection. If you've read my article on reference objects, you'll remember that the garbage collector compacts the heap after disposing of dead objects. If you have large blocks of heap memory allocated as buffers, they may get moved as part of compaction, and no matter how fast your CPU, that takes time; it's not something you want to do on every full collection. Since the direct buffer lives outside of the heap, it isn't affected by collections. On the other hand, every data access is a JNI call. Only benchmarking will tell you whether this helps or hurts your particular application.
What's using my native memory?
Once you have determined you are running out of native memory, the next logical question is: What's using that memory? Answering this question is hard because, by default, Windows and Linux do not store information about which code path is allocated a particular chunk of memory.
如何监控和排查non-heap 或 native memory leak
- IBM Support Assistant: https://www-01.ibm.com/marketing/iwm/iwm/web/reg/download.do?source=isa&S_PKG=isa5&lang=en_US&cp=UTF-8&dlmethod=http
- Preprocessor level: Dmalloc
- Linker level: Ccmalloc
- Runtime-linker level: NJAMD
- Emulator-based: memcheck
- JNI leaking: Valgrind memcheck http://www.oracle.com/technetwork/java/javase/memleaks-137499.html#gbyvk
- Java core file: http://www.javacodegeeks.com/2013/02/analysing-a-java-core-dump.html
- NMT Native Memory Tracking: https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html 注意目前（7u40）NMT只能用来分析HotSpot internal memory usage，不能分析第三方的JNI.
- GCMV: https://www.ibm.com/developerworks/java/jdk/tools/gcmv/ https://www.ibm.com/developerworks/community/blogs/troubleshootingjava/entry/gcmv_native_memory?lang=en