OOMAnalyser is a web page to analyse and explain the OOM message of a Linux kernel. The analysis is performed automatically in the browser. You get a summary and a list of (almost) all parameters with their values and a short explanation. Among them are also 2 diagrams that illustrate the memory usage.
OOMAnalyser consists of a web page where the OOM message is copied into the input field. JavaScript code extracts the data and displays the details. All processing takes place in the browser. No data is transferred to any server. Therefore, confidential OOM messages can also be analyzed, and the analysis can be performed in environments without an Internet connection.
This project is written in Python and uses Transcrypt to translate Python code into JavaScript.
Go back to "Step 1 - Enter your OOM message" to run a new analysis.
The OOM killer was automatically triggered to free memory, because the system couldn't satisfy the memory request. The OOM killer calculates a score for each process and terminates the process with the highest score to satisfy the original memory request.
The OOM killer was manually triggered (e.g. with "echo f > /proc/sysrq-trigger")
by a user with root privileges. There is no demand to free memory but the OOM killer started
nevertheless. The OOM killer calculates a score for each process and terminates the process with the
highest score.
The system has physical memory and swap space. That's total. ( out of ) physical memory and ( out of ) swap space are in use.
The system has physical memory and no swap space. ( out of ) physical memory are in use.
The process "" (PID ) requested a memory chunk of order from the "" memory zone. This is 2order pages == 2 pages == a == memory.
The request failed because the free memory would be below the memory low watermark after its completion. If this requirement was satisfied, the free memory would still be above the low memory watermark. The request failed because there is no free chunk in the current or higher order. The request failed, but the reason is unknown. This memory shortage triggers the OOM process.
The process "" (PID ) with an OOM score of has been terminated. It uses () of the resident memory.
Dynamic memory allocation is used by both the kernel and all applications. This leads to memory
fragmentation and is a common behavior.
The system memory is heavily fragmented, because all chunks with an order ≥
are in use. Allocation of larger contiguous
memory areas will fail.
The amount of fragmentation depends on the software run on the system and the best fix would be
to improve the software that causes heavy fragmentation. If that's not possible, compacting the
memory (memory defragmentation) can be triggered more often.
The system memory is not heavily fragmented, because chunks with an order ≥
are freely available.
Top this list are the actions that have the greatest impact on preventing OOMs.
1 to /proc/sys/vm/compact_memory.
/proc/sys/vm/watermark_scale_factor
/proc/sys/vm/watermark_boost_factor
3 to
/proc/sys/vm/zone_reclaim_mode
/proc/sys/vm/extfrag_threshold to start memory compaction earlier
The result of the analysis is displayed in three columns. The first column is used to name the property including the original OOM identifier in brackets. The extracted information is displayed in the second column. The last column contains further details and additional information.
| Trigger Process | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| (PID ) | This process requests memory and is triggering thereby the OOM situation. | |||||||||||
| Memory allocation flags (gfp_mask) |
These flags are also called GFP (get free pages) flags. They control the kernel-internal memory
allocation. Some OOM variants include the individual flags in addition to the hexadecimal representation. The flags are calculated if they are not specified. |
|||||||||||
| Node mask to show on which CPU Cores this process can run (nodemask) |
Bit mask indicating the cores on which the process can run. | |||||||||||
| Requested memory: order | The kernel specifies the requested number of pages as an exponent of a power of two. | |||||||||||
| Requested memory: number of pages | (2 pages) a per page | Requested memory in number of pages. | ||||||||||
| Requested memory: size | Requested memory in kBytes. | |||||||||||
| Requested memory: zone | Memory zone from which the requested storage chunk should come. | |||||||||||
| Requested memory: node |
First NUMA node with memory shortage watermark free < min in memory watermark
information.
Assumption that this is the node where the OOM was triggered. |
|||||||||||
| Adjust oom-killer score (oom_score_adj) |
This value is added to the badness score before it's used to determine the process to be killed. | |||||||||||
| Killed Process | ||||||||||||
| (PID ) | Process killed by Linux kernel to satisfy the memory request. | |||||||||||
| OOM Score (score) |
Programs with the highest OOM score are terminated first. | |||||||||||
| Virtual Memory (total_vm) |
Virtual memory used by this process. | |||||||||||
| Total resident anonymous memory (rss) |
All virtual process memory mapped into RAM. Normally, a process always requests more virtual memory
than it uses. TotalRSS = anon-rss + file-rss + shmem-rss |
|||||||||||
| Resident anonymous memory (anon-rss) |
Resident anonymous pages Part of the virtual process memory mapped into RAM. |
|||||||||||
| Resident file mapping memory (file-rss) |
Resident file mapping pages Files which have been mapped into RAM (with mmap(2).) |
|||||||||||
| Resident shared memory (shmem-rss) |
Resident shared memory pages This may include System V shared memory and shared anonymous memory. |
|||||||||||
| Memory Usage | ||||||||||||
| Graphs | ||||||||||||
| RAM Summary | ||||||||||||
| Swap Summary | ||||||||||||
| Swap Usage | ||||||||||||
| No swap space available | ||||||||||||
| Swap Total | Total amount of swap space available. [1] | |||||||||||
| Swap Free | Amount of swap space that is currently unused. [1] | |||||||||||
| Swap Cached | Memory that once was swapped out, is swapped back in but still also is in the swap file. (If memory pressure is high, these pages don't need to be swapped out again because they are already in the swap file. This saves I/O). [1] | |||||||||||
| Swap Used | Amount of used swap space w/o cached swap ( SwapUsed = SwapTotal - SwapFree -SwapCache)
|
|||||||||||
| Memory Pages | ||||||||||||
| RAM pages | Total number of RAM pages | |||||||||||
| HighMem/MovableOnly | Number of pages in the High Memory Area or marked movable for Contiguous Memory Allocator (CMA).
HighMem pages are also counted in the total page number. |
|||||||||||
| Reserved pages | Number of reserved pages | |||||||||||
| CMA reserved pages | 0 | Pages reserved for Contiguous Memory Allocator (CMA) | ||||||||||
| Pagetable Cache | 0 | Number of pages in pagetable cache | ||||||||||
| Number of pages with hardware errors | 0 | Pages with uncorrectable memory errors | ||||||||||
| Memory Usage Details | ||||||||||||
| Active anonymous memory (active_anon) |
Recently used anonymous memory. These memory pages are not normally swapped out. |
|||||||||||
| Inactive anonymous memory (inactive_anon) |
Least recently used anonymous memory. These memory pages can be swapped out. |
|||||||||||
| Isolated anonymous memory (isolated_anon) |
Memory isolation is used to separate memory between different virtual machines. | |||||||||||
| Active Pagecache (active_file) |
Pagecache that has been used more recently and usually not reclaimed unless absolutely necessary. | |||||||||||
| Inactive Pagecache (inactive_file) |
Pagecache which has been less recently used. It can be reclaimed without huge performance impact. | |||||||||||
| Isolated Pagecache (isolated_file) |
Memory isolation is used to separate memory between different virtual machines. | |||||||||||
| Unevictable Pages (unevictable) |
Unevictable memory. It can't be swapped out because the pages are owned by ramfs or protected by mlock(3) / shmctl(SHM_LOCK). Unevictable pages are managed by kernels LRU framework. | |||||||||||
| Dirty Pages (dirty) |
Memory which is waiting to get written back to the disk. [1] | |||||||||||
| Writeback (writeback) |
Memory which is actively being written back to the disk. [1] | |||||||||||
| Unstable (unstable) |
Not yet committed to stable storage. | |||||||||||
| Slab Reclaimable (slab_reclaimable) |
Slab is an in-kernel data structures cache. Part of Slab, that might be reclaimed, such as caches.
[1]
Additional details are listed in slabinfo(5) also. |
|||||||||||
| Slab Unreclaimable (slab_unreclaimable) |
Part of Slab, that cannot be reclaimed on memory pressure. [1] | |||||||||||
| Mapped (mapped) |
Files which have been mapped into memory (with mmap(2)), such as libraries. [1] | |||||||||||
| Shared Memory (shmem) |
Amount of memory consumed in tmpfs(5) filesystems. [1] | |||||||||||
| Pagetables (pagetables) |
Amount of memory dedicated to the lowest level of pagetables. [1] | |||||||||||
| Bounce (bounce) |
Memory used for block device "bounce buffers". [1] | |||||||||||
| Free pages (free) |
Free pages | |||||||||||
| Free per-CPU pages (free_pcp) |
Free number of pages per CPU | |||||||||||
| Free CMA pages (free_cma) |
Pages reserved but not used by Contiguous Memory Allocator (CMA) | |||||||||||
| Total Pagecache | Total number of pages in pagecache | |||||||||||
| Operating System | ||||||||||||
| Kernel | ||||||||||||
| Distribution | Guessed from the kernel version | |||||||||||
| Platform | Guessed from the kernel version | |||||||||||
| Page size | Extracted from DMA zone buddyinfo Guessed | |||||||||||
| Available Memory Chunks | ||||||||||||
| Memory Watermarks | ||||||||||||
| Process Table | ||||||||||||
|
||||||||||||
| Hardware Details | ||||||||||||
| Kernel Call Trace | ||||||||||||
| Entire OOM Message (click to hide) | ||||||||||||
Go back to "Step 1 - Enter your OOM message" to run a new analysis.
mm/omm_kill.c
at kernel.org or more comfortable
mm/omm_kill.c
at elixir.bootlin.com
mm/page_alloc.c:zone_watermark_ok()
at kernel.org or more comfortable
mm/page_alloc.c:zone_watermark_ok()
at elixir.bootlin.com
/proc/sys/vm/" contains explanations to some of the memory compaction settings.
OOMAnalyser.html in your favourite browser.Copyright (c) 2017-2024 Carsten Grohmann mail <add at here> carstengrohmann.de
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.