昨天南京到客户服务数据库的优化调整，其中新上线，经过审查alert.log当日志现在是在过去一段时间内取得，每隔几个小时的时间滞后，班会报似的内容：
IBM AIX on POWER Systems (64-bit)
There is new warning message in alert.log in 11.2.0.3 similar to
WARNING: Heavy swapping observed on system in last 5 mins.
pct of memory swapped in [2.08%] pct of memory swapped out [0.12%].
Please make sure there is no memory pressure and the SGA and PGA
are configured correctly. Look at DBRM trace file for more details.
On AIX platform this message can be seen even when there is no virtual memory swapping at all. --物理内存足够，并且根本没有使用swap交换空间
You may compare the vmstat from AIX level with DBRM trace file entries to see the differences.
The issue is caused by unpublished Bug:14731911.
Swap usage messages are based on statistics that do not reflect the actual usage.
The v$osstat does not reflect proper stats for the swap space paging.
解决方法：
Apply Patch:11801934 on top
of your IBM AIX on POWER Systems (64-bit) platform.
P.S: Bug is port-specific. --这个bug是针对端口指定的平台的
The issue is fixed in patchset 11.2.0.4 and release 12.1.
--说是在12.1的patch中修复了。但实际上12.1还是会有这个问题。会有ora-700错误，详见文档：[ID 1919850.1]
来看一下BUG：14731911的描写叙述：
|
| 类型 | B - Defect | 已在产品版本号中修复 | |
| 严重性 | 2 - Severe Loss of Service | 产品版本号 | 11.2.0.3 |
| 状态 | 96 - Closed, Duplicate Bug | 平台 | 212 - IBM AIX on POWER Systems (64-bit) |
| 创建时间 | 2012-10-8 | 平台版本号 | 6.1 |
| 更新时间 | 2014-10-11 | 基本 Bug | 11801934 |
| 数据库版本号 | 11.2.0.3 | 影响平台 | Port-Specific |
| 产品源 | Oracle |
id=14731911" style="font-size:14px; color:rgb(0,113,194); padding-top:5px; font-family:Helvetica,sans-serif">与此 Bug 相关的知识, 补丁程序和 Bug |
|
| 产品线 | Oracle Database Products | 系列 | Oracle Database Suite |
| 区域 | Oracle Database | 产品 | 5 - Oracle Database - Enterprise Edition |
Hdr: 14731911 11.2.0.3 RDBMS 11.2.0.3 VOS PRODID-5 PORTID-212 11801934
Abstract: FALSE SWAP WARNING MESSAGES PRINTED TO ALERT.LOG ON AIX *** 10/08/12 04:52 am ***
BUG TYPE CHOSEN
===============
Code
SubComponent: Virtual Operating System
======================================
DETAILED PROBLEM DESCRIPTION
============================
Oracle process seems to check wrong OS local statistic (which include also
FILESYSTEM caching etc.)
Alert log shows WARNING: Heavy swapping observed on system in last 5 mins.
pct of memory swapped in [2.08%] pct of memory swapped out [0.12%].
Please make sure there is no memory pressure and the SGA and PGA
are configured correctly. Look at DBRM trace file for more details.
but this is not reflected at OS level.
DIAGNOSTIC ANALYSIS
===================
1. nmon shows virtual memory swapping does not occur at all - see attached file --nmon根本没有监控到swap动作
2. Oracle Database Server is 11.2.0.3 and contains fix for 10220118
3. Server configuration
real mem: 144GB
lowest value of fre memory : 87,65 GB --剩余内存充足
4. DBRM seems to use a wrong OS statistics - trace file is attached
WORKAROUND?
===========
No
TECHNICAL IMPACT
================
Wrong diagnostic analyze.
Message is bothering customer's DBA when in fact the warning message is
misleading
RELATED ISSUES (bugs, forums, RFAs)
===================================
http://myforums.oracle.com/jive3/thread.jspa?threadID=1104581
10220118
HOW OFTEN DOES THE ISSUE REPRODUCE AT CUSTOMER SITE? ====================================================
Always
DOES THE ISSUE REPRODUCE INTERNALLY? ====================================
No
EXPLAIN WHY THE ISSUE WAS NOT TESTED INTERNALLY.
================================================
Unavailable Data Volume
IS A TESTCASE AVAILABLE?
========================
No
Link to IPS Package:
====================
not available
可是个人觉得这个bug的危害性并不大，仅仅仅仅是在alert.log日志中报了一个WARNING，并没有由于这个影响导致对数据库更加负面的影响，因此是否打补丁到11.2.0.4就见仁见智了。假设想让alert.log平安无事。那么就能够升级一下patch。
当然了，假设真的是由于OS内存吃紧造成的swap动作。就要差别对待了。由于此时的确会对数据库造成严重影响。要区分是否真的内存不足而非系统误报。那么主要还是通过nmon，topas，vmstat等监控工具来进行分析（linux下还能够用free监控）
AIX Platform
If your Platform is IBM-AIX then this is not the only possible reason for this alert log message. id=14731911&parent=DOCUMENT&sourceId=1452790.1">Bug 14731911
For IBM AIX on POWER Systems (64-bit), there is also next known port-specific bug:
with unpublished base bug:
Bug 11801934 : WRONG PAGE-IN AND PAGE-OUT OS VM STATS IN AIX.
在vmware平台中的这个WARNING信息。假设不是bug引起，则非常有可能和ora-04031/ora-04030相关，这个就严重多了
VMWare
Under VMWare, the messages may perhaps indicate a more serious issue, even when no memory related ORA-4031/ORA-4030 errors are reported.
Under circumstances, an instance in a virtual machine may be simply terminated by PMON due to error 471 without further errors in the alert log.
The OS logs may in such case report an out of memory condition like below:
/var/log/messages-20140629:Jun 27 18:29:06 vmh-msfc-dodp02 kernel: [1895074.304941] Out of memory: Kill process 42094 (oracle) score 391 or sacrifice child
/var/log/messages-20140629:Jun 27 18:29:06 vmh-msfc-dodp02 kernel: [1895074.305203] Killed process 42094, UID 303, (oracle) total-vm:189081588kB, anon-rss:27412kB, file-rss:109612
通常解决OS内存swap问题有下面几种方案：
vm.min_free_kbytes:Raising the value in /proc/sys/vm/min_free_kbytes will
cause the system to start reclaiming memory at an earlier time than it would have before.
vm.vfs_cache_pressure:At the default value of vfs_cache_pressure = 100 the kernel will attempt
to reclaim dentries and inodes at a “fair” rate with respect to pagecache and swapcache reclaim. Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. Increasing
vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes.
vm.swappiness:default 60,Apparently /proc/sys/vm/swappiness on
Red Hat Linux allows the admin to tune how aggressively the kernel swaps out processes'memory. Decreasing the swappiness setting may result in improved Directory performance as the kernel holds more of the server process in memory longer before swapping it
out.
设置下面值，以降低OOM(Out Of Memory)的可能性:
# Oracle-Validated setting for vm.min_free_kbytes is 51200 to avoid OOM killer
vm.min_free_kbytes = 51200
vm.swappiness = 40
vm.vfs_cache_pressure = 200
版权声明：本文博客原创文章，博客，未经同意，不得转载。