hystrix配置
配置下：
"hystrix.threadpool.default.keepAliveTimeMinutes": 1
"hystrix.threadpool.default.allowMaximumSizeToDivergeFromCoreSize": true
"hystrix.threadpool.default.coreSize": 100
"hystrix.threadpool.default.maximumSize": 200
运行10分钟：
[root@ip-172-29-206-104 applogs]# jcmd 2270 VM.native_memory summary scale=MB
2270: Native Memory Tracking: Total: reserved=4464MB, committed=3264MB
- Java Heap (reserved=2304MB, committed=2304MB)
(mmap: reserved=2304MB, committed=2304MB) - Class (reserved=1105MB, committed=91MB)
(classes #14894)
(malloc=5MB #33268)
(mmap: reserved=1100MB, committed=85MB) - Thread (reserved=671MB, committed=671MB)
(thread #666)
(stack: reserved=668MB, committed=668MB)
(malloc=2MB #3329)
(arena=1MB #1330) - Code (reserved=256MB, committed=71MB)
(malloc=12MB #17946)
(mmap: reserved=244MB, committed=58MB) - GC (reserved=87MB, committed=87MB)
(malloc=3MB #657)
(mmap: reserved=83MB, committed=83MB) - Compiler (reserved=1MB, committed=1MB)
(malloc=1MB #2082) - Internal (reserved=15MB, committed=15MB)
(malloc=15MB #30165) - Symbol (reserved=18MB, committed=18MB)
(malloc=15MB #163714)
(arena=3MB #1) - Native Memory Tracking (reserved=4MB, committed=4MB)
(tracking overhead=4MB) - Arena Chunk (reserved=3MB, committed=3MB)
(malloc=3MB) [root@ip-172-29-206-104 applogs]#
heap内存：
C[root@ip-172-29-206-104 applogs]# jstat -gc 2270 3000
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
16384.0 15872.0 14134.4 0.0 1147392.0 875335.0 1179648.0 130125.1 87208.0 84443.8 10408.0 9848.4 242 6.232 3 0.582 6.813
15872.0 15872.0 0.0 13996.8 1147392.0 82768.0 1179648.0 130237.2 87208.0 84443.8 10408.0 9848.4 243 6.257 3 0.582 6.839
15872.0 15872.0 0.0 13996.8 1147392.0 788559.4 1179648.0 130237.2 87208.0 84443.8 10408.0 9848.4 243 6.257 3 0.582 6.839
15872.0 15872.0 14883.6 0.0 1147904.0 167285.7 1179648.0 130285.2 87208.0 84443.8 10408.0 9848.4 244 6.278 3 0.582 6.859
15872.0 15872.0 14883.6 0.0 1147904.0 168735.1 1179648.0 130285.2 87208.0 84443.8 10408.0 9848.4 244 6.278 3 0.582 6.859
运行30分钟：
[root@ip-172-29-206-104 applogs]# jcmd 2270 VM.native_memory summary scale=MB
2270: Native Memory Tracking: Total: reserved=4603MB, committed=3404MB
- Java Heap (reserved=2304MB, committed=2304MB)
(mmap: reserved=2304MB, committed=2304MB) - Class (reserved=1105MB, committed=91MB)
(classes #14904)
(malloc=5MB #34390)
(mmap: reserved=1100MB, committed=85MB) - Thread (reserved=811MB, committed=811MB)
(thread #805)
(stack: reserved=807MB, committed=807MB)
(malloc=3MB #4029)
(arena=1MB #1610) - Code (reserved=256MB, committed=71MB)
(malloc=13MB #18351)
(mmap: reserved=244MB, committed=59MB) - GC (reserved=87MB, committed=87MB)
(malloc=3MB #658)
(mmap: reserved=83MB, committed=83MB) - Compiler (reserved=1MB, committed=1MB)
(malloc=1MB #2307) - Internal (reserved=16MB, committed=16MB)
(malloc=16MB #30918) - Symbol (reserved=18MB, committed=18MB)
(malloc=15MB #163761)
(arena=3MB #1) - Native Memory Tracking (reserved=4MB, committed=4MB)
(tracking overhead=4MB) [root@ip-172-29-206-104 applogs]#
heap：
[root@ip-172-29-206-104 applogs]# jstat -gc 2270 3000
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
15872.0 15872.0 13914.5 0.0 1147904.0 283049.1 1179648.0 185076.9 87464.0 84664.7 10408.0 9851.4 578 13.968 3 0.582 14.550
15872.0 15872.0 13914.5 0.0 1147904.0 675319.9 1179648.0 185076.9 87464.0 84664.7 10408.0 9851.4 578 13.968 3 0.582 14.550
15872.0 15872.0 0.0 15480.5 1147904.0 249504.6 1179648.0 185116.9 87464.0 84664.7 10408.0 9851.4 579 13.995 3 0.582 14.577
15872.0 16384.0 12641.9 0.0 1146880.0 149747.0 1179648.0 185140.9 87464.0 84665.2 10408.0 9851.4 580 14.014 3 0.582 14.596
15872.0 16384.0 12641.9 0.0 1146880.0 710224.5 1179648.0 185140.9 87464.0 84665.2 10408.0 9851.4 580 14.014 3 0.582 14.596
^C[root@ip-172-29-206-104 applogs]#
停止后：
线程数：
配置：
"hystrix.threadpool.default.keepAliveTimeMinutes": 1
"hystrix.threadpool.default.allowMaximumSizeToDivergeFromCoreSize": true
"hystrix.threadpool.default.coreSize": 10
"hystrix.threadpool.default.maximumSize": 200
运行10分钟后
[root@ip-172-29-206-104 applogs]# jcmd 12410 VM.native_memory summary scale=MB
12410: Native Memory Tracking: Total: reserved=4140MB, committed=2942MB
- Java Heap (reserved=2304MB, committed=2304MB)
(mmap: reserved=2304MB, committed=2304MB) - Class (reserved=1105MB, committed=92MB)
(classes #15124)
(malloc=5MB #34716)
(mmap: reserved=1100MB, committed=86MB) - Thread (reserved=343MB, committed=343MB)
(thread #341)
(stack: reserved=341MB, committed=341MB)
(malloc=1MB #1704)
(arena=1MB #680) - Code (reserved=256MB, committed=72MB)
(malloc=13MB #18164)
(mmap: reserved=244MB, committed=59MB) - GC (reserved=87MB, committed=87MB)
(malloc=3MB #852)
(mmap: reserved=83MB, committed=83MB) - Compiler (reserved=1MB, committed=1MB)
(malloc=1MB #2050) - Internal (reserved=17MB, committed=17MB)
(malloc=16MB #32870) - Symbol (reserved=18MB, committed=18MB)
(malloc=15MB #164149)
(arena=3MB #1) - Native Memory Tracking (reserved=4MB, committed=4MB)
(tracking overhead=4MB) - Arena Chunk (reserved=4MB, committed=4MB)
(malloc=4MB) [root@ip-172-29-206-104 applogs]#
配置：
[root@ip-172-29-206-104 applogs]# jcmd 12410 VM.native_memory summary scale=MB
12410: Native Memory Tracking: Total: reserved=4152MB, committed=2952MB
- Java Heap (reserved=2304MB, committed=2304MB)
(mmap: reserved=2304MB, committed=2304MB) - Class (reserved=1108MB, committed=92MB)
(classes #15132)
(malloc=6MB #35295)
(mmap: reserved=1102MB, committed=87MB) - Thread (reserved=356MB, committed=356MB)
(thread #354)
(stack: reserved=354MB, committed=354MB)
(malloc=1MB #1769)
(arena=1MB #706) - Code (reserved=257MB, committed=73MB)
(malloc=13MB #18377)
(mmap: reserved=244MB, committed=60MB) - GC (reserved=87MB, committed=87MB)
(malloc=3MB #853)
(mmap: reserved=83MB, committed=83MB) - Compiler (reserved=1MB, committed=1MB)
(malloc=1MB #1966) - Internal (reserved=17MB, committed=17MB)
(malloc=16MB #32946) - Symbol (reserved=18MB, committed=18MB)
(malloc=15MB #164173)
(arena=3MB #1) - Native Memory Tracking (reserved=4MB, committed=4MB)
(tracking overhead=4MB) - Arena Chunk (reserved=1MB, committed=1MB)
(malloc=1MB) [root@ip-172-29-206-104 applogs]# jstat -gc 12410 3000
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
16384.0 16384.0 13833.8 0.0 1146880.0 534503.7 1179648.0 217032.5 88724.0 85477.1 10684.0 9984.5 530 12.683 3 0.494 13.177
16384.0 16384.0 0.0 16070.7 1146880.0 574405.1 1179648.0 217127.6 88724.0 85478.4 10684.0 9984.5 531 12.704 3 0.494 13.198
16384.0 17408.0 15181.9 0.0 1144832.0 464593.0 1179648.0 217183.7 88724.0 85478.4 10684.0 9984.5 532 12.731 3 0.494 13.225
16896.0 17408.0 0.0 14509.7 1144832.0 419176.0 1179648.0 217199.7 88724.0 85478.4 10684.0 9984.5 533 12.758 3 0.494 13.252
16896.0 17408.0 15334.4 0.0 1144832.0 44737.8 1179648.0 217239.7 88724.0 85478.4 10684.0 9984.5 534 12.780 3 0.494 13.274
16896.0 17408.0 15334.4 0.0 1144832.0 757211.2 1179648.0 217239.7 88724.0 85478.4 10684.0 9984.5 534 12.780 3 0.494 13.274
17408.0 17408.0 0.0 14829.8 1144832.0 804482.2 1179648.0 217263.7 88724.0 85478.4 10684.0 9984.5 535 12.800 3 0.494 13.294
17408.0 17408.0 15166.5 0.0 1144832.0 465056.0 1179648.0 217431.8 88724.0 85478.4 10684.0 9984.5 536 12.826 3 0.494 13.320
^C[root@ip-172-29-206-104 applogs]#
停下来后
^C[root@ip-172-29-206-104 applogs]# jcmd 12410 VM.native_memory summary scale=MB
12410: Native Memory Tracking: Total: reserved=4007MB, committed=2810MB
- Java Heap (reserved=2304MB, committed=2304MB)
(mmap: reserved=2304MB, committed=2304MB) - Class (reserved=1108MB, committed=92MB)
(classes #15168)
(malloc=6MB #35582)
(mmap: reserved=1102MB, committed=87MB) - Thread (reserved=212MB, committed=212MB)
(thread #211)
(stack: reserved=211MB, committed=211MB)
(malloc=1MB #1054) - Code (reserved=257MB, committed=75MB)
(malloc=13MB #18704)
(mmap: reserved=244MB, committed=61MB) - GC (reserved=87MB, committed=87MB)
(malloc=3MB #866)
(mmap: reserved=83MB, committed=83MB) - Compiler (reserved=1MB, committed=1MB)
(malloc=1MB #2067) - Internal (reserved=17MB, committed=17MB)
(malloc=17MB #32299) - Symbol (reserved=18MB, committed=18MB)
(malloc=15MB #164213)
(arena=3MB #1) - Native Memory Tracking (reserved=4MB, committed=4MB)
(tracking overhead=4MB) [root@ip-172-29-206-104 applogs]# jstat -gc 12410 3000
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
17408.0 8192.0 0.0 7931.6 1145856.0 244561.9 1179648.0 227692.8 88980.0 85694.5 10684.0 10004.3 627 14.930 3 0.494 15.424
17408.0 8192.0 0.0 7931.6 1145856.0 246040.1 1179648.0 227692.8 88980.0 85694.5 10684.0 10004.3 627 14.930 3 0.494 15.424
17408.0 8192.0 0.0 7931.6 1145856.0 248604.8 1179648.0 227692.8 88980.0 85694.5 10684.0 10004.3 627 14.930 3 0.494 15.424
17408.0 8192.0 0.0 7931.6 1145856.0 250533.7 1179648.0 227692.8 88980.0 85694.5 10684.0 10004.3 627 14.930 3 0.494 15.424
17408.0 8192.0 0.0 7931.6 1145856.0 252890.4 1179648.0 227692.8 88980.0 85694.5 10684.0 10004.3 627 14.930 3 0.494 15.424
17408.0 8192.0 0.0 7931.6 1145856.0 254778.7 1179648.0 227692.8 88980.0 85694.5 10684.0 10004.3 627 14.930 3 0.494 15.424
17408.0 8192.0 0.0 7931.6 1145856.0 257137.2 1179648.0 227692.8 88980.0 85694.5 10684.0 10004.3 627 14.930 3 0.494 15.424
^C[root@ip-172-29-206-104 applogs]#
90分钟
[root@ip-172-29-206-104 applogs]# jcmd 4783 VM.native_memory summary scale=MB
4783: Native Memory Tracking: Total: reserved=4239MB, committed=3042MB
- Java Heap (reserved=2304MB, committed=2304MB)
(mmap: reserved=2304MB, committed=2304MB) - Class (reserved=1105MB, committed=91MB)
(classes #14901)
(malloc=5MB #34728)
(mmap: reserved=1100MB, committed=86MB) - Thread (reserved=448MB, committed=448MB)
(thread #445)
(stack: reserved=446MB, committed=446MB)
(malloc=1MB #2224)
(arena=1MB #888) - Code (reserved=257MB, committed=74MB)
(malloc=13MB #18479)
(mmap: reserved=244MB, committed=61MB) - GC (reserved=87MB, committed=87MB)
(malloc=3MB #656)
(mmap: reserved=83MB, committed=83MB) - Compiler (reserved=1MB, committed=1MB)
(malloc=1MB #1969) - Internal (reserved=15MB, committed=15MB)
(malloc=15MB #24605) - Symbol (reserved=18MB, committed=18MB)
(malloc=15MB #163771)
(arena=3MB #1) - Native Memory Tracking (reserved=4MB, committed=4MB)
(tracking overhead=4MB) [root@ip-172-29-206-104 applogs]# jstat -gc 4783 3000
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
17920.0 18432.0 0.0 14979.3 1142784.0 830986.9 1179648.0 280732.7 87640.0 84955.0 10368.0 9847.5 1281 34.591 3 0.538 35.129
17920.0 17920.0 14404.7 0.0 1143808.0 785266.3 1179648.0 281004.9 87640.0 84955.0 10368.0 9847.5 1282 34.615 3 0.538 35.153
17920.0 17920.0 0.0 15523.0 1143808.0 719577.6 1179648.0 281036.9 87640.0 84955.0 10368.0 9847.5 1283 34.638 3 0.538 35.176
17920.0 17920.0 14848.6 0.0 1143808.0 700206.5 1179648.0 281211.4 87640.0 84955.0 10368.0 9847.5 1284 34.660 3 0.538 35.199
17920.0 17920.0 0.0 16750.7 1143808.0 231795.8 1179648.0 281259.0 87640.0 84955.0 10368.0 9847.5 1285 34.688 3 0.538 35.226
17920.0 17920.0 15232.1 0.0 1143808.0 235731.5 1179648.0 281330.4 87640.0 84955.0 10368.0 9847.5 1286 34.713 3 0.538 35.251
17920.0 17920.0 15232.1 0.0 1143808.0 361054.2 1179648.0 281330.4 87640.0 84955.0 10368.0 9847.5 1286 34.713 3 0.538 35.251
17920.0 17920.0 15232.1 0.0 1143808.0 876187.4 1179648.0 281330.4 87640.0 84955.0 10368.0 9847.5 1286 34.713 3 0.538 35.251
17920.0 17920.0 0.0 14090.7 1143808.0 266485.3 1179648.0 281410.4 87640.0 84955.0 10368.0 9847.5 1287 34.736 3 0.538 35.275
17920.0 17920.0 16248.8 0.0 1143808.0 171034.3 1179648.0 281552.8 87640.0 84955.0 10368.0 9847.5 1288 34.774 3 0.538 35.312
18432.0 16896.0 0.0 16691.1 1143808.0 98403.1 1179648.0 281648.0 87640.0 84955.0 10368.0 9847.5 1289 34.803 3 0.538 35.342
18432.0 16896.0 0.0 16691.1 1143808.0 1000258.2 1179648.0 281648.0 87640.0 84955.0 10368.0 9847.5 1289 34.803 3 0.538 35.342
18432.0 17920.0 14650.0 0.0 1143296.0 501884.8 1179648.0 281680.0 87640.0 84955.0 10368.0 9847.5 1290 34.830 3 0.538 35.369
^C[root@ip-172-29-206-104 applogs]jstack 4783 >thread_dump_zi_pool_90m.txt
[root@ip-172-29-206-104 applogs]# sz thread_dump_zi_pool_90m.txt
[root@ip-172-29-206-104 applogs]
运行3小时后：
[root@ip-172-29-206-104 applogs]# jcmd 4783 VM.native_memory summary scale=MB
4783: Native Memory Tracking: Total: reserved=4248MB, committed=3053MB
- Java Heap (reserved=2304MB, committed=2304MB)
(mmap: reserved=2304MB, committed=2304MB) - Class (reserved=1105MB, committed=91MB)
(classes #14946)
(malloc=5MB #35224)
(mmap: reserved=1100MB, committed=86MB) - Thread (reserved=456MB, committed=456MB)
(thread #453)
(stack: reserved=454MB, committed=454MB)
(malloc=1MB #2264)
(arena=1MB #904) - Code (reserved=257MB, committed=76MB)
(malloc=13MB #18778)
(mmap: reserved=244MB, committed=63MB) - GC (reserved=87MB, committed=87MB)
(malloc=3MB #657)
(mmap: reserved=83MB, committed=83MB) - Compiler (reserved=1MB, committed=1MB)
(malloc=1MB #2011) - Internal (reserved=15MB, committed=15MB)
(malloc=15MB #24741) - Symbol (reserved=18MB, committed=18MB)
(malloc=15MB #164071)
(arena=3MB #1) - Native Memory Tracking (reserved=4MB, committed=4MB)
(tracking overhead=4MB) [root@ip-172-29-206-104 applogs]# jstat -gc 4783 3000
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
18432.0 18432.0 14415.1 0.0 1142784.0 851858.5 1179648.0 385107.3 88024.0 85297.6 10496.0 9877.1 2224 60.663 3 0.538 61.202
17920.0 18432.0 0.0 15092.2 1142784.0 589943.1 1179648.0 385375.9 88024.0 85297.6 10496.0 9877.1 2225 60.686 3 0.538 61.225
17920.0 17920.0 14179.5 0.0 1143808.0 371359.0 1179648.0 385399.9 88024.0 85297.6 10496.0 9877.1 2226 60.709 3 0.538 61.248
17920.0 17920.0 0.0 14082.2 1143808.0 286732.4 1179648.0 385763.3 88024.0 85297.6 10496.0 9877.1 2227 60.735 3 0.538 61.274
17920.0 17408.0 14786.5 0.0 1144320.0 225178.6 1179648.0 385931.3 88024.0 85297.6 10496.0 9877.1 2228 60.760 3 0.538 61.298
^C[root@ip-172-29-206-104 applogs]#
停止：
[root@ip-172-29-206-104 applogs]#
jcmd 4783 VM.native_memory summary scale=MB
4783: Native Memory Tracking: Total: reserved=4013MB, committed=2818MB
- Java Heap (reserved=2304MB, committed=2304MB)
(mmap: reserved=2304MB, committed=2304MB) - Class (reserved=1105MB, committed=91MB)
(classes #14946)
(malloc=5MB #35010)
(mmap: reserved=1100MB, committed=86MB) - Thread (reserved=222MB, committed=222MB)
(thread #221)
(stack: reserved=221MB, committed=221MB)
(malloc=1MB #1104) - Code (reserved=257MB, committed=76MB)
(malloc=13MB #18793)
(mmap: reserved=244MB, committed=63MB) - GC (reserved=87MB, committed=87MB)
(malloc=3MB #657)
(mmap: reserved=83MB, committed=83MB) - Compiler (reserved=1MB, committed=1MB)
(malloc=1MB #1713) - Internal (reserved=15MB, committed=15MB)
(malloc=15MB #23583) - Symbol (reserved=18MB, committed=18MB)
(malloc=15MB #164071)
(arena=3MB #1) - Native Memory Tracking (reserved=4MB, committed=4MB)
(tracking overhead=4MB) [root@ip-172-29-206-104 applogs]# jstat -gc 4783 3000
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
20992.0 7168.0 0.0 6724.2 1138688.0 486649.0 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
20992.0 7168.0 0.0 6724.2 1138688.0 488958.8 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
20992.0 7168.0 0.0 6724.2 1138688.0 490612.6 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
20992.0 7168.0 0.0 6724.2 1138688.0 493117.2 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
20992.0 7168.0 0.0 6724.2 1138688.0 503162.0 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
20992.0 7168.0 0.0 6724.2 1138688.0 505430.8 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
20992.0 7168.0 0.0 6724.2 1138688.0 507574.2 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
20992.0 7168.0 0.0 6724.2 1138688.0 509772.0 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
20992.0 7168.0 0.0 6724.2 1138688.0 511290.4 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
20992.0 7168.0 0.0 6724.2 1138688.0 514355.1 1179648.0 395663.0 88024.0 85312.2 10496.0 9877.1 2307 62.951 3 0.538 63.490
^C[root@ip-172-29-206-104 applogs]#
压18小时后
[root@ip---- applogs]# jmap -heap
Attaching to process ID , please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.171-b11 using thread-local object allocation.
Parallel GC with thread(s) Heap Configuration:
MinHeapFreeRatio =
MaxHeapFreeRatio =
MaxHeapSize = (.0MB)
NewSize = (.0MB)
MaxNewSize = (.0MB)
OldSize = (.0MB)
NewRatio =
SurvivorRatio =
MetaspaceSize = (.796875MB)
CompressedClassSpaceSize = (.0MB)
MaxMetaspaceSize = MB
G1HeapRegionSize = (.0MB) Heap Usage:
PS Young Generation
Eden Space:
capacity = (.0MB)
used = (.07801055908203MB)
free = (.921989440918MB)
35.02016008479947% used
From Space:
capacity = (.5MB)
used = (.9591827392578125MB)
free = (.5408172607421875MB)
47.230311802455354% used
To Space:
capacity = (.5MB)
used = (.0MB)
free = (.5MB)
0.0% used
PS Old Generation
capacity = (.0MB)
used = (.7806015014648MB)
free = (.21939849853516MB)
81.92539943589105% used interned Strings occupying bytes.
[root@ip---- applogs]# jstat -gc
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
10752.0 10752.0 5078.2 0.0 1158144.0 430235.0 1179648.0 966431.3 89048.0 86254.8 10496.0 9927.5 194.456 0.538 194.995
10752.0 10752.0 5078.2 0.0 1158144.0 434796.4 1179648.0 966431.3 89048.0 86254.8 10496.0 9927.5 194.456 0.538 194.995
10752.0 10752.0 5078.2 0.0 1158144.0 438717.1 1179648.0 966431.3 89048.0 86254.8 10496.0 9927.5 194.456 0.538 194.995
^C[root@ip---- applogs]
sa